Scala Machine Learning Projects

Step 8 - Measuring the likelihood of two documents

Now to get some more statistics, such as maximum likelihood or log likelihood on the document, we can use the following code:

if (ldaModel.isInstanceOf[DistributedLDAModel]) {    val distLDAModel = ldaModel.asInstanceOf[DistributedLDAModel]    val avgLogLikelihood = distLDAModel.logLikelihood / actualCorpusSize.toDouble    println("The average log likelihood of the training data: " +

avgLogLikelihood)    println()}

The preceding code calculates the average log likelihood of the LDA model as an instance of the distributed version of the LDA model:

The average log likelihood of the training data: -209692.79314860413

For more information on the likelihood measurement, interested readers should refer to  ...