Machine Learning: End-to-End guide for Java developers

Model assessment, evaluation, and comparisons

The key ideas discussed here are:

• How to assess or estimate the performance of the classifier on unseen datasets that it will be predicting on future unseen datasets.
• What are the metrics that we should use to assess the performance of the model?
• How do we compare algorithms if we have to choose between them?

Model assessment

In order to train the model(s), tune the model parameters, select the models, and finally estimate the predictive behavior of models on unseen data, we need many datasets. We cannot train the model on one set of data and estimate its behavior on the same set of data, as it will have a clear optimistic bias and estimations will be unlikely to match the behavior in the unseen data. So ...