Python Natural Language Processing

Other important concepts

In this section, we will look at those concepts that help us know how the training on our dataset using ML algorithms is going, how you should judge whether the generated ML-model will be able to generalize unseen scenarios or not, and what signs tell you that your ML-model can't generalize the unseen scenarios properly. Once you detect these situations, what steps should you take? What are the widely used evaluation matrices for NLP applications?

So, let's find answers to all these questions. I'm going to cover the following topics. We will look at all of them one by one:

• Bias-variance trade-off
• Underfitting
• Overfitting
• Evaluation matrix