Mastering Machine Learning for Penetration Testing

Detecting overfitting

To detect overfitting, it is highly recommended to split the initial dataset into a training set and a testing set. If the training set performs way better than the testing set, then we have a problem. Also, it is highly recommended to start with a simple algorithm and move on to more complex models later, checking whether upgrading the level of complexity was worth it. To defend against overfitting, we can use cross-validation. Cross-validation is the process of evaluating many machine learning techniques by training models with different subsets (k subsets).