Machine Learning Algorithms

Creating training and test sets

When a dataset is large enough, it's a good practice to split it into training and test sets; the former to be used for training the model and the latter to test its performances. In the following figure, there's a schematic representation of this process:

There are two main rules in performing such an operation:

• Both datasets must reflect the original distribution
• The original dataset must be randomly shuffled before the split phase in order to avoid a correlation between consequent elements

With scikit-learn, this can be achieved using the train_test_split() function:

from sklearn.model_selection import ...