Splitting data

The objective of machine learning is to develop an algorithm that performs well on future data, which, by definition, is unseen. How can we simulate the behavior of feature data? We split a dataset into training and test data. Some of the dataset is held back, and we develop our algorithm using the training data.

The first technique is to split data into a training dataset and a testing dataset. We use the training data to develop our algorithm. We then see how well the algorithm generalizes by applying the trained algorithm to the test data and quantifying the error rate.

The train_test_split() function, from scikit-learn (sklearn), makes splitting data easy, as seen in the following steps:

  1. We will import this function, and ...

Get Training Systems Using Python Statistical Modeling now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.