Deep Learning with PyTorch

Training, validation, and test split

It is best practice to split the data into three parts—training, validation, and test datasets. The best approach for using the holdout dataset is to:

1. Train the algorithm on the training dataset
2. Perform hyper parameter tuning based on the validation dataset
3. Perform the first two steps iteratively until the expected performance is achieved
4. After freezing the algorithm and the hyper parameters, evaluate it on the test dataset

Avoid splitting the data into two parts, as it may lead to an information leak. Training and testing it on the same dataset is a clear no-no as it does not guarantee algorithm generalization. There are three popular holdout strategies that can be used to split the data into training ...