Practical Convolutional Neural Networks

Target dataset is small but different from the original training dataset

If the target dataset is small but of a different type to the original – for example, the original dataset is dog images and the new (target) dataset is flower images – then do the following:

• Slice most of the initial layers of the network
• Add to the remaining pre-trained layers a new fully connected layer that matches the number of classes of the target dataset
• Randomize the weights of the new fully connected layer and freeze all the weights from the pre-trained network
• Train the network to update the weights of the new fully connected layer  

Since the dataset is small, overfitting is still a concern here as well. To overcome this, we will keep the weights of the ...