Inside Deep Learning

7 Autoencoding and self-supervision

This chapter covers

• Training without labels
• Autoencoding to project data
• Constraining networks with bottlenecks
• Adding noise to improve performance
• Predicting the next item to make generative models

You now know several approaches to specifying a neural network for classification and regression problems. These are the classic machine learning problems, where for each data point x (e.g., a picture of a fruit), we have an associated answer y (e.g., fresh or rotten). But what if we do not have a label y? Is there any useful way for us to learn? You should recognize this as an unsupervised learning scenario.

People are interested in self-supervision because labels are expensive. It is often much easier to get ...