Deep Learning By Example

The intuition behind TL

Let's build up the intuition behind TL by using the following teacher-student analogy. A teacher has many years of experience in the modules that he'she's teaching. On the other side, the students get a compact overview of the topic from the lectures that this teacher gives. So you can say that the teacher is transferring their knowledge in a concise and compact way to the students.

The same analogy of the teacher and students can be applied to our case of transferring knowledge in deep learning, or in neural networks in general. So our model learns some representations from the data, which is represented by the weights of the network. These learned representations/features (weights) can be transferred to another different ...