Modern Deep Learning Design and Application Development: Versatile Tools to Solve Deep Learning Problems

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022

A. YeModern Deep Learning Design and Application Developmenthttps://doi.org/10.1007/978-1-4842-7413-2_2

2. Pretraining Strategies and Transfer Learning

Andre Ye¹  

(1)

Redmond, WA, USA

 

Steal from anywhere that resonates with inspiration or fuels your imagination… And don’t bother concealing your thievery – celebrate if you feel like it. In any case, remember what Jean-Luc Godard said: “It’s not where you take things from – it’s where you take them to.”

—Jim Jarmusch, American film director¹

Obtaining a good deep neural network for most modern datasets is a difficult task. Optimizers need to traverse extraordinarily high-dimensional, jagged loss landscapes and ...