Vanishing/Exploding Gradients ProblemsXavier and He InitializationNonsaturating Activation FunctionsBatch NormalizationGradient ClippingReusing Pretrained LayersReusing a TensorFlow ModelReusing Models from Other FrameworksFreezing the Lower LayersCaching the Frozen LayersTweaking, Dropping, or Replacing the Upper LayersModel ZoosUnsupervised PretrainingPretraining on an Auxiliary TaskFaster OptimizersMomentum OptimizationNesterov Accelerated GradientAdaGradRMSPropAdam OptimizationLearning Rate SchedulingAvoiding Overfitting Through RegularizationEarly Stoppingℓ1 and ℓ2 RegularizationDropoutMax-Norm RegularizationData AugmentationPractical GuidelinesExercises