O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Getting started with deep learning

Book Description

Although interest in machine learning has reached a high point, lofty expectations often scuttle projects before they get very far. How can machine learning—especially deep neural networks—make a real difference in your organization? This lesson provides a practical overview of the foundations of neural networks and deep learning and also looks at deep networks.

This lesson is for you because you are a data scientist or software developer who wants to understand and be able to use efficient tools to implement programs capable of learning from data.

Table of Contents

  1. 1. Foundations of Neural Networks and Deep Learning
    1. Neural Networks
      1. The Biological Neuron
      2. The Perceptron
      3. Multilayer Feed-Forward Networks
    2. Training Neural Networks
      1. Backpropagation Learning
    3. Activation Functions
      1. Linear
      2. Sigmoid
      3. Tanh
      4. Hard Tanh
      5. Softmax
      6. Rectified Linear
    4. Loss Functions
      1. Loss Function Notation
      2. Loss Functions for Regression
      3. Loss Functions for Classification
      4. Loss Functions for Reconstruction
    5. Hyperparameters
      1. Learning Rate
      2. Regularization
      3. Momentum
      4. Sparsity
  2. 2. Fundamentals of Deep Networks
    1. Defining Deep Learning
      1. What Is Deep Learning?
      2. Organization of This Chapter
    2. Common Architectural Principles of Deep Networks
      1. Parameters
      2. Layers
      3. Activation Functions
      4. Loss Functions
      5. Optimization Algorithms
      6. Hyperparameters
      7. Summary
    3. Building Blocks of Deep Networks
      1. RBMs
      2. Autoencoders
      3. Variational Autoencoders
  3. 3. Major Architectures of Deep Networks
    1. Unsupervised Pretrained Networks
      1. Deep Belief Networks
      2. Generative Adversarial Networks
    2. Convolutional Neural Networks (CNNs)
      1. Biological Inspiration
      2. Intuition
      3. CNN Architecture Overview
      4. Input Layers
      5. Convolutional Layers
      6. Pooling Layers
      7. Fully Connected Layers
      8. Other Applications of CNNs
      9. CNNs of Note
      10. Summary
    3. Recurrent Neural Networks
      1. Modeling the Time Dimension
      2. 3D Volumetric Input
      3. Why Not Markov Models?
      4. General Recurrent Neural Network Architecture
      5. LSTM Networks
      6. Domain-Specific Applications and Blended Networks
    4. Recursive Neural Networks
      1. Network Architecture
      2. Varieties of Recursive Neural Networks
      3. Applications of Recursive Neural Networks
    5. Summary and Discussion
      1. Will Deep Learning Make Other Algorithms Obsolete?
      2. Different Problems Have Different Best Methods
      3. When Do I Need Deep Learning?