Deep Learning with Python, Second Edition

Deep Learning with Python, Second Edition

by Francois Chollet
Released November 2021
Publisher(s): Manning Publications
ISBN: 9781617296864

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Deep Learning with Python, Second Edition introduces the field of deep learning using Python and the powerful Keras library. In this revised and expanded new edition, Keras creator François Chollet offers insights for both novice and experienced machine learning practitioners. As you move through this book, you’ll build your understanding through intuitive explanations, crisp color illustrations, and clear examples. You’ll quickly pick up the skills you need to start developing deep-learning applications.

Table of contents

  1. Deep Learning with Python
  2. Copyright
  3. dedication
  4. brief contents
  5. contents
  6. front matter
    1. preface
    2. acknowledgments
    3. about this book
      1. Who should read this book
      2. About the code
      3. liveBook discussion forum
    4. about the author
    5. about the cover illustration
  7. 1 What is deep learning?
    1. 1.1 Artificial intelligence, machine learning, and deep learning
      1. 1.1.1 Artificial intelligence
      2. 1.1.2 Machine learning
      3. 1.1.3 Learning rules and representations from data
      4. 1.1.4 The “deep” in “deep learning”
      5. 1.1.5 Understanding how deep learning works, in three figures
      6. 1.1.6 What deep learning has achieved so far
      7. 1.1.7 Don’t believe the short-term hype
      8. 1.1.8 The promise of AI
    2. 1.2 Before deep learning: A brief history of machine learning
      1. 1.2.1 Probabilistic modeling
      2. 1.2.2 Early neural networks
      3. 1.2.3 Kernel methods
      4. 1.2.4 Decision trees, random forests, and gradient boosting machines
      5. 1.2.5 Back to neural networks
      6. 1.2.6 What makes deep learning different
      7. 1.2.7 The modern machine learning landscape
    3. 1.3 Why deep learning? Why now?
      1. 1.3.1 Hardware
      2. 1.3.2 Data
      3. 1.3.3 Algorithms
      4. 1.3.4 A new wave of investment
      5. 1.3.5 The democratization of deep learning
      6. 1.3.6 Will it last?
  8. 2 The mathematical building blocks of neural networks
    1. 2.1 A first look at a neural network
    2. 2.2 Data representations for neural networks
      1. 2.2.1 Scalars (rank-0 tensors)
      2. 2.2.2 Vectors (rank-1 tensors)
      3. 2.2.3 Matrices (rank-2 tensors)
      4. 2.2.4 Rank-3 and higher-rank tensors
      5. 2.2.5 Key attributes
      6. 2.2.6 Manipulating tensors in NumPy
      7. 2.2.7 The notion of data batches
      8. 2.2.8 Real-world examples of data tensors
      9. 2.2.9 Vector data
      10. 2.2.10 Timeseries data or sequence data
      11. 2.2.11 Image data
      12. 2.2.12 Video data
    3. 2.3 The gears of neural networks: Tensor operations
      1. 2.3.1 Element-wise operations
      2. 2.3.2 Broadcasting
      3. 2.3.3 Tensor product
      4. 2.3.4 Tensor reshaping
      5. 2.3.5 Geometric interpretation of tensor operations
      6. 2.3.6 A geometric interpretation of deep learning
    4. 2.4 The engine of neural networks: Gradient-based optimization
      1. 2.4.1 What’s a derivative?
      2. 2.4.2 Derivative of a tensor operation: The gradient
      3. 2.4.3 Stochastic gradient descent
      4. 2.4.4 Chaining derivatives: The Backpropagation algorithm
    5. 2.5 Looking back at our first example
      1. 2.5.1 Reimplementing our first example from scratch in TensorFlow
      2. 2.5.2 Running one training step
      3. 2.5.3 The full training loop
      4. 2.5.4 Evaluating the model
    6. Summary
  9. 3 Introduction to Keras and TensorFlow
    1. 3.1 What’s TensorFlow?
    2. 3.2 What’s Keras?
    3. 3.3 Keras and TensorFlow: A brief history
    4. 3.4 Setting up a deep learning workspace
      1. 3.4.1 Jupyter notebooks: The preferred way to run deep learning experiments
      2. 3.4.2 Using Colaboratory
    5. 3.5 First steps with TensorFlow
      1. 3.5.1 Constant tensors and variables
      2. 3.5.2 Tensor operations: Doing math in TensorFlow
      3. 3.5.3 A second look at the GradientTape API
      4. 3.5.4 An end-to-end example: A linear classifier in pure TensorFlow
    6. 3.6 Anatomy of a neural network: Understanding core Keras APIs
      1. 3.6.1 Layers: The building blocks of deep learning
      2. 3.6.2 From layers to models
      3. 3.6.3 The “compile” step: Configuring the learning process
      4. 3.6.4 Picking a loss function
      5. 3.6.5 Understanding the fit() method
      6. 3.6.6 Monitoring loss and metrics on validation data
      7. 3.6.7 Inference: Using a model after training
    7. Summary
  10. 4 Getting started with neural networks: Classification and regression
    1. 4.1 Classifying movie reviews: A binary classification example
      1. 4.1.1 The IMDB dataset
      2. 4.1.2 Preparing the data
      3. 4.1.3 Building your model
      4. 4.1.4 Validating your approach
      5. 4.1.5 Using a trained model to generate predictions on new data
      6. 4.1.6 Further experiments
      7. 4.1.7 Wrapping up
    2. 4.2 Classifying newswires: A multiclass classification example
      1. 4.2.1 The Reuters dataset
      2. 4.2.2 Preparing the data
      3. 4.2.3 Building your model
      4. 4.2.4 Validating your approach
      5. 4.2.5 Generating predictions on new data
      6. 4.2.6 A different way to handle the labels and the loss
      7. 4.2.7 The importance of having sufficiently large intermediate layers
      8. 4.2.8 Further experiments
      9. 4.2.9 Wrapping up
    3. 4.3 Predicting house prices: A regression example
      1. 4.3.1 The Boston housing price dataset
      2. 4.3.2 Preparing the data
      3. 4.3.3 Building your model
      4. 4.3.4 Validating your approach using K-fold validation
      5. 4.3.5 Generating predictions on new data
      6. 4.3.6 Wrapping up
    4. Summary
  11. 5 Fundamentals of machine learning
    1. 5.1 Generalization: The goal of machine learning
      1. 5.1.1 Underfitting and overfitting
      2. 5.1.2 The nature of generalization in deep learning
    2. 5.2 Evaluating machine learning models
      1. 5.2.1 Training, validation, and test sets
      2. 5.2.2 Beating a common-sense baseline
      3. 5.2.3 Things to keep in mind about model evaluation
    3. 5.3 Improving model fit
      1. 5.3.1 Tuning key gradient descent parameters
      2. 5.3.2 Leveraging better architecture priors
      3. 5.3.3 Increasing model capacity
    4. 5.4 Improving generalization
      1. 5.4.1 Dataset curation
      2. 5.4.2 Feature engineering
      3. 5.4.3 Using early stopping
      4. 5.4.4 Regularizing your model
    5. Summary
  12. 6 The universal workflow of machine learning
    1. 6.1 Define the task
      1. 6.1.1 Frame the problem
      2. 6.1.2 Collect a dataset
      3. 6.1.3 Understand your data
      4. 6.1.4 Choose a measure of success
    2. 6.2 Develop a model
      1. 6.2.1 Prepare the data
      2. 6.2.2 Choose an evaluation protocol
      3. 6.2.3 Beat a baseline
      4. 6.2.4 Scale up: Develop a model that overfits
      5. 6.2.5 Regularize and tune your model
    3. 6.3 Deploy the model
      1. 6.3.1 Explain your work to stakeholders and set expectations
      2. 6.3.2 Ship an inference model
      3. 6.3.3 Monitor your model in the wild
      4. 6.3.4 Maintain your model
    4. Summary
  13. 7 Working with Keras: A deep dive
    1. 7.1 A spectrum of workflows
    2. 7.2 Different ways to build Keras models
      1. 7.2.1 The Sequential model
      2. 7.2.2 The Functional API
      3. 7.2.3 Subclassing the Model class
      4. 7.2.4 Mixing and matching different components
      5. 7.2.5 Remember: Use the right tool for the job
    3. 7.3 Using built-in training and evaluation loops
      1. 7.3.1 Writing your own metrics
      2. 7.3.2 Using callbacks
      3. 7.3.3 Writing your own callbacks
      4. 7.3.4 Monitoring and visualization with TensorBoard
    4. 7.4 Writing your own training and evaluation loops
      1. 7.4.1 Training versus inference
      2. 7.4.2 Low-level usage of metrics
      3. 7.4.3 A complete training and evaluation loop
      4. 7.4.4 Make it fast with tf.function
      5. 7.4.5 Leveraging fit() with a custom training loop
    5. Summary
  14. 8 Introduction to deep learning for computer vision
    1. 8.1 Introduction to convnets
      1. 8.1.1 The convolution operation
      2. 8.1.2 The max-pooling operation
    2. 8.2 Training a convnet from scratch on a small dataset
      1. 8.2.1 The relevance of deep learning for small-data problems
      2. 8.2.2 Downloading the data
      3. 8.2.3 Building the model
      4. 8.2.4 Data preprocessing
      5. 8.2.5 Using data augmentation
    3. 8.3 Leveraging a pretrained model
      1. 8.3.1 Feature extraction with a pretrained model
      2. 8.3.2 Fine-tuning a pretrained model
    4. Summary
  15. 9 Advanced deep learning for computer vision
    1. 9.1 Three essential computer vision tasks
    2. 9.2 An image segmentation example
    3. 9.3 Modern convnet architecture patterns
      1. 9.3.1 Modularity, hierarchy, and reuse
      2. 9.3.2 Residual connections
      3. 9.3.3 Batch normalization
      4. 9.3.4 Depthwise separable convolutions
      5. 9.3.5 Putting it together: A mini Xception-like model
    4. 9.4 Interpreting what convnets learn
      1. 9.4.1 Visualizing intermediate activations
      2. 9.4.2 Visualizing convnet filters
      3. 9.4.3 Visualizing heatmaps of class activation
    5. Summary
  16. 10 Deep learning for timeseries
    1. 10.1 Different kinds of timeseries tasks
    2. 10.2 A temperature-forecasting example
      1. 10.2.1 Preparing the data
      2. 10.2.2 A common-sense, non-machine learning baseline
      3. 10.2.3 Let’s try a basic machine learning model
      4. 10.2.4 Let’s try a 1D convolutional model
      5. 10.2.5 A first recurrent baseline
    3. 10.3 Understanding recurrent neural networks
      1. 10.3.1 A recurrent layer in Keras
    4. 10.4 Advanced use of recurrent neural networks
      1. 10.4.1 Using recurrent dropout to fight overfitting
      2. 10.4.2 Stacking recurrent layers
      3. 10.4.3 Using bidirectional RNNs
      4. 10.4.4 Going even further
    5. Summary
  17. 11 Deep learning for text
    1. 11.1 Natural language processing: The bird’s eye view
    2. 11.2 Preparing text data
      1. 11.2.1 Text standardization
      2. 11.2.2 Text splitting (tokenization)
      3. 11.2.3 Vocabulary indexing
      4. 11.2.4 Using the TextVectorization layer
    3. 11.3 Two approaches for representing groups of words: Sets and sequences
      1. 11.3.1 Preparing the IMDB movie reviews data
      2. 11.3.2 Processing words as a set: The bag-of-words approach
      3. 11.3.3 Processing words as a sequence: The sequence model approach
    4. 11.4 The Transformer architecture
      1. 11.4.1 Understanding self-attention
      2. 11.4.2 Multi-head attention
      3. 11.4.3 The Transformer encoder
      4. 11.4.4 When to use sequence models over bag-of-words models
    5. 11.5 Beyond text classification: Sequence-to-sequence learning
      1. 11.5.1 A machine translation example
      2. 11.5.2 Sequence-to-sequence learning with RNNs
      3. 11.5.3 Sequence-to-sequence learning with Transformer
    6. Summary
  18. 12 Generative deep learning
    1. 12.1 Text generation
      1. 12.1.1 A brief history of generative deep learning for sequence generation
      2. 12.1.2 How do you generate sequence data?
      3. 12.1.3 The importance of the sampling strategy
      4. 12.1.4 Implementing text generation with Keras
      5. 12.1.5 A text-generation callback with variable-temperature sampling
      6. 12.1.6 Wrapping up
    2. 12.2 DeepDream
      1. 12.2.1 Implementing DeepDream in Keras
      2. 12.2.2 Wrapping up
    3. 12.3 Neural style transfer
      1. 12.3.1 The content loss
      2. 12.3.2 The style loss
      3. 12.3.3 Neural style transfer in Keras
      4. 12.3.4 Wrapping up
    4. 12.4 Generating images with variational autoencoders
      1. 12.4.1 Sampling from latent spaces of images
      2. 12.4.2 Concept vectors for image editing
      3. 12.4.3 Variational autoencoders
      4. 12.4.4 Implementing a VAE with Keras
      5. 12.4.5 Wrapping up
    5. 12.5 Introduction to generative adversarial networks
      1. 12.5.1 A schematic GAN implementation
      2. 12.5.2 A bag of tricks
      3. 12.5.3 Getting our hands on the CelebA dataset
      4. 12.5.4 The discriminator
      5. 12.5.5 The generator
      6. 12.5.6 The adversarial network
      7. 12.5.7 Wrapping up
    6. Summary
  19. 13 Best practices for the real world
    1. 13.1 Getting the most out of your models
      1. 13.1.1 Hyperparameter optimization
      2. 13.1.2 ensembling
    2. 13.2 Scaling-up model training
      1. 13.2.1 Speeding up training on GPU with mixed precision
      2. 13.2.2 Multi-GPU training
      3. 13.2.3 TPU training
    3. Summary
  20. 14 Conclusions
    1. 14.1 Key concepts in review
      1. 14.1.1 Various approaches to AI
      2. 14.1.2 What makes deep learning special within the field of machine learning
      3. 14.1.3 How to think about deep learning
      4. 14.1.4 Key enabling technologies
      5. 14.1.5 The universal machine learning workflow
      6. 14.1.6 Key network architectures
      7. 14.1.7 The space of possibilities
    2. 14.2 The limitations of deep learning
      1. 14.2.1 The risk of anthropomorphizing machine learning models
      2. 14.2.2 Automatons vs. intelligent agents
      3. 14.2.3 Local generalization vs. extreme generalization
      4. 14.2.4 The purpose of intelligence
      5. 14.2.5 Climbing the spectrum of generalization
    3. 14.3 Setting the course toward greater generality in AI
      1. 14.3.1 On the importance of setting the right objective: The shortcut rule
      2. 14.3.2 A new target
    4. 14.4 Implementing intelligence: The missing ingredients
      1. 14.4.1 Intelligence as sensitivity to abstract analogies
      2. 14.4.2 The two poles of abstraction
      3. 14.4.3 The missing half of the picture
    5. 14.5 The future of deep learning
      1. 14.5.1 Models as programs
      2. 14.5.2 Blending together deep learning and program synthesis
      3. 14.5.3 Lifelong learning and modular subroutine reuse
      4. 14.5.4 The long-term vision
    6. 14.6 Staying up to date in a fast-moving field
      1. 14.6.1 Practice on real-world problems using Kaggle
      2. 14.6.2 Read about the latest developments on arXiv
      3. 14.6.3 Explore the Keras ecosystem
    7. Final words
  21. index

Product information

  • Title: Deep Learning with Python, Second Edition
  • Author(s): Francois Chollet
  • Release date: November 2021
  • Publisher(s): Manning Publications
  • ISBN: 9781617296864