book

Deep Learning with Python

Name: Deep Learning with Python
Author: Francois Chollet
ISBN: 9781617294433

by Francois Chollet

December 2017

Intermediate to advanced

384 pages

11h 7m

English

Manning Publications

Read now

Unlock full access

Deep Learning with Python
François Chollet
Copyright
Brief Table of Contents
Table of Contents
Preface
Acknowledgments
About this Book
Who should read this bookRoadmapSoftware/hardware requirementsSource code
Book forum
About the Author
About the Cover

Part 1. Fundamentals of deep learning
Chapter 1. What is deep learning?
1.1. Artificial intelligence, machine learning, and deep learning1.1.1. Artificial intelligence1.1.2. Machine learning1.1.3. Learning representations from data1.1.4. The “deep” in deep learning1.1.5. Understanding how deep learning works, in three figures1.1.6. What deep learning has achieved so far1.1.7. Don’t believe the short-term hype1.1.8. The promise of AI1.2. Before deep learning: a brief history of machine learning1.2.1. Probabilistic modeling1.2.2. Early neural networks1.2.3. Kernel methods1.2.4. Decision trees, random forests, and gradient boosting machines1.2.5. Back to neural networks1.2.6. What makes deep learning different1.2.7. The modern machine-learning landscape1.3. Why deep learning? Why now?1.3.1. Hardware1.3.2. Data1.3.3. Algorithms1.3.4. A new wave of investment1.3.5. The democratization of deep learning1.3.6. Will it last?
Chapter 2. Before we begin: the mathematical building blocks of neural networks
2.1. A first look at a neural network2.2. Data representations for neural networks2.2.1. Scalars (0D tensors)2.2.2. Vectors (1D tensors)2.2.3. Matrices (2D tensors)2.2.4. 3D tensors and higher-dimensional tensors2.2.5. Key attributes2.2.6. Manipulating tensors in Numpy2.2.7. The notion of data batches2.2.8. Real-world examples of data tensors2.2.9. Vector data2.2.10. Timeseries data or sequence data2.2.11. Image data2.2.12. Video data2.3. The gears of neural networks: tensor operations2.3.1. Element-wise operations2.3.2. Broadcasting2.3.3. Tensor dot2.3.4. Tensor reshaping2.3.5. Geometric interpretation of tensor operations2.3.6. A geometric interpretation of deep learning2.4. The engine of neural networks: gradient-based optimization2.4.1. What’s a derivative?2.4.2. Derivative of a tensor operation: the gradient2.4.3. Stochastic gradient descent2.4.4. Chaining derivatives: the Backpropagation algorithm2.5. Looking back at our first example
Chapter 3. Getting started with neural networks
3.1. Anatomy of a neural network3.1.1. Layers: the building blocks of deep learning3.1.2. Models: networks of layers3.1.3. Loss functions and optimizers: keys to configuring the learning process3.2. Introduction to Keras3.2.1. Keras, TensorFlow, Theano, and CNTK3.2.2. Developing with Keras: a quick overview3.3. Setting up a deep-learning workstation3.3.1. Jupyter notebooks: the preferred way to run deep-learning experiments3.3.2. Getting Keras running: two options3.3.3. Running deep-learning jobs in the cloud: pros and cons3.3.4. What is the best GPU for deep learning?3.4. Classifying movie reviews: a binary classification example3.4.1. The IMDB dataset3.4.2. Preparing the data3.4.3. Building your network3.4.4. Validating your approach3.4.5. Using a trained network to generate predictions on new data3.4.6. Further experiments3.4.7. Wrapping up3.5. Classifying newswires: a multiclass classification example3.5.1. The Reuters dataset3.5.2. Preparing the data3.5.3. Building your network3.5.4. Validating your approach3.5.5. Generating predictions on new data3.5.6. A different way to handle the labels and the loss3.5.7. The importance of having sufficiently large intermediate layers3.5.8. Further experiments3.5.9. Wrapping up3.6. Predicting house prices: a regression example3.6.1. The Boston Housing Price dataset3.6.2. Preparing the data3.6.3. Building your network3.6.4. Validating your approach using K-fold validation3.6.5. Wrapping up
Chapter 4. Fundamentals of machine learning
4.1. Four branches of machine learning4.1.1. Supervised learning4.1.2. Unsupervised learning4.1.3. Self-supervised learning4.1.4. Reinforcement learning4.2. Evaluating machine-learning models4.2.1. Training, validation, and test sets4.2.2. Things to keep in mind4.3. Data preprocessing, feature engineering, and feature learning4.3.1. Data preprocessing for neural networks4.3.2. Feature engineering4.4. Overfitting and underfitting4.4.1. Reducing the network’s size4.4.2. Adding weight regularization4.4.3. Adding dropout4.5. The universal workflow of machine learning4.5.1. Defining the problem and assembling a dataset4.5.2. Choosing a measure of success4.5.3. Deciding on an evaluation protocol4.5.4. Preparing your data4.5.5. Developing a model that does better than a baseline4.5.6. Scaling up: developing a model that overfits4.5.7. Regularizing your model and tuning your hyperparameters
Part 2. Deep learning in practice
Chapter 5. Deep learning for computer vision
5.1. Introduction to convnets5.1.1. The convolution operation5.1.2. The max-pooling operation5.2. Training a convnet from scratch on a small dataset5.2.1. The relevance of deep learning for small-data problems5.2.2. Downloading the data5.2.3. Building your network5.2.4. Data preprocessing5.2.5. Using data augmentation5.3. Using a pretrained convnet5.3.1. Feature extraction5.3.2. Fine-tuning5.3.3. Wrapping up5.4. Visualizing what convnets learn5.4.1. Visualizing intermediate activations5.4.2. Visualizing convnet filters5.4.3. Visualizing heatmaps of class activation
Chapter 6. Deep learning for text and sequences
6.1. Working with text data6.1.1. One-hot encoding of words and characters6.1.2. Using word embeddings6.1.3. Putting it all together: from raw text to word embeddings6.1.4. Wrapping up6.2. Understanding recurrent neural networks6.2.1. A recurrent layer in Keras6.2.2. Understanding the LSTM and GRU layers6.2.3. A concrete LSTM example in Keras6.2.4. Wrapping up6.3. Advanced use of recurrent neural networks6.3.1. A temperature-forecasting problem6.3.2. Preparing the data6.3.3. A common-sense, non-machine-learning baseline6.3.4. A basic machine-learning approach6.3.5. A first recurrent baseline6.3.6. Using recurrent dropout to fight overfitting6.3.7. Stacking recurrent layers6.3.8. Using bidirectional RNNs6.3.9. Going even further6.3.10. Wrapping up6.4. Sequence processing with convnets6.4.1. Understanding 1D convolution for sequence data6.4.2. 1D pooling for sequence data6.4.3. Implementing a 1D convnet6.4.4. Combining CNNs and RNNs to process long sequences6.4.5. Wrapping up
Chapter 7. Advanced deep-learning best practices
7.1. Going beyond the Sequential model: the Keras functional API7.1.1. Introduction to the functional API7.1.2. Multi-input models7.1.3. Multi-output models7.1.4. Directed acyclic graphs of layers7.1.5. Layer weight sharing7.1.6. Models as layers7.1.7. Wrapping up7.2. Inspecting and monitoring deep-learning models using Keras callba- acks and TensorBoard7.2.1. Using callbacks to act on a model during training7.2.2. Introduction to TensorBoard: the TensorFlow visualization framework7.2.3. Wrapping up7.3. Getting the most out of your models7.3.1. Advanced architecture patterns7.3.2. Hyperparameter optimization7.3.3. Model ensembling7.3.4. Wrapping up
Chapter 8. Generative deep learning
8.1. Text generation with LSTM8.1.1. A brief history of generative recurrent networks8.1.2. How do you generate sequence data?8.1.3. The importance of the sampling strategy8.1.4. Implementing character-level LSTM text generation8.1.5. Wrapping up8.2. DeepDream8.2.1. Implementing DeepDream in Keras8.2.2. Wrapping up8.3. Neural style transfer8.3.1. The content loss8.3.2. The style loss8.3.3. Neural style transfer in Keras8.3.4. Wrapping up8.4. Generating images with variational autoencoders8.4.1. Sampling from latent spaces of images8.4.2. Concept vectors for image editing8.4.3. Variational autoencoders8.4.4. Wrapping up8.5. Introduction to generative adversarial networks8.5.1. A schematic GAN implementation8.5.2. A bag of tricks8.5.3. The generator8.5.4. The discriminator8.5.5. The adversarial network8.5.6. How to train your DCGAN8.5.7. Wrapping up
Chapter 9. Conclusions
9.1. Key concepts in review9.1.1. Various approaches to AI9.1.2. What makes deep learning special within the field of machine learning9.1.3. How to think about deep learning9.1.4. Key enabling technologies9.1.5. The universal machine-learning workflow9.1.6. Key network architectures9.1.7. The space of possibilities9.2. The limitations of deep learning9.2.1. The risk of anthropomorphizing machine-learning models9.2.2. Local generalization vs. extreme generalization9.2.3. Wrapping up9.3. The future of deep learning9.3.1. Models as programs9.3.2. Beyond backpropagation and differentiable layers9.3.3. Automated machine learning9.3.4. Lifelong learning and modular subroutine reuse9.3.5. The long-term vision9.4. Staying up to date in a fast-moving field9.4.1. Practice on real-world problems using Kaggle9.4.2. Read about the latest developments on arXiv9.4.3. Explore the Keras ecosystem9.5. Final words
Appendix A. Installing Keras and its dependencies on Ubuntu
A.1. Installing the Python scientific suiteA.2. Setting up GPU supportA.3. Installing Theano (optional)A.4. Installing Keras
Appendix B. Running Jupyter notebooks on an EC2 GPU instance
B.1. What are Jupyter notebooks? Why run Jupyter notebooks on AWS GPUs?B.2. Why would you not want to use Jupyter on AWS for deep learning?B.3. Setting up an AWS GPU instanceB.3.1. Configuring JupyterB.4. Installing KerasB.5. Setting up local port forwardingB.6. Using Jupyter from your local browser
Index
List of Figures
List of Tables
List of Listings

Content preview from Deep Learning with Python

Chapter 9. Conclusions

This chapter covers

Important takeaways from this book
The limitations of deep learning
The future of deep learning, machine learning, and AI
Resources for learning further and working in the field

You’ve almost reached the end of this book. This last chapter will summarize and review core concepts while also expanding your horizons beyond the relatively basic notions you’ve learned so far. Understanding deep learning and AI is a journey, and finishing this book is merely the first step on it. I want to make sure you realize this and are properly equipped to take the next steps of this journey on your own.

We’ll start with a bird’s-eye view of what you should take away from this book. This should refresh your memory regarding ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Deep Learning with Python, Second Edition

Publisher Resources

ISBN: 9781617294433Publisher Support Publisher Website Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Deep Learning with Python

by Francois Chollet

Chapter 9. Conclusions

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.