book

TensorFlow in Action

Name: TensorFlow in Action
Author: Thushan Ganegedara
ISBN: 9781617298349

by Thushan Ganegedara

October 2022

Intermediate to advanced

680 pages

19h 59m

English

Manning Publications

Read now

Unlock full access

TensorFlow in Action
Copyright
dedication
contents
front matter
prefaceacknowledgmentsabout this bookWho should read this book?How this book is organized: A roadmapAbout the codeliveBook discussion forumabout the authorabout the cover illustration
Part 1 Foundations of TensorFlow 2 and deep learning
1 The amazing world of TensorFlow
1.1 What is TensorFlow?1.1.1 An overview of popular components of TensorFlow1.1.2 Building and deploying a machine learning model1.2 GPU vs. CPU1.3 When and when not to use TensorFlow1.3.1 When to use TensorFlow1.3.2 When not to use TensorFlow1.4 What will this book teach you?1.4.1 TensorFlow fundamentals1.4.2 Deep learning algorithms1.4.3 Monitoring and optimization1.5 Who is this book for?1.6 Should we really care about Python and TensorFlow 2?Summary
2 TensorFlow 2
2.1 First steps with TensorFlow 22.1.1 How does TensorFlow operate under the hood?2.2 TensorFlow building blocks2.2.1 Understanding tf.Variable2.2.2 Understanding tf.Tensor2.2.3 Understanding tf.Operation2.3 Neural network-related computations in TensorFlow2.3.1 Matrix multiplication2.3.2 Convolution operation2.3.3 Pooling operationSummaryAnswers to exercises
3 Keras and data retrieval in TensorFlow 2
3.1 Keras model-building APIs3.1.1 Introducing the data set3.1.2 The Sequential API3.1.3 The functional API3.1.4 The sub-classing API3.2 Retrieving data for TensorFlow/Keras models3.2.1 tf.data API3.2.2 Keras DataGenerators3.2.3 tensorflow-datasets packageSummaryAnswers to exercises
4 Dipping toes in deep learning
4.1 Fully connected networks4.1.1 Understanding the data4.1.2 Autoencoder model4.2 Convolutional neural networks4.2.1 Understanding the data4.2.2 Implementing the network4.3 One step at a time: Recurrent neural networks (RNNs)4.3.1 Understanding the data4.3.2 Implementing the model4.3.3 Predicting future CO2 values with the trained modelSummaryAnswers to exercises

5 State-of-the-art in deep learning: Transformers
5.1 Representing text as numbers5.2 Understanding the Transformer model5.2.1 The encoder-decoder view of the Transformer5.2.2 Diving deeper5.2.3 Self-attention layer5.2.4 Understanding self-attention using scalars5.2.5 Self-attention as a cooking competition5.2.6 Masked self-attention layers5.2.7 Multi-head attention5.2.8 Fully connected layer5.2.9 Putting everything togetherSummaryAnswers to exercises
Part 2 Look ma, no hands! Deep networks in the real world
6 Teaching machines to see: Image classification with CNNs
6.1 Putting the data under the microscope: Exploratory data analysis6.1.1 The folder/file structure6.1.2 Understanding the classes in the data set6.1.3 Computing simple statistics on the data set6.2 Creating data pipelines using the Keras ImageDataGenerator6.3 Inception net: Implementing a state-of-the-art image classifier6.3.1 Recap on CNNs6.3.2 Inception net v16.3.3 Putting everything together6.3.4 Other Inception models6.4 Training the model and evaluating performanceSummaryAnswers to exercises
7 Teaching machines to see better: Improving CNNs and making them confess
7.1 Techniques for reducing overfitting7.1.1 Image data augmentation with Keras7.1.2 Dropout: Randomly switching off parts of your network to improve generalizability7.1.3 Early stopping: Halting the training process if the network starts to underperform7.2 Toward minimalism: Minception instead of Inception7.2.1 Implementing the stem7.2.2 Implementing Inception-ResNet type A block7.2.3 Implementing the Inception-ResNet type B block7.2.4 Implementing the reduction block7.2.5 Putting everything together7.2.6 Training Minception7.3 If you can't beat them, join ‘em: Using pretrained networks for enhancing performance7.3.1 Transfer learning: Reusing existing knowledge in deep neural networks7.4 Grad-CAM: Making CNNs confessSummaryAnswers to exercises
8 Telling things apart: Image segmentation
8.1 Understanding the data8.2 Getting serious: Defining a TensorFlow data pipeline8.2.1 Optimizing tf.data pipelines8.2.2 The final tf.data pipeline8.3 DeepLabv3: Using pretrained networks to segment images8.3.1 A quick overview of the ResNet-50 model8.3.2 Atrous convolution: Increasing the receptive field of convolution layers with holes8.3.3 Implementing DeepLab v3 using the Keras functional API8.3.4 Implementing the atrous spatial pyramid pooling module8.3.5 Putting everything together8.4 Compiling the model: Loss functions and evaluation metrics in image segmentation8.4.1 Loss functions8.4.2 Evaluation metrics8.5 Training the model8.6 Evaluating the modelSummaryAnswers to exercises
9 Natural language processing with TensorFlow: Sentiment analysis
9.1 What the text? Exploring and processing text9.2 Getting text ready for the model9.2.1 Splitting training/validation and testing data9.2.2 Analyze the vocabulary9.2.3 Analyzing the sequence length9.2.4 Text to words and then to numbers with Keras9.3 Defining an end-to-end NLP pipeline with TensorFlow9.4 Happy reviews mean happy customers: Sentiment analysis9.4.1 LSTM Networks9.4.2 Defining the final model9.5 Training and evaluating the model9.6 Injecting semantics with word vectors9.6.1 Word embeddings9.6.2 Defining the final model with word embeddings9.6.3 Training and evaluating the modelSummaryAnswers to exercises
10 Natural language processing with TensorFlow: Language modeling
10.1 Processing the data10.1.1 What is language modeling?10.1.2 Downloading and playing with data10.1.3 Too large vocabulary? N-grams to the rescue10.1.4 Tokenizing text10.1.5 Defining a tf.data pipeline10.2 GRUs in Wonderland: Generating text with deep learning10.3 Measuring the quality of the generated text10.4 Training and evaluating the language model10.5 Generating new text from the language model: Greedy decoding10.6 Beam search: Enhancing the predictive power of sequential modelsSummaryAnswers to exercises
Part 3 Advanced deep networks for complex problems
11 Sequence-to-sequence learning: Part 1
11.1 Understanding the machine translation data11.2 Writing an English-German seq2seq machine translator11.2.1 The TextVectorization layer11.2.2 Defining the TextVectorization layers for the seq2seq model11.2.3 Defining the encoder11.2.4 Defining the decoder and the final model11.2.5 Compiling the model11.3 Training and evaluating the model11.4 From training to inference: Defining the inference modelSummaryAnswers to exercises
12 Sequence-to-sequence learning: Part 2
12.1 Eyeballing the past: Improving our model with attention12.1.1 Implementing Bahdanau attention in TensorFlow12.1.2 Defining the final model12.1.3 Training the model12.2 Visualizing the attentionSummaryAnswers to exercises
13 Transformers
13.1 Transformers in more detail13.1.1 Revisiting the basic components of the Transformer13.1.2 Embeddings in the Transformer13.1.3 Residuals and normalization13.2 Using pretrained BERT for spam classification13.2.1 Understanding BERT13.2.2 Classifying spam with BERT in TensorFlow13.3 Question answering with Hugging Face’s Transformers13.3.1 Understanding the data13.3.2 Processing data13.3.3 Defining the DistilBERT model13.3.4 Training the model13.3.5 Ask BERT a questionSummaryAnswers to exercises
14 TensorBoard: Big brother of TensorFlow
14.1 Visualize data with TensorBoard14.2 Tracking and monitoring models with TensorBoard14.3 Using tf.summary to write custom metrics during model training14.4 Profiling models to detect performance bottlenecks14.4.1 Optimizing the input pipeline14.4.2 Mixed precision training14.5 Visualizing word vectors with the TensorBoardSummaryAnswers to exercises
15 TFX: MLOps and deploying models with TensorFlow
15.1 Writing a data pipeline with TFX15.1.1 Loading data from CSV files15.1.2 Generating basic statistics from the data15.1.3 Inferring the schema from data15.1.4 Converting data to features15.2 Training a simple regression neural network: TFX Trainer API15.2.1 Defining a Keras model15.2.2 Defining the model training15.2.3 SignatureDefs: Defining how models are used outside TensorFlow15.2.4 Training the Keras model with TFX Trainer15.3 Setting up Docker to serve a trained model15.4 Deploying the model and serving it through an API15.4.1 Validating the infrastructure15.4.2 Resolving the correct model15.4.3 Evaluating the model15.4.4 Pushing the final model15.4.5 Predicting with the TensorFlow serving APISummaryAnswers to exercises
appendix A Setting up the environment
A.1 In a Unix-based environmentA.1.1 Creating a virtual Python environment with Anaconda distribution (Ubuntu)A.1.2 Prerequisites for GPU support (Ubuntu)A.1.3 Notes on MacOSA.2 In Windows EnvironmentsA.2.1 Creating a Virtual Python Environment (Anaconda)A.2.2 Prerequisites for GPU supportA.3 Activating and deactivating the conda environmentA.4 Running the Jupyter Notebook server and creating notebooksA.5 Miscellaneous notes
appendix B Computer vision
B.1 Grad-CAM: Interpreting computer vision modelsB.2 Image segmentation: U-Net modelB.2.1 Understanding and defining the U-Net modelB.2.2 What’s better than an encoder? A pretrained encoder
appendix C Natural language processing
C.1 Touring around the zoo: Meeting other Transformer modelsC.1.1 Generative pre-training (GPT) model (2018)C.1.2 DistilBERT (2019)C.1.3 RoBERT/ToBERT (2019)C.1.4 BART (2019)C.1.5 XLNet (2020)C.1.6 Albert (2020)C.1.7 Reformer (2020)
index

Overview

Unlock the TensorFlow design secrets behind successful deep learning applications! Deep learning StackOverflow contributor Thushan Ganegedara teaches you the new features of TensorFlow 2 in this hands-on guide.
In TensorFlow in Action you will learn:

Fundamentals of TensorFlow
Implementing deep learning networks
Picking a high-level Keras API for model building with confidence
Writing comprehensive end-to-end data pipelines
Building models for computer vision and natural language processing
Utilizing pretrained NLP models
Recent algorithms including transformers, attention models, and ElMo

In TensorFlow in Action, you'll dig into the newest version of Google's amazing TensorFlow framework as you learn to create incredible deep learning applications. Author Thushan Ganegedara uses quirky stories, practical examples, and behind-the-scenes explanations to demystify concepts otherwise trapped in dense academic papers. As you dive into modern deep learning techniques like transformer and attention models, you’ll benefit from the unique insights of a top StackOverflow contributor for deep learning and NLP.

About the Technology
Google’s TensorFlow framework sits at the heart of modern deep learning. Boasting practical features like multi-GPU support, network data visualization, and easy production pipelines using TensorFlow Extended (TFX), TensorFlow provides the most efficient path to professional AI applications. And the Keras library, fully integrated into TensorFlow 2, makes it a snap to build and train even complex models for vision, language, and more.

About the Book
TensorFlow in Action teaches you to construct, train, and deploy deep learning models using TensorFlow 2. In this practical tutorial, you’ll build reusable skill hands-on as you create production-ready applications such as a French-to-English translator and a neural network that can write fiction. You’ll appreciate the in-depth explanations that go from DL basics to advanced applications in NLP, image processing, and MLOps, complete with important details that you’ll return to reference over and over.

What's Inside

Covers TensorFlow 2.9
Recent algorithms including transformers, attention models, and ElMo
Build on pretrained models
Writing end-to-end data pipelines with TFX

About the Reader
For Python programmers with basic deep learning skills.

About the Author
Thushan Ganegedara is a senior ML engineer at Canva and TensorFlow expert. He holds a PhD in machine learning from the University of Sydney.

Quotes
A nice dive into TensorFlow 2. It outlines modern techniques and reinforces them with concrete sample projects.
- Joshua A. McAdams, Google

Practical and hands-on. A valuable resource for practitioners and newbies.
- Amaresh Rajasekharan, IBM

Comprehensive. Covers advanced topics such as atrous convolution, Transformers, and MLOps.
- Wei Luo, Deakin University

Recommended for practitioners. There are code examples for every topic.
- Vidhya Vinay, Streamingo.ai

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Machine Learning with TensorFlow, Second Edition

Publisher Resources

ISBN: 9781617298349Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills