book

Keras 3

Name: Keras 3
ISBN: 9781807789817

by Rheinwerk Publishing, Inc, Mohammad Nauman

April 2026

Intermediate

633 pages

23h 43m

English

Packt Publishing

Read now

Unlock full access

Notes on Usage
1 Introduction
1.1 Overview of Deep Learning1.1.1 The Success of Deep Learning1.1.2 The Two Pillars Supporting These Breakthroughs1.1.3 Why Now?
1.2 Why Keras
1.3 The Structure of This Book
1.3.1 Text Boxes
1.4 How to Use This Book
1.4.1 Why Keras Installation Comes Later1.4.2 Suggested Reading Strategy
2 Introduction to the Core of Machine Learning
2.1 What Is Machine Learning?2.1.1 Helping the Machine Recognize Digits2.1.2 The Outdated Method—Rule-Based Learning2.1.3 Case Study in Machine Learning Code2.1.4 Structuring the Learning Process2.1.5 Analyzing the Machine’s Learning Process2.1.6 Lessons Learned from the Code Case Study
2.2 Types of Machine Learning
2.2.1 Supervised Learning2.2.2 Unsupervised Learning
2.3 The Magic Sauce: Reinforcement Learning
2.3.1 The Building Blocks of Reinforcement Learning2.3.2 Key Applications for Reinforcement Learning2.3.3 Challenges in Implementing Reinforcement Learning
2.4 Basics of Neural Networks
2.4.1 Core Components of Neural Networks2.4.2 The Unintuitive Process of Learning2.4.3 Clearing Up Some Misconceptions
2.5 Setting Up Your Environment

2.6 Summary
3 Fundamentals of Gradient Descent
3.1 Understanding Gradient Descent3.1.1 The Basic Setup3.1.2 Formalizing the Process: Training Phase3.1.3 From Training to Deployment: Making It Work in the Real World3.1.4 The Process of Learning3.1.5 Finding the Best Parameters: Minimizing the Loss3.1.6 Lifting the Assumptions of a Single Input Feature3.1.7 Higher Dimensions in Gradient Descent
3.2 Types of Gradient Descent: Batch, Stochastic, Mini-Batch
3.2.1 Contour Plots for Visualization of Gradient Descent3.2.2 Improving the Efficiency of Batch Gradient Descent3.2.3 A Paradox in the Use of Stochastic Gradient Descent
3.3 Learning Rate and Optimization
3.4 Implementing Gradient Descent in Code
3.4.1 Gradient Descent from Scratch3.4.2 Gradient Descent Using Keras
3.5 Summary
4 Classification Through Gradient Descent
4.1 Classification Basics4.1.1 Classification Problem Setup4.1.2 First Attempt Using Gradient Descent4.1.3 Second Attempt: Fixing the Issues in the First Attempt4.1.4 Third Attempt: Fixing the Loss Function4.1.5 Squishing Functions and Decision Boundaries4.1.6 Learning Process Summary
4.2 Nonlinear Relationships and Neural Networks
4.2.1 Feature Transformations4.2.2 The Kernel Trick, Logistic Regression, and All of Machine Learning
4.3 Binary vs. Multi-Class Classification
4.3.1 The One-vs-All Approach4.3.2 The Softmax Classifier
4.4 Loss Functions: Cross-Entropy
4.4.1 Categorical Cross-Entropy4.4.2 Pitfall to Avoid When Using Cross Entropies4.4.3 Sparse Categorical Cross-Entropy
4.5 Building a Classifier with Gradient Descent
4.5.1 Layers in Code4.5.2 Choice of Loss Function4.5.3 Parameter Counts4.5.4 The Density of Keras Code
4.6 Summary
5 Deep Dive into Keras
5.1 Introduction to Keras Framework5.1.1 The Philosophy Behind Keras: Making AI Human-Friendly5.1.2 Evolution Through Adaptability5.1.3 Keras 3.0: The Multi-Engine Framework5.1.4 Key Strengths of Keras5.1.5 Keras in the Real World
5.2 Setting Up Keras
5.2.1 Setting Up Python5.2.2 TensorFlow Installation and Points to Keep in Mind5.2.3 Setting Up CUDA for GPU Acceleration5.2.4 Installing Keras5.2.5 Using a GPU in Google Collaboratory
5.3 Building Your First Model
5.3.1 Why NumPy Matters for Machine Learning5.3.2 Symbolic Computation: The Magic Behind Neural Networks
5.4 Implementing Core Concepts in Keras: Gradient Descent and Classification
5.4.1 The Building Blocks of a Cat-Dog Classifier5.4.2 Getting and Fixing the Data5.4.3 Performance Optimization and Model Specification5.4.4 Evaluation Metrics5.4.5 Guiding the Training Through Callbacks and Checkpoints5.4.6 Evaluating the Model
5.5 Summary
6 Regularization Techniques
6.1 An Overview of Overfitting and Underfitting: Do You Need More Data?6.1.1 From Lines to Curves: Adding Polynomial Features6.1.2 Using Increasingly Complex Models6.1.3 The Balance of Complexity6.1.4 Regularization Term6.1.5 Adjusting the Complexity Knob6.1.6 Do I Need More Data6.1.7 Reporting the Final Results: Validation Set
6.2 Dropout: Concept and Implementation
6.2.1 The Problem: Co-Adaptation and Overfitting6.2.2 The Road to Memorization6.2.3 The Ensemble Intuition Behind Dropout6.2.4 Dropout Mechanics6.2.5 Finding the Sweet Spot: Dropout Rates in Practice6.2.6 Implementing Dropout in Pure Python6.2.7 Common Pitfalls and Debugging Tips
6.3 Other Regularization Methods: L1 and L2 Regularization
6.3.1 L1 Regularization (Lasso)6.3.2 Elastic Net: Combining L1 and L26.3.3 When Not to Use Regularization6.3.4 Practical Considerations: Dropout vs. L1/L2 in Neural Networks
6.4 Applying Regularization in Keras
6.4.1 Implementing L2 Regularization in Keras6.4.2 Dropout in Keras6.4.3 Beyond Basic Dropout: Specialized Variants6.4.4 Finding the Perfect Dropout Rate: A Systematic Approach
6.5 Summary
7 Convolutional Neural Networks
7.1 Introduction to Convolutional Neural Networks7.1.1 The Limitation of Fully Connected Networks7.1.2 The Learning Standstill7.1.3 Solving the Vanishing Gradient Problem7.1.4 Dense vs. Sparse Connections7.1.5 From Convolution to Neural Networks: The Conv2D Layer
7.2 Convolutional Layers, Pooling Layers and Fully Connected Layers
7.2.1 Core Implementation of a Convolutional Layer7.2.2 The Hidden Superpowers of Convolutional Layers7.2.3 Pooling Layers: The Image Simplifiers7.2.4 Global Pooling7.2.5 Bringing It All Together: Fully Connected Layers in CNNs
7.3 Implementing CNNs with Keras
7.3.1 Conv2D vs. Conv1D7.3.2 The Opposite of Convolution: Deconvolution Layers
7.4 The “Shapes” Problem
7.5 Case Study: Image Classification
7.6 Summary
8 Exploring the Keras Functional API
8.1 Overview of Keras Functional API8.1.1 The Information Bottleneck8.1.2 Networks as Directed Acyclic Graphs8.1.3 The Functional Programming Heritage8.1.4 Key Advantages of the Functional API
8.2 Building Complex Models with the Functional API
8.2.1 Overview of the Functional API Syntax8.2.2 Creating Models with the Functional API8.2.3 Best Practices for Complex Models8.2.4 Handling Multiple Inputs and Outputs8.2.5 Example: Building an Image Captioning Model8.2.6 Residual Connections8.2.7 Branching Architectures
8.3 Use Cases and Examples
8.3.1 Image Classification with ResNet8.3.2 Siamese Networks for Similarity Learning8.3.3 U-Net for Image Segmentation
8.4 Using Transfer Learning to Customize Models for Your Organization
8.4.1 The Why of Transfer Learning8.4.2 Leveraging Pretrained Models from Keras8.4.3 The Process of Transfer Learning8.4.4 Reloading an Existing Model
8.5 Summary
9 Understanding Transformers
9.1 The Theory Behind Transformers9.1.1 A Simple Time Series Example9.1.2 From Numbers to Words: The Challenge of Text Data9.1.3 GloVe: Learning the Language of Vectors9.1.4 A Gentle Introduction to Attention9.1.5 Why Transformers Revolutionized Natural Language Processing and Beyond
9.2 Components: Attention Mechanism, Encoder, Decoder
9.2.1 The Conversation Between Words9.2.2 Why Position Information Matters9.2.3 Encoder Structure: The Information Processing Powerhouse9.2.4 Decoder Structure: Creating New Sequences from Understanding
9.3 Implementing Transformers in Keras
9.3.1 The Encoder Block9.3.2 The Decoder Block9.3.3 The Transformer: Putting the Encoder and Decoder Together
9.4 Case Study: Large Language Model Chatbot
9.4.1 Structure of Modern Keras Transformer Models9.4.2 Working with Pretrained Models from Kaggle Hub
9.5 Summary
10 Reinforcement Learning: The Secret Sauce
10.1 Introduction to Reinforcement Learning10.1.1 The Problem of Learning by Doing10.1.2 Brief History and Major Breakthroughs10.1.3 Real-World Applications10.1.4 Challenges Unique to Reinforcement Learning
10.2 Key Concepts: Agents, Environments, Rewards
10.2.1 Structure of the Reinforcement Learning Framework10.2.2 Environment Design and State Representation10.2.3 Understanding Agents and Policy Functions10.2.4 Reward Engineering and Signal Design10.2.5 The Exploration vs. Exploitation Dilemma
10.3 Popular Algorithms: Q-Learning, Policy Gradients, and Deep Q-Networks
10.3.1 The Markov Decision Processes10.3.2 Value Functions and Q-Tables10.3.3 Building the Q-Table10.3.4 Q-Learning Algorithm: A Worked Example10.3.5 Q-Learning and Associated Issues10.3.6 The Limits of Tabular Q-Learning
10.4 Implementing Reinforcement Learning Models in Keras
10.4.1 Our First Reinforcement Learning Environment10.4.2 Implementing the Deep Q-Network Algorithm with Keras10.4.3 Experience Replay and Target Networks: The Foundations of Stable Deep Reinforcement Learning
10.5 Reinforcement Learning in Large Language Models
10.5.1 The Fundamental Challenge: Moving Beyond Prediction10.5.2 Challenges and Limitations10.5.3 Future Directions and Emerging Approaches
10.6 Summary
11 Autoencoders and Generative AI
11.1 Introduction to Autoencoders11.1.1 What Are Autoencoders?11.1.2 Autoencoder Architecture Deep Dive11.1.3 Building Your First Autoencoder in Keras11.1.4 Types of Autoencoders
11.2 Variational Autoencoders
11.2.1 Navigating the Space with Uncertainty11.2.2 Mathematical Framework of Variational Autoencoders11.2.3 Variational Autoencoder Implementation in Keras
11.3 Generative Adversarial Networks
11.3.1 The Adversarial Game11.3.2 Generative Adversarial Network Architecture11.3.3 Other Variations11.3.4 Generative Adversarial Network Implementation in Keras11.3.5 Implementation Challenges
11.4 Summary
12 Advanced Generative AI: Stable Diffusion
12.1 Theory Behind Stable Diffusion12.1.1 From Previous Generative Models to Diffusion12.1.2 Diffusion Process Fundamentals12.1.3 Reverse Diffusion: Learning to Denoise12.1.4 Connections to Physical Processes12.1.5 Denoising Diffusion Probabilistic Models12.1.6 Latent Diffusion and Stable Diffusion Architecture12.1.7 Cross-Attention: The Bridge Between Text and Images
12.2 How Stable Diffusion Uses Core Concepts
12.2.1 Efficient Diffusion Through Learned Representations12.2.2 Advanced Attention Mechanisms12.2.3 Training Strategies and Optimization
12.3 Implementing Stable Diffusion Models
12.3.1 Environment and Data Prep12.3.2 Setting Up an Evaluation Measure12.3.3 Model Description and Time-Step Encodings12.3.4 Diffusion and Reverse Diffusion12.3.5 The Generation Engine12.3.6 Following Progress in the Training Process
12.4 Case Study: Image Generation
12.4.1 Loading Pretrained Models from Keras Hub12.4.2 Loading Models Through Keras Hub12.4.3 Using Stable Diffusion Models12.4.4 Beyond Image Generation to More Complex Workflows
12.5 Summary
13 Recap of Key Concepts
13.1 Future Trends in Deep Learning13.1.1 Advanced Architecture Trajectories13.1.2 Reinforcement Learning Frontiers13.1.3 Generative AI Revolution
13.2 Tips for Staying Updated with Advancements
13.2.1 Technical Skills Maintenance13.2.2 Following Tutorials and Keras Codebase13.2.3 Research Consumption Strategy13.2.4 Community Engagement
13.3 Following the Latest Research
13.3.1 Technical Deep Dives13.3.2 Practical Research Integration13.3.3 Parting Words
The Author
Index
Service Pages
Legal Notes

Content preview from Keras 3

9.3 Implementing Transformers in Keras

Now that we’ve explored the theoretical underpinnings of transformers and dissected their key components, it’s time to roll up our sleeves and bring these concepts to life through code. The architectural diagrams and mathematical formulations we’ve examined are about to transform from abstract concepts into working neural networks that you can train and experiment with. In the previous chapter, we discovered how the Keras Functional API gives us the flexibility to create complex model architectures with multiple inputs, branches, and shared layers. That foundation will prove invaluable now as we construct transformers, which feature intricate patterns of connections and information flow. Transformers ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781807789817

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Keras 3

by Rheinwerk Publishing, Inc, Mohammad Nauman

9.3 Implementing Transformers in Keras

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.