book

Generative AI

Name: Generative AI
Author: T V Geetha
ISBN: 9789367130520

by T V Geetha

January 2025

Intermediate to advanced

272 pages

6h 9m

English

Pearson India

Read now

Unlock full access

Cover
About Pearson
Title Page
Preface
Acknowledgements
About the Author
Course Structure
Chapter 1: Introduction
1.1 Generative Artificial Intelligence (Generative AI)1.1.1 Quotes about Generative AI1.1.2 Generative AI is Disruptive1.2 From AI, Machine Learning, Deep Learning to Generative AI1.3 Basics of Machine Learning1.3.1 Supervised and Unsupervised learning – A Detailed Comparison1.3.2 Comparing Traditional Machine Learning and Generative AI Models71.4 Deep Learning1.4.1 Comparing Deep Learning and Generative AI Models1.5 Working of Generative AI1.5.1 Steps of Generative Modelling1.6 History of Generative AI1.6.1 Early Foundations (1950s–1960s)1.6.2 Rule-Based Systems (1970s–1980s)1.6.3 Statistical and Probabilistic Methods (1990s–2000s)1.6.4 Neural Networks and Deep Learning (2010s–Present)1.6.5 Current Trends and Challenges1.7 Benefits of Generative AI1.7.1 Generic Benefits1.7.2 Benefits for Business1.8 Future of Generative AI1.8.1 Trends and Opportunities1.8.2 Predictions about Generative AI 1.8.3 Role of Generative AI and Humans in the Workplace
Chapter 2: Deep Generative Models
2.1 Introduction2.2 Discriminative Models2.3 Basics of Generative Models2.4 Differences between Discriminative and Generative Models2.5 Traditional Generative Models2.6 Deep Generative Models2.6.1 Introduction 2.7 Probabilistic Modelling and Deep Generative Models2.7.1 Probability Foundations2.8 Characteristics of Deep Generative Models2.8.1 Simple Flow of Deep Generative Model2.9 Bayes’ Theorem and Generative Models2.10 Need for Deep Generative Models2.11 Deep Generative Modelling Framework2.11.1 Approaches to Handling the Intractable Normalization Constant
Chapter 3: Classification of Deep Generative Models
3.1 Foundations of Deep Generative Models3.1.1 Probability Density Estimation 3.1.2 Sampling3.1.3 Latent Space 3.1.4 Inference3.1.5 Loss Functions3.1.6 Optimization3.2 Classification of Deep Generative Models3.2.1 Classification of Generative AI models – General vs Task Specific3.2.2 Classification of General Generative AI Models – Computation of Input Probability Distribution3.2.3 Classification of Deep Generative Models based on Working Concept

Chapter 4: Autoregressive Deep Generative Models
4.1 Introduction4.2 Basics of Autoregressive Generative Model4.2.1 Conditions Imposed4.3 Mathematical Basis of Autoregressive Models4.4 Working of Autoregressive Model4.4.1 Evaluation of Autoregressive Models4.5 Types of Autoregressive Generative Models4.5.1 Neural Autoregressive Density Estimator (NADE)4.5.2 PixelCNN4.5.3 PixelRNN4.5.4 PixelCNNs vs PixelRNNs4.6 Autoregressive Models – Applications4.7 Factors in the Use of Autoregressive Models 4.8 Advantages and Limitations of Autoregressive Models4.8.1 Advantages4.8.2 Limitations
Chapter 5: Variational Autoencoders
5.1 Introduction5.2 Dimensionality Reduction and Autoencoders5.2.1 Dimensionality Reduction5.2.2 Autoencoders5.2.3 Architecture of Autoencoders5.2.4 Limitations of Autoencoders for Content Generation5.3 Variational Autoencoders5.3.1 Variational Inference5.3.2 Objective Function5.3.3 Latent Space Representation5.3.4 VAEs and Generative AI5.4 Probabilistic Framework of Variational Autoencoders5.5 Training Process of Variational Autoencoders5.6 Advantages and Disadvantages of Variational Autoencoders5.6.1 Advantages5.6.2 Disadvantages5.7 Applications of Variational Autoencoders5.7.1 Image Generation5.7.2 Data Generation and Augmentation5.7.3 Data Compression5.7.4 Feature Learning and Representation5.7.5 Anomaly Detection5.7.6 Image Denoising and Inpainting
Chapter 6: Generative Adversarial Networks
6.1 Introduction6.1.1 GANs – Supervised or Unsupervised Learning?6.1.2 Working of GANs6.2 Mathematical Basics of GANs6.3 Applications of GANs6.4 Types of GANs6.5 Advantages, Limitations and Challenges of GANs6.5.1 Advantages6.5.2 Limitations6.5.3 Challenges
Chapter 7: Diffusion Based Generative Models
7.1 Introduction7.2 Diffusion Generative Models7.3 Mechanism of Diffusion Generative Models7.4 Theoretical Aspects of Diffusion-based Generative Models7.4.1 Forward Diffusion Process7.4.2 Reverse Diffusion Process7.4.3 Training the Diffusion Model7.5 Types of Diffusion Generative Models7.6 Advantages of Diffusion Models7.7 Limitations of Diffusion Models7.8 Applications of Diffusion Models
Chapter 8: Flow-based and Energy-based Generative Models
8.1 Introduction to Flow-based Generative Models8.2 Mechanism of Normalizing Flow-based Models8.3 Mathematical Aspects of Flow-based Models8.4 Types of Flow-based Models8.5 Advantages and Disadvantages of Flow-based Models8.6 Applications – Flow-based Models8.7 Introduction to Energy-based Generative Models8.8 Mechanism of Energy-based Models8.9 Types of Energy-based Models8.10 Advantages and Disadvantages of Energy-based Models8.11 Applications of Energy-based Models
Chapter 9: Transformer Models
9.1 Introduction9.2 Word Embedding, Contextual Word Embedding and Sub-word Representation9.3 Basis of Transformers9.3.1 Language Models9.3.2 Attention Mechanism9.3.3 Encoder-Decoder Model and Attention Mechanism9.3.4 Attention and Query-key-value Triple9.4 Introduction to Transformer Model9.5 Architecture of Transformer Models 9.6 Core Components and Working of Transformers9.7 Training and Inferencing of Transformers 9.8 Salient Features of Transformer Architecture9.9 Applications of Transformer Models
Chapter 10: Pre-Trained Language Models and Large Language Models
10.1 Pre-trained Language Models10.1.1 Introduction10.1.2 Pre-trained Representations 10.1.3 Language Modelling and Pre-trained Models10.1.4 Advantages of Pre-Trained Models10.2 Fine-Tuning pre-trained Language Models10.2.1 Methods of Fine-Tuning10.2.2 Advantages of Fine-Tuning 10.3 Large Language Models10.3.1 Building Blocks of Large Language Models 10.3.2 Working of Large Language Models10.3.3 Fine-tuning and Large Language Models10.3.4 Prompt Engineering and Large Language Models10.3.5 Advantages of Large Language Models10.3.6 Limitations of Large Language Models10.4 Types of Large Language Models10.5 Generative Pre-trained Transformer (GPT)10.5.1 Pre-Training 10.5.2 Use of Decoder Component of Transformer10.5.3 Training of GPT 10.5.4 Fine Tuning of GPT10.5.5 Examples of the Use of GPT10.6 Bidirectional Encoder Representations from Transformers (BERT)10.6.1 Working of BERT10.6.2 BERT and Sub-word Tokenization10.6.3 Pre-Training BERT10.6.4 Fine-Tuning BERT for NLP Tasks10.6.5 BERT and GPT10.7 T5 Models10.7.1 Working of T5 10.7.2 T5 as a Unified Framework10.8 BART10.9 Embedding from Language Models (ELMo)10.9.1 Character Embedding10.9.2 ELMo ARCHITECTURE
Chapter 11: Generative AI – Applications, Tools, Evaluation and Ethical Perspectives
11.1 Generative AI Applications – Introduction11.1.1 Generative AI Applications – Text Models11.1.2 Generative AI Applications – Audio Models 11.1.3 Generative AI Applications – Conversational Models11.1.4 Generative AI Applications – Data Augmentation11.1.5 Generative AI Applications – Image and Video Applications11.1.6 Generative AI Applications – Software and Coding11.1.7 Generative AI Applications – Domain Areas11.2 Generative AI – Tools11.2.1 Text-to-Text Tools11.2.2 Text to Image Tools11.2.3 Personal Business Assistant Tools11.2.4 Music and Video Generation Tools11.3 Evaluation of Generative AI Models11.3.1 Introduction11.3.2 Evaluation and AI Lifecycle11.3.3 Evaluation Metrics11.4 Ethical Perspectives of Generative AI11.4.1 Introduction11.4.2 Ethical Aspects and Large Language Models11.4.3 Ethical Aspects and Generative AI11.4.4 Challenges and Future Directions of Ethical Generative AI
References

Content preview from Generative AI

CHAPTER 10 Pre-Trained Language Models and Large Language Models

10.1 Pre-trained Language Models

10.1.1 Introduction

In recent times, there has been a focus on pre-training followed by fine-tuning in Natural language processing (NLP) and its applications. The concept of pre-training is related to transfer learning that is the reuse of knowledge (parameters of the model) learnt while training a model for one or more tasks and applying it to novel tasks. Pre-training is described as modelling on a large dataset for a particular task X, and then fine-tuning the learned model on a smaller dataset for task Y. Here, we assume that the task Y is related in some way to task X, and hence some of the neural representations generated by the model for ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9789367130520

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Generative AI

by T V Geetha

CHAPTER 10