book

Generative AI and LLMs

Name: Generative AI and LLMs
ISBN: 9783111425511

by S. Balasubramaniam, Seifedine Kadry, Aruchamy Prasanth, Rajesh Kumar Dhanaraj

September 2024

Intermediate to advanced

275 pages

10h 6m

English

De Gruyter

Read now

Unlock full access

About the Editors
List of Contributors
A. Ashwini, Rubia J. Jency, H. Sehina, B. Sundaravadivazhagan 1 Unveiling the Power of Generative AI: A Journey into Large Language Models
1.1 Overview of Generative AI and Large Language Models1.2 Fundamental Concepts1.2.1 Probability Distribution1.2.2 Neural Networks1.2.3 Generative Adversarial Networks (GANs)1.2.4 Variational Autoencoders (VAEs)1.2.5 Transfer Learning1.2.6 Transformer Architecture1.3 Algorithms Used in Generative Models1.3.1 Recurrent Neural Networks (RNNs)1.3.2 Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU)1.3.3 Bidirectional RNNs (BRNNs)1.3.4 Power of Convolutional Neural Networks (CNNs)1.3.5 Activation Functions Used in Generative Models1.3.6 Optimization Techniques for Generative Modeling1.4 Text Generation1.5 Pretraining and Fine-Tuning of LLM Models1.6 Impact on Generative AI and LLM1.7 Application of LLMs1.7.1 Natural Language Understanding (NLU)1.7.2 Text Generation and Creative Writing1.7.3 Language Translation1.7.4 Text Summarization1.7.5 Dialogue Systems1.7.6 Content Generation and Personalization1.7.7 Medical and Scientific Research1.8 Challenges and Limitations1.8.1 Bias and Fairness1.8.2 Ethical Use1.8.3 Privacy Concerns1.8.4 Computational Resources1.8.5 Environmental Impact1.8.6 Interpretability and Transparency1.8.7 Data Quality and Diversity1.9 Future Directions1.9.1 Continued Scale and Performance Improvements1.9.2 Multimodal Capabilities1.9.3 Contextual Adaptation and Personalization1.9.5 Ethical and Responsible AI Development1.9.6 Human–AI Collaboration1.9.7 Domain-Specific and Specialized Applications1.9.8 Interdisciplinary Research and Collaboration1.10 Conclusion
A. Ashwini, V. Kavitha, S. Balasubramaniam, Seifedine Kadry 2 Early Roots of Generative AI Models and LLM: A Diverse Landscape
2.1 Introduction to Rule-Based Approaches2.2 Emergence of Statistical Language Models2.2.1 Evolutionary Steps of Statistical Language Models2.3 Early Experiments on Neural Network2.4 Pioneering Architectures in Language Modeling2.4.1 Recurrent Neural Networks (RNNs)2.4.2 Long Short-Term Memory (LSTM) Networks2.4.3 Transformer Architecture2.4.4 Bidirectional Encoder Representations from Transformers (BERT)2.4.5 Generative Pretrained Transformer (GPT)2.5 Integration of Expert Systems with Language Models2.5.1 Knowledge Representation2.5.2 Semantic Parsing and Ontology Development2.5.3 Data Preprocessing and Feature Engineering2.5.4 Training Hybrid Models2.5.5 Evaluation and Validation2.5.6 Deployment and Application Integration2.5.7 Continuous Maintenance2.6 Impact on Early Generative AI2.6.1 Natural Language Processing (NLP)2.6.2 Content Generation and Creativity2.6.3 Drug Discovery and Healthcare2.7 Theoretical Foundations and Hybrid Approaches2.7.1 Probability Theory2.7.2 Information Theory2.7.3 Computational Linguistics2.7.4 Rule-Based Preprocessing2.7.5 Hybrid Architectures2.7.6 Ensemble Methods2.8 Limitations and Challenges2.9 Bridge to Modern Large Language Models (LLMs)2.9.1 Fine-Tuning of Parameters2.9.2 Scale and Size2.9.3 Applications and Impact2.10 Conclusion
C. Arun, S. Karthick, S. Selvakumara Samy, B. Hariharan, Po-Ming Lee 3 Generative AI Models and LLM: Training Techniques and Evaluation Metrics
3.1 Introduction3.1.1 Layers of Generative AI Model3.2 Generative AI Model and LLM Training Techniques3.2.1 Generative Adversarial Networks (GANs)3.2.2 Conditional GAN3.2.3 Deep Convolutional GAN (DCGAN)3.2.4 Pix2Pix GAN3.2.5 Cycle GAN3.3 Variational Autoencoder3.4 Transformer Models3.4.1 BERT3.4.2 GPT3.5 LangChain3.6 Diffusion Model3.7 Flow-Based Models3.8 Evaluation Metrics3.8.1 Inception Score (IS)3.8.2 Frechet Inception Distance3.8.3 CLIP3.8.4 Perplexity3.8.5 BLEU Score3.8.6 ROUGE3.8.7 METEOR3.8.8 BERT3.8.9 GPT Score3.8.10 Levenshtein Similarity Ratio3.8.11 MoverScore3.9 Conclusion
M. Abinaya, G. Vadivu, S. Balasubramaniam, Seifedine Kadry 4 Importance of Prompt Engineering in Generative AI Models
4.1 Introduction4.1.1 Defining Prompts in Generative AI4.1.2 Development of Prompt Engineering4.2 Theoretical Underpinnings of Prompt Engineering4.2.1 Prompt Design and Linguistic Theory4.2.2 Cognitive Science4.2.3 Computational Linguistics Approaches in Making Prompts for AI Models4.3 Methodologies in Prompt Engineering4.3.1 Template-Based Prompting4.3.2 Constraint-Based Prompt Design4.3.3 Reinforcement Learning Techniques4.4 Empirical Studies and Case Examples4.4.1 Text Generation: Prompts for Creative Writing4.4.2 Empirical Studies and Evaluation of Prompt Engineering Techniques4.4.3 Knowledge Distillation in Prompt Engineering4.5 Examining the Influence of Prompts: Multidisciplinary Views4.5.1 Cognitive Perspectives on Prompt-Model Interaction4.5.2 The Significance of Prompt Design in Sociology4.5.3 Human–Computer Interaction4.5.4 Ethical and Social Implications4.5.5 Empirical Research and Illustrative Case Studies4.6 Interdisciplinary Perspectives on Prompt Engineering4.6.1 Psychological Insights4.6.2 Sociological Perspectives4.6.3 HCI Perspectives4.7 Future Directions and Challenges4.7 Emerging Trends in Prompt Engineering4.7.1 Addressing Limitations and Ethical Considerations4.7.2 Opportunities for Interdisciplinary Research and Collaboration4.8 Prospects and Difficulties4.8.1 New Developments in Quick Engineering4.8.2 Taking Ethical and Limitation Considerations into Account4.8.3 Possibilities for Multidisciplinary Study and Cooperation4.9 Conclusion
Anitha Velu, Raghu Ramamoorthy, S. M. Manasa, A. Prasanth 5 LLM Pretraining Methods
5.1 Introduction5.1.1 Smart Factory5.1.2 Advantages of Pretraining in LLM5.2 Steps for Training LLM Models5.3 Study of Pretraining in LLM5.3.1 Data Collection5.3.2 Data Preprocessing5.3.3 Pretraining Task5.3.4 Evaluating the Pretrained Model5.3.5 Next-Word Prediction5.4 Effect of Pretraining on LLM5.5 Key Considerations for Pretraining LLM5.6 Characteristics of LLM Pretraining5.7 Some Use Cases of LLM Pretraining5.8 Summary
S. Aathilakshmi, G. Sivapriya, T. Manikandan 6 LLM Fine-Tuning: Instruction and Parameter-Efficient Fine-Tuning (PEFT)
6.1 Introduction6.2 LLM Fine-Tuning: Instruction and Parameter Efficient Fine-Tuning6.2.1 Selecting a pretrained LLM model6.2.2 Various Approaches to Fine-Tune LLMs6.2.3 Unsupervised Versus Supervised Fine-Tuning (SFT)6.3 Reinforcement Learning from Human Feedback (RLHF)6.4 Parameter-Efficient Fine-Tuning (PEFT)6.4.1 Advantages of Low-Rank Adaptation (LoRA) Method6.5 PEFT Methods6.6 LoRA: Low-Rank Adaption Method6.7 QLoRA: Quantized Low-Rank Adaption Method6.7.1 Four-bit Normal Float (NF4)6.7.2 Key Steps in QLoRA6.8 Conclusion
Dawn Sivan, K. Satheesh Kumar, Veena Raj, Rajan Jose 7 Reinforcement Learning from Human Feedback (RLHF)
7.1 Introduction7.2 Foundations of Reinforcement Learning7.2.1 Key Components of an RL System7.2.2 RL Workflow7.2.3 Benefits and Challenges of RL7.2.4 The Intersection of RL and LLMs7.3 Transitioning to RLHF7.3.1 Challenges in RLHF for LLMs in Niche Domains7.3.2 Working Principle of RLHF7.4 Impact of RLHF on Tailoring LLMs: Case Studies7.4.1 Enhancing Conversational Agents with RLHF7.4.2 Refining Language Translation Models for Accuracy and Fluency7.4.3 Creative Content Generation for Specific Industries7.5 Ethical Considerations in RLHF for LLMs7.6 RLHF Derivatives7.6.1 The Llama-2 Model7.6.2 Safe RLHF7.6.3 Reinforcement Learning with AI Feedback (RLAIF)7.7 Conclusion
A. Ashwini, J. Manoj Prabhakar, Seifedine Kadry 8 Exploring the Applications on Generative AI and LLM
8.1 Overview to Generative AI8.2 Meta Learning Fundamentals for Adaptive Scientific Modeling8.2.1 Key Principles of Meta Learning for Adaptive Scientific Modeling8.3 Automatic Hypothesis Generation with Generative Models8.3.1 Data Representation8.3.2 Model Training8.3.3 Hypothesis Generation8.3.4 Evaluation and Validation8.3.5 Iterative Refinement8.4 Quantum Computing Concepts in Generative Models8.4.1 Quantum Generative Models8.4.2 Variational Quantum Circuit (VQC) Models8.4.3 Quantum Boltzmann Machines8.4.4 Quantum Variational Autoencoders (QVAEs)8.4.5 Quantum Boltzmann Generative Adversarial Networks (QB-GANs)8.4.6 Quantum Annealers for Sampling8.5 Real-Time Collaboration with Generative Models8.5.1 Interactive Interfaces8.5.2 Shared Workspaces8.5.3 Dynamic Feedback Loops8.5.4 Multimodal Outputs8.5.5 Customizable Models8.5.6 Privacy and Security8.5.7 Scalability and Performance8.5.8 Integrating Effective Tools8.6 Implementation of Privacy-Preserving Techniques8.6.1 Differential Privacy8.6.2 Federated Learning8.6.3 Homomorphic Encryption8.6.4 Secure Multiparty Computation (SMPC)8.6.5 Generative Adversarial Privacy (GAP)8.6.6 Data Perturbation8.6.7 Model Watermarking8.6.8 Privacy-Preserving Evaluation8.7 Enhancing Scientific Visualization Techniques8.8 Leveraging Blockchain for Trust and Transparency8.8.1 Model Verification and Trustworthiness8.8.2 Data Origin and Property8.8.3 Transparent Learning Processes8.8.4 Decentralized Model Management8.8.5 Transparent Network Outputs8.9 Conclusion and Future Directions

Mani Deepak Choudhry, M. Sundarrajan, Karthic Sundaram, K. Rama Abirami 9 Bias and Fairness in Generative AI
9.1 Introduction9.1.1 Bias9.1.2 Fairness9.2 Bias: Sources, Impact, and Mitigation Strategies9.2.1 Sources of Bias9.2.2 Impact of Bias9.2.3 Methodologies of Mitigation for AI Bias9.3 Fairness: Metrics and Mitigation Strategies9.3.1 Sources of Fairness9.3.2 Metrics of Fairness in AI9.3.3 Methodologies of Mitigation for AI Fairness9.4 Conclusion
M. Abinaya, G. Vadivu, B. Sundaravadivazhagan 10 Future Directions and Open Problems in Generative AI
10.1 Introduction10.1.1 Overview of Generative AI10.2 Importance of Exploring GenAI10.2.1 Improving Sample Quality and Diversity and Challenges of Sample Generation10.2.2 Sample Quality Enhancement10.2.3 Diversity Strategies10.3 Improving Control and Interpretability in Generative AI10.4 Ethical Challenges in Generative AI10.5 Expanding Generative Frameworks10.5.1 Difficulties in Growing Generative Models10.5.2 Methods for Educating Extensive Models10.5.3 Possible Advances in Scalability in the Future10.6 Semantic Gap10.7 Innovative Architectures10.7.1 Developments in Training Approaches10.7.2 Making Use of Distributed Computing10.8 Research Areas in Generative AI10.9 Industry Perspectives and Case Studies10.9.1 Real-World Applications10.9.2 Success Stories and Challenges10.9.3 Insights on Future Directions10.9.4 Future Challenges and Opportunities10.9.5 Ethical Considerations in Generative AI10.9.6 Human-Centric Design in Generative AI10.10 Conclusion
Pankaj Rahi, Mayur Dilip Jakhete, Anurag Anand Duvey 11 Optimizing Sustainable Project Management Life Cycle Using Generative AI Modeling
11.1 Introduction11.1.1 What Is Generative AI and Its Architecture?11.1.2 Types of Generative Artificial Intelligence (GenAI)11.1.3 Core Procedures of Enhanced AI Models11.2 Literature Review11.2.1 Generative AI for Optimizing Product Management Life Cycle11.2.2 Use Cases11.2.3 Benefits of GenAI in Project Organization Activities11.3 Current Issues in Project/Product Life Cycle Management Using GenAI11.4 Optimizing the GenAI Made for Edge Devices in the Near Future11.5 Conclusion
L. B. Reshmi, R. Vipin Raj, S. Balasubramaniam, K. Satheesh Kumar 12 Generative AI and LLM: Case Study in Finance
12.1 Introduction12.1.1 Understanding Generative AI and Large Language Models (LLMs)12.1.2 Language Models in Finance12.1.3 Applications of Language Models in Finance12.2 Challenges and Ethical Considerations for Language Models in Finance12.2.1 Misinformation and False News12.2.2 Data Privacy and Security12.2.3 Data Quality and Bias12.2.4 Risk Management12.2.5 Absence of Domain Knowledge12.2.6 Limited Multilingual Capabilities12.2.7 Hallucinations12.2.8 Inadequate Knowledge of Human Behavior12.2.9 Ethical Issues12.2.10 Continuous Monitoring and Improvement12.3 Major FinTech Models12.3.1 BloombergGPT12.3.2 FinGPT-HPC12.3.3 FinBERT12.3.4 T5: Text-to-Text Transfer Transformer12.4 Conclusion and Future Directions
Rajiv Iyer, Vedprakash C. Maralapalle, Poornima Mahesh, Deepak Patil 13 Generative AI and LLM: Case Study in E-Commerce
13.1 Introduction13.2 Significance of AI in E-Commerce13.2.1 Transformative Impact of AI in E-Commerce13.2.2 Key Applications of AI in E-Commerce13.2.3 Benefits of AI Adoption in E-Commerce13.2.4 Challenges and Future Implications13.2.5 Future Directions in AI-Driven E-Commerce13.2.6 Theoretical Foundations13.3 Case Studies13.3.1 Personalized Product Recommendations13.3.2 Natural Language Processing for Customer Interactions13.3.3 Content Generation for Marketing Campaigns13.3.4 Fraud Detection and Prevention13.4 Implementation Strategies13.4.1 Best Practices: How to Tie Generative AI and LLM to E-Commerce13.4.2 Exploring Use of Generative AI and LLM13.4.3 Implementation and Integration Best Practices13.4.4 Benefits of Integration13.4.5 Integration Challenges and Solutions13.4.6 Case Studies and Success Stories13.4.7 Future Opportunities13.4.8 Efficient Methods for Introducing Generative AI and LLM in E-Commerce13.4.9 Ethical Challenges and Data Protection13.4.10 Ethical Considerations13.4.11 Data Privacy Concerns13.5 Future Trends in E-Commerce13.5.1 Online Shopping Has Proven to Be an Efficient, Sustainable, and Profitable Form of Sales13.5.2 AI-Powered Personalization13.5.3 Evolution of AI Chatbots and Conversational AI13.5.4 Visual Search and Personalized Recommendations13.5.5 Providing Trust and Transparency by Blockchain Technology13.5.6 What Paths We May Take and Obstacles We Will Encounter?13.5.7 Identification and Exploring the Possible Limitations and Risks13.6 Conclusion
Index

Content preview from Generative AI and LLMs

7 Reinforcement Learning from Human Feedback (RLHF)

Dawn Sivan

K. Satheesh Kumar

Veena Raj

Rajan Jose

Abstract

This chapter primarily focuses on the introduction of reinforcement learning from human feedback (RLHF), an approach by which artificial intelligence (AI) models learn from human feedback, within the realm of generative AI and large language models (LLMs). The foundation principles of RLHF are explored, emphasizing its crucial role in implementing RLHF for specialized applications. The associated challenges and their possible solutions are discussed, ensuring the efficient integration of RLHF with LLMs. The methodologies for implementing LLMs with human feedback, such as advanced reward design and iterative model refinement, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9783111425511

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Generative AI and LLMs

by S. Balasubramaniam, Seifedine Kadry, Aruchamy Prasanth, Rajesh Kumar Dhanaraj