book

Intermediate Python and Large Language Models

Name: Intermediate Python and Large Language Models
Author: Dilyan Grigorov
ISBN: 9798868814754

by Dilyan Grigorov

June 2025

Intermediate to advanced

342 pages

6h 44m

English

Apress

Read now

Unlock full access

Intermediate Python and Large Language Models
Introduction
Acknowledgments
Table of Contents
About the Author
About the Technical Reviewer
1. LangChain and Python: Basics
LangChain Basics and Basic ComponentsChainsPromptsMemoryTools and AgentsRetrieval-Augmented Generation (RAG)Data LoadersIntegrations and ExtensibilityLLM Outputs and PostprocessingLangChain InstallationPackages OverviewInstalling from SourceHow to Prompt?Prompt EngineeringRole PromptingFew-Shot PromptingKey BenefitsAlternating Human/AI MessagesChain PromptingKey CharacteristicsSteps for Chain PromptingUsing Chain Prompting in LangChainAdditional BenefitsChain-of-Thought PromptingAdvanced Tips for Effective Prompt EngineeringWhat Are Chains?Chain ComponentsChain Types1. Simple Chain2. Sequential Chain3. Conversational Chain4. Multi-input ChainWhy Multi-input?5. Multi-output ChainWhy It’s a Multi-output Chain6. Router ChainWhy Router Chain?7. Control Flow ChainKey Features of the Control Flow Chain8. Retrieval-Aware Chain9. Agent ChainBreakdown of Key Steps of This More Complicated Code10. Parallel ChainWhat Does the Code Do?Key Features of RunnableParallel11. Custom ChainKey Features of a Custom ChainWhat Does This Code Do?Conclusion
2. LangChain and Python: Advanced Components
LangChain MemoryUnderstanding LangChain’s Memory ModuleKey Capabilities of the Memory ModuleWhy Memory MattersWhen to Use the Memory ModuleCore Processes in the Memory System: Reading and WritingStructuring a Memory SystemLangChain Memory TypesConversationBufferMemoryConversationBufferWindowMemoryConversationSummaryMemoryConversation Summary Buffer MemoryConversation Token Buffer MemoryKnowledgeGraphMemoryEntityMemoryVectorStoreMemorySelecting the Appropriate Memory TypeImplementing Memory in LangChainLangChain Document LoadersCommon Document LoadersSpecialized Document LoadersLangChain Embedding ModelsLangChain Indexes and RetrieversIndexes in LangChain: Structure and TypesTypes of IndexesRetrievers in LangChain: Querying and OptimizationEnd-to-End Workflow: From Indexing to RetrievalReal-World Applications of LangChain Indexes and RetrieversUsing LangChain Indexing APITechnical Structure of the Indexing APIDeletion Modes and Content MaintenanceRequirements and CompatibilityImportant ConsiderationsAgents in LangChainDefining AgentsTypes of Agents in LangChainTools As Extensions of Language ModelsContent Generation vs. Reasoning EnginesExploring Autonomous Agents: AutoGPT and BabyAGIWhat Is AutoGPT?How Does AutoGPT Work?What Is BabyAGI?Key Features of BabyAGIA Practical Implementation of BabyAGILLM Models in LangChainChat ModelsAI21 LabsAlibaba Cloud PAI EASAnthropicAnyscaleAzure OpenAIAzure ML EndpointBaidu QianfanAWS BedrockCohereDatabricksDeepInfraEden AIEverlyAIFireworksGigaChatGoogle AIGoogle Cloud Vertex AIGPTRouterGroqChatHuggingFaceIBM watsonx.aiJinaChatKineticaLiteLLMLiteLLM RouterLlama 2 ChatLlama APILlamaEdgeLlama.cppmaritalkMiniMaxMistralAIMLXMoonshotNaverNVIDIA AI EndpointsChatOCIModelDeploymentOCIGenAIChatOctoAIOllamaOpenAIPerplexityPremAIPromptLayer ChatOpenAISambaNovaCloudSambaStudioSnowflake CortexsolarSparkLLM ChatNebula (Symbl.ai)Tongyi QwenUpstagevLLM ChatVolc Enging MaasYandexGPTSupported LLMsLLMs vs. Chat ModelsLarge Language Models (LLMs)Key Characteristics of LLMsChat ModelsKey Characteristics of Chat ModelsInstruct ModelsKey Benefits of Instruct ModelsA Comprehensive List of Popular Instruct ModelsSummary
3. Building Advanced Applications Powered by LLMs with LangChain and Python
App 1: YouTube Video SummarizerHow to Build the AppStep 1: Get Your OpenAI API KeyStep 2: Run the Following CommandsStep 3: Execute the Following CommandStep 4: Import the Whisper Model and Process the VideoStep 5: Read the Written Content in a FileStep 6: Use LangChain to Split a Text File into Smaller ChunksStep 7: Summarize the Preprocessed ContentStep 8: Define a Prompt Template Using LangChain's PromptTemplateStep 9: Summarization PipelineApp 2: Chat with a GitHub RepositoryHow It WorksStep 1: Select a GitHub Repository and Download It As ZipStep 2: Install All Libraries RequiredStep 3: Import the Libraries and Obtain the Needed API KeysStep 4: Get Repository ContentStep 5: Fetch All FilesStep-by-Step ExplanationStep 6: Creating a Searchable DatabaseStep-by-Step ExplanationStep 7: Creating the Actual Chatting Feature FunctionStep-by-Step ExplanationApp 3: Financial Report Analysis AppKey FeaturesStep 1: Install All Required LibrariesStep 2: Set Up OpenAI API Key and Add It to the CodeStep 3: Import All Required LibrariesStep 4: Process Financial ReportsStep 5: Preparing and Indexing Text Data for Efficient Retrieval Using AI-Powered Search and Question Answering (QA)1. Splitting the Extracted Text into Smaller Chunks2. Generating Text Embeddings3. Storing and Indexing the Text Chunks in a FAISS Database4. Setting Up the AI-Powered Retrieval and QA SystemStep 6: Ask a QuestionApp 4: Automate and Enhance Your Blog Posts with LangChain and Google SearchStep 1: Install All Required LibrariesStep 2: Define Three Variables—Title, Text All, and Text to ChangeStep 3: Define Your API KeysHow to Get Your API Keys and IDStep 4: Generate Search ResultsStep 5: Get Search ResultsStep 6: Find the Most Relevant ResultsStep 7: Split into ChunksStep 7: Create EmbeddingsStep 8: Extend the SentenceApp 6: YouTube Scriptwriting ToolStep 1: Install All Required Libraries and Import ThemStep 2: Authenticate in Google Drive As We Use Google Colab and Insert Your OpenAI API KeyStep 3: Download Your Desired YouTube Video, Extract the Audio, and Convert It to MP3Step 4: Transcribe AudioStep 5: Generate OutlineStep 6: Expand the ScriptStep 7: Combine All and Run the ToolApp 7: Email GeneratorKey FeaturesStep 1: Install All Required Libraries and Import ThemStep 2: Generate Response with OpenAIStep 3: Combine All Together and Generate EmailApp 8: CSV Data Analysis AppStep 1: Install All Required Libraries and Import ThemStep 2: Generate and Add Your OpenAI API KeyStep 3: Load Your CSV FileStep 4: Create a LangChain AgentApp 9: Knowledge Base Voice AssistantStep 1: Install the Required Libraries and Import ThemStep 2: Generate and Add Your OpenAI API KeyStep 3: Develop Voice InteractionStep 4: Load Knowledge Base from the Web and Create the QA ChainStep 5: Combine Them All TogetherApp 10: Analyzing Codebase with LangChainStep 1: Install All Required LibrariesStep 2: Generate and Add Your OpenAI API KeyStep 3: Upload and Load the FilesStep 4: Create and Store Code EmbeddingsStep 5: Create Retriever and Retrieval ChainApp 11: Recommender System with LangChainHow It WorksStep 1: Install and Import the Required LibrariesStep 2: Generate and Add Your OpenAI API Key and Then Import All Libraries RequiredStep 3: Load Up Some Sample DataStep 4: Convert Data into LangChain Document Format and Generate EmbeddingsStep 5: Define an Advanced Retrieval FunctionStep 6: Integrate a QA System Using LangChainStep 7: Set Up an AI Conversational AgentStep 8: Test the SystemApp 12: PDF Files ChatbotStep 1: Install All Required LibrariesStep 2: Generate and Add Your OpenAI API KeyStep 3: Upload Your PDF Files, Access Them, and Create a Vector Store DatabaseStep 4: Create a Chatbot with MemoryStep 5: Ask the Chatbot and Receive an AnswerSummary
4. Deploying LLM-Powered Applications
Integrating LLMs into Web and Mobile ApplicationsHosted ModelsHow Hosted Models WorkPrepackaged ModelsDeploying Open Source ModelsSteps to Deploy Open Source ModelsFactors to Consider When Choosing a MethodLLM Cloud Deployment and Scalability ConsiderationsDeployment ArchitectureInfrastructurePerformanceCost ManagementMonitoringHigh Availability and Fault ToleranceCompliance and EthicsTools for Deploying LLMsModel Hosting FrameworksExample: Saving a Model Locally, Uploading It to Hugging Face, and Calling ItOptimization ToolsONNX ExampleCloud ServicesAWS SageMaker ExampleOrchestration and ScalingEdge and Mobile DeploymentAPIs for Hosted ModelsDistributed Inference and Fine-TuningMonitoring and ObservabilityLLM Inference Challenges: A Comprehensive ExplorationLatency in InferenceComputational Demands and Resource ConstraintsScalability and MultitenancyThe Trade-Off Between Batching and ResponsivenessModel Parallelism and Distributed SystemsCost EfficiencyReliability and RobustnessEthical and Security ConsiderationsLLM Memory OptimizationThe Memory Challenges of LLMsKey Memory Optimization TechniquesTrade-Offs in Memory OptimizationFuture DirectionsLLM CompressionQuantizationPruningKnowledge DistillationLow-Rank FactorizationSparsity-Inducing TechniquesCompression Challenges and Trade-OffsFuture Directions in LLM CompressionAttention Layer OptimizationScheduling Optimization in LLM DeploymentSummary

5. Building and Fine-Tuning LLMs
Architecture of Large Language Models (LLMs)At the Foundation of Any LLM Lies the Process of TokenizationSelf-Attention MechanismLayer Normalization and Residual ConnectionsTransformer BlocksAt the Output Layer, LLMs Operate Differently Depending on Their DesignVariations in LLM ArchitecturesFine-Tuning Strategies and ConsiderationsWhat Is LLM Fine-Tuning?Data Requirements for Fine-TuningLLM Fine-Tuning TechniquesPrimary Approaches to Fine-Tuninga. Feature Extraction (Repurposing)b. Fine-Tuning Embedding ModelsWhy Fine-Tune Embedding Models?How to Fine-Tune Embedding ModelsPopular Techniques for Fine-Tuning Embeddingsc. Full Model Fine-TuningStriking the BalanceProminent Fine-Tuning Methodsa. Supervised Fine-TuningKey Techniques in Supervised Fine-Tuningb. Reinforcement Learning from Human Feedback (RLHF)Key Techniques in RLHFFine-Tuning Process and Best Practicesa. Data Preparationb. Choosing the Right Pretrained Modelc. Identifying the Right Parameters for Fine-Tuningd. ValidationEvaluation Metrics and Benchmarks for Fine-Tuning LLMsClassification Tasks (e.g., Sentiment Analysis, Intent Detection)Text Generation Tasks (e.g., Summarization, Translation, Dialogue)Reasoning and Question AnsweringDialogue and Chatbot EvaluationGeneral Model EvaluationCommon BenchmarksLanguage UnderstandingSummarizationMachine TranslationQuestion AnsweringDialogueRetrieval and Retrieval-Augmented Generation (RAG)Best Practicese. Detect Bias, Fairness, and Groundedness of LLMsGroundednessBiasFairnessPractical Steps to ImplementDetecting Data Drift When Fine-Tuningf. Model Iterationg. Model DeploymentFine-Tuning Applicationsa. Sentiment Analysisb. Chatbotsc. SummarizationAdvanced Fine-Tuning Techniques for LLMsLow-Rank Adaptation (LoRA)Prompt TuningFederated Fine-TuningWhen to Not Use LLM Fine-TuningPretrained Models Are Already SufficientInsufficient or Low-Quality DataHigh Computational Costs and Resource ConstraintsRegulatory, Privacy, and Ethical ConstraintsMaintaining Model VersatilityTask Scope Is Uncertain or EvolvingHigh-Risk Scenarios Requiring Predictability and StabilityEthics and Bias in AI and LLMsUnderstanding AI Ethics and Its Relevance to LLMsCore Ethical Challenges in LLMsBias in Language ModelsPrivacy and Data UsageTransparency and AccountabilityMisinformation and ManipulationEnvironmental ImpactPromoting Fairness and Equity in LLMsAddressing Broader Ethical ConcernsResponsible AI DevelopmentRegulation and Policy for Ethical AIFuture DirectionsLLM Fine-Tuning ExampleStep 1: Loading DatasetStep 2: TokenizationStep 3: Training and Evaluation SetsStep 4: Adapting the ModelStep 5: Fine-Tuning the ModelStep 6: EvaluationWhat Happens Internally During Fine-TuningConclusion
Index

Content preview from Intermediate Python and Large Language Models

D. GrigorovIntermediate Python and Large Language Modelshttps://doi.org/10.1007/979-8-8688-1475-4_4

4. Deploying LLM-Powered Applications

Dilyan Grigorov¹

(1)

Varna, Varna, Bulgaria

The deployment of large language models (LLMs) marks a pivotal step in transforming cutting-edge AI research into impactful real-world applications. Whether enabling conversational agents, automating content creation, or driving decision-making tools, LLMs unlock opportunities for innovation across industries. However, deploying these powerful models is far from straightforward. It requires navigating a landscape of technical challenges, architectural choices, and optimization ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9798868814754

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Intermediate Python and Large Language Models

by Dilyan Grigorov

4. Deploying LLM-Powered Applications

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.