book

Building Generative AI Services with FastAPI

Name: Building Generative AI Services with FastAPI
Author: Alireza Parandeh
ISBN: 9781098160302

by Alireza Parandeh

April 2025

Intermediate to advanced

530 pages

12h 39m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Includes

Quizzes

Foreword
Preface
Objective and ApproachPrerequisitesBook StructureHow to Read This BookHardware and Software RequirementsConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
I. Developing AI Services
1. Introduction
What Is Generative AI?Why Generative AI Services Will Power Future ApplicationsFacilitating the Creative ProcessSuggesting Contextually Relevant SolutionsPersonalizing the User ExperienceMinimizing Delay in Resolving Customer QueriesActing as an Interface to Complex SystemsAutomating Manual Administrative TasksScaling and Democratizing Content GenerationHow to Build a Generative AI ServiceWhy Build Generative AI Services with FastAPI?What Prevents the Adoption of Generative AI ServicesOverview of the Capstone ProjectSummary
2. Getting Started with FastAPI
Introduction to FastAPISetting Up Your Development EnvironmentInstalling Python, FastAPI, and Required PackagesCreating a Simple FastAPI Web ServerFastAPI Features and AdvantagesInspired by Flask Routing PatternHandling Asynchronous and Synchronous OperationsBuilt-In Support for Background TasksCustom Middleware and CORS SupportFreedom to Customize Any Service LayerData Validation and SerializationRich Ecosystem of Plug-InsAutomatic DocumentationDependency Injection SystemLifespan EventsSecurity and Authentication ComponentsBidirectional Web Socket, GraphQL, and Custom Response SupportModern Python and IDE Integration with Sensible DefaultsFastAPI Project StructuresFlat StructureNested StructureModular StructureProgressive Reorganization of Your FastAPI ProjectOnion/Layered Application Design PatternComparing FastAPI to Other Python Web FrameworksFastAPI LimitationsInefficient Model Memory ManagementLimited Number of ThreadsRestricted to Global Interpreter LockLack of Support for Micro-Batch Processing Inference RequestsCannot Efficiently Split AI Workloads Between CPU and GPUDependency ConflictsLack of Support for Resource-Intensive AI WorkloadsSetting Up a Managed Python Environment and ToolingSummary
3. AI Integration and Model Serving
Serving Generative ModelsLanguage ModelsAudio ModelsVision ModelsVideo Models3D ModelsStrategies for Serving Generative AI ModelsBe Model Agnostic: Swap Models on Every RequestBe Compute Efficient: Preload Models with the FastAPI LifespanBe Lean: Serve Models ExternallyThe Role of Middleware in Service MonitoringSummaryAdditional References
4. Implementing Type-Safe AI Services
Introduction to Type SafetyImplementing Type SafetyType AnnotationsUsing AnnotatedDataclassesPydantic ModelsHow to Use PydanticCompound Pydantic ModelsField Constraints and ValidatorsCustom Field and Model ValidatorsComputed FieldsModel Export and SerializationParsing Environment Variables with PydanticDataclasses or Pydantic Models in FastAPISummary
II. Communicating with External Systems
5. Achieving Concurrency in AI Workloads
Optimizing GenAI Services for Multiple UsersOptimizing for I/O Tasks with Asynchronous ProgrammingSynchronous Versus Asynchronous (Async) ExecutionAsync Programming with Model Provider APIsEvent Loop and Thread Pool in FastAPIBlocking the Main ServerProject: Talk to the Web (Web Scraper)Project: Talk to Documents (RAG)Optimizing Model Serving for Memory- and Compute-Bound AI Inference TasksCompute-Bound OperationsExternalizing Model ServingManaging Long-Running AI Inference TasksSummaryAdditional References
6. Real-Time Communication with Generative Models
Web Communication MechanismsRegular/Short PollingLong PollingServer-Sent EventsWebSocketComparing Communication MechanismsImplementing SSE EndpointsSSE with GET RequestSSE with POST RequestImplementing WS EndpointsStreaming LLM Outputs with WebSocketHandling WebSocket ExceptionsDesigning APIs for StreamingSummary

7. Integrating Databases into AI Services
The Role of a DatabaseDatabase SystemsProject: Storing User Conversations with an LLM in a Relational DatabaseDefining ORM ModelsCreating a Database Engine and Session ManagementImplementing CRUD EndpointsRepository and Services Design PatternManaging Database Schemas ChangesStoring Data When Working with Real-Time StreamsSummary
III. Securing, Optimizing, Testing, and Deploying AI Services
8. Authentication and Authorization
Authentication and AuthorizationAuthentication MethodsBasic AuthenticationJSON Web Tokens (JWT) AuthenticationImplementing OAuth AuthenticationOAuth Authentication with GitHubOAuth2 Flow TypesAuthorizationAuthorization ModelsRole-Based Access ControlRelationship-Based Access ControlAttribute-Based Access ControlHybrid Authorization ModelsSummary
9. Securing AI Services
Usage Moderation and Abuse ProtectionGuardrailsInput GuardrailsOutput GuardrailsGuardrail ThresholdsImplementing a Moderation GuardrailAPI Rate Limiting and ThrottlingImplementing Rate Limits in FastAPIThrottling Real-Time StreamsSummary
10. Optimizing AI Services
Optimization TechniquesBatch ProcessingCachingModel QuantizationStructured OutputsPrompt EngineeringFine-TuningSummary
11. Testing AI Services
The Importance of TestingSoftware TestingTypes of TestsThe Biggest Challenge in Testing SoftwarePlanning TestsTest DimensionsTest DataTest PhasesTest EnvironmentsTesting StrategiesChallenges of Testing GenAI ServicesVariability of Outputs (Flakiness)Performance and Resource Constraints (Slow and Expensive)RegressionBiasAdversarial AttacksUnbound Testing CoverageProject: Implementing Tests for a RAG SystemUnit TestsIntegration TestingEnd-to-End TestingSummary
12. Deployment of AI Services
Deployment OptionsDeploying to Virtual MachinesDeploying to Serverless FunctionsDeploying to Managed App PlatformsDeploying with ContainersContainerization with DockerDocker ArchitectureBuilding Docker ImagesContainer RegistriesContainer Filesystem and Docker LayersDocker StorageDocker NetworkingEnabling GPU DriverDocker ComposeEnabling GPU Access in Docker ComposeOptimizing Docker Imagesdocker initSummary
Afterword
Index
About the Author

Content preview from Building Generative AI Services with FastAPI

About the Author

Alireza Parandeh is a chartered engineer with the UK Engineering Council and a certified developer and data scientist with Microsoft and Google. He has led engineering teams at both multinational consultancies and tech startups across various markets to build and deploy AI-enabled software. As an AI advocate, Ali started London’s Beginners Machine Learning meetup in 2018 to help people break into data and AI careers through hands-on workshops and community events. With a passion for education, he has also taught multiple software engineering bootcamps for Code First Girls, empowering more women to establish their tech and AI careers.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Building Generative AI Agents: Using LangGraph, AutoGen, and CrewAI

Publisher Resources

ISBN: 9781098160296Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Building Generative AI Services with FastAPI

by Alireza Parandeh

About the Author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.