Building Generative AI Services with FastAPI

Chapter 3. AI Integration and Model Serving

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. The GitHub repo is available at https://github.com/Ali-Parandeh/building-generative-ai-services.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

Chapter Goals

In this chapter, you will learn:

• The intuition of how different GenAI models work

• How to integrate and serve generative model into FastAPI

• How to work with text, image, audio, video and 3D models

• How to model outputs within an API service

• How to preload models on application startup by leveraging the FastAPI lifespan events

• How to quickly build a user interface for prototyping

• Several model serving strategies in FastAPI

• How to leverage FastAPI background tasks to handle long-running model inferences

• How to leverage middlewares for service monitoring

In this chapter, you will learn the mechanisms of various GenAI models and how to serve them in a FastAPI application. Additionally, using the Streamlit UI package, you will create a simple browser client for interacting with the model serving endpoints. We will explore ...