Practical MLflow for Generative AI on Databricks

Chapter 4. Building and Versioning a Tool-Calling Agent

In traditional machine learning (ML), rigorous logging and versioning are non-negotiable, and this is even more vital for generative AI (GenAI) applications. GenAI applications are often more complex than traditional ML projects, involving the integration of multiple components like large language models (LLMs), vector databases, tools, and prompt templates. This complexity makes reproducibility, traceability, and objective performance evaluation essential. Without structured tools to version and track this intricate system, you are essentially flying blind. Identifying which change degraded performance, diagnosing variations, or reverting to a previously working version becomes a frustrating guessing game.

As we saw in Chapter 2, MLflow provides the capabilities for managing this complexity, enabling detailed logging and versioning for each iteration of a GenAI application to support an evaluation-driven development process. In this chapter, we’ll learn how to build a tool-calling agent using MLflow, LangChain, and Databricks functionalities. This chapter on versioning lays the foundation for more advanced topics that we will cover in subsequent chapters.

What Is a Tool-Calling Agent?

One of the simplest design patterns for developing GenAI applications is a chain. This approach operates like a workflow, where multiple components are executed in a predefined sequence. For this reason, it is often referred to as a deterministic ...