Retrieval Augmented Generation in Production with Haystack

Chapter 2. Optimizing AI

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be finalized later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

On multiple occasions, I’ve heard feedback from customers that LLM responses are too generic, or obviously AI generated. As humans, we are very sensitive to small discrepancies, and with the numerous options available, a low quality application makes it highly likely that the customer switches to another provider. In order to ensure high quality applications that attract customers, you need to be able to measure performance, and make improvements to performance

That satisfies customers. In this chapter, we will learn how to evaluate and optimize RAG applications. Specifically, we will go through how to evaluate RAG applications, and the levers of choice for improving the performance of our application, based on these evaluations.

RAG based applications in particular, have 6 distinct components to be optimized, based on use-case - including document extraction, chunking mechanism, embedding model, database, ...