Retrieval Augmented Generation in Production with Haystack

Chapter 3. Observable AI

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be finalized later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

Now that you’ve deployed your AI application, it is time to sit back, relax, and let customers have a seamless experience with your model. Seamless because after all - haven’t you evaluated your model offline on representative data and load tested it prior to deployment in production? Well that is often not the case. In traditional software applications, we care mostly about operational metrics (latency and throughput). But for AI applications, in addition to operational metrics, we also care about quality and performance.

Here’s an example of a case where performance is impacted. Let’s say, for example, that a product website builds a recommendation system. The performance is great initially, as customers find recommendations useful and sales go up. But a week later, performance starts to go down. It is so bad that the model did worse than the previous simplistic model. What happened here? A few weeks of ...