AI and ML for Coders in PyTorch

Chapter 18. Introduction to RAG

Remember that first time you chatted with an LLM like ChatGPT—and how it was extremely insightful about things you didn’t expect it to know? I had worked with LLMs before the release of ChatGPT and on projects that highlighted LLM abilities, and I still was surprised by what they could do. Remember the famous on-stage demonstration by Google, where the CEO had a conversation with the planet Pluto? It was one of those fundamental mind shifts in the possibilities of AI that we’re still exploring as it continues to evolve.

But, despite all that brilliance, there were still limitations, and the more I and others worked with LLMs, the more we encountered them. The transformer-based architecture that we discussed in Chapter 15 was brilliant at snarfing up text data, creating QKV mappings from it, and learning how to artificially understand the semantics of the text as a result. But despite the volume of text used to build those mappings, there was—and always is—one blind spot: private data. In particular, if there is data that you want to work with that the model was not trained on, you’re at a major risk of hallucination!

Gaining skills to help mitigate this blind spot could potentially be the most valuable thing you can do as a software developer.

For this chapter, I want you to think about AI models and in particular large generative models like LLMs differently. Stop seeing them as intelligent and knowledgeable and start seeing them as utilities