Hands-On RAG for Production

Foreword by Jim Dowling

Ilya Sutskever, the leading figure in the development of large language models (LLMs), claimed that because LLMs can accurately predict the next token, they understand the underlying reality that led to the creation of that token. In other words, LLMs have an internal model of the world based on language. The LLM’s internal model can reason about anything in the world, provided it is first transformed into language the LLM was trained on. The LLM has encyclopedic knowledge of the world and can answer queries using the huge volume of knowledge it acquired from the vast number of documents it was trained on.

But if you want an LLM to provide insights on anything that happened after its training cutoff date, you need to include all the relevant information (known as context) in your prompt to the LLM so that it can answer the question. LLMs can even learn and generalize from the context you provide, in what is known as in-context learning. But if you converse directly with an LLM (not via a chatbot), it will be like talking to Leonard Shelby from Memento, who tragically could form no new long-term memories. Chatbots give you the illusion that the LLM has memory as they provide the full conversation as context in every prompt.

In a way, the LLMs are like computers that only have ROM but no RAM to make new memories. Just as Leonard Shelby got creative by using his body to store memories as tattoos (mementos), LLMs can use external systems as memory that can ...