Generative AI with Python

6.3    Advanced Techniques

Although our simple RAG system is impressive, there’s still a lot of room for improvement. Opportunities for improvement can be found at different stages in the pipeline. For instance, some improvements can be made to the indexing pipeline.

Practically, we can improve the data before we add it to our vector database, which we’ll cover when we discuss advanced pre-retrieval techniques in Section 6.3.1. Other techniques tackle the problem at the retrieval step of the pipeline, which we cover in Section 6.3.2. Unsurprisingly, the last step of the pipeline can be improved as well; we’ll explore improvements in the generation step in Section 6.3.3.

6.3.1    Advanced Preretrieval Techniques

If the RAG system fails to fetch ...