Generative AI with Python

5.5    Embeddings

Let’s visualize where we are in the process. We’ve loaded one or more documents. We’ve broken them down into small manageable pieces. But up to this point, we’re dealing with texts—human-readable texts. We need to translate this text into a machine-understandable representation of text in a process called embedding. As shown in Figure 5.10, text chunks are converted into numerical vectors.

Figure 5.10     Embedding Process

In Section 5.4, you learned what an embedding vector was. A short recap: an embedding is a numerical vector that represents the meaning of a word or of a complete sentence or paragraph. This vector does not ...