12.4 Creating an Index
As we saw in Section 12.1, LlamaIndex provides a very convenient function for converting documents into a searchable index: VectorStoreIndex.from_documents. The function merely requires the list of documents as a parameter. LlamaIndex performs some interesting steps in the background, which we’ll take a closer look at in this section.
First, LlamaIndex nodes are created from the LlamaIndex documents. For this purpose, a text splitter is used such as the SentenceSplitter. In contrast to an “ordinary” text splitter, this one attaches importance to not splitting a sentence into multiple nodes.
To give you a feeling for how the splitting of documents into nodes using the SentenceSplitter can look in numbers, we’ve counted ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access