How text analysis works

We have seen an overview of Solr's text analysis; now let's learn how Solr actually implements the analysis process. Here are the common steps normally used by Solr in an analysis process:

  • Removing stop words:
    • Stop words (common letters/words) such as a, an, the, at, to, for, and so on are removed from the text string; so Solr will not give results for these common words. These words are configured in a text file (for example, stopwords_en.txt) and this file needs to be imported in the analysis configuration.
  • Adding synonyms:
    • Solr reads synonyms from the text file (synonyms) and adds them to the token stream. All synonyms need to be preconfigured in the text file, such as Football and SoccerCountry and Nation ...

Get Mastering Apache Solr 7.x now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.