Token filters

The main function of a token filter is to add, modify, or delete the characters of the output tokens from the tokenizer. There are approximately 50 built-in token filters. We'll cover some popular token filters in the following table. You can find out more and learn about the remaining token filters at https://www.elastic.co/guide/en/elasticsearch/reference/7.x/analysis-tokenfilters.html. Each example token filter in the following table uses a standard tokenizer and a specified token filter. Note that no character filter is applied:

Token filter
asciifolding Input text "Ÿőű'ľľ ľőνė Ȅľȁśťĩćŝėȁŕćĥ 7.0"
Description

This transforms the terms when letters, numbers, and unicode symbols are not in the first 127 ASCII ...

Get Advanced Elasticsearch 7.0 now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.