Learning Elasticsearch

Merging segments

By default, the refresh process creates a new segment every second. This will result in the creation of lots of segments. As a search on the shard has to go through all the segments in the shard, having lots of segments will slow down the search performance.

Segments also need lots of resources, such as file handlers, CPU, disk space, and memory. Since segments are immutable, any updates and deletes are only soft deleted due to which we will eventually run out of disk space. To decrease the number of segments, Lucene merges the segments of similar size into a bigger segment. While merging the segments, the documents that are marked as deleted are not copied to the merged segment. Until the segments merge, the document is ...