ElasticSearchSink
Another common target to stream data to be searched in NRT is Elasticsearch. Elasticsearch is also a clustered searching platform based on Lucene, like Solr. It is often used along with the logstash project (to create structured logs) and the Kibana project (a web UI for searches). This trio is often referred to as the acronym ELK (Elasticsearch/Logstash/Kibana).
Note
Here are the project home pages for the ELK stack that can give you a much better overview than I can in a few short pages:
- Elasticsearch: http://elasticsearch.org/
- Logstash: http://logstash.net/
- Kibana: http://www.elasticsearch.org/overview/kibana/
In Elasticsearch, data is grouped into indices. You can think of these as being equivalent to databases in a single MySQL ...
Get Apache Flume: Distributed Log Collection for Hadoop - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
