May 2017
Beginner to intermediate
596 pages
15h 2m
English
Content preview from Data Lake for Enterprises
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Data indexing from Hive
Now that we can visualize all the data loaded into Hadoop via Hive tables, we have complete customer data in Hadoop. The address and contacts data is there in both Elasticsearch and Hadoop, using Flink pipeline. Also, customer profile data is available in Hadoop, using Sqoop job. But, we don't have customer profile data in Elasticsearch.
For this, we can export the Hive data as Elasticsearch indices. This can be achieved by using ES-Hadoop framework, which is part of Elastic Stack.
For ES-Hadoop framework to work with Hive, a quick setup and configuration are required, as summarized here:
- Download the ES-Hadoop binaries from the following location using the following command
wget http://download.elastic.co/hadoop/elasticsearch-hadoop-5.4.0.zip ...