Building Data Streaming Applications with Apache Kafka

Expose hive table

We will create hive table over base directory where a streaming record is getting pushed on HDFS. This will help us track the number of fraud records being generated over time:

hive> create database packt;hive> create external table packt.teststream (iprecords STRING) LOCATION '/user/packt/streaming/fraudips';

You can also expose hive tables on top of the incoming data which is being pushed to Kafka topic in order to track the percentage of IPs being detected as fraud from an overall record. Create one more table and add the following line to your streaming application explained later:

ipRecords.dstream().saveAsTextFiles("hdfs://localhost:8020/user/packt/streaming/iprecords", "");

Also, create the following table in hive:  ...