Data Lake for Enterprises

Avro Data as HIVE Table

In order to create Avro data from the contacts text file, we will make use of INTERNAL Hive table. They come with in-built mechanisms to convert text data to Avro data. However, having an EXTERNAL table for Avro data would be more practical as discussed before from integration perspective with Sqoop, Flume and Flink.

In order to see this in action we will need to execute the following steps:

1. Avro schema for contacts namely contact.ascv (schema file) could be represented as shown the following command:

{   "namespace": "example.avro",   "type": "record",   "name": "Contact",   "fields": [      {"name": "id", "type": "string"},      {"name": "cell",  "type": "string"},      {"name": "phone", "type": "string"}, {"name": "email", "type": "string"} ...