Data Lake for Enterprises

Data preparation

For Hadoop examples, we will expand the data generation utility of customer profile even further. For batch processes it is generally expected that the volume of data is quite substantial.

Hence we will try and generate about 2 million customer records and their related information for batch processing.

We shall use the same data generation utility as before for populating the data in the database, the only difference being that we will be generating 2 million customer records in the DB. Generating these many rows in relational database may take some time depending on the machine configuration.

If you would like to cleanup the PostgreSQL database before running the data generation utilities, please run the following queries, ...