Data Lake for Enterprises

Data partitioning

Data partitioning is a very common technique by that data stored in a persistent store is segregated into sections for faster queries and easy management and monitoring (being segregated, the data to be managed becomes small and more manageable). This is a technique used in RDBMS and it's the same that could be employed in HDFS. HDFS, being a file system, partitioning is achieved using file partitioning. For an enterprise wide Data Lake, data partitioning is a needed aspect that needs serious thought and consideration. In previous section we did see different zones in data storage, this section expands on that aspect by bringing in the implementation aspect using data partitioning.

A table stored in HDFS has a number of ...