Designing Big Data Platforms

10Data Security

After reading this chapter, you should be able to:

• Learn infrastructure security patterns
• Understand data privacy and techniques
• Describe prevalent law enforcement on data
• Learn Big Data security solutions

Infrastructure security and data privacy are a must but a hurdle for any organization at the same time. Organizations want to take advantage of Big Data technologies while addressing security and privacy concerns. Big Data makes matters a bit more complicated as various data sources with different policies meet at the same data storage. Large volumes of data from partners, customers, subjects, and products meet in the same place. Big Data platforms are double‐edged swords. The aim here is to ease the development of insights as easily as possible and to mitigate any vulnerabilities. While designing Big Data platforms, concerns about infrastructure security, data privacy, and addressing law enforcement arise.

10.1 Infrastructure Security

Securing Big Data platforms requires securing underlying infrastructure components, which mostly require distributed computing. Distributed computing means partial data per node but much more complexity. The more complex the systems are, the easier the systems to have security flaws and issues. Recently, distributed computing frameworks or storages that empowered most of Big Data infrastructures lacked proper security protocols. Distributed computing frameworks need to provide security during computation between nodes. ...