In this chapter, we expose the modern architecture challenges facing the SMACK stack (Apache Spark, Mesos, Akka, Cassandra, and Kafka). Also, we present dynamic processing environment problems to see which conditions are suitable and which are not.
This chapter covers the following:
Why we need a pipeline architecture for big data
The Lambda Architecture concept
ETL and its dark side