Modern Big Data Architectures

CHAPTER 2Evolution of IT Architectures and Paradigms

2.1 Evolution of IT Architectures

Over recent decades corporate IT architectures have evolved significantly. Starting from the large monolith application, through the introduction of web services and the emergence of the Service Oriented Architecture (SOA), which has evolved into microservices, we went through the wide adoption of cloud computing and have now reached the popularity of edge computing, the Internet of Things and cyber-physical systems. Each of these steps required a change in the way we produced, processed, stored, and analyzed the data, which will be explored in the subsequent sections of this chapter.

2.1.1 Monolith

Back in the 1990s corporate systems were built mainly as large monolith applications. They were based on a number of tightly coupled modules with strong interdependencies. This caused high development and maintenance costs. At the beginning of the software development process it is beneficial to have all the building blocks in one place, but as the system grows, it becomes tedious to track all the internal dependencies and the code base becomes hard to manage. The growing size and complexity of a monolith impacts all software life cycle steps influencing design, development, testing, and deployment.

Each design and development decision taken in a monolith system has long lasting consequences. This phenomenon is well described by the term technical debt coined by Cunningham Cunningham [1993]. ...