Managing Complexity of Information Systems: The Value of Simplicity

Chapter 2Complexity, Simplicity,and Abstraction

 

 

 

Recursion is the root of computation since it tradesdescription for time.

Alan Jay Perlis — Epigrams on Programming

 

2.1. What does information theory tell us?

We start our journey through complexity and simplicity concepts with mathematics or, more precisely, information theory. This might seem an exotic topic if what we have in mind are applications to the IT world. However, concepts that will be at the core of our future preoccupations, information, randomness, and especially complexity have all been under close scrutiny by mathematicians for over more than half a century now. In their hands, these concepts have evolved into a set of ideas, which is both deep and robust. Moreover, information theory is actually one of those few areas where mathematics succeeded in rigorously formalizing imprecise, almost philosophical concepts, such as complexity and information, to which they bring a unique insight. It would thus seem unreasonable for us to overlook this body of knowledge altogether. These information theory concepts form a collection of metaphors that will help us build a healthy intuition that will prove helpful later when we venture into less rigorous but more practical IT concepts. As we shall see, this first look at the subject, through mathematical glasses, also highlights a number of important issues and limitations, which occur as soon as one seriously attempts to define complexity.

As information theory is a highly ...