Autonomous Learning Systems: From Data Streams to Knowledge in Real-time

2

Fundamentals of Probability Theory

It is a truth very certain that when it is not in our power to determine what is true we ought to follow what is most probable.

(R. Descartes in Discourse on Method)

I will never believe that God plays dice with the Universe.

(A. Einstein)

Probability theory is one of the methodologies to represent and tackle some types of uncertainties (specifically, randomness). It was mainly developed in the eighteenth century with main contributions from the mathematicians such as Blaise Pascal (1623–1662), Pierre de Fermat (1601–1665), Daniel Bernoulli (1700–1782) and later the British clergymen Thomas Bayes (1701–1761) addressing problems of gambling and insurance. A huge role to make it more mathematics based and scientific was played by the Russian mathematician Andrey A. Markov (1856–1922) and the Soviet academic Andrey N. Kolmogorov (1903–1987).

This theory is widely used and read in most prestigious universities. It is the view of the author of this book that it indeed provides an elegant framework to describe random processes and data, but is overused, because the assumptions on which it is based (random nature of the processes and events, independence of the data samples from one another and, often required, normal or parameterised distributions) rarely (or, more precisely, never fully) hold in practice. A possible explanation why probability theory is accepted much more widely than, for example, fuzzy logic theory is, perhaps, its philosophical ...