Big Data

Big Data and the SP Theory of Intelligence

 163

6.10 Veracity: Managing Errors and Uncertainties in Data

In building a statistical model from any data source, one must often deal with the

fact that data are imperfect. Real-world data are corrupted with noise. Such noise

can be either systematic (i.e., having a bias) or random (stochastic). Measurement

processes are inherently noisy, data can be recorded with error, and parts of the

data may be missing. [12, p. 99]

Organizations face huge challenges as they attempt to get their arms around the

complex interactions between natural and human-made systems. The enemy is

uncertainty. In the past, since computing ...