Principles and Practice of Big Data, 2nd Edition

12

Finding the Clues in Large Collections of Data

Abstract

It is often assumed that Big Data resources are too large and complex for human comprehension. The analysis of Big Data is best left to software programs. Not so. When data analysts go straight to complex calculations, before they perform simple estimations, they will find themselves accepting wildly ridiculous conclusions. There is nothing quite like a simple, intuitive estimate to yank an overly-eager analyst back to reality. Often the simple act of looking at a stripped-down version of the problem opens a new approach that can drastically reduce computation time. In some situations analysts will find that a point is reached when higher refinements in methods yield diminishing returns. ...