Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics

CHAPTER 6

The Evolution of Analytic Tools and Methods

Is it possible to build a house using nothing but hand tools and following blueprints from decades ago? Sure. But very few people would want to do that given the modern power tools and updated blueprints available today. Similarly, analytic professionals can continue to build analytic processes using nothing but custom code and traditional methodologies. However, if time is taken to understand the options now available, few would choose to do so. Just as a home can be built with less manual effort and more features than a generation ago, so, too, can analytic processes be.

Analytic professionals have used a range of tools over the years that enabled them to prepare data for analysis, execute analytic algorithms, and assess the results. It’s no surprise that the depth and functionality of these tools have increased. In addition to much richer user interfaces, tools now automate or streamline common tasks. As a result, analytic professionals end up with more time to focus on analysis. When combining new tools and methods with the evolved scalability and processes discussed in Chapters 4 and 5, organizations will be well positioned to tame big data.

In this chapter, we’ll cover several topics related to how analytic professionals have changed their approaches to building analytic processes to better leverage the advances in tools and scalability available to them. Topics will include ensemble models, commodity models, and text ...