Big Data: Understanding How Data Powers Big Business

Chapter 10

Solution Engineering

You are now ready to tie all the previous exercises and envisioning work into a solution. But what is meant by a “solution,” and what special skills and processes are necessary to architect a solution?

The trouble with big data is that there is no one shiny technical solution. You can't just install Hadoop, predictive analytics, or a data appliance and assume it will provide a big data solution. The data industry has struggled with this dilemma before, as data warehousing and business intelligence technologies sought relevance within organizations over the past 10 to 15 years. To be successful with big data and advanced analytics—like its brethren data warehousing and business intelligence before it—requires a new engineering skill, something called solution engineering.

There's engineering for many disciplines—system engineering, electrical engineering, mechanical engineering—so why not solution engineering? Solution engineering would be defined as:

A process for identifying and breaking down an organization's key business initiatives into its business enabling capabilities and supporting technology components in order to support an organization's decision-making and data monetization efforts.

Let's take a look at the steps of the solution engineering process.

Note

Some key materials and graphics have been duplicated from previous chapters in order to make this a stand-alone chapter.

The Solution Engineering Process

Surprisingly, the solution ...