Designing Big Data Platforms

8Data Science

After reading this chapter, you should be able to

• Describe data science applications.
• Understand the life cycle of data science.
• Use various data science tools.
• Use tooling for data science production.

Companies want to use any data for competitive advantage. Manual analysis of big data is difficult and tedious. An alternative might be applying data science techniques to analyze the data automatically. Data science applications' Big Data might provide customer satisfaction and retention gains with the data science tools and algorithms on the rise. Therefore, a modern Big Data platform should embrace data science from the bottom up.

The data science process includes several techniques. From data to actionable results, one might employ a mix of strategies. Some strategies might involve business understanding, domain knowledge, and creativity. Some strategies are relatively straightforward as it is left to the underlying algorithm or tooling like pattern discovery. Although there have been significant data science developments, the number of easily applicable techniques is arguably stable. This chapter discusses data science techniques, applies them through various technologies, and discusses deploying data science projects on production.

8.1 Data Science Applications

Data science has been a vital technology for organizations to decide, predict, and discover. Industries ranging from finance, manufacturing, transport, e‐commerce, education, travel, healthcare, ...