Architecting Data and Machine Learning Platforms

Chapter 3. Designing for Your Data Team

When designing a data platform, there are several technical aspects to take into consideration: performance, cost, operational overhead, operational excellence, integration of new analytical and ML approaches, etc. However, these technical aspects will fall to the wayside if you don’t address the culture of the company—new technologies require a willingness from employees to change their mental models and ways of working. Another key aspect to keep in mind is the skills that existing employees currently possess and will need to pick up. In some cases, employees who learn new skills and change their way of working will end up in a different role than the role they had before the data platform was in place.

In this chapter, we explore how organizations can plan and orchestrate these changes in mental models, workflows, technical skills, and roles. Every organization is unique, and so building a data platform will involve devising a granular plan for each division and employee in it. In this chapter, we describe what such a granular plan would look like for different types of organizations.

Classifying Data Processing Organizations

Organizations can succeed by employing different strategies based on their talent. There is no universal “best” approach. A sports team with a strong defense should play to their strengths and focus on defense, not try to copy the offense of a team with skilled offensive players. Similarly, if your organization ...