Chapter 4. Choosing the Technical Infrastructure
So your team is well placed in the organization and has the right people on it. You are working to create a strong culture of providing value, and you have methods for managing the tasks. But those tasks require your data scientists to be doing technical work, which means they need the tools to do so. These include questions like what programming languages to use (R, Python, or something else); what kinds of databases to store information in; if models should be deployed as APIs by the engineering team or as batch scripts run by the data science team; and more. A data science leader has to be heavily involved in choosing which technology a team should use and deciding when it’s time to switch between them.
How to Make Decisions on Technical Infrastructure
The actual process for making decisions about a team’s technical infrastructure is as important as the actual decisions. The leader of the data science team may be making these decisions directly, or it may be a person on the team like the technical lead or a principal data scientist who has the final call. With any of these sorts of decisions, there is a spectrum of options for how the decision is made (see Figure 4-1). On one side of the spectrum is authoritarianism, the idea that all decisions within the team are solely made by the person in charge. On the other side is anarchy, the idea that anyone on the team can make whatever decision they personally feel is best.