Effective Data Science Infrastructure

2 The toolchain of data science

This chapter covers

• The key activities that the data scientist engages in on a daily basis
• The essential toolchain that makes the data scientist productive
• The role of workflows in the infrastructure stack

Every profession has its tools of the trade. If you are a carpenter, you need saws, rulers, and chisels. If you are a dentist, you need mirrors, drills, and syringes. If you are a data scientist, what are the essential tools that you need in your daily job?

Obviously, you need a computer. But what’s the purpose of the computer? Should it be used to run heavy computation, train models, and such, or should it be just a relatively dumb terminal for typing code and analyzing results? Because production applications ...