Software Engineering for Data Scientists

Chapter 12. Automation and Deployment

Software engineers love to automate boring, repetitive tasks. Automation is a skill that you can learn and apply to the code you write as a data scientist. In this chapter, I’ll show you some tools you can use to automate some of the steps in your coding workflow such as linting and testing. I’ll also describe the process of deploying code to a production system. Even if deployments aren’t handled by your team, you’ll likely find it useful to know the basics of how the process works.

At first, tools for automation and deployment may seem complex and a little intimidating. Many tools are available, and all have their own terminology. But often, it’s just the initial setup that is complex. Once you have set up these tools, they run without any action needed from you.

Automating your coding processes helps ensure they are standardized and reproducible. You can make sure that your code is always linted, formatted, and tested before it reaches a production system, or even before it is committed to version control. You can also make sure that the details are standardized, for example, by making sure that you are always using the same linter with the same settings. Automation becomes even more useful when you’re deploying your code frequently, and it can save you a huge amount of time.

Automation also helps ensure that your processes are standard across your team. If tests are automated, everyone is more likely to run them, and there will be fewer ...