97 Things Every Data Engineer Should Know

Chapter 30. Engineering Reproducible Data Science Projects

Dr. Tianhui Michael Li

Just like any scientific field, data science is built on reproducibility. In a reproducible project, someone else (including the future you) can re-create your results by running a simple command. On the one hand, this means that you should check your analysis code into a source-control tool like Git. On the other, it also means following DevOps best practices like including dependency lists in machine-readable forms (like requirements.txt for pip or environment.yml for Conda). You might go one step further and use a Dockerfile. The commands needed to install and run the analysis should also be included. Finally, make sure that you clearly document what to run in a README.md file, or preferably in a job runner like Make.

Another important piece of reproducibility is eliminating something we’ll call algorithmic randomness from your pipeline in order to maintain consistency. If your data is being subsetted from a larger dataset or your analysis depends on an initial random condition (many of your favorite ones do), you’re depending on a random number generator. This can cause the same analysis to yield different results—so make sure your generator is tied to a random seed that’s checked into version control. This ensures that your work can be reproduced, and any variation in your results can be attributed ...