When optimizing a data science project, it is crucial to look at the entire analytics pipeline and understand the time and effort spent at each stage and their interdependencies.
Let's simplify the problem to decrease the execution time of the software implemented in a particular language. We won't worry about shuffling around large blocks of data, say, from a production database to the analytics data store.
At its most abstract level, the execution time of your code is a function of the code itself and the hardware used to run it. If you want to decrease the time required to run your code, you can upgrade the hardware, modify the software, or do both.
For optimization purposes, we want to start ...