The Care and Feeding of Data Scientists

Chapter 5. To Agile or Not to Agile

You’ve recruited and hired your team. You’ve set them up for continuous learning to keep them engaged and performing their best for your business. Next, you’ll want to assess how you’re going to work together as a team to achieve those fantastic business outcomes your boss has been dreaming of (and telling their boss to expect…).

Over the past few decades, “Agile” has taken the software world by storm and become a cottage industry in and of itself. Data science teams in companies with lots of software engineers face a choice about how they’re going to work: to Agile or not to Agile? Perhaps even more so, data science teams that don’t work regularly with engineering teams should ask themselves the question: Should we learn from the experiences of our engineering sisters and brothers, or does data science require a different way of working?

In this chapter, we make a distinction between the Agile mindset or philosophy and all of the trappings that have become the Agile process. We maintain that although the Agile mindset is a great fit for data science work, teams must beware of adopting all of the formality of the Agile process lest it suck the life (and ultimately the exploratory creativity) out of a data science team.

History of the Agile World, Part I

The Agile movement traces its origins back to the Agile Manifesto, which was written by a group of software engineering process thinkers at a ski resort in Utah in 2001. In its summary, the ...