Bad Data Handbook

Chapter 15. The Dark Side of Data Science

Marck Vaisman

More often than not, data scientists hit roadblocks that do not necessarily arise from problems with data itself, but from organizational and technical issues. This chapter focuses on some of these issues and provides practical advice on dealing with them, both from human and technical perspectives. The anecdotes and examples in this chapter are drawn from real-world experiences working with many clients over the last five years and helping them overcome many of these challenges.

Although the ideas that are presented in this chapter are not new, the main purpose is to highlight common pitfalls that can derail analytical efforts. When put into context, these guidelines will help both data scientists and organizations be successful.

Avoid These Pitfalls

The subject of running a successful analytics organization has been explored in the past. There are many books, articles, and opinions written about it and this will not be addressed here. However, if you would like to be successful in executing and/or managing analytical efforts within your organization, you should not heed the “commandments” listed below.

1. Know nothing about thy data

2. Thou shalt provide your data scientists with a single tool for all tasks

3. Thou shalt analyze for analysis’ sake only

4. Thou shalt compartmentalize learnings

5. Thou shalt expect omnipotence from data scientists

These commandments attempt to cluster-related ideas, which I will explore in the following sections. If ...