Data Science at the Command Line

We data scientists love to create exciting data visualizations and insightful statistical models. However, before we get to that point, usually much effort goes into obtaining, scrubbing, and exploring the required data.

The command line, although invented decades ago, is an amazing environment for performing such data science tasks. By combining small, yet powerful, command-line tools you can quickly explore your data and hack together prototypes. New tools such as GNU Parallel, jq, and Drake allow you to use the command line for today's data challenges. Even if you're already comfortable processing data with, for example, R or Python, being able to also leverage the power of the command line can make you a more efficient data scientist.

We will make use of the Data Science Toolbox, which is a free, open-source virtual environment that allows everybody to get started with data science in minutes. The Data Science Toolbox runs not only on Linux, but also on Mac OS X and Microsoft Windows, so everybody can participate with this hands-on webcast.

In about two hours we will cover the following subjects:

• Essential concepts of the *nix command line;
• Setting up the Data Science Toolbox;
• Integrating the command line with IPython and R;
• Filters such as cut, grep, sed, and awk;
• Scraping websites using curl, scrape, xml2json, and jq;
• Managing your data science workflow using Drake;
• Parallelizing and distributing data-intensive pipelines using GNU Parallel;
• Turning existing Python, R and Java code into reusable command-line tools;
• Creating data visualizations and statistical models.

Whether you're entirely new to the command line or already dreaming in shell scripts, by the end of this webcast you will have a solid understanding of how to leverage the power of the command line for your next data science project.