Apache Sqoop Cookbook

Chapter 2. Importing Data

The next few chapters, starting with this one, are devoted to transferring data from your relational database or warehouse system to the Hadoop ecosystem. In this chapter we will cover the basic use cases of Sqoop, describing various situations where you have data in a single table in a database system (e.g., MySQL or Oracle) that you want to transfer into the Hadoop ecosystem.

We will be describing various Sqoop features through examples that you can copy and paste to the console and then run. In order to do so, you will need to first set up your relational database. For the purpose of this book, we will use a MySQL database with the account sqoop and password sqoop. We will be connecting to a database named sqoop. You can easily create the credentials using the script mysql.credentials.sql uploaded to the GitHub project associated with this book.

You can always change the examples if you want to use different credentials or connect to a different relational system (e.g., Oracle, PostgreSQL, Microsoft SQL Server, or any others). Further details will be provided later in the book. As Sqoop is focused primarily on transferring data, we need to have some data already available in the database before running the Sqoop commands. To have something to start with, we’ve created the table cities containing a few cities from around the world (see Table 2-1). You can use the script mysql.tables.sql from the aforementioned GitHub project to create and populate all tables ...