O'Reilly logo

Hadoop Operations and Cluster Management Cookbook by Shumin Guo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Importing data to HDFS

If our Big Data is on the local filesystem, we need to move it to HDFS. In this section, we will list steps to move data from the local filesystem to the HDFS filesystem.

Getting ready

We assume that our Hadoop cluster has been properly configured and all the Hadoop daemons are running without any issues. And we assume that the data on the local system is in the directory /data.

How to do it...

Perform the following steps to import data to HDFS:

  1. Use the following command to create a data directory on HDFS:
    hadoop fs -mkdir data
    

    This command will create a directory /user/hduser/data in the HDFS filesystem.

  2. Copy the data file from the local directory to HDFS using the following command:
    hadoop fs -cp file:///data/datafile /user/hduser/data ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required