Loading data from a local machine to HDFS

In this recipe, we are going to load data from a local machine's disk to HDFS.

Getting ready

To perform this recipe, you should have an already Hadoop running cluster.

How to do it...

Performing this recipe is as simple as copying data from one folder to another. There are a couple of ways to copy data from the local machine to HDFS.

  • Using the copyFromLocal command
    • To copy the file on HDFS, let's first create a directory on HDFS and then copy the file. Here are the commands to do this:
      hadoop fs -mkdir /mydir1
      hadoop fs -copyFromLocal /usr/local/hadoop/LICENSE.txt /mydir1
      
  • Using the put command
    • We will first create the directory, and then put the local file in HDFS:
      hadoop fs -mkdir /mydir2
      hadoop fs -put /usr/local/hadoop/LICENSE.txt ...

Get Hadoop: Data Processing and Modelling now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.