Mastering Apache Storm

Setting up HDFS

Follow these steps for setting up HDFS:

1. Make directories for holding the namenode and datanode data:

$ mkdir -p ~/mydata/hdfs/namenode
$ mkdir -p ~/mydata/hdfs/datanode  

2. Specify the namenode port in the $HADOOP_CONF_DIR/core-site.xml file by adding the following property inside the <configuration> tag:

<property> 
        <name>fs.default.name</name> 
        <value>hdfs://localhost:19000</value> 
   <!-- The default port for HDFS is 9000, but we are using 19000 Storm-Yarn uses port 9000 for its application master --> 
</property> 

3. Specify the namenode and datanode directory in the $HADOOP_CONF_DIR/hdfs-site.xml file by adding the following property inside the <configuration> tag:

<property> <name>dfs.replication</name> <value>1</value> <!-- ...