Time for action – adding a rack awareness script
We can enhance the default flat rack configuration by creating a script that derives the rack location for each host.
- Create a script in the Hadoop user's home directory on the NameNode host called
rack-script.sh
, containing the following text. Remember to change the IP address to one of your HDFS nodes.#!/bin/bash if [ $1 = "10.0.0.101" ]; then echo -n "/rack1 " else echo -n "/default-rack " fi
- Make this script executable.
$ chmod +x rack-script.sh
- Add the following property to
core-site.xml
on the NameNode host:<property> <name>topology.script.file.name</name> <value>/home/Hadoop/rack-script.sh</value> </property>
- Restart HDFS.
$ start-dfs.sh
- Check the filesystem via
fsck
.$ Hadoop fsck –rack
The ...
Get Hadoop Beginner's Guide now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.