You can perform the following steps for the configuration of YARN:
- Start Hadoop NameNode, secondary NameNode, and DataNode
- Alter yarn-env.sh.
- Add the following under the definition of YARN_CONF_DIR:
export HADOOP_CONF_DIR="${HADOOP_CONF_DIR:-$YARN_HOME/etc/hadoop}" export HADOOP_COMMON_HOME="${HADOOP_COMMON_HOME:-$YARN_HOME}" export HADOOP_HDFS_HOME="${HADOOP_HDFS_HOME:-$YARN_HOME}"
- Alter yarn-site.xml:
<?xml version="1.0"?> <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce.shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> ...