In this recipe, we will touch upon MapReduce parameters and see how we can optimize them.
For this recipe, you will again need a running cluster with HDFS and YARN. Users must have completed the recipe Configuring YARN for performance recipe.
master1.cyrus.comand switch to the
dfs.blocksize. This can be configured as follows:
<property> <name>mapreduce.task.io.sort.mb</name> <value>200</value> </property>