O'Reilly logo

Apache Mesos Essentials by Dharmesh Kakadia

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Advanced configuration for Hadoop on Mesos

Hadoop has many parameters that affect its performance and fault tolerance behavior. You may want to tune the following options suitable for your cluster and workload.

Task resource allocation

This is used to control the amount of resources allocated to Hadoop tasks. By setting the following parameters in conf/core-site.xml, Mesos will be instructed to assign an appropriate CPU, memory (in MB), and diskspace (in MB) resources.

Note

Note that the memory includes the JVM overhead (~10%).

<property> <name>mapred.mesos.slot.cpus</name> <value>1</value> </property> <property> <name>mapred.mesos.slot.mem</name> <value>1024</value> </property> <property> <name>mapred.mesos.slot.disk</name> <value>1024</value> </property> ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required