O'Reilly logo

HBase: The Definitive Guide, 2nd Edition by Lars George

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 10. Performance Tuning

Thus far, you have seen how to set up a cluster and make use of it. Using HBase in production often requires that you turn many knobs to make it hum as expected. This chapter covers various advanced techniques for tuning a cluster and testing it repeatedly to verify its performance.

Heap Tuning

In this section we are going to discuss two topics: sizing of the Java VM heap overall, and the subsequent splitting of said heap for various uses once the servers run.

Java Heap Sizing

Before Java 8, you were forced to set, at least, the maximum size of the JVM using the provided configuration files, or the built in default of up to 1 GB of memory was used—which is not useful in the context of HBase region servers, and considering the memory available on modern servers. More specifically, the JVM used to set (and still does for 32bit VMs) its minimum heap size to 1/64th of the available physical memory, and 1/4th of the latter, but only up to 1 GB, for the maximum heap size. You can override both using the following JVM parameters:

-Xms10g -Xmx10g

Another option provided by HBase is its hbase-env.sh configuration file, and setting the included HBASE_HEAPSIZE variable to what you need. The file currently has the following, commented out line as a starting point:

...
# The maximum amount of heap to use. Default is left to JVM default.
# export HBASE_HEAPSIZE=1G
...

One point of recent discussion is setting Xms to the same value as Xmx, or not, and setting ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required