Getting Started with Storm

Chapter 7. Using Non-JVM Languages with Storm

Sometimes you want to use languages that aren’t based on the JVM to implement a Storm project, either because you feel more comfortable with another language or you want to use a library written in that language.

Storm is implemented in Java, and all the spouts and bolts that you’ve seen in this book were written in Java as well. So is it possible to use languages like Python, Ruby, or even JavaScript to write spouts and bolts? The answer is yes! It is possible using something called the multilang protocol.

The multilang protocol is a special protocol implemented in Storm that uses standard input and standard output as a channel of communication with a process that does the job of a spout or a bolt. Messages are passed through this channel encoded as JSON or as lines of plain text.

Let’s take a look at a simple example of a spout and a bolt in a non-JVM language. You’ll have a spout that generates numbers from 1 to 10,000 and a bolt that filters for prime numbers, both written in PHP.

Note

In this example, we check for prime numbers in a naive way. There are much better implementations, but they are also more complex and out of the scope of this example.

There is an official implementation of a PHP DSL for Storm. In this chapter, we’ll show our implementation as an example. First of all, define the topology.

...
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("numbers-generator", new NumberGeneratorSpout(1, 10000));
builder ...