Apache Flume: Distributed Log Collection for Hadoop - Second Edition

Starting up with "Hello, World!"

No technical book would be complete without a Hello, World! example. Here is the configuration file we'll be using:

agent.sources=s1
agent.channels=c1
agent.sinks=k1

agent.sources.s1.type=netcat
agent.sources.s1.channels=c1
agent.sources.s1.bind=0.0.0.0
agent.sources.s1.port=12345

agent.channels.c1.type=memory

agent.sinks.k1.type=logger
agent.sinks.k1.channel=c1

Here, I've defined one agent (called agent) who has a source named s1, a channel named c1, and a sink named k1.

The s1 source's type is netcat, which simply opens a socket listening for events (one line of text per event). It requires two parameters: a bind IP and a port number. In this example, we are using 0.0.0.0 for a bind address (the Java convention ...