Time for action – capturing the output of a command to a flat file

Let's show this in action, along the way demonstrating a new kind of source as well.

  1. Create the following file as agent2.conf within the Flume working directory:
    agent2.sources = execsource agent2.sinks = filesink agent2.channels = filechannel agent2.sources.execsource.type = exec agent2.sources.execsource.command = cat /home/hadoop/message agent2.sinks.filesink.type = FILE_ROLL agent2.sinks.filesink.sink.directory = /home/hadoop/flume/files agent2.sinks.filesink.sink.rollInterval = 0 agent2.channels.filechannel.type = file agent2.channels.filechannel.checkpointDir = /home/hadoop/flume/fc/checkpoint agent2.channels.filechannel.dataDirs = /home/hadoop/flume/fc/data agent2.sources.execsource.channels ...

Get Hadoop: Data Processing and Modelling now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial