bash Cookbook

Saving a Copy of Output Even While Using It As Input

Problem

You want to debug a long sequence of piped I/O, such as:

$ cat my* | tr 'a-z' 'A-Z' | uniq | awk -f transform.awk | wc

How can you see what is happening between uniq and awk without disrupting the pipe?

Solution

The solution to these problems is to use what plumbers call a T-joint in the pipes. For bash, that means using the tee command to split the output into two identical streams, one that is written to a file and the other that is written to standard out, so as to continue the sending of data along the pipes.

For this example where we’d like to debug a long string of pipes, we insert the tee command between uniq and awk:

$ ... uniq | tee /tmp/x.x | awk -f transform.awk ...

Discussion

The tee command writes the output to the filename specified as its parameter and also write that same output to standard out. In our example, that sends a copy to /tmp/x.x and also sends the same data to awk, the command to which the output of tee is connected via the | pipe symbol.

Don’t worry about what each different piece of the command line is doing in these examples; we just want to illustrate how tee can be used in any sequence of commands.

Let’s back up just a bit and start with a simpler command line. Suppose you’d just like to save the output from a long-running command for later reference, while at the same time seeing it on the screen. After all, a command like:

find / -name '*.c' -print | less

could find a lot of C source files, so it ...