Unix: Visual QuickStart Guide

Eliminating duplicates with uniq

If you've sorted files using the handy-dandy sort command, you might end up with results that have duplicates in them. Heck, you might have files with duplicates anyway. At any rate, here's how to find and work with them. As Code Listing 6.18 shows, you can get rid of duplicate lines by using the uniq command (short for "unique") in conjunction with sort.

To eliminate duplicates with uniq:

• sort address.book | uniq

  At the shell prompt, type sort and the file name, then type | uniq to pipe the output to uniq. The output of uniq will not contain any duplicated entries (Code Listing 6.18).

Tip

uniq only finds identical, adjacent (sorted) lines. For example, if you have both Jones and jones in your address book, ...