O'Reilly logo

Linux and Unix Shell Programming by David Tansley

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

11.3. Using uniq

The purpose of uniq is to strip or suppress duplicate lines from a text file. Uniq assumes the file has been sorted to get the correct results, though this is not set in stone: you can use any type of unordered text and you can even compare ordinary lines of text if you wish.

You may think that this is a bit like the unique option in the sort command. Well, it is in a way, but with one important difference. The sort’s uniq option gets rid of all duplicates, whereas the uniq command does not get rid of all duplicates. What is a repeated line then? In uniq ’s case it means any line that is continuously repeated without a break. Here’s an example.

						$ pg myfile.txt 
May Day 
May Day 
May Day 
Going Down 
May Day.
					

Uniq sees the first ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required