O'Reilly logo

Linux Pocket Guide, 2nd Edition by Daniel J. Barrett

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Name

uniq — stdin  stdout  - file  -- opt  --help  --version

Synopsis

uniq [options] [files]

The uniq command operates on consecutive, duplicate lines of text. For example, if you have a file myfile:

$ cat myfile
a
b
b
c
b

then uniq would detect and process (in whatever way you specify) the two consecutive b’s, but not the third b.

$ uniq myfile
a
b
c
b

uniq is often used after sorting a file:

$ sort myfile | uniq
a
b
c

In this case, only a single b remains because all three were made adjacent by sort, then collapsed to one by uniq. Also, you can count duplicate lines instead of eliminating them:

$ sort myfile | uniq -c
      1 a
      3 b
      1 c

Useful options

-c

Count adjacent duplicate lines.

-i

Case-insensitive operation.

-u

Print unique lines only.

-d

Print duplicate lines only.

-s N

Ignore the first N characters on each line when detecting duplicates.

-f N

Ignore the first N whitespace-separated fields on each line when detecting duplicates.

-w N

Consider only the first N characters on each line when detecting duplicates. If used with -s or -f, sort will ignore the specified number of characters or fields first, then consider the next N characters.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required