<pre><code> MonadLUG man page of the month presentation: uniq 8 February 2007 uniq removes duplicate lines from a sorted file. sample file: in.txt a, one a, one b, two B, Two c, three b, one Simplest example: uniq <<n.txt a, one b, two B, Two c, three b, one But there's much more! Show how many times each line occurs: uniq -c <in.txt 2 a, one 1 b, two 1 B, Two 1 c, three 1 b, one (yes, the indent is in the generated output file) Print only the duplicate lines: uniq -d <in.txt a, one Count lines while ignoring case: uniq -d -c <in.txt 2 a, one Print unique lines -- suppress duplicates:uniq -u <in.txt b, two B, Two c, three b, one And with case-insensitivity: uniq -u -i <in.txt c, three b, one Skip first field, check uniq on the second: uniq -f 1 <in.txt a, one b, two B, Two c, three b, one Now sort case insensitive on second field: sort -k 2 <in.txt | uniq -f 1 Note field/column origins different for sort and uniq. B, Two a, one c, three b, two Of course, we could have also used plain sort: sort -k 2 -u <in.txt B, Two a, one c, three b, two Plus case insensitivity and count: cat in.txt | sort -k 2 -f | uniq -c -f 1 -i 3 a, one 1 c, three 2 B, Two And, for our finale, sort ignoring leading blanks and numerically to get uniq lines listed by number of times they appear. cat in.txt | sort -k 2 -f | uniq -c -f 1 -i | sort -b -n 1 c, three 2 B, Two 3 a, one Refer to your friendly man page or Google for linux uniq tutorial for lots of information. Presented by Ray Côté Appropriate Solutions, Inc. "We Build Software" http://www.AppropriateSolutions.com </pre> -- %MAINWEB%.TedRoche - 08 Mar 2007
This topic: GNHLUG
>
WebHome
>
RecentEvents
>
MPoMuniqHandout
Topic revision: r1 - 2007-03-08 - TedRoche
All content is Copyright © 1999-2025 by, and the property of, the contributing authors.
Questions, comments, or concerns?
Contact GNHLUG
.
All use of this site subject to our
Legal Notice
(includes Terms of Service).