Keys, adjacency, and an expression language

sort compares a key, and by default the key is the whole line

sort reads the files you name, or standard input if you name none, and writes the ordered result to standard output. It never sorts a file in place. The comparison is made on a sort key, which by default is the entire line read left to right as text — so 002 sorts before 010 because the character 0 is followed by 0 in one and 1 in the other, and the comparison stops there.

Here is the ten-line fixture, a stock number followed by a city. Note that 002 Brandon appears twice.

004 Kelowna
002 Brandon
010 Nanaimo
002 Brandon
007 Vernon
001 Yorkton
009 Brandon
003 Kelowna
006 Iqaluit
008 Nanaimo

Plain sort gives the ten lines in stock-number order, with the duplicate pair adjacent:

001 Yorkton
002 Brandon
002 Brandon
003 Kelowna
004 Kelowna
006 Iqaluit
007 Vernon
008 Nanaimo
009 Brandon
010 Nanaimo

The four invocations

sort -u sorts, then removes lines that are duplicates of the line before them. Nine lines come out — one 002 Brandon is gone. The unit of comparison is the whole line, not a field: two lines with the same city but different stock numbers are not duplicates and both survive.

sort -k 2 changes the key. -k START[,END] numbers fields from one, and with no END the key runs from the start of field 2 to the end of the line. Here field 2 is the city, so the numeric prefix stops mattering:

002 Brandon
002 Brandon
009 Brandon
006 Iqaluit
003 Kelowna
004 Kelowna
008 Nanaimo
010 Nanaimo
007 Vernon
001 Yorkton

When two keys tie, sort falls back to comparing whole lines, which is why the three Brandon rows come out 002, 002, 009 rather than in the order they appeared in the file.

sort -r reverses the collation order — it does not reverse the file. Sorting then reversing gives 010 Nanaimo first and 001 Yorkton last; feeding the file to tac would give something entirely different, because that reverses the original line order rather than the sorted one.

sort -n compares the leading number numerically instead of as text. On this fixture the output is identical to plain sort — and that is the interesting part of the question, not a coincidence to be waved away. The stock numbers are zero-padded to three digits, so every key is the same width, and for equal-width digit strings a left-to-right text comparison and a numeric comparison agree on every pair.

Strip the padding and they part company immediately. Given the three lines 9 Vernon, 10 Brandon and 2 Kelowna, plain sort produces 10 Brandon, 2 Kelowna, 9 Vernon — because 1 sorts before 2 sorts before 9, one character at a time. sort -n produces 2 Kelowna, 9 Vernon, 10 Brandon. Zero padding is what makes the two agree; unpadded numbers is where you need -n.

uniq only ever sees its neighbour

uniq collapses a run of identical adjacent lines into one. It holds a single previous line, compares, and moves on — so a line that recurs later in the file, with anything at all in between, is not a duplicate as far as uniq is concerned.

Take seven lines: blue green red blue green green red. Plain uniq yields blue green red blue green red — only the adjacent green green pair collapsed. uniq -d, which prints only lines that begin a run of two or more, yields just green. Yet blue, green and red each appear three, three and two times in the file.

Sort first and the picture is correct. sort | uniq gives blue green red, and sort | uniq -d gives blue green red — all three genuinely are duplicated. That pipeline is the answer to any find-the-repeated-lines question. The other flags are -c to prefix each line with its count, -u for only the lines that appear exactly once, and -i to compare case-insensitively.

find is an expression language

find directory-list expression walks each named directory recursively and evaluates the expression against every file it meets. The criteria are predicates: -name pattern (quote it, always), -type f for regular files, -newer file, -size, -perm, -user, -inum, -links. The actions are -print (the default on modern systems), -exec CMD {} \; where {} is the current pathname and the semicolon must be escaped, and -ok, which is -exec with a confirmation prompt.

The part people skip is that these compose with boolean operators. Juxtaposition is AND, -o is OR, ! is NOT, and \( \) groups — escaped and surrounded by spaces, because the parentheses are shell metacharacters. Build one criterion at a time:

find . -name '*.mp4'
find . -type f -name '*.mp4'
find ~ \( -name '*.ps' -o -name '*.gif' \) -a -newer report.tex

The third reads: under my home directory, files whose name ends .ps or .gif, and which are newer than report.tex. Drop the grouping and precedence bites — AND binds tighter than OR, so -name '*.ps' -o -name '*.gif' -newer report.tex means any PostScript file at all, or a GIF that is newer, which is not the question that was asked.

Finally, find composes with grep through a pipe, because find writes pathnames to standard output one per line and grep filters lines:

find . -type f -name '*.mp4' | grep -i moonlanding

One command would do it — find . -type f -name '*moonlanding*.mp4' — but the two-tool form is worth writing once, because it makes the boundary explicit: find decides which files exist and match a criterion, grep decides which of those lines of text to keep.