Memra

Why `2>&1` has to come last

◈ 11 cards

Redirect stdout and stderr independently or together, and apply the left-to-right rule that makes `cmd > f 2>&1` and `cmd 2>&1 > f` two different commands.

Two rules, and everything follows from them

Rule one: the redirections on a command line are processed strictly left to right. The shell walks the line, and each operator acts on the descriptor table as it stands at that instant.

Rule two: m>&n makes descriptor m a duplicate of descriptor n — right now. It copies a pointer. It is not a subscription: if n is re-pointed later, m does not follow it.

Put the two together and the fact most often got wrong in this course falls out on its own.

Worked example: the same two operators, two orders

Start from the table every command inherits — 0, 1 and 2 all on the terminal.

cmd > f 2>&1. First > f: slot 1 now points at f. Then 2>&1: copy whatever slot 1 holds into slot 2 — and slot 1 holds f. Both streams land in f. This is the form people mean when they say send everything to the log.

cmd 2>&1 > f. First 2>&1: copy whatever slot 1 holds into slot 2 — and at this instant slot 1 is still the terminal. So slot 2 is now pointed at the terminal, which is where it already pointed; the operator has done nothing visible. Then > f: slot 1 moves to f, and slot 2 is never consulted again. Result: output in f, errors still on your screen.

The second form is not always a mistake. It is exactly the idiom for keeping the diagnostics visible while capturing the output — you want to know immediately that one of three input files was unreadable, rather than discover it in a log next week. It is only a mistake when the author meant the first form, which is most of the time.

The same rule, proved from the other end

Write sort 0< students 1> sorted 2> sort.err and then delete students. Where does the message sort: cannot read: students appear?

On your screen — and the reason has nothing to do with sort. The shell processes 0< students first, fails to open the file, and reports the failure itself. At that moment descriptor 2 is still the terminal, because the 2> operator is further to the right and has not been reached yet. Reorder the line to sort 2> sort.err 0< students 1> sorted and the same message lands in the file.

That is the same left-to-right rule seen from the other side: it governs not only where the command's output goes, but where the shell's own failure to set the command up gets reported.

The forms worth having by heart

  • cmd > f 2>&1 — everything into f. The order is mandatory.
  • cmd > out 2> err — two streams, two files. No duplication is involved, so no ordering question arises.
  • cmd >> out 2>> err — the appending version of the same.
  • cmd > /dev/null 2>&1 — total silence. Reverse the two operators and the errors still print.
  • echo 'usage: report FILE' 1>&2 — inside a script, put your own diagnostic on stderr where a diagnostic belongs, so a caller who redirects your stdout into a file still sees it.

The C shell cannot express any of this

csh and tcsh have no descriptor syntax at all — no 2>, no 1>&2, no m>&n. Their entire vocabulary here is >& (both streams to a file), >>& (append both) and |& (pipe both). To get stdout and stderr into different files you need a subshell:

(cmd > out) >& err

Read it in order and it is the same left-to-right rule again. The outer shell points both its streams at err; the subshell is created after that and inherits both; then the subshell additionally moves its own stdout to out. Net result: results in one file, diagnostics in the other. If a question asks you to separate the two streams under csh and you reach for 2>, you have written a syntax error.

012A: > fttyfttyA: 2>&1ttyffB: 2>&1ttyttyttyB: > fttyfttyRow B1 copied slot 1 while it was still the terminal.
Rows A are `cmd > f 2>&1`; rows B are `cmd 2>&1 > f`. Slot 1 ends up at `f` in both. Only slot 2 disagrees, and it disagrees because of when it was copied.
NORMAL ~/memra/learn/comp-325/stderr-descriptors-and-the-order-rule utf-8 LF