Three files are already open

Three files you never opened

A UNIX command does not open your keyboard and does not open your screen. By the time its main() starts, three files are already open on its behalf, and the program refers to each of them by a small non-negative integer called a file descriptor:

• stdin, descriptor 0 — where the command reads input. The terminal keyboard, by default.
• stdout, descriptor 1 — where it writes results. The terminal screen, by default.
• stderr, descriptor 2 — where it writes diagnostics. Also the terminal screen, by default.

Those three numbers are guaranteed, not conventions a particular programmer chose. That guarantee is what makes the rest of this module possible: if sort writes its results to descriptor 1 without knowing or caring what descriptor 1 is attached to, then anyone who can change what slot 1 points at can send sort output anywhere at all — without touching a line of sort.

Worked example: following descriptor 1 out of a running ls

Run ls at a terminal and follow the number 1 all the way out to the hardware. There are three tables in the chain, and each one holds something the other two cannot.

Step 1 — the per-process file descriptor table. Every process has its own, and it is a plain array. Slot 1 holds a pointer. That is all a descriptor is: an index into this array. Because the array is per-process, your ls and my ls can both use the number 1 for completely different destinations, at the same time, without any coordination.

Step 2 — the system-wide file table. Slot 1 points at an entry here, and that entry holds two things a descriptor number cannot: the access mode the file was opened with (read, write, append) and the current offset — how far into the file the next read or write will land. One entry can be shared by several descriptors, in one process or across several, and sharing an entry means sharing the offset.

Step 3 — the inode table. The file-table entry points at an in-memory inode, which is the file itself: its type, its permission bits, its link count, its size, and the addresses of its data blocks. For ls writing to a terminal, that inode describes a character-special device, and its data blocks are a device driver.

So ls writes to 1; the kernel walks slot 1, to a file-table entry, to an inode, to the terminal driver, to the glass. Nothing inside ls knows any of that happened.

Trace it yourself: descriptor 0 for cat < roster.txt

The same walk, one slot over. The shell opens roster.txt for reading and points slot 0 at the resulting file-table entry, whose offset starts at 0 and whose inode is the ordinary file on disk. cat reads descriptor 0 until end-of-file, and each read moves the offset in the middle table forward. Slots 1 and 2 were never touched, so the text lands on your screen and any complaint lands there too.

Why the offset cannot live in the inode

This is the question the three-table chain exists to answer, and it is worth being able to argue in one breath. A file has exactly one inode. Two processes reading that file at the same time must be at different places in it, or your head would drag my tail along behind it. So the position has to be per-open, not per-file — and the system-wide file table is the only table in the chain that is per-open. Put the offset in the inode and every reader of a file would be forced to share one cursor.

The converse matters just as much. When a shell duplicates a descriptor — which is exactly what 2>&1 does, one lesson from here — both descriptors end up pointing at the same file-table entry, so they share one offset and their writes interleave cleanly instead of landing on top of each other.

Reading your own table from the shell

On Linux, /proc/PID/fd is a directory of symbolic links, one per open descriptor of that process, and /dev/fd is the same view of whatever process is asking. Listing either is as close as the shell gets to printing your own descriptor table. Descriptors above 2 are yours to use: exec 3< roster.txt opens a file on descriptor 3 of the shell itself and leaves it open until you close it again with exec 3<&-.