Memra

Asterisk, question mark, and who actually expands them

◈ 9 cards

The shell expands a filename pattern before the command starts, so the command never sees the pattern — which explains every surprising thing globbing does.

The single most useful fact about globbing

The command never sees the pattern. The shell expands *.c into a list of matching filenames, sorts them, and passes that list as ordinary arguments. By the time ls or rm or cc starts running, there is no asterisk left anywhere — it was consumed by a program that had already finished its job.

Almost every confusing thing about globbing follows from that one sentence, so it is worth proving to yourself rather than believing.

Worked example — one directory, three commands

Throughout this lesson the working directory holds exactly six files:

Makefile  alpha.c  beta.c  gamma.c  notes.txt  readme

Start with a command that has nothing to do with files at all:

echo *.c

This prints alpha.c beta.c gamma.c. echo does not know what a file is; it prints its arguments. So those three names must have been given to it, and the only program that could have produced them from *.c is the shell.

Now the same pattern with a command that does read the directory:

ls *.c

ls here receives three argumentsalpha.c, beta.c, gamma.c — exactly the words echo printed. It stats each one and prints it. It is not matching a pattern; it never had one.

Finally, quote the pattern:

ls "*.c"

This fails: ls: cannot access '*.c': No such file or directory. The quotes stopped the expansion, so ls received one argument — the three-character string *.c — and dutifully looked for a file with that name. The error message is ls telling you, accurately, that it was handed a name it could not find. This is the clearest possible demonstration of the division of labour.

What the pattern characters actually mean

  • * matches zero or more characters. Zero matters: lab* matches the name lab itself.
  • ? matches exactly one character. lab?.c does not match lab.c, because there is nothing for the question mark to consume.
  • […] matches exactly one character from a set — the whole subject of the next lesson.

Matching is case sensitive, like everything else in UNIX: *.c does not match alpha.C.

When nothing matches

Run rm * in an empty directory and the shell reports something like rm: cannot remove '*': No such file or directory. Read that carefully — the error names the asterisk. In the Bourne family's default behaviour, a pattern that matches nothing is not removed and is not an error: the shell passes the word through unexpanded, and rm receives a one-character filename * that does not exist.

That is why the message says no such file rather than nothing to do. It is also a small warning: a command whose glob matched nothing still runs, with the pattern text as a literal argument. In a script, that has caused real damage.

Why this ordering is a feature

Because expansion happens once, in one place, every command on the system gets pattern matching for free and they all agree about what a pattern means. cp, a program written in 1972, and a Python script you wrote this morning both accept *.c identically, and neither contains a line of matching code. Compare that with a world where each program implements its own wildcards — which is precisely the world the other major desktop operating system's command interpreter grew up in, and precisely why its behaviour varies per command.

the shell holds the patternthe pattern is goneplain filenamesyou typels *.cthe shellreads the directory, sorts matchesls is startedargv = ls alpha.c beta.c gamma.cls opens each nameno pattern involvedA quoted pattern skips stage two andarrives at ls as a literal name.
The pattern dies at stage two. Nothing to the right of the shell has ever seen an asterisk — which is why the command cannot be blamed for, or fixed to change, how a glob behaves.
NORMAL ~/memra/learn/comp-325/globbing-is-done-by-the-shell utf-8 LF