Unquoted variables, command substitution and the exit status

The shell is still globbing, even inside your variables

Make a directory containing exactly two files, John1 and John2, and run these two lines:

$ name='John*'
$ echo $name
John1 John2
$ echo "$name"
John*

The variable holds the same five characters both times. The difference is what the shell does after it has substituted them.

Expansion is a pipeline of stages, and it runs in a fixed order:

1. Parameter expansion — $name is replaced by its value, John*. This happens inside double quotes too.
2. Word splitting — the result is chopped at every run of whitespace listed in IFS. Double quotes suppress this stage.
3. Pathname expansion — any word still containing *, ? or [...] is matched against the filenames in the directory, and replaced by the matches. Double quotes suppress this stage too.

So echo $name reaches stage 3 with the word John* and comes out holding two filenames, while echo "$name" skips stages 2 and 3 entirely and prints the literal. This is exactly the globbing rule from Module 3, applied one level later: the shell globs what is on the command line after substitution, not what you typed.

The practical consequence is the reason this course quotes every expansion from here on. An unquoted $f breaks the moment a filename contains a space, and it breaks catastrophically the moment a filename contains a *. Neither is exotic; both arrive with the first file a user names badly.

Capturing a command's output

A variable can be given the output of a command rather than a literal:

today=$(date +%Y-%m-%d)
echo "Report for $today"

What the shell does, in order: fork a subshell; run the command in it with standard output connected to a pipe rather than the terminal; read everything the command wrote; strip every trailing newline — all of them, not just the last; and put the resulting text where the substitution was. Standard error is not captured, so an error message still appears on your terminal.

After substitution the result is an ordinary piece of the command line, which means it is word-split and globbed like anything else unless you quote it. files=$(ls) followed by echo $files will glob; echo "$files" will not.

There are two spellings and they mean the same thing:

today=`date +%Y-%m-%d`
today=$(date +%Y-%m-%d)

The backquote form is what the textbook uses throughout and what the exam paper will print, so you must be able to read it. The dollar-parenthesis form is what to write in new scripts: it nests without escaping, so $(basename$(pwd)) is legal, where the backquote equivalent needs a backslash before each inner backquote and becomes unreadable at two levels deep.

One trap the paper likes. A command substitution standing alone on a line is not a way to print output. The shell replaces the substitution with the command's output and then tries to run that output as a command. Put date in backquotes on a line by itself and you get something like Wed: command not found, because the first word of the date string was taken as a program name.

The exit status

Every command that finishes hands the shell a small integer. Zero means success; anything non-zero means some kind of failure, and a program is free to use different non-zero values for different failures. The shell keeps the most recent one in $?:

$ grep -q root /etc/passwd
$ echo $?
0
$ grep -q nosuchuser /etc/passwd
$ echo $?
1

This is the channel that if, while and until read, and it is why test is a command rather than syntax — the next lesson but two builds on it.

Two cautions. First, $? is overwritten by every command, including the echo you use to look at it, so if you need the status twice, save it immediately: status=$?. Second, $? is the Bourne spelling. The C shell keeps its exit status in $status, and mixing the two up is a reliable way to lose marks on a question that asks you to convert a script between the two shells.

Your script hands a status back to its caller with exit. exit 0 for success, a non-zero value for each distinguishable failure. If you leave exit out, the script exits with the status of the last command it ran, which is a coincidence rather than a decision.