Putting a command’s input inside the script

A third source of standard input

A command that reads standard input usually gets it from the terminal or from a file. A here document is the third possibility: the input is written inside the script, between the command line and a terminator you choose.

#!/bin/sh
grep -i "$1" << PHONES
Ada Lovelace   403-555-0142
Grace Hopper   780-555-0199
Ken Thompson   604-555-0170
PHONES

The << operator says: everything from the next line up to (but not including) a line consisting of exactly PHONES is the standard input of grep. Run ./phone.sh hopper and grep -i matches one line and prints it.

The marker is arbitrary. PHONES, END, EOF, XYZ — the only requirement is that it never appears on a line of its own inside the data. Uppercase is a convention that makes it visible, not a rule.

Why embed the data at all? The script becomes self-contained: there is no second file to ship with it, to lose, or to have permissions wrong on. And the command reads the lines straight from the shell instead of the shell opening, reading and closing an external file, which on a small dataset is genuinely faster. The obvious trade is that the data is now version-controlled with the code and cannot be edited without editing the script.

The shell substitutes into the body — unless you stop it

By default the body of a here document is treated like the inside of a double-quoted string: parameter substitution and command substitution both happen before the text reaches the command.

cat << REPORT
Logged in as $USER
Generated `date "+%Y-%m-%d"`
REPORT

That prints your login name and today’s date. It is how a script generates a filled-in report, a config file, or a mail message.

Quote the opening marker and every substitution is suppressed — the body arrives at the command exactly as typed:

cat << 'REPORT'
Logged in as $USER
REPORT

Now the output contains the six literal characters $USER. Any quoting of the opening marker does it: 'TAG', "TAG", even \TAG. This is how you write a here document whose body is itself shell code — a generated script, an awk program, a block of documentation about the shell.

The three sharp edges

One — the terminator must be alone on its line, with nothing around it. Not indented, no trailing spaces, no comment. A single invisible trailing blank means the shell never recognises the terminator, reads on to the end of the file looking for it, swallows the rest of your script as data, and reports an unexpected end of file. The error appears at the bottom of the script; the fault is at the marker.

Two — <<- strips leading tabs, and tabs only. The dashed form exists so a here document can be indented to match the if or while it sits inside: leading tab characters are removed from every body line and from the terminator line, so the terminator may be indented too. Leading spaces are not touched. Indent the body with four spaces under <<- and all four spaces are still there when the command reads it; indent the terminator with spaces and the here document never terminates at all. An editor configured to insert spaces instead of tabs breaks <<- silently, which is why many people avoid it entirely and simply keep the body flush left.

Three — redirections and pipes belong on the command line. A here document is the input of a command, so everything else about that command — its error redirection, the pipeline it feeds — must be written before the <<, not after the terminator. When you need a whole pipeline to be fed, parenthesise it so the shell treats it as one command:

(grep -i "$1" 2> errors | sort) << PHONES
Ada Lovelace   403-555-0142
Grace Hopper   780-555-0199
PHONES

The here document now feeds the head of the pipeline, grep; sort receives grep’s output as usual; and the error redirection lands where you meant it. Written without the parentheses, sort and the redirection get separated from the input you were trying to give them.

Worked example — the self-contained lookup

Put the three parts together and you have a phone-book script that depends on nothing but itself: a here document holding the data, a quoted marker if the data contains dollar signs, and a parenthesised pipeline so multiple hits come out sorted. That script is a single file you can mail to somebody, and it works on the far end because the data travelled with the code.