Editing a stream you never open

A non-interactive editor

sed is a stream editor. It reads its input one line at a time, applies the editing commands you gave it to that line, prints the result, and moves on. Nothing is opened, nothing is loaded into memory, and — the fact people are most often surprised by — nothing is written back to your file. sed writes to standard output. If you want the change on disk you redirect the output somewhere, or you ask for in-place editing explicitly.

That per-line cycle is the whole execution model, and it is worth stating precisely because every sed behaviour follows from it:

1. Read the next line into the pattern space.
2. For each command in the script, check whether the command's address selects this line. If it does, run the command.
3. Print the pattern space — unless -n was given.
4. Go back to step 1.

An address is either a line number, a $ meaning the last line, or a regular expression between slashes. No address means every line.

Worked example — four commands, one config file

Here is a five-line file called relay.conf. The third line is indented; two lines are comments; the word port appears five times.

# relay configuration
port = 8080
    timeout = 30
# the port here replaced the old port
backup_port = 8080

Command one — substitute the first match on each line.

sed 's/port/PORT/' relay.conf

The s command takes the form s/regex/replacement/. Every line comes out, because there is no address and no -n. Line 2 becomes PORT = 8080, line 5 becomes backup_PORT = 8080 — note that sed neither knows nor cares that it matched inside a longer word — and line 4 becomes # the PORT here replaced the old port. Only the first occurrence on that line changed. Line 3 has no match and is printed untouched.

Command two — substitute every match on each line.

sed 's/port/PORT/g' relay.conf

The g flag goes after the closing delimiter and means global-within-the-line. Now line 4 comes out as # the PORT here replaced the old PORT. That single character is the only difference between the two runs, and it is the thing most worth internalising about s: without g, one substitution per line.

Command three — delete by address.

sed '/^#/d' relay.conf

Here the address is the regular expression ^# and the command is d, delete. The three non-comment lines are printed and the two comment lines are not. This is the removal half of the Assignment 2 script, and the counting half is grep -c '^#' relay.conf, which reports 2.

Command four — select instead of delete.

sed -n '/^#/p' relay.conf

-n switches off the automatic print at step 3, so nothing comes out unless a command explicitly prints it. p prints. The result is only the two comment lines — the exact complement of command three. Leave -n off and each comment line appears twice: once from p and once from the automatic print.

Two more addresses worth having

Stripping leading whitespace is one substitution: match a run of blanks pinned to the start of the line, and replace it with nothing.

sed 's/^[[:space:]]*//' relay.conf

The indented timeout line loses its four spaces; every other line is unchanged, because * allows zero blanks and replacing zero characters with nothing is a no-op.

Addresses can also be numbers or ranges. sed -n '2,4p' relay.conf prints lines two through four and nothing else — the head/tail combination in one command. And sed '1d' drops the first line, which is how you strip a header off a report before feeding it to something else.