Shellcode, SetUID binaries, and privilege escalation

From crash to code

Overwriting a return address with a random value produces a crash. Overwriting it with an address the attacker chose, pointing at machine code the attacker also supplied, produces arbitrary code execution. That supplied code is called shellcode, after its traditional payload: launch a command shell, historically by having the process call execve on /bin/sh, so the attacker gets an interactive prompt on the target machine.

The crucial property is inherited, not chosen: shellcode runs with the privileges of the attacked process. It is machine code, so it is specific to a processor architecture and usually to an operating system too — which is why real attacks target one particular build of one particular piece of software rather than "Linux" or "servers".

Three constraints, three techniques

What makes shellcode interesting pedagogically is that each of its three constraints forces a technique. This is a causal chain, not a list of three facts to memorise.

Constraint 1: it must be position independent. The shellcode does not know the address it will be loaded at, because the buffer's address depends on how deep the call stack happened to be. It therefore cannot contain absolute references to its own constant data. The technique this forces is the call/jump trick: place a call instruction immediately before the constant data, so that executing the call pushes the address of that data onto the stack — where the shellcode can pop it and use it as a base pointer.

Constraint 2: it must contain no NUL bytes. The payload usually arrives through a string-handling routine, and every one of those stops at the first zero byte. Any instruction encoding that contains a 00 truncates the payload right there. The technique this forces is manufacturing zeros at run time — for example, exclusive-ORing a register with itself to obtain zero, rather than loading the constant 0 and embedding a NUL in the instruction stream. Notice that this constraint applies to the return address too: as the previous lesson's exercise showed, a 64-bit user-space address is largely zero bytes, which is a real obstacle on 64-bit targets.

Constraint 3: the start address cannot be predicted precisely. The attacker must supply a return address, but only knows the buffer's location approximately. The technique this forces is the NOP sled: prefix the payload with a long run of do-nothing instructions and aim the return address at the middle of that run. A guess wrong by less than half the sled's length still lands inside it, and execution simply slides down the sled into the payload.

Worked example — the same bug, twice

A spooling utility contains an unchecked strcpy of argv[1] into a 64-byte local buffer. Run it as an ordinary user and overflow it, and one of two things happens.

Case A: the binary has ordinary permissions. The process runs with the attacker's own real and effective user IDs. The overflow succeeds, the shellcode runs, and a shell appears — owned by the attacker, with exactly the privileges the attacker already had. Nothing was gained. This is a crash class bug, and at worst a denial of service.

Case B: the binary is installed SetUID root (-rwsr-xr-x root root). Now the kernel gives the process an effective UID of 0 at exec time, because that is what the SetUID bit means. The overflow succeeds identically — but the shell it spawns inherits effective UID 0. A file that the attacker's own account could not read a moment ago is now readable. This is privilege escalation, and it is the answer to "how may a buffer overflow be exploited".

That is the whole argument for treating every SetUID-root program as a potential path to unrestricted access. The vulnerability is identical in both cases; only the privilege the process was granted differs.

Beyond interactive utilities

Two realistic target classes matter more in practice than a local SetUID binary.

Network daemons. A server accepts a connection, spawns a child with the connection on its standard input and output, and processes attacker-supplied bytes. Shellcode launched there gets the network connection as its terminal — this was the shape of the 1988 Internet worm's overflow.

Document-format decoders. Here the input is the file being decoded, delivered by e-mail attachment or fetched by a web page. There is no interactive session for a shell to attach to, so packaged shellcode typically opens a connection back to the attacker instead. The same packages can flush firewall rules and break out of a chroot jail, which is why "but we run it in a sandbox" is not by itself an answer.