The artificial neuron & the perceptron

The artificial neuron

An artificial neuron has four parts: inputs $x_i$, real-valued weights $w_i$, an activation $\text{net} = \sum_i w_i x_i$ (the weighted sum), and a threshold/activation function $f$ that turns net into an output. The classic hard-limiter fires $+1$ when $\sum w_i x_i \ge t$ and $-1$ otherwise; folding the threshold in as a bias weight on a constant $x_0 = 1$ lets the same rule read $\sum_{i\ge 0} w_i x_i \ge 0$.

McCulloch-Pitts (1943) showed such neurons compute any logic function. An AND gate uses weights $[-2, +1, +1]$ on $[\text{bias}, x, y]$: only $(1,1)$ makes $x+y-2 \ge 0$. An OR gate uses $[-1, +1, +1]$: any input with a $1$ fires. By composition you get every Boolean function — neural computation is Turing-complete in principle. But MP neurons are *hand-wired*, not learned.

The perceptron and its learning rule

The perceptron (Rosenblatt 1958) makes a single-layer threshold neuron *learnable*. After each example it nudges the weights toward the desired output $d$:

$$\Delta w_i = c\,\big(d - \text{sign}(\textstyle\sum w x)\big)\,x_i,$$

where $c$ is the learning rate. If the output already matches $d$, $\Delta w_i = 0$; otherwise weights on the active inputs move to fix the error. The perceptron convergence theorem guarantees: *if a weight vector that classifies all examples exists, the rule will find it.*

Linear separability and XOR — the famous wall

The catch in the theorem is the word *if*. A perceptron computes a single hyperplane decision boundary, so it can only learn linearly separable problems — ones where a straight line (in 2D) splits the classes. AND and OR are linearly separable; XOR is not. The four XOR points $\{(0,0)\!:\!0,(0,1)\!:\!1,(1,0)\!:\!1,(1,1)\!:\!0\}$ cannot be split by any single line — the two positive points sit on a diagonal with the negatives on the other diagonal. No weights satisfy all four inequalities, so the perceptron rule oscillates forever on XOR.

Minsky and Papert’s 1969 analysis of exactly this limitation stalled neural-network research for years — until the hidden-layer fix of L8.6. The exercise trains a perceptron to 100% on AND and then watches it *fail* on XOR.