The artificial neuron & the perceptron

◈ 4 cards

The neuron model (Σw·x, threshold), McCulloch-Pitts logic gates, the perceptron delta-rule update, linear separability, and the XOR problem.

The artificial neuron

An artificial neuron has four parts: inputs $x_{i}$ , real-valued weights $w_{i}$ , an activation $net = \sum_{i} w_{i} x_{i}$ (the weighted sum), and a threshold/activation function $f$ that turns net into an output. The classic hard-limiter fires $+ 1$ when $\sum w_{i} x_{i} \geq t$ and $- 1$ otherwise; folding the threshold in as a bias weight on a constant $x_{0} = 1$ lets the same rule read $\sum_{i \geq 0} w_{i} x_{i} \geq 0$ .

McCulloch-Pitts (1943) showed such neurons compute any logic function. An AND gate uses weights $[- 2, + 1, + 1]$ on $[bias, x, y]$ : only $(1, 1)$ makes $x + y - 2 \geq 0$ . An OR gate uses $[- 1, + 1, + 1]$ : any input with a $1$ fires. By composition you get every Boolean function — neural computation is Turing-complete in principle. But MP neurons are hand-wired, not learned.

The perceptron and its learning rule

The perceptron (Rosenblatt 1958) makes a single-layer threshold neuron learnable. After each example it nudges the weights toward the desired output $d$ :

$Δ w_{i} = c (d - sign (\sum w x)) x_{i},$

where $c$ is the learning rate. If the output already matches $d$ , $Δ w_{i} = 0$ ; otherwise weights on the active inputs move to fix the error. The perceptron convergence theorem guarantees: if a weight vector that classifies all examples exists, the rule will find it.

Linear separability and XOR — the famous wall

The catch in the theorem is the word if. A perceptron computes a single hyperplane decision boundary, so it can only learn linearly separable problems — ones where a straight line (in 2D) splits the classes. AND and OR are linearly separable; XOR is not. The four XOR points ${(0, 0) : 0, (0, 1) : 1, (1, 0) : 1, (1, 1) : 0}$ cannot be split by any single line — the two positive points sit on a diagonal with the negatives on the other diagonal. No weights satisfy all four inequalities, so the perceptron rule oscillates forever on XOR.

Minsky and Papert’s 1969 analysis of exactly this limitation stalled neural-network research for years — until the hidden-layer fix of L8.6. The exercise trains a perceptron to 100% on AND and then watches it fail on XOR.

Read each column as four points in the input square. For AND the single 1 sits in one corner and for OR the single 0 does, so in both cases one straight line fences the 1s off from the 0s. XOR’s 1s sit at (0,1) and (1,0) — opposite corners, with the 0s on the other diagonal — and no line can put both 1s on one side. That geometric fact, not any weakness of the learning rule, is what defeats the perceptron.