Bayesian belief networks (breadth)

The structure: a DAG plus CPTs

Naive Bayes assumes *everything* is independent; full Bayes assumes *nothing* is. A Bayesian belief network (BBN) is the principled middle ground. It is a directed acyclic graph (DAG) whose nodes are random variables and whose directed edges encode *causal influence*, with a conditional probability table (CPT) attached to each node giving $P(\text{node} \mid \text{its parents})$.

The defining property is the Markov assumption: *each node is conditionally independent of all its non-descendants given its parents.* This is what makes a BBN compact — it lets the full joint distribution factor into a product of small per-node CPTs: $$P(x_1, \dots, x_n) = \prod_{i} P\!\left(x_i \mid \text{parents}(x_i)\right).$$ Instead of one exponential $2^n$-entry table, you store one small table per node — exponential savings whenever each node has few parents.

d-separation: when is influence blocked?

d-separation is the graphical test for conditional independence: are $A$ and $B$ independent *given* a set of observed nodes? It depends on the three connection patterns along the path between them:

- Serial $A \to V \to B$ — observing the middle node $V$ blocks the influence (knowing $V$ makes $A$ and $B$ independent). - Diverging $A \leftarrow V \to B$ (common cause) — observing $V$ blocks the correlation between $A$ and $B$. - Converging $A \to V \leftarrow B$ (common effect) — the *opposite* rule: $A$ and $B$ are independent when $V$ (and all its descendants) is unobserved; observing $V$ creates a correlation. This is ‘explaining away’: two competing causes of the same observed effect become *negatively* correlated — confirming one cause makes the other less needed. (Explaining away is exactly the abductive reasoning from L7.4, now formalized.)

Worked example — burglary → alarm → call

A three-node chain: a Burglary ($B$) may trigger the Alarm ($A$), and the alarm may prompt a neighbour’s Call ($C$). We are given the prior $P(B)$, the CPT $P(A \mid B)$, and the CPT $P(C \mid A)$, and we want the posterior $P(B \mid C)$ — how likely a burglary is, given that we got a call. By Bayes, $$P(B \mid C) = \frac{P(B, C)}{P(C)} = \frac{\sum_A P(B)\,P(A\mid B)\,P(C\mid A)}{\sum_{B}\sum_A P(B)\,P(A\mid B)\,P(C\mid A)}.$$ This is inference by enumeration: write the joint via the factorization, then sum out the hidden variable $A$ in both numerator and denominator. With a tiny burglary prior ($0.001$), the posterior stays small even after a call — a correct and sobering result (most alarm calls are not burglaries). The code exercise enumerates exactly this and prints $P(B \mid C)$.