Bayesian belief networks (breadth)

The structure: a DAG plus CPTs

Naive Bayes assumes everything is independent; full Bayes assumes nothing is. A Bayesian belief network (BBN) is the principled middle ground. It is a directed acyclic graph (DAG) whose nodes are random variables and whose directed edges encode causal influence, with a conditional probability table (CPT) attached to each node giving $P(\text{node}\mid \text{its parents})$.

The defining property is the Markov assumption: each node is conditionally independent of all its non-descendants given its parents. This is what makes a BBN compact — it lets the full joint distribution factor into a product of small per-node CPTs: $$P(x_1,\dots ,x_n)=\prod _{i}P\left(x_i\mid \text{parents}(x_i)\right).$$ Instead of one exponential $2^n$-entry table, you store one small table per node — exponential savings whenever each node has few parents.

d-separation: when is influence blocked?

d-separation is the graphical test for conditional independence: are $A$ and $B$ independent given a set of observed nodes? It depends on the three connection patterns along the path between them:

• Serial $A\to V\to B$ — observing the middle node $V$ blocks the influence (knowing $V$ makes $A$ and $B$ independent).
• Diverging $A\leftarrow V\to B$ (common cause) — observing $V$ blocks the correlation between $A$ and $B$.
• Converging $A\to V\leftarrow B$ (common effect) — the opposite rule: $A$ and $B$ are independent when $V$ (and all its descendants) is unobserved; observing $V$ creates a correlation. This is ‘explaining away’: two competing causes of the same observed effect become negatively correlated — confirming one cause makes the other less needed. (Explaining away is exactly the abductive reasoning from L7.4, now formalized.)

Worked example — burglary → alarm → call

A three-node chain: a Burglary ($B$) may trigger the Alarm ($A$), and the alarm may prompt a neighbour’s Call ($C$). We are given the prior $P(B)$, the CPT $P(A\mid B)$, and the CPT $P(C\mid A)$, and we want the posterior $P(B\mid C)$ — how likely a burglary is, given that we got a call. By Bayes, $$P(B\mid C)=\frac{P(B,C)}{P(C)}=\frac{{\sum }_{A}P(B)P(A\mid B)P(C\mid A)}{{\sum }_B{\sum }_{A}P(B)P(A\mid B)P(C\mid A)}.$$ This is inference by enumeration: write the joint via the factorization, then sum out the hidden variable $A$ in both numerator and denominator. With a tiny burglary prior ($0.001$), the posterior stays small even after a call — a correct and sobering result (most alarm calls are not burglaries). The code exercise enumerates exactly this and prints $P(B\mid C)$.