ML & short-answer drills

The four ML short-answers, drilled to crispness

These carry full marks for a tight, correct paragraph — no code required. Memorize the structure of each.

Q5 — inductive reasoning

Inductive reasoning generalizes from specific observed instances to a general rule — it reasons *bottom-up*, from examples to a hypothesis. This is the opposite of deductive reasoning, which applies a general rule to a specific case and is truth-preserving (a valid deduction with true premises has a true conclusion). Induction is not truth-preserving: the conclusion can be false even when every observation is true, because no finite sample logically guarantees the general rule (Hume's problem). This is why every learner needs an inductive bias — a prior preference that picks one generalization out of the infinitely many consistent with the data.

*Example:* after seeing many white swans, you induce *"all swans are white"* — a useful generalization that a single black swan would falsify. Machine learning is applied induction: from a labelled training set the learner induces a classifier that it hopes generalizes to unseen instances of the same population.

Q7 — three advantages each of rule-, case-, and model-based reasoning

Rule-based (heuristic if-then) reasoning: 1. Captures expert heuristics directly as readable if-then rules. 2. Easy to modify — add, remove, or edit one rule without touching the rest (knowledge separated from control). 3. Focused, traceable explanation — the *why* is the current rule, the *how* is the chain of rules fired.

Case-based reasoning (CBR): 1. Avoids the knowledge-acquisition bottleneck — collect past cases from records instead of interviewing experts for rules. 2. Learns from experience — each solved problem is stored, so the system improves over time. 3. Avoids repeated search / reasoning — reuse an adapted past solution instead of solving from scratch.

Model-based reasoning (MBR): 1. Robust to novel faults — reasons from a model of how the system *should* work, not from pre-enumerated symptom rules. 2. Causal (deep) explanation — it explains *why* using first principles of structure and function. 3. Transfers across tasks/devices — the same structural model supports diagnosis, prediction, and design.

Q9 — backpropagation in a multilayer network

Backprop trains a multilayer net by gradient descent on the output error, in two passes:

1. Forward pass. Feed the input forward; each neuron computes a weighted sum $net = \sum_k w_k x_k$ and a differentiable activation, the sigmoid $f(net) = \dfrac{1}{1 + e^{-net}}$, whose derivative is the convenient $f'(net) = f(net)\,(1 - f(net))$. This yields the network output $O$.
2. Backward pass. Compute the output error and propagate blame backward via the chain rule. For an output node $i$: $\delta_i = (d_i - O_i)\,O_i(1 - O_i)$ and $\Delta w_{ki} = c\,\delta_i\,x_k$. For a hidden node $i$: $\Delta w_{ki} = c\,O_i(1 - O_i)\big(\sum_j \delta_j w_{ij}\big)x_k$ — its blame is the sum of the downstream errors $\delta_j$ weighted by the connections $w_{ij}$ it feeds. Each weight is nudged down the error gradient by learning rate $c$, and the process repeats over epochs until error is small.

The hidden layer is what lets the network solve problems a single perceptron cannot — famously XOR, which is not linearly separable. The differentiable sigmoid is the key enabler: a hard threshold has no gradient, so there would be nothing to descend. (Weakness: gradient descent can stall in local minima.)

Expert-system limits (likely follow-up)

Expert systems are brittle at the edges of their knowledge (no graceful degradation outside the rules), have no deep/causal model of their domain (heuristic rules, not first principles), are costly to build (the knowledge-acquisition bottleneck), and cannot easily recognize when a problem falls outside their competence.