Probabilistic Graphical Models

Reasoning under uncertainty about one variable is the Bayesian story. But real problems involve many interrelated uncertain variables — symptoms and diseases, causes and effects, signals and states — and the full joint distribution over all of them is astronomically large. Probabilistic graphical models (PGMs) make this tractable with a beautiful insight: draw the dependencies between variables as a graph, and that graph both compresses the distribution into manageable pieces and tells you how to reason with it. It's where probability theory meets graph theory, and it underlies a great deal of modern probabilistic AI.

It ties together threads you've already met — it's Bayesian, its graphs are the same DAGs from causal inference, and learning its structure is causal discovery. This page is the core idea (independence as a graph), Bayesian networks, how the graph factorises the joint, and how you reason with evidence.

Taming the impossible joint

Everything you'd want to know about a set of variables is, in principle, in their joint distribution — the probability of every combination of values. The problem is size: for just 30 yes/no variables, the joint has over a billion entries. You can't store it, let alone estimate it from data or compute with it. Some structure has to be exploited, and the structure that saves us is independence — most variables don't directly depend on most others.

Conditional independence as a graph

The central idea: a graph encodes which variables are (conditionally) independent of which. Each variable is a node; an edge between two nodes means a direct dependency; a missing edge asserts a conditional independence ("once I know its parents, this variable tells me nothing more about that one"). The graph is a compact, human-readable map of the dependency structure — and crucially, those independence assumptions are exactly what shrink the impossible joint into something small and computable.

Bayesian networks

The most common PGM is the Bayesian network (or belief network): a directed acyclic graph (DAG, the same structure as a causal diagram) where each node depends only on its parents. Each node carries a small table — its probability given its parents — and that local information, plus the graph, fully specifies the entire joint distribution.

A classic Bayesian network. Rain and Sprinkler each independently can make the Grass wet; Rain also influences whether the Sprinkler runs. The graph says Grass depends only on Rain and Sprinkler — so the huge joint factorises into three small local pieces.

The classic toy example: Rain and a Sprinkler can each make the Grass wet, and rain also affects whether the sprinkler runs. Three nodes, a few small tables — and you've captured a full joint distribution you can reason about.

The factorisation: why it's tractable

The mathematical payoff is the factorisation. A Bayesian network says the full joint distribution is just the product of each variable's probability given its parents:

P(X_1, \dots, X_n) = \prod_{i=1}^{n} P\big(X_i \mid \text{parents}(X_i)\big)

This is the whole magic in one line. Instead of one gigantic table over all variables, you store many small tables (one per node, sized only by its few parents). The billion-entry joint for 30 variables collapses to a handful of small tables — storable, learnable, and computable. The graph's missing edges are precisely the independence assumptions that license this factorisation: structure buys tractability.

Reasoning with evidence: inference

The point of building the model is inference — answering questions of the form $P(\text{query} \mid \text{evidence})$ : "given the grass is wet and it's cloudy, what's the probability it rained?" You clamp the observed variables to their values and compute the updated distribution over the ones you care about — Bayesian belief-updating, propagated through the network.

Learning the model

A PGM has two things to learn: the parameters (the probability tables, given the graph — usually straightforward from data) and the structure (the graph itself — which edges exist). Learning the structure from observational data is exactly causal discovery, with all its difficulty (you often recover only an equivalence class). In practice the structure frequently comes from domain knowledge — an expert draws the dependency graph — and only the parameters are learned, which is a real strength: PGMs let you encode what you know and learn the rest.

Where it shows up in my work

Transparent reasoning under uncertainty

PGMs matter most where there are many interrelated uncertain variables and you need reasoning that's both principled and explainable. Unlike a black-box model, a Bayesian networkshows its reasoning — the graph is an auditable map of what depends on what, and an inference like "given this evidence, here's the updated probability" can be traced through it. In an accountable setting that transparency is genuinely valuable: you can defend not just the answer but the structure of the reasoning.

It also unifies several pages: it's Bayesian updating with many variables, its DAGs are the causal-inference diagrams, learning its structure is causal discovery, and its inference echoes GNN message passing. The discipline is respecting the inference cost — exact reasoning is hard on tangled graphs, so you lean on approximation — and that the model is only as good as the dependency structure you encode.

Refresh in 60 seconds

The full joint distribution over many variables is astronomically large — PGMs tame it by exploiting independence.
A graph encodes conditional independence: nodes = variables, a missing edge = an independence. A compact, readable dependency map.
A Bayesian network is a DAG where each node depends on its parents; with small per-node tables it specifies the whole joint.
The factorisation $P(X_1..X_n)=\prod_i P(X_i\mid \text{parents})$ turns one giant table into many small ones — storable, learnable, computable.
Inference = $P(\text{query}\mid\text{evidence})$ . Easy on trees (belief propagation); NP-hard in general → MCMC / approximate.
Learn parameters (easy) + structure (= causal discovery, or from domain knowledge). Strength: transparent, explainable reasoning.

The independence-as-graph idea, the Bayesian-network factorisation, and the exact-vs-approximate inference framing reflect current probabilistic-graphical-model references alongside coursework.