Quantile Regression

Ordinary regression answers one question: how does a predictor move the average of the outcome? That's useful — and it can hide almost everything that matters. The average effect of a policy might be small while it helps the top a lot and the bottom not at all; the spread of waiting times might widen with demand even as the mean holds steady. Quantile regression goes beyond the average: it models how a predictor affects any chosen point of the distribution — the median, the 10th percentile, the 90th — so you see the effect on the whole response, not just its centre of mass.

It's a distinct and surprisingly practical tool — and it links neatly to two neighbours: its loss function makes it naturally robust, and modelling the tails connects to extreme value theory and prediction intervals. This page is the idea, the clever loss that powers it, how to read its output, and where it earns its place.

Beyond the average

The limitation of mean regression is that the conditional mean is a single summary, and a single summary can't capture how a relationship changes across the distribution. Two scenarios with the same mean effect can be completely different: a predictor might shift everyone up equally, or lift the top while leaving the bottom flat, or increase the spread without moving the centre. Ordinary least-squares reports the same average for all three and is blind to the difference.

Yet the difference is often the whole point — in equity ("does this help the worst-off, or just the already-advantaged?"), in risk ("how bad is the bad case, not the typical case?"), and in service guarantees ("what's the 95th-percentile wait, not the average wait?"). Quantile regression is built to answer exactly these.

Modelling a conditional quantile

The idea is a direct generalisation. Where ordinary regression models the conditional mean of $Y$ given $X$ , quantile regression models a conditional quantile $\tau$ — for example $\tau = 0.5$ (the median), or $0.9$ (the 90th percentile) — as a function of the predictors. Fit it at several quantiles and you get a family of lines describing how the bottom, the middle, and the top of the outcome each respond to the predictors. You've modelled the whole conditional distribution, not just its mean.

The pinball loss

The mechanism is one elegant change of loss function. Ordinary regression minimises squared error (which targets the mean); quantile regression minimises an asymmetric absolute error — the pinball (or check) loss — that targets a chosen quantile:

L_\tau(r) = \begin{cases} \tau\,r & r \ge 0 \\[3pt] (\tau - 1)\,r & r < 0 \end{cases}

The asymmetry is the whole trick. For $\tau = 0.9$ , under-predictions (the true value is above the line) are penalised 9× harder than over-predictions — so the fitted line is pushed up until only ~10% of points lie above it: the 90th percentile. Tune $\tau$ and you target any quantile. And because it's built on absolute (not squared) error, quantile regression is naturally robust to outliers — median regression ( $\tau = 0.5$ ) is exactly least-absolute-deviations, the robust cousin of least-squares.

Reading the fanning lines

The real insight comes from fitting several quantiles at once and looking at the lines together:

What the quantile lines reveal. If they stay parallel, the spread is constant — the predictor shifts the whole distribution. If they fan apart, the spread grows with the predictor (heteroscedasticity) — something a single mean line completely hides.

If the quantile lines are roughly parallel, the predictor shifts the whole distribution equally (constant spread). If they fan apart, the spread grows with the predictor — heteroscedasticity — meaning the predictor affects not just the level but the variability. That fanning is invisible to a single mean line, and it's frequently the most important finding: "as X increases, outcomes don't just rise, they become more unequal."

What it's good for

Quantile regression earns its place wherever the spread or the tail matters as much as the centre:

Prediction intervals — fit the 5th and 95th quantiles and you have a direct, honest interval ("90% of cases fall between these") — a close relative of conformal prediction.
Risk & tails — modelling the 99th percentile of losses or delays directly, where the extreme is the concern, not the average.
Equity analysis — does an effect differ for the bottom vs the top of the distribution? (Does a programme lift the worst-off, or only the already-doing-well?) The mean can't tell you; quantile regression can.

The honest limits

A couple of caveats keep it honest:

Where it shows up in my work

When the average isn't the question

A lot of the most important questions in government analysis aren't about the average — they're about the distribution: the worst-case wait or delay (the 95th percentile, not the mean), whether an intervention helps the worst-off or only the already-advantaged (equity), and how unequal outcomes are and whether that's widening. Quantile regression answers these directly where mean regression is blind, and the fanning lines are a powerful way to show that a predictor increases not just the level but the inequality of an outcome.

It also pairs with neighbours I lean on: it's naturally robust (median regression = least absolute deviations), it gives honest prediction intervals (fit two quantiles), and for the genuine tail it hands off to extreme value theory. Knowing the average is rarely the whole story — quantile regression is how you ask about the rest of it.

Refresh in 60 seconds

Ordinary regression models the mean — which hides how a predictor affects the bottom vs the top. Quantile regression models any conditional quantile (median, 90th…).
It minimises the pinball / check loss — asymmetric absolute error; for τ=0.9, under-predictions cost 9× more, so the line sits at the 90th percentile.
Built on absolute (not squared) error → naturally robust; median regression = least absolute deviations.
Fit several quantiles and read them together: parallel = constant spread; fanning = heteroscedasticity (the predictor changes the spread/inequality — invisible to a mean line).
Great for prediction intervals (fit two quantiles), risk/tails, and equity (effect on the worst-off vs the top).
Caveats: quantile crossing, and extreme quantiles need more data — hand off to EVT for the genuine tail.

The conditional-quantile idea, the pinball-loss asymmetry, the heteroscedasticity reading, and the quantile-crossing / tail-data caveats reflect current quantile-regression references alongside coursework.