Graph Neural Networks

Deep learning conquered images and text by exploiting their structure — a grid of pixels, a sequence of words. But an enormous amount of important data isn't a grid or a sequence; it's a graph: people connected to people, accounts to transactions, molecules, knowledge. Graph neural networks (GNNs) are the deep-learning architecture built for exactly this — they learn directly on a graph's connected structure, by a deceptively simple idea: every node updates its understanding of itself by listening to its neighbours, over and over.

It's the natural meeting point of two pages already in this section — the network analysis that studies graph structure, and the deep learning that learns representations — and a genuinely modern tool. This page is why ordinary nets don't fit graphs, the message-passing mechanism at the core, what a GNN actually learns, and where it pays off.

Learning on connected data

The promise is to apply deep learning's power — learning useful representations automatically — to data whose meaning lives in its connections. The classic network-analysis page computed structural features by hand (degree, centrality, communities); a GNN learns the right representation of each node from the graph, combining a node's own features with the pattern of its connections. It's the hand-crafted-features → learned-features leap, applied to relational data.

Why grids and sequences don't fit

A CNN works because an image has a fixed grid — every pixel has the same number of neighbours in the same positions, so a filter can slide across it. An RNN/transformer works because text is an ordered sequence. A graph has neither: nodes have wildly different numbers of neighbours, and there's no natural ordering of them. You can't slide a fixed filter over something with arbitrary, irregular connectivity.

So GNNs need an operation that's permutation-invariant (the answer can't depend on the arbitrary order you list a node's neighbours) and works for any number of neighbours. That operation is message passing.

Message passing: the core mechanism

The heart of (almost) every GNN is a three-step loop, repeated once per layer:

Message passing, one layer. The centre node gathers a 'message' from each neighbour (its current features), aggregates them with a permutation-invariant operation (sum/mean/max), and updates its own representation by combining that with its previous state. Stack layers and information flows from further and further away.

Message — each neighbour sends its current feature vector (optionally transformed).
Aggregate — the node combines all incoming messages with a permutation-invariant function (sum, mean, or max — so order doesn't matter and any count works).
Update — the node forms its new representation by combining the aggregated message with its own previous state (through a small neural network).

h_v^{(k+1)} = \text{UPDATE}\Big(h_v^{(k)},\; \underset{u \in N(v)}{\text{AGGREGATE}}\big(h_u^{(k)}\big)\Big)

One layer lets each node see its immediate neighbours; stack layers and information propagates further — after $k$ layers, a node's representation reflects its $k$ -hop neighbourhood. That spreading of information across the graph is the whole trick.

What a GNN learns

The output is a learned embedding for each node — a vector that captures both the node's own features and its position/role in the graph structure. Two nodes with similar features and similar neighbourhoods end up with similar embeddings. That's the powerful part: the model discovers, from data, what aspects of the structure matter — rather than you guessing which hand-crafted centrality measure to use. Those embeddings then feed a final layer for whatever task you care about.

GCN, GraphSAGE, GAT

The popular GNN variants are all the same message-passing idea with a different aggregation step:

GCN (graph convolutional network) — averages the neighbours' messages (a normalised mean). The simple, foundational version.
GraphSAGE — uses learnable aggregation and crucially samples a fixed number of neighbours, so it scales to huge graphs and can generalise to nodes unseen during training.
GAT (graph attention network) — applies attention so the node weights some neighbours more than others (not all neighbours are equally relevant) — the same attention idea that powers transformers, on a graph.

The unifying view is worth remembering: every GNN layer is just "let nodes talk to their neighbours," and the variants differ only in how they listen.

What it's used for

GNNs handle the three natural graph tasks:

Node classification — label a node from its features and neighbourhood (is this account fraudulent? what topic is this paper?).
Link prediction — predict missing or future edges (will these two people connect? — the engine behind friend/product recommendations and completing a knowledge graph).
Graph classification — label a whole graph (is this molecule toxic?).

The honest limits

GNNs are powerful but have characteristic failure modes:

Where it shows up in my work

Learning on the graph, not just measuring it

Where intelligence and integrity work models entities and their relationships as a graph, a GNN is the step up from measuring the structure (hand-picked centralities) to learning from it — combining a node's own attributes with its connection pattern to spot, say, a fraudulent account by the company it keeps, or predict a hidden link in an entity network. Link prediction in particular (suggesting connections that probably exist but aren't recorded) is directly useful.

What keeps it honest is knowing it's message passing — nodes listening to neighbours — so its power and its over-smoothing limit both come from that averaging, and it's only worth it when the relationships carry the signal. It sits at the intersection of deep learning, network analysis, and knowledge graphs — and is the modern way to do machine learning directly on connected data.

Refresh in 60 seconds

The message-passing framework, the GCN/GraphSAGE/GAT variants, and the over-smoothing limit reflect current GNN references alongside deep-learning coursework.