Synthetic Data

Three of the demos on this site run on data that never existed. The rows look like lab results, street addresses or a month of daily check-ins, and none of them came from a person or a partner organisation. Each time that was a choice, because the real data could not leave the place it lived. This note covers how a generator makes rows like these, what it keeps and what it quietly drops, and the checks I run before I trust a synthetic table.

The idea

Synthetic data is drawn from a model of the data rather than copied out of the data. The model can be written by hand, as a set of rules and distributions you choose, or it can be fitted, learned from real rows. Either way the synthetic rows are new draws, so no row has to belong to anyone.

It gets used for two quite different jobs. The first is as a stand-in for data you are not allowed to share: patient records, a partner's lab exports, anything under a privacy or confidentiality promise. The second is as test data where you control the truth. If you plant a fault in row 412 on purpose, you know exactly what a good detector should flag, which real data never tells you. The second job does not need any real data at all.

The catch is the same in both. A generator can only give back what its model holds. A relationship between two columns that the model leaves out is missing from every synthetic row, and drawing a million rows will not bring it back. The opposite failure is just as real: a generator that holds too much is a copy of the real rows with extra steps. So every synthetic table faces two questions that pull against each other. Is it close enough to the real thing to be useful, which is its fidelity? And is it far enough from the real rows to be safe, which is its disclosure risk?

The maths

Real rows come from some unknown distribution $P$ . A generator fits a model $\hat{P}_\theta$ to them and then draws new rows from the model.

x_1, \dots, x_n \sim P \quad\longrightarrow\quad \hat{P}_\theta \quad\longrightarrow\quad \tilde{x}_1, \dots, \tilde{x}_m \sim \hat{P}_\theta

Whatever the fitted model cannot express, no synthetic row will contain.

For fidelity, compare the two samples as distributions rather than comparing a few summary numbers. A plain way is to cut the space into $J$ cells, take the share of real rows $p_j$ and synthetic rows $q_j$ in each, and add up the differences. This is the total variation distance.

\operatorname{TV}(p, q) = \frac{1}{2} \sum_{j=1}^{J} \lvert p_j - q_j \rvert

0 means the shares match in every cell. 1 means the two samples never share a cell.

A score on its own means little, because two honest samples from the same source also differ. The baseline is a second, fresh sample of real data, held out from the generator. If the synthetic rows land near the fresh sample's score, the generator is about as good as it can be on this measure. If they score much better than fresh real data, be suspicious, because the likeliest way to beat a fresh sample is to copy the training rows.

For disclosure, the simplest check is the distance to the closest record. For each synthetic row, find the nearest real row.

d_i = \min_{1 \le j \le n} \lVert \tilde{x}_i - x_j \rVert

Compare the spread of these distances with the same distances for fresh real rows.

Fresh real rows sit at some typical distance from the training rows. Synthetic rows that sit much closer than that are leaking individual records, and a distance of exactly zero means a real row has been published as it was.

When the target is a mix of categories rather than a cloud of points, the fidelity check becomes a goodness-of-fit test. Count the observed rows $O_k$ in each of $K$ categories, compare them with the counts the target mix expects, $E_k$ , and report the effect size next to the p-value so a large sample cannot make a trivial gap look important.

\chi^2 = \sum_{k=1}^{K} \frac{(O_k - E_k)^2}{E_k}, \qquad w = \sqrt{\chi^2 / n}

Pearson's test with K − 1 degrees of freedom, valid when every expected count is at least about 5.

Try it

The real data in the widget is made up too: 100 rows from three clusters. Each generator learns from those rows and draws 100 new ones. The table scores the synthetic rows next to a fresh draw from the same three clusters, the best result any generator could hope for.

Generator

σ, noise added to each copied row0.05

Seed 1

Real rows
Synthetic rows

Faint lines mark the 5 × 5 grid the shape gap is counted on.

The real rows have a correlation of 0.55.

The Normal generator gives a correlation of 0.61 against the real 0.55. Its shape gap of 0.53 is well above the 0.16 a fresh real sample scores, so the shape is lost. Its rows sit further from real rows than fresh real data does (0.174 against 0.033), so little is copied.

Scores for a fresh real sample and for the synthetic rows, each measured against the real rows.
Rows	r	Shape gap	Nearest real	Copies
Fresh real sample	0.57	0.16	0.033	0
Normal generator	0.61	0.53	0.174	0

Every dataset here comes from a seeded generator, so a seed and a setting always give the same rows. Real rows are rings, synthetic rows are dots.

Start on Normal. Its correlation lands close to the real one, yet its shape gap is the worst of the three, because one normal distribution fills the empty space between the clusters. Matching a summary statistic proves very little.
Switch to Shuffle. Each column keeps its exact values, the correlation falls to about zero, and a cluster appears in the top-left corner where the real data has none.
Switch to Copy and drag σ to 0. The shape gap now beats the fresh real sample and all 100 rows are copies. Perfect fidelity is the privacy failure.
Raise σ slowly. Near 0.05 the median distance to the nearest real row matches the fresh sample. Past 0.2 the clusters blur and the shape gap climbs.
On some seeds, including the first, Shuffle rebuilds one real row exactly, purely by chance. A generator does not have to copy on purpose to publish a real record.

Where I used it

SA Mock Address Lab · HPLC QC Lab · Daybook

SA Mock Address Lab started as a command-line tool I wrote in August 2025 to make mock South Australian addresses for software testing. Porting it to the web in 2026 showed that its remoteness and socio-economic weights had never been applied, so every suburb was equally likely. The revival keeps that uniform mode as built, adds the promised weighting from ABS remoteness and SEIFA data, and checks each sample against its target mix with Wilson intervals, a goodness-of-fit test and Cohen's w. Every output is stamped as synthetic test data, because a generated address can still match a real one by chance.

HPLC QC Lab rebuilds our MAST30034 industry capstone to run in the browser on synthetic data only. None of the partner's data is used. Every sheet, chart and cluster comes from a seeded generator that mimics the shape of the lab's exports and plants faults on purpose, so the right answer is known in advance.

On my Moodist write-up, the demo is Daybook, a separate concept demo with made-up questions and four weeks of synthetic history. Moodist belongs to the University of Melbourne, and none of its data appears on this site.

Easy to get wrong

Five traps

Calling synthetic data anonymous by default. A generator fitted closely enough gives real rows back, and unusual rows are the easiest to recover. Check the distance to the closest record before release. A formal guarantee such as differential privacy has to be designed into the generator, it does not come for free.
Matching the summary and stopping. Means, spreads and one correlation can all agree while the joint shape is wrong, which is exactly what the fitted normal does above.
Reading a score without a baseline. Two real samples never match perfectly either. Without a fresh real sample to compare against, you cannot tell ordinary sampling noise from a generator that has missed something.
Drawing conclusions about the world from synthetic rows. A pattern the model never held cannot show up, so an analysis on synthetic data will report it as absent. Synthetic data is for building and testing the pipeline. The findings still need the real data.
Forgetting the seed. A bug found in unseeded synthetic data may never appear again. Both labs above take a seed, so the same settings give the same rows on any machine.

Sources

Synthetic Data: what, why and how?Jordon, Szpruch, Houssiau and others, arXiv:2205.03257, 2022.A broad, readable report on what synthetic data can and cannot promise, written for the Royal Society. Good on the fidelity, utility and privacy trade-off.
synthpop: Bespoke Creation of Synthetic Data in RNowok, Raab and Dibben, Journal of Statistical Software, volume 74, issue 11, 2016.A practical package and paper from the official statistics world. It fits one column at a time, conditioned on the ones before, and comes with utility checks.
Synthetic Data: Anonymisation Groundhog DayStadler, Oprisanu and Troncoso, USENIX Security Symposium, 2022.The paper to read before calling a synthetic release private. It attacks several generators and shows that unusual records stay exposed.