/knowledge/data-architecture
Data Architecture & the Modern Data Stack
Before you can analyse data, it has to live somewhere and flow from where it's created to where it's used. How an organisation arranges that — warehouse, lake, or lakehouse; ETL or ELT — quietly shapes everything an analyst can do.
- Studied
- Data Architecture & the Modern Data StackIn practice · where data flows
- When
- Data engineering · ongoing
- Applied in
- Designing sound data flows
- Read / Refreshed
- ~14 min read2026-06-26
Every analysis rests on a question most people never see: where does the data live, and how did it get there? Between the systems that create data (an app, a sensor, a form) and the analyst who uses it sits an entire architecture — storage, pipelines, and transformation — and the choices made there quietly determine what's possible, how fresh the data is, and how much an analyst can trust it. Data architecture is the design of that flow, and the modern data stack is the now-standard cloud-based way to build it.
It's worth knowing even if you never build one, because understanding how data is stored and served is what lets you work with it sensibly — and spot when the architecture, not your analysis, is the problem. This page is the practical landscape: the storage options, the great ETL→ELT shift, and the components of the modern stack. It builds on the database systems page.
01
Where data lives and how it flows
Think of it as a journey with stages: data is created in source systems, then ingested into central storage, transformed into clean, usable shapes, and finally served to dashboards, models, and analysts. Every data platform is some arrangement of those four stages, and the interesting decisions are where the storage and the transformation happen — which is exactly what's changed dramatically over the last decade.
02
Two jobs: OLTP vs OLAP
The foundational split, from the database page: databases do two very different jobs, and you don't want to mix them.
- OLTP (transactional) — the operational databases that run the business: fast, tiny reads and writes ("record this sale", "update this account"). Optimised for many small operations.
- OLAP (analytical) — systems built to analyse: scan and aggregate huge volumes ("total sales by region this year"). Optimised for big questions, not small updates.
Running heavy analytics on the live transactional database would cripple the application, so the whole point of an analytical architecture is to move data out of the operational systems into a place built for analysis. That place is the warehouse — or the lake, or the lakehouse.
03
Warehouse, lake & lakehouse
The three big storage paradigms, and the arc from one to the next:
- Data warehouse — stores structured, cleaned data in a defined schema (schema-on-write: shape it before you store it). Excellent for fast, reliable BI and SQL — but rigid, and it doesn't suit raw or unstructured data.
- Data lake — stores everything, raw and in any format (schema-on-read: store first, impose structure when you query). Cheap and flexible — but easily becomes a "data swamp": ungoverned, undocumented, hard to trust.
- Data lakehouse — the modern convergence: lake-style cheap, flexible storage of raw data plus warehouse-style structure, transactions, and governance layered on top. It aims to give you one platform for both raw and analysis-ready data, and it's the dominant 2020s design.
04
ETL vs ELT: the shift that changed everything
One of the biggest practical changes is the order of two letters. The classic approach was ETL — Extract, Transform, Load: pull data out, clean and reshape it before loading it into the warehouse. You had to, because warehouse storage and compute were scarce and expensive.
Cheap, scalable cloud storage flipped this to ELT — Extract, Load, Transform: load the raw data into the warehouse first, then transform it inside the warehouse using its (now abundant) compute. The advantages are real: you keep the raw data (so you can re-transform it later when needs change), transformations are version-controlled SQL rather than opaque pipelines, and it's faster and more flexible. This single re-ordering is what made the modern stack possible.
05
The modern data stack
The "modern data stack" is the now-standard, modular, cloud-native set of tools assembled around ELT. Each layer is a specialised, swappable component:
The transformation layer deserves a name: dbt (data build tool) became the industry standard by letting analysts write transformations as version-controlled, tested, modular SQL— bringing the software-engineering discipline of the reproducibility page to data modelling. An orchestrator schedules and chains the whole flow, and BI tools sit on top for the dashboards.
06
Bronze, silver, gold: the medallion
A popular way to organise the transformation inside the lakehouse is the medallion architecture — data flows through three quality tiers:
- Bronze — raw, as-ingested data, untouched (your faithful record of what arrived).
- Silver — cleaned, validated, conformed (the data preparation tier — deduplicated, typed, joined).
- Gold — business-level aggregates and features, analysis-ready (what dashboards and models actually consume).
It's a clean, progressive refinement — each tier improves quality and structure — and keeping the raw bronze layer is itself a reproducibility win: you can always rebuild silver and gold from the original truth.
07
Centralised vs data mesh
A final organisational question: should one central data team own everything, or not? The traditional model centralises — one team, one warehouse, one source of truth — which is simple to govern but can become a bottleneck as an organisation grows. Data mesh is the counter-idea: decentralise ownership so each domain team owns its own data as a product, with governance handled in a federated way. It's more an organisational philosophy than a technology, and it suits large, complex organisations more than small ones — a trade-off between central control and domain autonomy, not a universal upgrade.
08
Where it shows up in my work
09
Refresh in 60 seconds
The warehouse/lake/lakehouse arc, the ETL→ELT shift, the modern-data-stack (dbt) components, and the medallion and data-mesh patterns reflect current data-architecture references alongside hands-on work.