How Schema Evolution Actually Works (and why it's a coordination problem more than a technical one)
The technical mechanics that make schema evolution possible, and the coordination realities that make it work in practice.
Pipeline Patterns · Teaching
How systems actually work.
Essays on the mechanics underneath analytical data systems: storage, execution, modeling, correctness, and the tradeoffs that shape real work. New writing lands first on Substack.
What Durability Actually Means (and how it translates to analytical systems)
By the end, you should understand what durability guarantees, what it costs to provide, and how the concern translates to analytical systems where the mechanics are mostly invisible.
What (ACID) Consistency Actually Means
By the end of this article, you will understand what consistency actually guarantees, why it's often dismissed as a "redundant" property in ACID, and what it doesn't cover.
What Atomicity Actually Means
By the end, you should understand what atomicity guarantees, what it protects against, how databases actually implement it, and what its costs are.
How Modern Table Formats Actually Work (and what they mean for your data architecture)
By the end, you will understand how table formats actually work, where Iceberg, Delta, and Hudi actually differ, and when the lakehouse architecture earns its complexity versus when it doesn't.
How Columnar Storage Actually Works
By the end of this article you will be able to explain why columnar storage is a standard for analytical workloads.
How BigQuery Actually Executes a Query (and why most optimization advice misses half the picture)
By the end, you should be able to look at a BigQuery execution plan and tell a story about what the engine is doing in physical terms.