In progress
Production System Foundations
Most incidents trace to missing fundamentals, not missing scale patterns — boring production competence is the highest-leverage engineering.
0% complete0 / 19 chapters
Coming soon
Knowledge maps, not tutorials
Atlases of production engineering — prose for humans, ontology for machines.
The atlas library
Each atlas is a deep, code-first book — but they share one knowledge graph, so a concept you learn in one shows up, connected, in the others.
Most incidents trace to missing fundamentals, not missing scale patterns — boring production competence is the highest-leverage engineering.
Architectural patterns proven at consumer scale — prose for humans, ontology for machines.
Client engineering is distributed systems on a fleet of devices you don't control, shipped through a deploy pipeline you don't own.
Teams adopt events for decoupling and inherit distributed-systems problems they didn't sign up for; this atlas teaches when events earn their cost.
A data platform is the system that turns raw operational exhaust into trusted business memory.
State has to live somewhere with guarantees that fit the workload — most "SQL vs NoSQL" debates are workload questions in disguise, and the fewest stores that satisfy the invariants wins.
Observability is not three kinds of telemetry; it's the ability to ask new questions of a system without shipping new code.
Security is a systems property, not a gate — learnable by the same engineers who learned caching and retries.
Quality is not testing at the end; it's a portfolio of validation investments, each with a cost and an ROI.
Models are the easy part — ML systems fail as systems: data, serving, feedback loops, drift.
Relevance is a product problem wearing infrastructure clothes.
The algorithm is a minority of the outcome — inventory, cold start, feedback loops, and measurement dominate.
A/B testing is easy; trustworthy experimentation at org scale is a systems and culture problem.
GPUs are a scarce, expensive resource you redesign around — the accelerator as a supply chain, not a server; the hardware substrate beneath the AI-era stack.
Production LLM systems are 10% model calls and 90% systems around a component that is confidently wrong.
Agents are distributed systems where one node is stochastic; autonomy is a budget you spend, not a feature you enable.
AI doesn't remove engineering judgment; it relocates it — from writing code to specifying, verifying, and owning systems you didn't type.
Platforms are products with customers who are free to leave; golden paths beat mandates.
Enterprise software is different physics — buyer ≠ user, tenant heterogeneity, and decade-long lifecycles reshape every architectural decision.
Org design is system design — Conway's law runs in both directions, and most "technical" problems are org structures leaking into architecture.
When software has to sense, decide, and act in the physical world, it inherits constraints no cloud scale prepares you for — physical time, lying sensors, and actuation with no undo; edge/cloud partitioning is where you spend or save them.
The knowledge graph
Concepts aren't isolated chapters — they're a graph. Every concept links to the chapters, books, papers and engineering blogs that touch it.
Ask Zenorator
Every answer is grounded in the books, chapters, papers and blogs in the graph — and cited, so you can read the source.
◇ Answer
A retry is a request you make because a previous one failed. On its own that's harmless — the problem is correlation: when a dependency slows down, every caller fails at roughly the same moment, and every caller retries at roughly the same moment.1
That doubles or triples load on the exact system that was already struggling. It gets slower, more requests time out, and the retried load grows again — a feedback loop that takes down a service that was only briefly degraded.2
The fixes break the correlation and cap the amplification: jittered backoff so retries spread out in time, retry budgets so a client spends only a small fraction of its traffic on retries, and circuit breakers so a sick dependency gets room to recover.3
Sources
Related concepts
Search
One bar, everywhere. Search concepts, patterns, books, chapters, companies, technologies, papers and blogs — results show how each one connects to the rest of the atlas.
connects → Retries, Kafka, Tail Latency
Event-Driven Systems · references Reactive Streams
mitigates → cascading failure
foundation for → backpressure
On the roadmap