But by early 2026, the physical cost of running these models caught up with the business reality.

Look at the events of the last quarter. OpenAI shut down Sora and backed out of their Disney partnership. Microsoft is tightening its cloud budget. And a leaked internal dashboard shared by The Signal from Anthropic showed just how unsustainable the generative AI business model is right now.

We aren’t entering an era of unlimited personal AI. We’re actually going back to the 1970s model, the era of the IBM Mainframe. The hallucination seems officially over 🙂

AI isn’t a SaaS

Software-as-a-Service (SaaS) is a great business model because the marginal cost of serving one more user is effectively zero.

AI inference isn’t SaaS. It scales linearly. Every prompt requires physical hardware to do work.

To understand why the flat-rate AI subscription is failing, look at the leaked Anthropic dashboard. An enterprise power user on a $200/month “Pro” tier ran an autonomous coding loop. In 23 days, that single user consumed 1.1 billion tokens and triggered 9,221 sub-agent tasks.

The actual compute cost of running those inferences on Anthropic’s GPU clusters was $27,000. Anthropic took a 135x loss on a single customer in less than a month.

Analyzing a 100-page PDF or running an autonomous agent isn’t a simple database query. It requires firing up clusters of GPUs and executing billions of operations. You can’t sell that kind of compute for a flat fee and hope to make it up in volume. Volume is exactly what causes the losses.

The Death of Sora

If you want proof that the subsidy is over, look at Sora.

OpenAI killed their flagship video generation model less than six months after its public launch. The tech press talked about “safety concerns” and “copyright,” but the reality was the Cost of Goods Sold (COGS).

Generating 60 frames per second of photorealistic video requires massive compute. Keeping the Sora clusters running for 500,000 active users burned an estimated $1 million a day in electricity and GPU depreciation.

They tried to pivot to the enterprise by signing a $1 billion partnership with Disney. But Disney realized that offloading their rendering pipeline to OpenAI’s servers was actually more expensive than doing it in-house. The unit economics didn’t make sense, so the servers were shut down.

OpenAI’s “Risk Factor”

The cloud providers have woken up. For years, Microsoft subsidized OpenAI’s compute to gain market share. That era of cheap infrastructure is over.

In a recent financial disclosure, OpenAI explicitly listed their reliance on Microsoft Azure’s compute pricing as a “Risk Factor.”

Wall Street is demanding a return on the billions poured into data centers. Investors are forcing AI labs to drop unprofitable consumer tools and focus entirely on enterprise contracts that actually pay the bills.

Edge vs. Mainframe

To fix the broken math, the AI market is splitting into two distinct tiers.

The middle ground which is $200/month frontier web app is disappearing.

Tier 1: The Consumer Edge

Consumers will get smaller, 8-billion parameter models running locally on their phones and laptops (Apple Silicon, Snapdragon NPUs etc).

These models are good enough for basic tasks like grammar correction and summarizing emails, but they aren’t capable of deep reasoning.

Why the shift?

Because by pushing the model to the edge, companies offload the cost of the compute and electricity directly onto the user’s battery. It is the only way the consumer unit economics work.

Tier 2: The AI Mainframe (Enterprise)

True frontier AI, massive models capable of deep, autonomous workflows will become bespoke enterprise tools.

These won’t be accessible via a casual web interface.

They will be sold via multi-million-dollar B2B contracts to pharmaceutical companies running protein simulations, and quant hedge funds executing trading logic.

They are the only businesses with gross margins high enough to afford the true, unsubsidized cost of compute.

Conclusion

The idea that a solo developer in a garage will have the exact same compute power as the CTO of JPMorgan isn’t realistic.

The physics of data centers dictate otherwise.

As a Software Architect, you need to plan defensively. Relying on cloud providers to subsidize your application’s heavy reasoning is a risk.

Build your systems around cheap, local, open-source models for the basic plumbing such as routing, classification, and simple tasks.

Treat frontier API calls as an expensive, highly constrained physical resource.

Use them only when absolutely necessary and plan for AI like heavy industrial equipment.

The era of cheap, subsidized compute is over.

Take a breath.

I know what your LinkedIn and Twitter feeds look like right now.

Every time you open your phone, an influencer in a rented sports car or a sleek home office is screaming at you.

They claim that AI is replacing software engineers tomorrow, and if you don’t buy their “AI Mastery Guide” today, your career is over before it begins.

I see the panic in computer science cohorts and junior developers. It is entirely valid.

But you need to understand that this anxiety is not a natural reaction to the tech market. It is a manufactured emotion, meticulously engineered by marketers to make you feel inadequate so you will open your wallet.

In any gold rush, the people who actually get rich are the ones selling the shovels. Today’s “AI Influencers” are selling cheap, plastic shovels to desperate, anxious students.

Let’s tear down their selling tactics like an architecture.

The Scam

If you want to be an engineer, you need to learn how to reverse-engineer systems.

Let’s look at the system design of the influencer business model.

When you see an ad for a “Master ChatGPT in 30 Days” course priced at ₹500 or $5, you think you are buying an educational product.

You are not. You are participating in a Customer Acquisition Cost (CAC) mechanism.

That ₹500 course is just the entry point to a sales funnel. The content inside is nothing but regurgitated Twitter threads, repackaged into a PDF to create artificial complexity.

It is designed to give you a dopamine hit of “productivity” while making you feel like you are still missing the real secret.

Then, the trap springs. Once you finish the cheap course, the email sequence begins.

They tell you that to actually secure a six-figure job, you need to join their exclusive “AI Mastermind” or buy the advanced bootcamp for ₹50,000.

You are being farmed for your attention and your anxiety.

The Delusion

The core lie holding this entire grift together is the concept of the “Prompt Engineer.”

Let me give you the hard truth

Prompt Engineering is not a sustainable technical career.

A prompt is simply a natural language API call. You are no more an “engineer” for typing instructions into ChatGPT than you are a “Search Engineer” for typing queries into Google.

More importantly, it is a rapidly depreciating asset.

Two years ago, getting an LLM to output a specific JSON format required paragraphs of complex “jailbreak” constraints and magical phrasing.

Today, frontier models like Claude 3.5 Sonnet, GPT-4o, and Gemini 1.5 Pro natively understand intent, reason through logic, and output strict JSON out of the box.

The models are getting smarter at interpreting human ambiguity. As the compilers (the LLMs) get better at understanding natural language, the need for complex, esoteric “prompts” disappears.

Buying a course on prompting is investing your limited time and money into a dying skill.

The Trap

Part of the illusion relies heavily on visual spectacle.

The influencer will open a video by shouting,

“I just generated a 50-page PowerPoint, a PDF marketing brochure, and 12 photorealistic images in under three minutes using these SECRET tools!”

They will show you a curated list of glossy AI web apps or tools that automatically build slide decks, generate stock photos, or format PDFs.

They will try to convince you that knowing which button to click on these websites is a highly monetizable, technical skill.

Let me be brutally clear.

Using a SaaS wrapper to generate a PDF makes you an end-user, not an engineer.

You are not building software. You are consuming it.

Knowing how to type a prompt into an image generator or a presentation-maker is equivalent to claiming you are a Database Architect because you know how to sort a column in Microsoft Excel.

It is a neat productivity trick for a marketing manager, but it has absolutely zero engineering value.

When you buy a course to learn these tools, you are paying for a glorified software tutorial, not an engineering curriculum.

The Real AI Engineering

Contrast the influencer fantasy with the brutal reality of what the industry actually pays for.

Companies do not pay AI Engineers $150,000 a year to type clever sentences into a web interface or generate a pretty slide deck.

They pay engineers to build resilient, scalable systems around non-deterministic intelligence.

Real AI engineering is systems engineering. It is building deterministic RAG (Retrieval-Augmented Generation) pipelines. It is managing the severe latency of vector databases. It is calculating the unit economics of token costs versus API throughput.

An actual AI Engineer spends their day setting up strict guardrails to catch LLM hallucinations before they reach a customer.

They write middleware to mask Personally Identifiable Information (PII) before it hits a third-party API.

They write exponential backoff algorithms to handle 502 Bad Gateway errors and 429 Too Many Requests rate limits from OpenAI.

They build intelligent chunking strategies to fit a massive PDF into a 100k context window without destroying the semantic meaning.

If a course does not teach you how to handle network failures, rate limits, or context window chunking, it is not an AI engineering course.

It is a typing class.

Where you should learn?

So, where do you actually go to learn?

You ignore the influencers and you go to the raw, unpolished sources.

Here is your actual roadmap, and it is almost entirely free.

The Math & Mechanics

Go to YouTube and search for Andrej Karpathy’s “Neural Networks: Zero to Hero” series. Karpathy was a founding member of OpenAI and the Director of AI at Tesla.

He will teach you how to build a neural network from scratch using Python.

Understanding the underlying calculus and matrix multiplication is infinitely more valuable than memorizing a prompt template.

The Applied Systems

Stop buying courses and start reading official documentation.

Take Grow with Google AI Lessons.

Read the Anthropic documentation.

Read the OpenAI Cookbooks.

If you need structured courses, take Andrew Ng’s DeepLearning AI classes.

They are built by actual AI researchers, not Twitter marketers.

The Framework

Want to build AI apps?

Go read the raw GitHub documentation for LangChain, LlamaIndex, or Hugging Face.

Build a simple Python script that takes a local text file, converts it into embeddings, stores it in a local ChromaDB, and queries it.

You will learn more in one weekend of fighting Python dependency errors than in a year of watching influencer videos.

Conclusion

True technical leverage does not live behind a paywall on an Instagram ad.

It lives in your ability to sit in a quiet room, read difficult technical documentation, and build things that break until you figure out how to fix them.

A career in software engineering is a 40-year marathon.

Technologies will rise and fall. Frameworks will die.

Do not let marketers rush you into buying plastic shovels.

Ignore the noise. Protect your wallet. Focus on the fundamentals.

Last month, I was brought in to consult for a generative AI startup that had just closed a massive Seed round. They had built an “AI Customer Success Agent” that looked incredible in staging.

Then I opened their Anthropic billing dashboard.

Their Cost of Goods Sold (COGS) wasn’t just high. It was terminal. Every time a user asked their bot a question, the company lost money.

When I tore down their architecture to find the leak, I realized they hadn’t actually built a software company. They had built an unoptimized API proxy.

And this is not an isolated incident.

Look at the startup graveyard from the last two years. The AI Copywriter. The AI PDF Chatbot. The AI Email Assistant. They launched to deafening hype, dominated Product Hunt, and convinced investors they had cracked a new market.

Twelve months later, they were quietly shuttered or sold for parts.

Their entire product architecture consisted of prepending a system prompt to a user input and piping it directly to an LLM provider. They had no defensible moat, and their gross margins were systematically eaten alive by the underlying compute provider.

The Maths of Margin Erosion

Software-as-a-Service (SaaS) valuations are predicated on 80 percent gross margins and a near-zero marginal cost of replication.

If your COGS scales perfectly linearly with every single user click, you do not have a software business. You have a subsidized consulting firm.

The RAG Trap

Let’s examine the exact architecture that was killing my client. A standard enterprise RAG (Retrieval-Augmented Generation) implementation. You deploy a customer support bot handling a modest 10,000 conversations a day.

You write a dense, 2,000-token system prompt detailing the company’s tone and rules. Every time a user asks a question, your vector database retrieves 5,000 tokens of documentation context. Before the LLM generates a single word, your baseline payload is 7,000 input tokens.

At premium model pricing (roughly $10 to $15 per million input tokens for models like Claude 3 Opus or GPT-4), that single interaction costs about $0.10 just to read the prompt. Multiply that by 10,000 conversations, factoring in average multi-turn context window inflation, and you are burning $1,500+ a day.

You are bleeding $500,000 a year purely on input tokens before factoring in output costs, vector database hosting, or salaries.

The Output Multiplier

The input tax is just the entry fee. The output tax is where margins actually die.

Generating tokens requires significantly more compute than reading them. Providers charge 3x to 5x more for output tokens. If your bot generates a helpful 500-word response, that single output costs another $0.015.

Users do not ask one question; they iterate. A four-message conversation easily compounds to 30,000 total tokens processed. You are paying a premium cloud tax on every single syllable your system outputs, 24 hours a day.

The Latency Tax

Financial bleed is only the first failure mode. The second is Time to First Token (TTFT).

TTFT

TTFT is the ultimate metric in AI architecture. When you wire your frontend directly to an external LLM provider, you surrender total control of your application’s physics.

You inherit their network round-trips. You inherit their TLS handshakes. You inherit their peak-hour queue delays. You are at the mercy of raw 2-to-5-second inference times.

Human beings abandon interfaces that take more than 400 milliseconds to react. If your application takes five seconds to stream the first word because us-east-1 is saturated, your users will leave.

The Streaming Band-Aid

Many developers try to hide this latency by streaming tokens to the UI. This is a visual band-aid, not an architectural fix.

Streaming gives the illusion of speed, but it does not change the physical time it takes to complete the task.

You cannot out-prompt the speed of light.

Prompt engineering cannot fix an overloaded cloud endpoint. If a backend task requires parsing JSON from an LLM response before executing a database query, streaming is useless.

You are simply blocked for five seconds.

The Self-Hosting Fallacy

Faced with massive API bills and latency spikes, engineering teams typically experience a knee-jerk reaction

“Our API bill is too high. Let’s buy an H100 and host Llama 3 ourselves.”

This is an ego trip disguised as a financial strategy.

The Silicon Math

Generating a million tokens on a hosted, heavily optimized API endpoint costs pennies for smaller models. Renting a single A100 or H100 GPU node costs upwards of $80 to $150 a day.

Because user traffic is spiky and unpredictable, that expensive silicon will sit completely idle 80 percent of the time.

You are paying for maximum capacity, but only utilizing a fraction of it.

The DevOps Nightmare

Running LLMs in production is not like running a Node.js server. It is a grueling battle with KV cache memory fragmentation, continuous batching algorithms, and CUDA out-of-memory crashes.

To keep a self-hosted model running efficiently, you need specialized AI infrastructure engineers. Those engineers cost $250,000 a year.

Unless you process tens of millions of tokens daily at a perfectly consistent utilization rate, or you have strict air-gapped compliance requirements, self-hosting will bankrupt you faster than using LLM APIs.

You trade variable API costs for massive fixed CapEx and severe operational friction.

Architecting the Defensive Moat

To survive the LLM API tax, architects must build a ruthless abstraction layer between their application and the model provider.

You need an AI Gateway.

Tactic 1: Semantic Caching

Users ask the exact same questions constantly. You should not pay Anthropic 1,000 times a day to answer “What is your refund policy?”

Instead, pass the user’s query through a cheap, sub-millisecond embedding model. Store that vector in a Redis cache alongside the LLM’s final generated response.

When the next user asks “How do I get my money back?”, perform a cosine similarity search.

If the mathematical match exceeds a 0.95 threshold, serve the cached string instantly. Your compute cost drops to $0.

Your TTFT drops to 50 milliseconds.

Tactic 2: Intelligent Routing (The Cascade)

Stop using premium frontier models for everything. A massive percentage of application logic involves basic classification, sentiment analysis, or summarization.

Your gateway should route trivial tasks to ultra-cheap, high-speed models like Gemini Flash, Llama 3 8B, or Claude Haiku.

Reserve the expensive, heavy reasoning models strictly for complex escalation paths. Implement a cascade, try the cheap model first, and only trigger the expensive model if the output fails a deterministic validation check.

Tactic 3: Context Pruning

Stop treating the LLM context window as a dumping ground.

Throwing 50 pages of PDF text into an API call because “the model supports 1M tokens” is financial suicide.

Implement strict context pruning.

Use a fast Cross-Encoder to re-rank your vector search results. Strip out boilerplate text, HTML tags, and redundant paragraphs before assembling the final prompt.

Sending exactly 800 highly relevant tokens is infinitely cheaper and yields better model accuracy than blindly dumping 8,000 tokens.

Tactic 4: Circuit Breakers

External APIs fail. Rate limits get exhausted. If your application crashes when LLM API throws a 502 Bad Gateway, your architecture is brittle.

Your gateway must implement strict circuit breakers.

If the primary provider times out or degrades, the gateway must instantly and transparently reroute the exact payload to a backup provider (e.g., failing over from OpenAI to Anthropic).

Graceful degradation is a mandatory requirement, not a feature.

Conclusion

Large Language Models are not magic brains. They are commodity compute.

The value of your engineering team is not in writing a clever system prompt. The value is in building the caching, routing, pruning, and abstraction infrastructure that makes running that prompt economically viable.

If you build a thin wrapper, the API provider will eventually consume your margins.

Build the moat, or prepare to join the graveyard.

Last week, I sat in an architecture review with a client who had just secured Series A funding. They were building a standard Retrieval-Augmented Generation (RAG) pipeline for their internal documents.

Before anyone had written a single line of backend logic, the engineering lead proudly displayed a slide proposing a six-figure enterprise contract with a dedicated Vector Database.

When I asked why we weren’t just using Postgres, the answer was immediate

“Because this is an AI app. You need a Vector DB for AI.”

This is the exact kind of hype-driven development that destroys startup runways.

At the physical level, Vector Databases are not magic AI boxes. They do not understand “meaning,” “context,” or “semantics.”

They are simply C++ or Rust memory allocators navigating multi-dimensional mathematical graphs.

To do this quickly, they trade exact accuracy for speed, and they force you to pay an absolute fortune in RAM to do it.

Before you sign a massive SaaS contract for a dedicated vector engine, you need to understand the physics of the hardware you are renting.

The Brute Force Math

To understand why traditional databases fail at vector search, you have to look at the math of a “Vector.”

An OpenAI text-embedding-3-small embedding is a 1,536-dimensional array of floating-point numbers. In physical memory, a single 32-bit float consumes 4 bytes.

1,536 dimensions × 4 bytes = 6,144 bytes (6 KB) per vector

If you want to find the closest vectors to a user’s query using exact math, a process called Exact K-Nearest Neighbors (KNN) which your CPU must calculate the cosine similarity between the query vector and every single row in the database.

Let’s do the memory bandwidth math on a small 1-million-row table

1,000,000 rows × 6 KB = 6.1 Gigabytes of data

To answer one user query,

• the CPU must pull 6.1 GB of data from RAM through the memory bus,

• load it into the L1 cache,

• and execute millions of AVX-512 SIMD (Single Instruction, Multiple Data) dot-product operations.

Even on modern DDR5 RAM peaking at 50 GB/s of bandwidth, a single concurrent user doing an Exact KNN search will consume 12% of your entire server’s memory bandwidth.

If you get 10 concurrent searches, your CPU is completely starved for data. The system grinds to an absolute halt.

Even the traditional B-Trees cannot save you.

A B-Tree relies on 1-dimensional inequalities (Is X > 10? Go right). You cannot sort or bisect 1,536 dimensions simultaneously.

HNSW (Hierarchical Navigable Small World)

If Exact KNN locks up the CPU, how do Vector DBs return results in 20 milliseconds?

They don’t do exact searches. They cheat.

Vector databases rely on Approximate Nearest Neighbor (ANN) algorithms.

They accept that finding the perfect match is computationally impossible at scale, so they settle for finding a very good match almost instantly.

The undisputed king of these algorithms is HNSW.

Do not let the academic name intimidate you. HNSW is just a multi-layered skip-list mapped over a proximity graph.

Imagine you are driving from New York to a specific house in a suburb of Los Angeles

The Top Layer (Interstates)

You don’t take local roads across the country. You get on an interstate.

In HNSW, the top layer has very few nodes, but they have long-distance links.

The search algorithm drops in here and makes massive, cross-graph jumps toward the general cluster of the target.

The Middle Layers (City Roads)

Once you are near Los Angeles, you drop down a layer.

There are more nodes here, connected by shorter links.

You navigate to the correct neighborhood.

The Bottom Layer (Local Streets)

You drop to the base layer, which contains every single vector in the database, and you traverse the local streets until you hit the closest possible house (a local minimum).

HNSW is a masterpiece of algorithmic engineering. It reduces a catastrophic O(N) full table scan into a blisteringly fast O(log N) graph traversal.

But it comes with a brutal physical cost called Pointer Chasing.

Why NVMe SSDs Hate HNSW (Pointer Chasing)

Traditional relational databases are famous for being disk-friendly because they exploit Locality of Reference.

B-Tree nodes are packed cleanly into contiguous 8KB blocks (pages) on your SSD. When Postgres needs an index node, it pulls a single 8KB block into memory, and all the sequential keys are right there next to each other.

HNSW graphs are the exact opposite.

An HNSW index is a giant, chaotic web of pointers (memory addresses) pointing to other pointers across multiple layers.

The nodes are scattered randomly across the heap during insertion.

Traversing this graph means jumping wildly from memory address to memory address.

If your HNSW index does not fit in RAM and is forced onto a Solid-State Drive, following those pointers requires thousands of Random Disk Reads per query.

A standard NVMe SSD takes roughly 100 microseconds (µs) to complete a random 4KB read. If an HNSW search requires 200 graph hops to find the nearest neighbor

200 hops × 100 µs = 20 milliseconds of pure disk latency

That sounds fast, until you realize this is for one query.

If your app is doing 1,000 queries per second, your SSD must sustain 200,000 random IOPS.

Even the most expensive AWS io2 Block Express volumes will buckle under that kind of random I/O queue depth.

Your 20ms latency will instantly spike to 2 seconds as the disk controllers choke.

Furthermore, pointer chasing completely defeats the OS Page Cache and the CPU’s hardware prefetchers.

The CPU cannot guess which memory address the graph will jump to next, resulting in continuous L3 cache misses.

The RAM Tax

Here is the physical reality of vector search

To get the sub-50ms latency that Vector DBs advertise, the entire HNSW index must physically live in RAM.

This brings us to the invoice (cost of the RAM).

RAM is exponentially more expensive than NVMe storage. Let’s do the math on a production-scale deployment of 100 million vectors.

• Vector Payload - 100,000,000 × 6 KB (1536-dim floats) = 600 GB

• HNSW Graph Overhead - Each node in HNSW maintains bidirectional pointers to its neighbors across multiple layers. This pointer overhead usually adds 30% to 50% to the base vector size. Let’s add 200 GB.

To serve 100 million vectors, you need 800 GB of RAM.

Storing 800 GB on a standard Postgres SSD costs about $65 a month. Holding 800 GB in an AWS r6a.24xlarge memory-optimized instance costs $5,300 a month.

When you buy a dedicated Vector Database, you are not buying better “AI.”

You are buying an expensive fleet of high-RAM cloud instances to hold a massive, chaotic pointer graph in volatile memory because the algorithm physically cannot survive on a disk.

What you should do?

Do not subsidize a SaaS company’s valuation because you think standard infrastructure can’t handle vector math.

Here is the architectural framework you should use before signing a vendor contract

Use Postgres (pgvector)

If you have fewer than 5 million vectors, you do not have a scale problem. You have a standard CRUD problem.

Install the pgvector extension on your existing Postgres instance.

Postgres is perfectly capable of building an HNSW index and holding it in its shared_buffers (RAM).

You save thousands of dollars, you eliminate a fragile network hop in your infrastructure, and you keep your relational data and your embeddings in the exact same ACID-compliant transaction block.

You can JOIN your semantic search results directly against your user permission tables in a single query.

Use a Dedicated Vector DB (Pinecone, Milvus, Qdrant)

You only graduate to a dedicated vector engine when your index physically outgrows the RAM limits of a single, massive database instance.

When you hit 50 million, 100 million, or a billion vectors, a single Postgres node will OOM (Out of Memory) trying to fit the HNSW graph into shared_buffers.

That is the exact moment you pay a premium for a distributed vector database.

You are paying them to shard the massive HNSW graph across multiple clustered nodes and handle the distributed scatter-gather networking required to query it.

Conclusion

Architecture requires alignment between the physical realities of the hardware and the economic realities of the business.

HNSW is a brilliant algorithm, but it is an unapologetic memory glutton.

It defeats disk I/O, thrashes CPU caches, and demands expensive DDR5 RAM to function at scale.

Start with Postgres, monitor your memory saturation, and scale out to a distributed vector engine only when the physics of the graph demand it.

Don’t buy a distributed system until you have a distributed problem.

Nobody gets fired for choosing Postgres.

It is the safest, most defensible technical decision you can make. Until it isn’t. Especially when you are building a highly concurrent, event-driven logger.

You pick it because it handles everything perfectly on day one, completely ignoring that you are buying a general-purpose engine for what might be a highly specialized problem.

We do this constantly in software. We slap a UUIDv4 on a primary key because it is convenient, blindly accepting the B-Tree fragmentation it guarantees at scale.

We spin up Postgres because it is the industry standard, ignoring what its storage engine actually does to our specific data shape.

Convenience dictates the architecture. Physics dictates the bill.

The cost of MVCC Write

Look at how Postgres manages concurrency. Multi-Version Concurrency Control (MVCC) is a brilliant mechanism. It allows readers and writers to operate simultaneously by creating new tuple versions instead of modifying data in place.

Let’s be absolutely clear.

If you are building a standard B2B SaaS application, with users, organizations, billing tables, and dynamic records, Postgres is flawless. If you are building a banking ledger where balances update constantly, MVCC is your best friend.

But if you are building an event log, a sensor stream, or an audit trail, MVCC is a parasite.

Consider a standard telemetry or event logging table. The schema looks entirely innocent

CREATE TABLE user_events (
    event_id UUID PRIMARY KEY,
    user_id BIGINT,
    event_type VARCHAR(50),
    payload JSONB,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Executed 50,000 times a second
INSERT INTO user_events (event_id, user_id, event_type, payload)
VALUES ('...', 123, 'click', '{"button": "signup"}');

To the developer, this is a simple, lightweight append operation. To the Postgres storage engine, this is a massive overhead event.

Your rows are immutable the second they hit the disk. Yet, every single insert carries a hidden 23-byte tuple header (xmin, xmax, cmin, cmax) tracking version visibility for concurrency that will never happen.

At 50,000 inserts a second, you are paying a massive, constant write tax on data that will never change. You are generating gigabytes of Write-Ahead Log (WAL) just to track ghosts.

8KB at a Time

When you do actually update or delete records in a highly active table, the architecture fights you in a completely different way.

-- You think you are modifying data in place. You aren't.
UPDATE user_sessions 
SET last_seen = NOW() 
WHERE session_id = 'abc-123';

Postgres does not delete old rows. It leaves the dead tuple sitting exactly where it was on the physical 8KB page. It writes the new, updated row to a new location. A background worker called autovacuum is supposed to sweep through eventually and mark that old space as reusable.

Under a heavy write load, autovacuum loses the race. Dead tuples accumulate faster than the database can process them.

Because live and dead records share the exact same physical pages, the query planner cannot skip the graveyard. If your table has 10 million live rows and 40 million dead ones, your hardware is pulling massive amounts of garbage from the disk into your buffer pool just to find the active data. The engine literally chokes on its own history.

As Andy Pavlo at CMU has pointed out, if someone were building a new MVCC database today, they would not implement it the way Postgres does. Its append-only storage design is a relic of the 1980s that predates log-structured storage patterns entirely.

The Snapshot Collision

One day someone asks for a dashboard. You run a long analytical query against this live operational data.

-- The Data Analyst runs this at 9:00 AM
SELECT date_trunc('hour', created_at), count(*)
FROM user_events
GROUP BY 1
ORDER BY 1;

Running a daily sales report on a 5-gigabyte SaaS database is fine. Running a 2-hour aggregation on a table absorbing 50,000 sensor readings a second is a death sentence.

To guarantee consistent reads, Postgres holds the transaction snapshot open. While that two-hour report runs, autovacuum is paralyzed.

It cannot clean up any dead tuples that fall behind that active transaction horizon. Your dashboard is actively blocking garbage collection for the entire operational database.

Every UPDATE happening on your site during those two hours leaves a permanent scar on the disk until the report finishes.

Friction generates heat. Heat generates wear. Wear destroys your query latency. Postgres was simply not designed to serve heavy OLTP and OLAP from the same instance under load.

The Optimization Treadmill

This is when you step onto the Optimization Treadmill. You do not fix the architecture. You treat the symptoms.

Step 1 - The Index Trap

Write throughput dips, so you add a composite index to speed up the read queries causing the locks. You just amplified your write penalty. Every insert now has to update multiple B-Trees.

Step 2 - The Vacuum Illusion

The table bloats, so you aggressively tune the background workers.

ALTER TABLE user_sessions 
SET (autovacuum_vacuum_scale_factor = 0.01);

You force the database to vacuum constantly. You are now burning precious CPU cycles fighting your own insert rate.

Step 3 - Hardware Scaling

CPU spikes, so you upgrade the instance class. You double the RAM. You provision higher IOPS.

Every single one of these interventions is the technically correct response to a real symptom. You are doing exactly what an experienced DBA would do. But none of it changes your trajectory. You are fighting an architectural ceiling, not a misconfiguration.

The treadmill only speeds up. Each fix buys you a few months of runway before the database collects its tax again.

How to Spot the Treadmill Early

The most critical decision you can make is recognizing the treadmill before you get on it. Stop treating Postgres as a zero-cost default. Look at the physical reality of your data before you write the schema.

• Are your writes predominantly inserts with no updates?

• Are the rows permanently immutable once written?

• Is your data volume growing by 50 percent year over year?

• Do you need to run heavy analytics against the same live table?

If you nod your head to any of those, the architectural mismatch is already baked in. The only variable is how much engineering time and AWS budget you burn before you stop asking how to tune the database, and start asking the question you should have asked on day one.

Are we actually building a general-purpose OLTP system?

The HTAP Escape

If you recognize the treadmill early, you can escape it. If you actually need to serve heavy OLTP and OLAP concurrently from the same system without melting your disks, you need an architecture designed for Hybrid Transactional/Analytical Processing (HTAP).

When Postgres is no longer the default, you have three primary paths

1. SingleStore

SingleStore (formerly MemSQL) solves the OLTP/OLAP collision by physically splitting the storage engine under the hood. It uses an in-memory rowstore for blazing-fast transactional inserts and a disk-based columnstore for your analytical queries.

It is true HTAP. You get millisecond ingest and sub-second aggregations on the exact same cluster. It speaks the MySQL wire protocol, making integration trivial.

But, it is proprietary and expensive. The in-memory rowstore demands that your operational working set fits entirely in RAM. You are trading software friction for a massive hardware bill.

2. TiDB

TiDB separates compute from storage entirely. It uses a Raft-based distributed key-value store (TiKV) for your transactional inserts, and asynchronously replicates that data to a columnar engine (TiFlash) for analytics. The smart query planner automatically routes your dashboard queries to the column nodes and your point-writes to the row nodes.

It scales horizontally forever. You can run massive analytical reports against TiFlash without ever locking an operational row in TiKV.

But, it has severe operational complexity. You do not just “spin up” TiDB. Running the control plane, compute nodes, row storage nodes, and column storage nodes requires a dedicated infrastructure team. It is a Ferrari; you have to hire the mechanics.

3. Kafka + ClickHouse (Split Stack)

If you cannot justify HTAP licensing or operational overhead, stop trying to force both workloads into one database. Write your immutable events to an append-only log (Kafka). Consume that log into a pure columnar database (ClickHouse) for your dashboards.

The physics align perfectly. Kafka’s sequential writes effortlessly absorb the OLTP load. ClickHouse’s columnar compression destroys the OLAP load. Both systems run at absolute maximum efficiency.

But, you now maintain two distinct distributed systems and the pipeline connecting them. Eventual consistency is guaranteed; immediate consistency is impossible.

Conclusion

There is a recurring theme in engineering failures, we mistake popularity for universal fit.

Postgres is arguably the greatest relational database ever built. But it is a machine with specific operating parameters.

When you force an append-only, high-velocity stream into an MVCC engine, you are not testing the limits of Postgres. You are testing the limits of your infrastructure budget.

Stop letting convenience dictate your architecture. The cost of learning a specialized database like ClickHouse or Kafka today is infinitely lower than the cost of fighting the Optimization Treadmill tomorrow.

Physics always wins. Choose your database accordingly.

The adoption of UUIDv4 as a primary key in OLTP databases has become a widespread default, driven by ORM convenience and distributed system requirements.

This paper demonstrates that UUIDv4 primary keys guarantee severe, predictable performance degradation as table size grows. We introduce the Buffer Saturation Ratio (BSR) and derive a closed-form threshold (N^*) that predicts exactly when write latency will spike.

We prove that this degradation is a physical limitation of B-Tree mechanics under random insertion, compounded by Write Amplification Factor (WAF) penalties, rather than a hardware or traffic-scaling failure.

The Anatomy of a Deferred Crisis

Modern frameworks make it trivial to assign a random, 128-bit string as a primary key. Distributed ID generation solves immediate engineering headaches. It prevents sequence contention. It hides entity counts from users.

But it introduces a delayed tax on your storage layer.

Engineers rarely observe the cost of UUIDv4 during the first year of a project. Databases aggressively cache active indexes in RAM. As long as the primary key index fits entirely within the database buffer pool, random inserts perform acceptably.

The crisis arrives silently. The index quietly outgrows the allocated memory. The database begins reading cold pages from the physical disk to complete routine inserts. Write latency jumps by two orders of magnitude. The typical engineering response, upgrading the instance class which fails to resolve the underlying physics.

B-Tree Mechanics and the Fill Factor Penalty

Relational databases use B-Tree variants to store indexes. Sequential keys, such as BigInt sequences or UUIDv7, append new records to the rightmost edge of the tree. The database keeps that single, active leaf node hot in memory.

UUIDv4 values possess zero temporal locality. Every insert targets a random location in the B-Tree.

When you insert a record into a random leaf node that is already full, the database executes a page split. It allocates a new page (16KB in MySQL InnoDB, 8KB in PostgreSQL), transfers half the records, and updates the parent nodes.

Sequential inserts achieve a high index fill factor (f ≈ 0.90 to 0.94). Random UUIDv4 inserts guarantee continuous page splits, driving the equilibrium fill factor down to approximately 0.50. You are permanently storing half-empty pages. Your index requires twice the RAM just to exist.

The Buffer Saturation Ratio (BSR)

To predict performance failure, we must track the relationship between index size and available memory. We define the Buffer Saturation Ratio (BSR)

Where B_pool is the configured memory for caching (InnoDB Buffer Pool or PostgreSQL shared_buffers), and I_size is the total size of the primary key index.

When BSR ≥ 1.0, the entire index resides in memory. Writes are fast.
When BSR < 1.0 , the index exceeds available RAM.

At BSR < 1.0, every random insert carries a cache miss probability (P_miss)

When a cache miss occurs, the database evicts a page to load the target leaf node from disk. Because UUIDv4 targets pages uniformly, the newly loaded page is highly unlikely to be reused before it is evicted again. This creates LRU eviction thrashing.

Your expected insert latency (L_insert) becomes completely dominated by disk I/O

Deriving the Threshold Row Count (N^*)

We can calculate the exact row count where a database will fall off the performance cliff. First, we model the projected index size

Where

• N = Number of rows

• K_size= Key size in bytes (16 for binary UUID, 36 for char UUID, 8 for BigInt)

• P_overhead = Pointer overhead (roughly 8 bytes)

• f = Fill factor (0.50 for UUIDv4, 0.94 for BigInt)

To find the threshold row count (N^*), we set I_size equal to B_pool (BSR = 1.0) and solve for N

This equation allows architects to project the exact lifespan of a database schema configuration.

Write Amplification Factor (WAF)

Random inserts punish the storage layer beyond just cache misses. They generate massive Write Amplification Factor (WAF) in the Write-Ahead Log (WAL).

The write amplification for a UUIDv4 index follows a logarithmic curve

Where m is the B-Tree order (roughly 80 for InnoDB at 16KB pages with 200-byte rows), N is total rows, and M is the number of rows that fit in the buffer pool.

Your total write amplification multiplies by your secondary indexes (k)

MySQL InnoDB uses clustered indexes. The massive, random primary key is duplicated into the leaf nodes of every secondary index. PostgreSQL uses heap tables, bypassing clustered index bloat, but secondary indexes still suffer the brutal page split penalty and identical WAF multipliers.

Case study

One of my client ran a package scan event log using a char(36) UUIDv4 primary key on a 16 GB RDS instance equipped with a 12 GB buffer pool.

During the first year, at 50 million rows, performance remained nominal. Their BSR sat at 8.0. The buffer pool easily absorbed the random writes.

The table eventually reached 500 million rows. The char(36) key, combined with the 0.50 fill factor, swelled the index size to roughly 15 GB.

Their BSR dropped to 0.80. Write latency instantly climbed from 2 ms to 200 ms. CPU utilization on the RDS instance pinned at 90 percent, entirely consumed by managing LRU evictions.

They upgraded the RDS instance class twice. Neither upgrade resolved the latency. Each vertical scaling event raised CPU, RAM, and the buffer pool proportionally, but the BSR remained strictly below 1.0. The index remained larger than the pool.

Hardware scaling cannot outrun mathematical thresholds. The only viable solution was migrating the primary key to a sequential format.

The N^* Decision Matrix

Applying the N^* formula to standard infrastructure configurations reveals the severe capacity penalty of UUIDv4 strings

A 32 GB server running a char(36) UUIDv4 index chokes at 273 million rows. The exact same hardware running a BigInt handles 2.25 billion rows. You are paying a 400 percent RAM tax for a framework default.

Conclusion

UUIDv4 primary keys constitute a systemic architectural flaw in high-throughput OLTP databases. They actively defeat database caching mechanisms and maximize physical disk I/O.

Engineers must stop treating database performance degradation as a generic symptom of traffic scaling. Use the N^* threshold formula to project your schema’s structural limits before executing a CREATE TABLE statement.

If distributed ID generation is a hard requirement, adopt temporally sequential identifiers like UUIDv7 or ULID. Align your data structures with the physical constraints of B-Tree mechanics.

Read the paper here

https://zenodo.org/records/19034540

It happens every five years. A new CTO arrives at a Global 500 bank.

They look at the “Green Screen” terminals, the 40-year-old IBM Z/OS mainframes, and the millions of lines of COBOL.

They cringe.

“This is legacy,” they declare.

“This is technical debt. We are going to rewrite the Core Ledger in Java microservices. We will be cloud-native.”

Three years and $50 million later, the project is quietly cancelled.

The CTO moves on to a different company.

The COBOL remains.

Why?

It’s not because the bank is lazy. It’s not because we lost the source code.

It is because the fundamental architecture of modern programming languages is hostile to the requirements of global finance.

You are not fighting code, you are fighting how computers do math.

The Math Problem (IEEE 754)

When you write float or double in Java, Python, C++, or Go, you are using IEEE 754 Binary Floating Point arithmetic.

This standard is designed for scientific calculation, measuring the distance to a star or the velocity of a particle.

It prioritizes range and speed over absolute precision.

The Rounding Error

Open your browser console or a Node.js shell and type

0.1 + 0.2
// The result: 0.30000000000000004

In a physics simulation, that 0.00000000000000004 is noise.

In a banking ledger processing trillions of dollars a day, that error is a regulatory violation, a failed audit, and potentially a lawsuit.

Fixed-Point Decimal (COBOL Way)

COBOL was not designed for science.

It was designed for Business (Common Business Oriented Language).

It does not use Binary Floating Point for money. It uses Fixed-Point arithmetic stored as Binary-Coded Decimal (BCD).

In COBOL, you define a variable like this

01 ACCOUNT-BALANCE PIC S9(13)V99 COMP-3.

• 01: Level Number. In COBOL, 01 represents a top-level record or variable. Think of it like const or let at the root scope.

• ACCOUNT-BALANCE: The Variable Name. (COBOL uses kebab-case because it was invented before snake_case or camelCase won the war).

• PIC: Picture Clause. This tells the compiler exactly what the data looks like.

• S: Signed. This bit tracks if the number is positive or negative.

• 9(13): Numeric Integers. The number 9 represents a digit. (13) means “allocate space for 13 of them.”

• V: Virtual Decimal. This is the magic. It tells the CPU “assume a decimal point here,” but it does not store a dot character in memory. It saves space.

• 99: Precision. Allocate exactly 2 digits for cents.

• COMP-3: Packed Decimal (BCD). This is the instruction to store 2 digits per byte (using 4 bits each), rather than the standard 1 byte per character. This is what enables the hardware math precision.

When COBOL calculates 0.1 + 0.2, it does not convert them to binary approximations.

It calculates them in base-10, digit by digit, often utilizing specific hardware instructions on the mainframe (Decimal Floating Point units) that x86 architectures have historically lacked or emulated poorly.

The result is exactly 0.3. Always.

The Cost of Emulation

You must be wondering can’t we do this in Java?

Yes, using java.math.BigDecimal.

But BigDecimal is a software object. It adds memory overhead and CPU cycles for every single calculation.

COBOL operates on money at the instruction-set level. When you are processing 50,000 transactions per second (TPS), that overhead isn’t just a performance hit.

It’s an infrastructure bill.

True Transactionality (The CICS)

The second reason rewrites fail is the misunderstanding of “Transactionality.”

Modern web development is obsessed with “Statelessness.”

You send a REST request, the server forgets you, and you send another.

State is hard.

The Mainframe world runs on CICS (Customer Information Control System). CICS is an application server that manages transactions with ACID properties (Atomicity, Consistency, Isolation, Durability) as a law of physics.

Lets take an example of ATM withdrawal

1. Check Balance.

2. Debit Ledger.

3. Dispense Cash.

4. Log Audit Trail.

In CICS, if step 3 fails (the cash jams), the entire transaction rolls back instantly. The ledger is never touched.

The state remains consistent.

The Microservices Nightmare

In a distributed microservices architecture, you break these steps into different services.

• Service A (Ledger) debits the account.

• Service B (ATM) fails to dispense.

• Now Service A must “compensate” (undo) the transaction.

You have moved from immediate consistency to Eventual Consistency.

You are now managing “Distributed Transactions” and “Sagas.”

So question to you is, do you want your checking account balance to be “Eventually Consistent”?

Or do you want it to be Correct?

Scale Up vs. Scale Out

The philosophy of the modern cloud is Scale Out.

“If the server is overloaded, spin up 100 more cheap nodes. If a node fails, kill it and retry.”

The philosophy of the Mainframe is Scale Up.

“This single machine will process everything. It will not fail. If a CPU dies, a backup CPU takes over without the operating system even noticing.”

Mainframes utilize specialized I/O Processors (Channel Subsystems).

The main CPU doesn’t waste time reading from a disk or talking to the network card. It delegates that to a sub-processor and keeps crunching numbers. This allows mainframes to run at 100% CPU utilization for years without throttling.

Try running a Linux server at 100% CPU for an hour, it will become unresponsive.

Finally, the verdict is

• For Twitter, Scale Out. If a tweet fails to load, nobody loses money.

• For the Global Economy, Scale Up. Reliability is not optional.

Why this matters to You?

Even if you never touch a line of COBOL, the principles of the Mainframe apply to your modern stack.

Never Use Floats for Money

If you are building a FinTech app in JavaScript or Python, do not use standard number types for currency.

const price = 19.99; // Is Wrong

Use libraries like decimal.js, Python’s decimal module, or store values as Integers (cents) and format them only for display

(const priceInCents = 1999;)

Respect the Monolith

We have been trained to think “Microservices = Good” and “Monolith = Bad.”

But if your application requires strict ACID compliance (inventory management, voting systems, banking), a well-structured Monolith with a single database transaction is infinitely less buggy than a distributed system trying to coordinate state across ten services.

“Legacy” is a Success Metric

Code becomes “legacy” only if it survives.

If you are looking at a system that has been running for 30 years, do not mock it.

Learn from it. It has survived market crashes, leap years, and user stupidity for decades.

It is doing something right.

Conclusion

Rewriting a Core Banking System is rarely a “refactoring” effort. It is an archaeological excavation.

That “ugly” COBOL code contains 40 years of edge cases, the tax law change of 1992, the negative interest rate handling of 2015, the specific rounding rule required by the Bank of Japan.

If you rewrite it, you will miss these rules.

You will introduce bugs that were solved in 1985.

Don’t replace the Mainframe. Modernize it.

Wrap the COBOL logic in REST APIs (using Z/OS Connect).

Let the Java frontend look pretty, but let the Iron down in the basement handle the math.

Respect the Old Gods.

They are still holding up the sky.

Right now, in boardrooms and daily stand-ups across the world, executives are signing off on millions for “AI Modernization,” while engineers are blindly feeding legacy scripts into ChatGPT to refactor core business logic.

The pitch is intoxicatingly simple, take a 10,000-line C++ monolith, paste it into an LLM window, and ask it to output pristine, idiomatic Python, Go or Rust microservices.

This is not engineering. This is reckless gambling.

The fundamental flaw in this strategy, whether you are a VP buying an enterprise AI tool or a Senior Engineer writing a migration script, is a misunderstanding of what an LLM actually is.

Large Language Models are probabilistic text generators. They are not compilers. They do not possess an underlying, deterministic model of the code they are ingesting. They do not mathematically understand state mutation, variable shadowing, or lexical scope.

They simply predict the next most likely token based on latent space vectors.

An LLM might translate a complex legacy while loop correctly 99 times. But on the 100th time, distracted by a weirdly named variable or an obscure GOTO statement, it will silently hallucinate. It will drop a crucial state mutation. It will misinterpret the lexical scope of a nested variable.

When you are migrating a critical system such as, a core banking ledger or a flight control system, a 99% success rate is another term for a catastrophic production outage.

Let’s look at an example of how a pure LLM migration breaks.

Smart LLMs rarely make basic syntax errors anymore, instead, they make semantic errors by translating text perfectly while fundamentally changing how the computer manages memory which we will see in the example below

Imagine a pricing calculation in legacy C++ using a struct. By default, C++ passes structs by value (meaning it creates a safe copy).

struct Trade { 
    double price; 
};

// Passed by VALUE. Modifies a local copy for a "what-if" simulation.
void simulateDiscount(Trade t) { 
    t.price *= 0.9; 
    logSimulation("Discounted price would be: ", t.price);
}

// ... elsewhere in the system ...
Trade myTrade = { 100.0 };
simulateDiscount(myTrade);

// myTrade.price is STILL 100.0 here. The original is safely untouched.
processPayment(myTrade); // Processes the actual, full price.

An engineer pastes this into an LLM and asks for an “idiomatic Python” translation. The LLM confidently spits out beautifully formatted Python

class Trade:
    def __init__(self, price: float):
        self.price = price

# Passed by REFERENCE. Modifies the original object!
def simulate_discount(t: Trade):
    t.price *= 0.9
    log_simulation("Discounted price would be: ", t.price)

# ... elsewhere in the system ...
my_trade = Trade(100.0)
simulate_discount(my_trade)

# my_trade.price is now 90.0! 
process_payment(my_trade) 
# You just undercharged the client by 10% because of a simulation!

The LLM got a 100% on the syntax. The code is highly readable.

But it completely corrupted your ledger.

The LLM didn’t know why this was wrong because it doesn’t build a memory execution graph. It mapped a C++ function signature to a Python function signature. It completely failed to realize it just changed a safe, immutable pass-by-value operation into a highly destructive pass-by-reference mutation.

Why AST matters

To translate code safely, we have to return to Computer Science 101.

Code is not text. Treating code as a string of characters is the original issue of pure-LLM migration.

Code is a tree. Specifically, it is an Abstract Syntax Tree (AST).

When a traditional compiler reads your code, the very first thing it does is strip away the formatting, the whitespace, and the text, converting the logic into a strict, hierarchical graph of nodes and edges.

If we parse our legacy C++ simulateDiscount(Trade t) function into an AST, the parser generates a structural map that looks something like this

{
  "node_type": "FunctionDeclaration",
  "identifier": "simulateDiscount",
  "parameters": [
    {
      "node_type": "Parameter",
      "identifier": "t",
      "data_type": "Trade",
      "memory_model": "PASS_BY_VALUE" // <-- The Lifesaver
    }
  ],
  "body": [ ... ]
}

Notice the memory_model metadata.

The AST isn’t guessing based on text patterns, it mathematically knows that this specific C++ language construct creates a local copy.

When an AST-driven migration maps this to Python, it compares the source memory model (PASS_BY_VALUE) to the target language’s default memory model (in Python, objects are passed by reference).

Because it detects this mismatch, the structural translation engine intervenes before the LLM can make a mistake.

It physically forces a constraint into the generated Python code to preserve the semantic contract of the original architecture.

The resulting code structurally generated by the tool will look like this

import copy

def simulate_discount(t: Trade):
    # AST-enforced safety boundary to preserve C++ pass-by-value
    t_local = copy.copy(t) 
    t_local.price *= 0.9
    log_simulation("Discounted price would be: ", t_local.price)

A pure LLM drops the constraint because it just sees words.

An AST preserves the constraint because it enforces mathematical logic.

Probabilistic models guess. ASTs prove.

The Hybrid approach

If pure LLMs are dangerous, and pure AST translation (source-to-source compilers/transpilers) produces ugly, unmaintainable “machine code,” what is the solution?

If you are building an enterprise-grade migration tool, like the local AI agent architectures we build at BinaryBox, you must use a hybrid pipeline. You do not ask the AI to write code from scratch. You force it to operate within a deterministic pipeline

Parse

Use a traditional, deterministic parser (like Tree-sitter) to generate the AST of the legacy C++, or Java codebase.

This locks the exact structure, memory model, and control flow into a mathematical graph.

Analyze

Pass specific, isolated subtrees of the AST to the LLM.

Instead of asking the AI to “rewrite this file,” you ask it to identify intent.

“Analyze this AST node block. Is this an implementation of a bubble sort? Is this a deprecated synchronous network call?”

Generate

Use the AST to deterministically generate the new syntax, utilizing the LLM only to provide modern idiomatic mapping.

As shown above, if the AST dictates a pass-by-value parameter, the LLM is constrained to generating code that enforces a clone or copy in the target language.

The AST guarantees the structure. The AI translates the semantics.

Why you should care?

Why does this architectural distinction matter to a CTO?

Because of QA costs.

If you use a pure LLM to translate a 500,000-line monolith, you cannot trust the output.

Because the system is probabilistic, human Senior Engineers must manually read, verify, and test every single line of the generated code to ensure no silent bugs were introduced.

The time and salary cost of having a Principal Engineer QA 500,000 lines of AI-generated spaghetti is often higher than the cost of having them rewrite it by hand.

The ROI of the migration drops to zero.

An AST+AI hybrid approach mathematically guarantees structural equivalence.

If the AST parser proves that all control flows and state mutations have been preserved in the new language, your engineers do not need to read every line.

They only need to review the architectural patterns and idiomatic choices.

You eliminate the operational risk of silent logic drops, and you cut the migration timeline from years to months.

Conclusion

There is a recurring theme in resilient system design, Architecture requires constraints.

We saw this in the Editor Wars. Atom gave developers total freedom to touch the DOM, resulting in chaotic performance. VS Code built a strict Extension Host, a structural prison that constrained plugins and guaranteed a flawless user experience.

AI agents are no different. If you give an LLM the freedom to write whatever text it wants, it will eventually write a bug that bankrupts your company.

To do a reliable AI code migration, you must build a structural prison.

The Abstract Syntax Tree (AST) is that prison. You lock the LLM inside the boundaries of the AST, forcing it to translate node-by-node, bounded by the laws of lexical scope, memory models, and control flow.

Freedom is chaos. Constraints are reliability.

Stop trying to use a magic wand, and start building a compiler.

In my previous Deep Dive, I tried to write 1,000,000 records per second to PostgreSQL running on an AWS c8g.48xlarge instance backed by Provisioned IOPS SSDs (io2 Block Express).

The database locked up. The queue depth exploded. The disk, a $30,000/month NVMe SSD simply couldn’t physically accept the write signals fast enough.

We had to abandon persistent storage entirely and switch to a Redis cluster.

We traded durability for speed, accepting that a power failure would vaporize millions of transactions in an instant.

But here’s the part that breaks most engineers’ mental models

Apache Kafka handles 1 million writes per second on cheap, spinning hard drives.

Not NVMe. Not even SATA SSDs. Actual magnetic platters with mechanical arms. The kind of physical, spinning rust drives you can buy for $40 at Amazon.

How is this possible?

The answer isn’t “Kafka is written in a faster language” (it runs on the JVM, which is notoriously heavy).

The answer isn’t “Kafka uses better compression.”

The answer is physics.

Kafka doesn’t fight the hard drive. It exploits it. This is the story of how Kafka “cheats” by respecting the fundamental constraints of hardware, while traditional databases try to bend reality and lose.

RAM is Fast, Disk is Slow?

Every engineer “knows” RAM is faster than disk. But at scale, throughput beats latency. Sequential disk can outrun random RAM.

Let’s challenge the conventional wisdom directly. We are all taught these standard latency numbers

• RAM ~100 nanoseconds access time

• SSD ~100 microseconds access time (1,000x slower)

• HDD ~10 milliseconds access time (100,000x slower)

These numbers are factually correct. But for high-throughput workloads, they are completely irrelevant. The missing context in that table is Random vs. Sequential I/O.

Those latency numbers assume random access, your application is jumping to arbitrary memory addresses or disk sectors.

But when you switch to sequential access, the story completely flips.

Let’s look at Sequential Read/Write Throughput

• RAM (DDR4) ~20-50 GB/sec

• NVMe SSD ~3-7 GB/sec

• SATA SSD ~500 MB/sec

• 7200 RPM HDD ~200 MB/sec

Here is the key insight.

A standard, cheap hard drive doing pure sequential I/O can easily saturate a 1 Gbps network link. If your system bottleneck is network throughput (which it absolutely is at 1,000,000 requests per second), the magnetic disk is actually fast enough.

Let’s look at the real comparison between our failed experiment and Kafka’s architecture.

Postgres doing 1M random inserts

Each insert updates multiple B-Tree indexes. Each index update requires seeking to a random page on the disk. Even on an enterprise SSD, a random seek takes ~100 microseconds.

1,000,000 random seeks × 100 microseconds = 100 seconds of pure seek time.

It is mathematically impossible to process that in one second.

Kafka doing 1M sequential appends

Kafka writes to the end of a log file. There is no seeking. A modern hard drive sequentially writes at ~200 MB/sec.

1,000,000 writes × 1KB each = 1 GB.

At sequential speeds, that takes roughly 5 seconds on a single cheap disk, and is trivially parallelized across 5-10 disks in a JBOD (Just a Bunch of Disks) configuration to handle it in 1 second.

The lesson here. Sequential disk beats random RAM when throughput matters more than latency.

This is why Kafka doesn’t need NVMe. It just needs sequential access patterns.

What’s the throughput of your production database doing sequential scans vs. indexed lookups? If you don’t know, you’re optimizing blind.

How Kafka Enforces Sequential I/O

Unlike relational databases, which rely heavily on B-Trees to enable fast random lookups, Kafka is built around a single, aggressively simple data structure. The Commit Log.

When a message arrives at a Kafka broker, the system does exactly one thing, it appends the message to the end of the current log segment (a raw file on disk).

• It never updates existing entries.

• It never seeks backward to modify a state.

• It writes in large batches (saving multiple messages in a single system call).

Why does this work so perfectly?

Hard drives have a mechanical read/write head. To read or write data, that head must physically move across the platter to find the correct sector.

This is why random I/O is so devastatingly slow, the mechanical arm is constantly repositioning. It is physically vibrating.

But when you strictly append to the end of a file, the head drops into position and stays there. The disk controller can stop worrying about seek times and optimize entirely for pure throughput.

You are essentially turning a hard drive into a firehose.

If you add batching, writing 100 messages per write() syscall instead of 1, you reduce the CPU context-switch overhead by 100x while keeping the disk arm perfectly still.

The Trade-off Kafka Makes

Databases optimize for flexibility, random access (give me record ID 47293), updates in place (change this user’s email), and complex queries (JOIN across three tables).

Kafka completely abandons this. It optimizes strictly for append-only writes (add this event), sequential reads (replay messages in order), and time-based queries (give me all events from 2:00 PM).

This is a conscious architectural choice, not magic. By refusing to support random updates, Kafka gets to use the fastest possible I/O pattern the hardware offers.

Look at Netflix. They log every single user interaction (play, pause, seek, stop) to Kafka. At peak, that is hundreds of thousands of events per second from millions of concurrent users.

Netflix doesn’t need to query “what did user X do exactly 4 seconds ago?” in real-time. They need to capture the firehose of data and process it asynchronously.

A B-Tree database would collapse under that write load. Kafka’s append-only log absorbs it effortlessly.

Look at your highest-volume write workload.

Is it actually appending events, or are you using INSERT/UPDATE simply because “that’s what databases do”?

Kafka Doesn’t Manage Memory

If you write a standard application that interacts with a disk and a network, your data flow generally looks like this

1. Read data from disk into a kernel buffer.

2. Copy data from the kernel buffer to application memory (like the JVM heap).

3. Process the data.

4. Copy data from application memory back to a kernel socket buffer.

5. Write to the network socket.

At extreme throughput, this standard pattern creates two massive system bottlenecks.

1. Garbage Collection Death

Every message object allocated in the JVM heap must eventually be garbage collected.

If you are pushing 1,000,000 messages per second through application memory, the Garbage Collector cannot keep up.

You will experience massive “stop-the-world” pauses that instantly kill your throughput and trigger network timeouts.

2. Double Buffering

Your data exists in two places at once, kernel memory (the OS page cache) and application memory (the JVM heap).

You are wasting RAM, and more importantly, you are wasting CPU cycles copying the exact same bytes back and forth between user space and kernel space.

Kafka’s Solution

Bypass Application Memory Entirely.

Kafka does not attempt to manage a complex internal buffer pool. It relies entirely on the Linux OS Page Cache.

When Kafka writes a message, it calls write() to append to the log file. The OS buffers this write in the page cache in RAM. Kafka immediately returns a success acknowledgment to the producer.

The Linux kernel flushes that page cache to the physical disk asynchronously in the background.

When Kafka reads a message, the OS loads the file into the page cache. Kafka references the page cache directly. The message data is never copied into the JVM heap. Therefore, there is no garbage collection penalty.

Modern operating systems are ruthlessly efficient at managing file caches. Linux will gladly use 100% of your free RAM as a page cache. Kafka doesn’t try to outsmart the OS, it defers to the kernel.

The practical result?

A Kafka broker with 64 GB of RAM effectively has ~4 GB dedicated to the JVM heap (which is tiny), and ~60 GB dedicated to the OS page cache.

Consumers reading recent data get RAM-speed access because the OS serves it directly from the cache. Older messages fall out of cache and are read from disk, but because it’s sequential, it remains incredibly fast.

Postgres must manage its own complex buffer pool because it supports random updates, ACID transactions, and row-level locking.

Kafka can rely entirely on the OS because it only does sequential access.

Zero-Copy

This brings us to the centerpiece of Kafka’s architecture.

The fastest code is the code that never runs. Zero-Copy means the CPU doesn’t touch your data.

That’s why it’s fast.

The Traditional Data Path (4 Copies)

When a consumer requests a batch of messages from Kafka, a naive implementation would execute the following path

1. Disk → Kernel Buffer - Via DMA - Direct Memory Access. Hardware does the work.

2. Kernel Buffer → Application Buffer - CPU copies data from kernel space into the Kafka JVM heap.

3. Application Buffer → Socket Buffer - CPU copies data from the JVM back to the kernel network stack.

4. Socket Buffer → NIC - Via DMA to the Network Interface Card.

At each copy boundary, you suffer. CPU cycles are wasted. You force expensive context switches between user space and kernel space. You pollute the CPU L1/L2 caches, evicting hot application state just to make room for transient message bytes that are passing through.

At scale, serving 1,000,000 messages/sec means copying 1 GB/sec four times. That is 4 GB/sec of memory bandwidth consumed just moving the exact same bytes around the motherboard.

On our massive c8g.48xlarge server, the CPU would be saturated just copying data, doing absolutely zero actual processing.

The Zero-Copy Solution

sendfile() Linux provides the sendfile() syscall (and its cousin splice()) to solve this exact bottleneck.

// Traditional approach (CPU intensive)
read(file_fd, buffer, size);           // Copy from kernel to app
write(socket_fd, buffer, size);        // Copy from app to kernel

// Zero-copy approach (Hardware accelerated)
sendfile(socket_fd, file_fd, offset, size);  // Copy directly kernel-to-kernel

What actually happens under the hood when Kafka uses zero-copy?

1. Disk → Kernel Page Cache - DMA read.

2. Page Cache → Socket Buffer - A kernel-to-kernel copy. User-space is completely bypassed.

3. Socket Buffer → NIC - DMA write.

The message data never enters Kafka’s application memory. Kafka simply issues a command to the Linux kernel,

“Take 500 bytes from file descriptor Z at offset X, and stream them directly into network socket W.”

The kernel and the DMA controllers handle everything.

Instead of 4 copies, you get 2 copies (and both are heavily hardware-accelerated). Instead of 4 expensive context switches, you get 1. Instead of thrashing the CPU cache, you keep it pristine for actual orchestration logic.

LinkedIn engineering published benchmarks demonstrating that zero-copy improves Kafka’s throughput by 2-3x for consumer reads. At 1M messages/sec, that is the literal difference between needing a cluster of 3 massive servers versus 1 server.

Why traditional message queues can’t do this

RabbitMQ, ActiveMQ, and traditional enterprise queues usually transform messages (adding headers, parsing routing keys), encrypt payloads in the application layer, or apply middleware.

All of these actions require the message to be pulled into application memory so the CPU can inspect and alter the bytes.

Kafka’s messages are opaque byte arrays. Kafka does not parse them, it does not transform them, and it does not care about their contents.

This architectural constraint allows Kafka to use zero-copy. The broker is just a dumb, incredibly fast pipe moving bytes from a disk to a network card.

How many times is your data copied between receiving it and sending it? Every single copy is CPU waste you are paying for on your AWS bill.

When to Use This Pattern

Understanding how Kafka works is only half the battle. You need to know when to apply these principles, append-only logs and zero-copy transfers to your own systems.

This pattern WORKS when

Write-heavy, read-sequential workloads

Event logging, audit trails, analytics ingestion pipelines, and background job queues.

Messages are opaque blobs

You don’t need the broker to parse, transform, or route based on content. Consumers handle the deserialization.

Recent data is hot, old data is cold

99% of your reads are for data written in the last few minutes (guaranteeing a page cache hit). Occasional historical reads (requiring a disk seek) are acceptable.

Durability matters, but immediate consistency does not

Relying on the OS page cache flush (write-behind caching) is “good enough,” and you don’t need to force an fsync() to the physical platter on every single write.

This pattern DOES NOT work when

Random access queries

“Give me user 47293’s profile.” (Use a traditional database).

Low-latency single-message processing

If you need sub-millisecond latency per message, Kafka isn’t the tool. Zero-copy optimizes for massive batch throughput, not single-message latency.

Message transformation in the broker

If your broker must decrypt, dynamically route, or mutate messages, you cannot use zero-copy because you must pull the data into application memory.

Before reaching for Postgres, Redis, or MongoDB for a high-volume endpoint, ask yourself

“Am I appending events, or am I updating records?”

If your workload is append-mostly and sequential-read, you are leaving 10x performance on the table by using a general-purpose B-Tree database.

Consider Kafka for event streams, ClickHouse for analytics, or InfluxDB for time-series metrics.

All of them use append-only logs. All of them respect sequential I/O.

Conclusion

In my 1M RPS test, Postgres failed not because it was poorly designed, but because it was designed for a entirely different problem space.

Postgres optimizes for maximum flexibility, random updates, complex queries, and strict ACID guarantees. To deliver this flexibility, it must use B-Trees, endure random I/O, and manage its own application buffers.

Kafka optimizes for maximum throughput, append-only writes, sequential reads, and eventual consistency. To deliver this throughput, it uses commit logs, demands sequential I/O, and relies entirely on kernel-managed caching.

Neither system is “better.” They solve different physical problems.

The lesson here isn’t “use Kafka instead of Postgres.”

The lesson is, understand the physics of your hardware, and then choose the data structure that ruthlessly exploits it.

Sequential disk is faster than random RAM. Zero-copy is faster than application processing. The Linux OS page cache is smarter than your hand-rolled buffer pool.

Stop fighting the metal. Start respecting it. When you align your architecture with the strict constraints of the underlying hardware, you don’t need to scale out to hundreds of servers. You can handle 1 million writes per second on a $40 hard drive.

That’s not magic. That’s engineering.

March 2024. Andres Freund, a Microsoft engineer and PostgreSQL developer, is working from home when he notices something odd.

His SSH logins are taking 500 milliseconds longer than normal.

Most engineers would ignore this. Network latency. A busy server. Maybe restart the daemon and forget about it.

Freund didn’t. He broke out the profiler and debugged it.

He traced the micro-delay to liblzma. A ubiquitous compression library used by OpenSSH (via systemd).

He found that recent versions contained a backdoor so sophisticated that it had bypassed

• Automated security scanners

• Code reviews from major Linux distributions

• Penetration testing

• Static analysis tools

• The “many eyes” of the open-source community

The backdoor had been planted over 2.5 years by a nation-state actor who earned the trust of the maintainer, contributed legitimate code, and then injected malware into the build process itself.

If Freund had ignored that 500ms delay, every major Linux distribution would have shipped SSH servers with a pre-installed, pre-authentication remote access backdoor.

Every CISO in the world relies on million-dollar security budgets, automated scanners, and penetration tests. A backdoor that took 2.5 years to plant was caught by one engineer who thought, “Huh, that CPU spike is weird.”

This is the story of why your security budget missed what one curious engineer caught by accident. And why “free” open-source software might be the most expensive dependency in your infrastructure.

The Myth

Let’s challenge the core religion of modern software engineering.

Linus’s Law states

“Given enough eyeballs, all bugs are shallow.”

This has been gospel in the tech industry for 30 years. It’s why enterprise companies confidently build trillion-dollar infrastructures on open-source software.

The assumption is that open source is inherently more secure than proprietary code because anyone can audit it. Bad actors can’t hide in plain sight when millions of developers are watching.

Here is the reality of XZ Utils

• Downloads - 1.4 million per day.

• Usage - Core dependency for every major Linux distribution (Debian, Ubuntu, Fedora, Red Hat).

• Criticality - Required for SSH authentication on virtually every server on the internet.

• Maintainers - ONE unpaid volunteer (Lasse Collin).

• Annual Budget - $0.

• Last Security Audit - Never.

According to the Open Source Security and Risk Analysis (OSSRA) report, 87% of commercial applications contain open-source components.

Yet, less than 3% of organizations perform regular security audits of their dependencies, and the average application relies on 500+ transitive dependencies.

Linus’s Law assumes the eyes are looking. But nobody audits compression libraries. Nobody reviews arcane build scripts. Nobody sponsors the maintainer working nights and weekends for zero pay.

Your production infrastructure runs on code that hasn’t been reviewed since it was written. The “many eyes” aren’t watching. They’re assuming someone else is. This collective delusion is exactly what made the XZ attack possible.

How many dependencies does your main application have? How many of those have you personally reviewed in the last year? If the answer is ‘zero,’ you’re trusting strangers with your production environment.

How Jia Tan Bypassed the Eyeballs

To understand why your scanners failed, you have to understand the mechanics of the attack. It was a masterpiece of both social and software engineering.

Phase 1 - Social Engineering the Maintainer

The attack didn’t start with code. It started with economics.

Lasse Collin had maintained XZ Utils alone, unpaid, for over 15 years. He had a full-time job. XZ was his nights-and-weekends project. In 2021, a coordinated pressure campaign began. Multiple sock-puppet accounts created hostile issues on the XZ mailing list

• “Lasse is unresponsive to patches.”

• “This project is effectively unmaintained.”

• “We need new leadership or XZ will die.”

The manufactured urgency worked. Lasse, burnt out and dealing with personal health issues, accepted help.

Enter “Jia Tan”

A seemingly legitimate open-source contributor who had been making small, helpful commits for months. Jia earned trust slowly. He fixed real bugs. He responded to issues professionally. He became indispensable. After two years of free labor, Lasse granted Jia Tan co-maintainer status and release access.

The initial vulnerability wasn’t a buffer overflow. It was the economics. One unpaid volunteer maintaining critical infrastructure for billions of users. The attackers didn’t need a zero-day. They just needed patience.

Phase 2 - Hiding the Payload

Traditional malware injection is obvious: You add malicious code to .c source files. But Jia Tan knew code reviewers read source files. Even casual contributors glance at GitHub diffs.

Jia Tan’s solution was brilliant. Don’t hide the backdoor in the source code. Hide it where nobody looks.

The payload was concealed inside binary test files, specifically, .xz compressed blobs used for the test suite to verify the compressor was working. Developers assume test data is benign. Nobody reviews binary blobs. The malicious test files sat in the repository for months, completely invisible to the human eye.

Phase 3 - Build-Time Injection

The source code remained clean. Your Veracode or Snyk scanners found nothing suspicious in the .c files. But Jia Tan heavily obfuscated and modified the build scripts (configure.ac and Makefile.am).

During the make compilation process on a Debian or RPM build serve

1. The build script extracted the hidden payload from the binary test files.

2. It decrypted the payload and injected the backdoor directly into the compiled binary (liblzma.so).

The source code never contained the malware, it only existed in the build artifact.

Your build process is a black box.

You review code, but do you verify the compiled binary matches the source? If not, how would you detect a build-time injection?

Phase 4 - The IFUNC/GOT Hijacking

Once the malicious liblzma.so was loaded into memory by the SSH daemon, it executed a sophisticated Linux exploitation technique

IFUNC (Indirect Function) Resolvers and GOT (Global Offset Table) Hijacking.

Linux supports IFUNC, a mechanism where a function’s implementation is chosen dynamically at runtime based on CPU capabilities (e.g., choosing an AVX-512 optimized version if the hardware supports it).

The backdoor registered a malicious IFUNC resolver that executed during the dynamic linking phase, before main() even runs.

This resolver modified the Global Offset Table, replacing the pointer to OpenSSH’s RSA_public_decrypt() function with a pointer to the attacker’s own code.

The backdoor allowed remote code execution using a specific Ed448 cryptographic key controlled by the attacker. No valid SSH credentials were required.

It was entirely stealthy, it checked if the process was /usr/sbin/sshd so it wouldn’t crash normal tarball extractions, and it self-destructed if it detected profiling or debugging tools. (It only failed to hide from Freund’s profiler because of a tiny CPU cycle overhead bug).

Modern security assumes you can trust the build process. Code review, static analysis, and fuzzing all operate on source code.

But if the build itself is compromised, none of that matters. This was an attack on the software supply chain, not the software itself.

The Root Cause

Let’s shift from the technical to the economic. The brutal math of open-source maintenance is the actual root cause of CVE-2024-3094.

Let’s quantify what Lasse Collin was managing

• Impact - Billions of devices worldwide.

• Critical dependencies - OpenSSH, systemd, dpkg, rpm.

• Maintainer compensation - $0/year.

• Support burden - Thousands of emails, bug reports, and feature requests from corporations demanding free support.

The Fortune 500 runs mission-critical infrastructure on a foundation of unpaid labor.

• OpenSSL - Maintained by ~10 people with an Annual Budget of ~$1M (via grants) and used by Billions of users.

• curl - Maintained by Daniel Stenberg (mostly solo) with an annual budget of ~100k (sponsors) and used by Billions of users.

• SQLite - Maintained by D. Richard Hipp + 2 contractors with a Self-funded budget and used by Billions of users.

• XZ Utils (pre-attack) - Maintained by 1 unpaid volunteer with $0 budget and used by Billions of users.

The sock-puppet pressure campaign worked because Lasse was genuinely overwhelmed. The complaints about slow response times were real. “Helpful” contributors offered relief, and no one else (not Amazon, not Google, not Microsoft) was stepping up to help him. Handing off maintenance seemed like the responsible thing to do.

If XZ Utils disappeared tomorrow and you had to build a replacement in-house, how much would it cost?

A conservative estimate

3 senior engineers × 2 years × $200K = $1.2 million.

That is the real enterprise value Lasse Collin was providing. For free. While working a separate full-time job.

Companies will happily pay $50,000 annually for a Datadog license to monitor their infrastructure.

But they won’t pay $50,000 to sponsor the unpaid maintainer of the compression library that every server in their data center depends on. Until the maintainer burns out. Hands over access. And a nation-state backdoor ends up in production.

If your most critical open-source dependency disappeared tomorrow, how much would it cost to replace it? That’s what the maintainer is saving you. Are you paying them anything?

What you should do now?

The era of blind trust in npm install and apt-get upgrade is over.

Here is the framework for securing your supply chain Monday morning

Audit Your Dependency Tree

Don’t just look for known CVEs. Identify maintenance risk.

• Run dependency analysis (npm audit, pip-audit, cargo audit).

• Find dependencies with fewer than 3 active maintainers.

• Check the last commit date (>1 year without activity is a massive red flag).

• Red Flags - A single maintainer, hundreds of unresolved issues, or a maintainer posting “looking for new ownership.”

Pin and Vendor Critical Dependencies

Stop trusting the internet to compile your software.

Stop using latest or ^1.0.0 version ranges in production. Stop auto-updating dependencies without review.

Pin exact versions with SHA checksums. Use lock files (package-lock.json, Cargo.lock). For ultra-critical infrastructure libraries, vendor them (copy the source into your own repository) and build them from source.

Zero Trust for the Build Process

If the build is compromised, the code review doesn’t matter.

• Implement Reproducible Builds (the exact same source code must produce the exact same byte-for-byte binary).

• Isolate your build environments in ephemeral containers or VMs.

• Generate a Software Bill of Materials (SBOM) for every release.

• Verify that downloaded packages match published checksums and ensure no unexpected network requests happen during your CI/CD pipeline.

Fund the Maintainers

Funding maintainers isn’t charity. It’s cheaper than breach response, cheaper than in-house development, and cheaper than explaining to your board why you ignored supply chain risk.

• If a library is critical to your business, sponsor it via GitHub Sponsors, Tidelift, or Open Collective.

• The Goal - $50K–$100K/year for infrastructure-level libraries.

• This reduces maintainer burnout, creates accountability, and gives you input on roadmap and priority bug fixes.

Treat Open Source Like Vendors

Apply the exact same due diligence to your open-source libraries that you apply to a SaaS vendor. Monitor dependencies like production services. Alert on new releases, review changelogs before upgrading, and test updates in staging.

You cannot audit every line of every dependency. You have 500+ transitive dependencies, and they change constantly. But you CAN control your build process, fund the maintainers of critical libraries, and treat “free” software as unpaid labor with systemic risk.

Conclusion

The XZ backdoor wasn’t a failure of code. It was a failure of economics.

Jia Tan didn’t find a zero-day or exploit a race condition. He exploited the fact that trillion-dollar companies depend on unpaid volunteers, and then act surprised when those volunteers burn out and hand over the keys.

Linus’s Law ”with enough eyeballs, all bugs are shallow” assumes the eyeballs are actually looking. But nobody reviews compression libraries. Nobody audits build scripts. Nobody funds the maintainer. Until something breaks.

The next time your CFO questions a $500,000 budget for open-source sponsorships, show them the XZ timeline.

Show them the 2.5 years of patient infiltration. Show them that one burnt-out maintainer was all that stood between production systems and a global, nation-state backdoor.

“Free” software isn’t free. You’re just deferring the payment. The only question is

Will you pay before the breach, or after?

Every dependency in your stack is either funded, or it’s a countdown timer.

Credit to Veritasium for their exceptional video breakdown of the Jia Tan timeline, which served as the foundation for this architectural teardown. Watch it here

Every CTO has a slide in their pitch deck that says: “We will just auto-scale.”
It is the lie we tell investors to hide the fact that we haven’t thought about physics.

To test the limits of modern cloud infrastructure, I recently provisioned a monster. I spun up an AWS c8g.48xlarge, the “Beast.”

• 192 vCPUs (ARM-based Graviton).

• 384 GB of DDR5 RAM.

• 50 Gbps baseline bandwidth.

The goal?

1,000,000 Requests Per Second (RPS) on a single instance.

The result? We hit it.

But the bottleneck wasn’t the code logic. It wasn’t the database.

It was the AWS billing department.

The Compute Tax (Node.js vs. C++)

We started with the industry standard, Node.js.

Node is famous for its non-blocking I/O. It’s supposed to be fast.

But when you throw 600,000 concurrent connections at it, the V8 engine starts to cough.

The Node.js Ceiling (~500k RPS)

Even with 192 cores, our Node cluster capped out at roughly 500,000 requests per second.

Why? Overhead.

At this volume, the cost of creating a JavaScript object, managing the closure scope, and running the Garbage Collector becomes heavier than the actual request logic. The CPU wasn’t processing user data; it was managing V8 memory pointers.

The C++ Fix (1M RPS)

To unlock the hardware, we had to strip away the runtime.

We rewrote the service in C++ using the Drogon framework.

Drogon is brutal. It maps threads directly to cores. It has zero garbage collection overhead.

The result, 1.2 Million RPS at 80% CPU utilization.

This forces a massive economic trade-off.

• Option A (Node.js) - You pay for 2x the hardware (two c8g.48xlarge servers) but you can hire JavaScript developers for $120k/year.

• Option B (C++) - You pay for 1x hardware, but you need C++ systems engineers who cost $200k/year and take 6 months to hire.

At small scale, hardware is cheap.

At massive scale, hardware is expensive.

Pick your poison.

Disk vs. RAM

Hitting the endpoint is easy. Saving the data is a suicide mission.

Try executing INSERT INTO logs 1,000,000 times per second on Postgres.

We tried writing to a Provisioned IOPS SSD (EBS io2 Block Express).

The disk latency spiked immediately.

The queue depth exploded. The SSD simply couldn't physically accept the write signals fast enough.

The database locked up.

To provision the 250,000+ IOPS required to even attempt this would cost roughly $30,000/month for a single volume.

The only way to absorb 1M RPS is to never touch the disk. We switched to a Redis Cluster.

RAM is orders of magnitude faster than NVMe. We could ingest the traffic easily.

The major trade-off was durability.

If the power fails on that rack in the data center, the last few seconds of data, potentially millions of transactions, are gone forever.

You are trading $25,000/month in EBS costs for the risk of data loss.

Is your data worth that much?

Bandwidth

We fixed the CPU. We fixed the Storage.

Then we got the bill for the network cable.

During a 30-minute stress test, the “Beast” moved roughly 60 Terabytes of data.

(1M RPS × 30KB payload × 1800 seconds ≈ 54 TB).

Public Traffic vs. Internal Traffic

The cost depends entirely on where the data goes.

Internal VPC (Inter-AZ, the test)

60,000 GB × $0.02 = $1,200 for the 30-minute test.

Public Internet (The Reality)

If these were real users downloading data over the internet, you pay AWS standard egress rates which is ~$0.09 per GB

60,000 GB × $0.09 = $5,400.

That is $5,400 for a single 30-minute test.

• 60,000 GB × $0.09 = $5,400.

• That is $5,400 for a 30-minute test.

If you sustain 1M RPS for a month?

The cost will be ~$7.9 Million/month (based on 730 hours).

This is a huge cost for the Bandwidth alone.

You can optimize your C++ code all you want.

You cannot optimize the size of a byte. If your business model involves high-throughput data transfer (video, logging, telemetry), the CPU cost is a rounding error.

The bandwidth will bankrupt you.

The Price of 1 Million Req /Sec

Let’s look at the actual receipt for this “experiment.”

The “Beast” Server (c8g.48xlarge)

• $10.89 per hour. (Cheap!)

The Load Generator Fleet

• It took 60 c5.large instances just to generate enough traffic to crash the Beast.

• $40.00 per hour.

The Failed Storage

• We provisioned a high-IOPS EBS volume for 2 hours before giving up. $200 (pro-rated).

The Bandwidth

• Internal Scenario (Inter-AZ)

  • Even moving data between zones costs money.

  • $1,200 (30 mins) × 2 = $2,400/hr.

• Public Scenario (Internet)

  • $5,400 (30 mins) × 2 = $10,800/hr.

Total Cost Summary (1 Hour)

• Total Cost (Internal Test) - $11 + $40 + $200 + $2,400 = ~$2,651.

• Total Cost (Public Internet) - $11 + $40 + $200 + $10,800 = ~$11,051.

  • This is estimated as we don’t have such money lying around to do the actual test.

To calculate a sustained month cost we can use the below formula

(Cost of Bandwidth + Cost of Compute) X 730 hours

Sustained Monthly Cost (Internet)

• AWS - ($10,800 bandwidth + $51 compute) × 730 hours = ~$7,921,000/month.

• GCP - (~$9,600 bandwidth + $51 compute) × 730 hours = ~$7,045,000/month. (Est. $0.08/GB egress)

• Azure - (~$9,600 bandwidth + $51 compute) × 730 hours = ~$7,045,000/month. (Est. $0.08/GB egress)

• Bare Metal - ~$140,000/month. (See breakdown below)

Bare Metal consideration

To achieve this without the “Cloud Tax,” you move to wholesale IP Transit.

Traffic Load

120TB/hour converts to roughly 266 Gbps sustained throughput.

Bandwidth Cost

~$0.50 per Mbps - Wholesale IP Transit (e.g., Cogent, Hurricane Electric) rough cost at this volume.

• 266,000 Mbps × $0.50 = $133,000/month.

Hardware (CapEx)

• 1x High-End Server (Dual AMD EPYC 96-core, 384GB RAM, 2x 100GbE NICs): ~$25,000 (one-time).

• ~$700/month - Spread over 3 years.

Colocation Fee

• ~$2,000/month - Rack space, power (2kW), and cross-connects.

Conclusion

1 Million Requests Per Second is a vanity metric.

Technically, it is solvable with C++, RAM, and big iron.

Economically, it is a different beast entirely.

If you are building a system, do not ask “Can we handle 1M RPS?”

Ask, “Is the user paying us enough to cover the $0.09/GB egress tax on 1M RPS?”

If the answer is no, stop optimizing your code and start optimizing your business model.

In the history of software engineering, there is no story more tragic than that of Atom.

In 2014, GitHub unveiled Atom to the world with a tagline that promised utopia, “A hackable text editor for the 21st Century.”

To realize this vision, the GitHub engineering team performed a miracle.

They built a runtime that allowed developers to create desktop applications using the web stack (HTML, CSS, JavaScript). They called it Atom Shell.

Today we know it as Electron.

Atom didn’t just build an editor, they built the foundation for the next decade of software.

Slack, Discord, Figma, they all stand on the shoulders of Atom Shell.

But in a twist of cruel irony, a sleeping giant in Redmond picked up this exact tool, identified the one fatal flaw Atom refused to fix, and used it to build Visual Studio Code.

Atom provided the gun, the bullets, and the blueprint.

Microsoft simply pulled the trigger.

The Single-Threaded Trap

To understand why Atom died, we must look past the “bloat” complaints and analyze the Concurrency Model.

Atom’s core philosophy was Total Freedom.

They wanted the editor to be truly hackable.

This meant that a plugin developer could access everything.

They had direct access to the DOM. They shared the global scope.

Crucially, they ran on the Main Thread.

In a single-threaded environment like JavaScript, the Main Thread is holy ground.

It is responsible for event loops, layout calculations, and painting pixels.

By allowing plugins to inhabit this space, Atom created a zero-sum game for CPU cycles.

You install a “Bracket Matcher” plugin.

You type a bracket. The plugin wakes up to find the matching pair. It runs a synchronous search that takes 50ms.

For those 50ms, the UI renderer is blocked. The cursor stops blinking. The character you typed hangs in the buffer.

The “heaviness” users complained about was not memory usage, it was Thread Contention.

Atom relied on the “goodwill” of plugin authors to write non-blocking code. It was an architectural gamble that assumed developers would be disciplined.

They were not.

VS Code’s Architecture (Extension Host)

When Microsoft architected VS Code, led by Erich Gamma (one of the “Gang of Four”), they looked at Atom’s freedom and rejected it.

They prioritized Isolation over Hackability.

They introduced the Extension Host Process.

In VS Code, the editor UI runs in the Main Process and all the extensions run in a completely separate Node.js process.

There is a hard, impassable barrier between them.

• An extension cannot access the DOM.

• An extension cannot access the window object.

• An extension cannot freeze the renderer.

If a VS Code extension enters an infinite loop or tries to mine crypto, the Extension Host might crash, but the user can still type. The cursor still blinks.

The “Hot Path” (the keystroke loop) is protected at all costs.

RPC vs. Shared State

This decision created a massive engineering trade-off.

Atom’s Model (Shared State)

Latency was near zero. A plugin can read the document buffer instantly from memory.

But it has volatile stability. One bad plugin kills the app.

VS Code’s Model (RPC)

Latency was high overhead. Since the plugin is in a different process, data must be serialized into JSON, sent over an IPC (Inter-Process Communication) pipe, deserialized, processed, reserialized, and sent back.

But it guaranteed stability. App will keep running even a bad plugin is installed.

Why constraints won?

VS Code bet that Input Latency was the only metric that mattered.

Users can tolerate a 100ms delay in syntax highlighting appearing.

But, they cannot tolerate a 10ms delay in a character appearing on screen.

By forcing communication through Asynchronous RPC, VS Code enforced “Mechanical Sympathy.” It acknowledged the reality of the JavaScript event loop and refused to fight it.

Atom tried to pretend the Main Thread was infinite.

Constraints are Features

The downfall of Atom teaches us a critical lesson in System Design

Constraints are often more valuable than capabilities.

Atom failed because it offered Total Freedom.

It gave developers a foot-gun, and developers proceeded to shoot the performance in the foot.

VS Code succeeded because it imposed Strict Constraints.

It built a sandbox that prevented developers from hurting the user experience, even if they wanted to.

Conclusion

In 2022, GitHub (now owned by Microsoft) officially sunset Atom.

Atom did not die in vain. It was the martyr of the modern web desktop.

It proved the hypothesis that Electron could work.

It served as the glorious, messy prototype that paved the way for the production-grade architecture of VS Code.

Atom died so the ecosystem could live. And in the ruthless history of engineering, the prototype always gets deprecated.

We love indexes. They are the first tool we reach for when a query is slow.

A SELECT takes 4 seconds?

Add an index. Problem solved. Response time drops to 12 milliseconds.

You feel like a hero.

So you add another. And another.

A composite index for that dashboard query.

A partial index for that admin report.

A unique index on that email column.

Before you know it, your table has 12 indexes and your insert latency has quietly tripled.

Nobody notices. Not yet.

This article is about the other side of the indexing story, the side that almost nobody talks about.

Not "how to create the right index" but "how your indexes are silently destroying your write performance."

What happens when you INSERT a row?

To understand why over-indexing is dangerous, you need to understand what the database actually does when you insert a single row.

On a table with zero indexes (a heap table), an INSERT is almost trivially cheap. The database finds a page with free space, writes the row, and it is done. This is mostly a sequential memory operation. It is fast.

Now add a Primary Key.

That key is backed by a B-Tree index. The database must now also navigate that B-Tree, find the correct leaf page, and insert the key. If the leaf page is full, the database performs a page split, it allocates a new page, redistributes half the entries, and updates the parent node pointers.

This is the same mechanics I described in my UUID v4 article.

Now add a second index. And a third.

For every single INSERT, the database must repeat this B-Tree navigation and insertion process once per index. Five indexes means five separate B-Tree insertions for every row you write.

The critical insight is this, the database cannot skip any of them. Even if an index exists only to serve a query that runs once a month, it must be updated on every single write.

The Multiplier Effect

Markus Winand, author of SQL Performance Explained and the site use-the-index-luke.com, published a benchmark that should be pinned to every engineer's wall

“A table with zero indexes can be 100x faster to insert into than a table with a single index. And each additional index degrades insert performance further in a roughly linear fashion.”

Think about that.

You go from zero indexes to one, and your insert speed drops by two orders of magnitude. The actual data write is trivial. The index maintenance is the expensive part.

Now consider what happens in production. You have a table with 8 indexes. You deploy 10 application instances.

Each instance is doing 500 writes per second to that table. Every write triggers 8 separate B-Tree insertions.

That is 40,000 index operations per second hitting your database.

And when you wonder why your RDS CPU is at 85%, nobody thinks to look at the index count.

It costs beyond Write Latency

The write speed penalty is only the beginning. Over-indexing creates a cascade of secondary problems that are even harder to diagnose.

WAL Amplification.

In PostgreSQL, every change to an index page generates Write-Ahead Log (WAL) entries. More indexes means more WAL data.

That WAL is shipped to your replicas. It is archived for point-in-time recovery. It is processed by your backup systems.

Unnecessary indexes do not just slow down your primary, they increase replication lag, inflate your backup storage, and slow down disaster recovery.

Buffer Pool Pressure

Every index occupies pages in the shared buffer pool (PostgreSQL) or InnoDB buffer pool (MySQL).

Indexes you never query still compete for memory with the indexes you actually need.

When your buffer pool is full of cold, unused index pages, the hot pages that serve your real queries get evicted.

This causes more disk I/O for the queries you are trying to make fast.

The irony, an unused index can make your other indexes slower.

Vacuum and Maintenance Overhead

In PostgreSQL, VACUUM must process every index on a table to clean up dead tuples.

A table with 2 indexes vacuums twice as fast as a table with 8.

If autovacuum falls behind because it is spending all its time maintaining unnecessary indexes, you get table bloat, dead rows pile up, the table grows on disk, and sequential scans (which ignore indexes entirely) get slower and slower.

Query Planner Confusion

Every index gives the query planner another option to consider.

With many indexes, the planner spends more time evaluating plans, and the chance of it choosing a suboptimal plan increases.

PostgreSQL's planner has been measured to slow down measurably when a table has a large number of indexes, especially without prepared statements.

How Over-Indexing happens

We are not doing it intentionally for sure.

Nobody sets out to over-index a table. It happens gradually, through a series of individually reasonable decisions.

The "Just Add an Index" Reflex

A query is slow.

Someone runs EXPLAIN ANALYZE, sees a sequential scan, and creates an index.

The query gets faster.

Nobody checks if an existing index already covers the query, or if the query itself could be restructured.

Composite Index Duplication

You have an index on (user_id, created_at).

Someone creates a new index on (user_id) because their query only filters by user_id.

But a composite index on (user_id, created_at) already supports lookups by user_id alone, it is a "leftmost prefix" index.

The new index is pure waste.

Abandoned Feature Indexes

A feature was built six months ago.

It had specific queries that needed indexes. The feature was removed or rewritten.

The indexes stayed. Nobody went back to clean them up.

The codebase forgot. The database remembers.

ORM-Generated Indexes

ORMs like Sequlize, Django, Rails, and Hibernate often generate indexes automatically on every foreign key, on every column used in a unique_together, sometimes on columns that are barely queried.

Engineers trust the ORM defaults without auditing them.

Let me tell you a story.

The story (it’s not a fiction though)

I once worked with a SaaS company that had an events table tracking user activity.

The table received about 3,000 inserts per second during peak hours.

Over two years, different teams had added indexes for various reporting queries, admin dashboards, and one-off data exports.

The table had 14 indexes.

The symptom, insert latency would spike to 800ms during peak traffic, triggering timeouts in the application layer.

They had already upgraded the RDS instance twice. They were looking at sharding.

I ran a simple query against pg_stat_user_indexes

SELECT
    schemaname || '.' || relname AS table,
    indexrelname AS index,
    idx_scan AS times_used,
    pg_size_pretty(pg_relation_size(indexrelid)) AS size
FROM pg_stat_user_indexes
WHERE relname = 'events'
ORDER BY idx_scan ASC;

Six of the fourteen indexes had zero scans since the last statistics reset (which was three months prior).

Two more had fewer than 10 scans total, they were backing queries that had been removed from the codebase months ago.

The total size of those 8 unused indexes was 11 GB.

Sitting in memory. Being updated 3,000 times per second.

Generating WAL. Slowing down vacuum.

Contributing nothing.

We dropped them. Insert latency fell from 800ms peaks to under 50ms.

CPU dropped from 80% to 35%. They cancelled the sharding project.

The fix was not adding something. It was removing something.

How to Find and Kill Unused Indexes

Here is a practical process for auditing your indexes in PostgreSQL.

Step 1 - Check When Stats Were Last Reset

Before trusting scan counts, verify that your statistics have been accumulating long enough to be meaningful.

SELECT
    stats_reset,
    age(now(), stats_reset) AS stats_age
FROM pg_stat_database
WHERE datname = current_database();

If the stats were reset yesterday, the scan counts are not reliable. Wait at least one full business cycle (ideally a month) before making decisions.

Step 2 - List All Indexes by Usage

SELECT
    schemaname || '.' || relname AS table,
    indexrelname AS index,
    idx_scan AS scans,
    pg_size_pretty(pg_relation_size(indexrelid)) AS size,
    pg_size_pretty(pg_total_relation_size(relid)) AS table_total_size
FROM pg_stat_user_indexes
ORDER BY idx_scan ASC;

Anything with idx_scan = 0 over a multi-week window is a candidate for removal.

Anything with very low scan counts (single digits) should be investigated.

Step 3 - Check for Duplicate and Overlapping Indexes

A composite index on (a, b) makes a standalone index on (a) redundant for most queries. Look for these overlaps.

SELECT
    pg_size_pretty(sum(pg_relation_size(idx))::bigint) AS total_size,
    (array_agg(idx))[1] AS index_1,
    (array_agg(idx))[2] AS index_2
FROM (
    SELECT
        indexrelid::regclass AS idx,
        (indrelid::regclass)::text AS table_name,
        (indkey::text) AS columns
    FROM pg_index
) sub
GROUP BY table_name, columns
HAVING count(*) > 1;

Step 4 - Drop with a Safety Net

Do not drop indexes directly in production. First, make them invisible to the planner (PostgreSQL 17+)

-- PostgreSQL 17+ only
ALTER INDEX idx_events_legacy SET (deduplicate_items = off);
-- Or simply: monitor after dropping

For older PostgreSQL versions, the safest approach is, drop the index, monitor for one to two weeks, and keep the CREATE INDEX statement saved so you can recreate it if a query regresses.

-- Save the DDL first
SELECT pg_get_indexdef(indexrelid)
FROM pg_index
WHERE indexrelid = 'idx_events_legacy'::regclass;

-- Then drop
DROP INDEX CONCURRENTLY idx_events_legacy;

Always use DROP INDEX CONCURRENTLY to avoid locking the table.

The Rule of Thumb

There is no magic number for how many indexes a table should have. But there is a principle

Every index must justify its existence against the write cost it imposes.

If you cannot point to a specific, frequently-executed query that needs an index, that index should not exist.

"We might need it someday" is not a justification.

Creating an index takes seconds. The cost of maintaining a useless one runs 24 hours a day, 365 days a year.

For write-heavy tables (logging, events, analytics, time-series), aim for the absolute minimum, a primary key and perhaps one or two carefully chosen indexes.

For read-heavy tables with infrequent writes (reference data, configuration, product catalogs), you can afford more indexes.

The question is never "Will this index make my reads faster?" The answer is almost always yes.

The real question is "Is the read improvement worth the write cost across every insert, update, and delete this table will ever process?"

Conclusion

Indexes are not free.

Every index you create is a standing instruction to the database, "Every time any row in this table changes, do extra work."

Most engineers treat indexes as read-side optimizations and forget they are write-side penalties.

The next time you reach for CREATE INDEX, stop and ask three questions

1. Is there already an index that covers this query? Check for leftmost-prefix matches on existing composite indexes.

2. How often does this query actually run? If it runs once a day, you probably do not need an index. If it runs 10,000 times a second, you probably do.

3. How write-heavy is this table? A table receiving thousands of inserts per second pays a much higher tax per index than a table that changes once an hour.

And once a quarter, run the audit queries above.

Find your dead indexes.

Drop them.

Your database will thank you with lower CPU, faster writes, less replication lag, and a smaller storage bill.

Stop hoarding indexes. Start earning them.

To a user like us, launching an application is a singular atomic event.

We double-click the App icon, and App appears. Simple.

To a Windows kernel engineer, this makes perfect sense.

You call CreateProcess(). You pass it the path to the executable, some flags, and the OS allocates memory, loads the .exe file from the disk, and sets the instruction pointer to the entry point.

It creates a brand new thing from scratch. It is sane. It is logical.

To a Linux kernel engineer, this is boring.

In the Unix world, creating a new process is not an act of construction, it is an act of biological division followed by a brain transplant.

We don’t just “start” a program. We clone the current program, and then we lobotomize the clone.

Enter the fork() and execve() dance.

fork()

In Linux, there is (almost) only one way to create a new process, you have to duplicate an existing one.

When your shell (bash/zsh) wants to run ls, it calls the system call fork().
This function takes zero arguments.

pid_t pid = fork();

When the CPU executes this instruction, the kernel performs Mitosis. It creates an exact duplicate of the current process.

• Memory - It copies the Stack, the Heap, and the Global variables (Logically, via Copy-on-Write).

• File Descriptors - It copies the table of open files.

• CPU State - It copies the registers and, crucially, the Instruction Pointer.

The “Multiverse” Moment

This is the part that breaks the brain of every Computer Science 101 student.

When fork() returns, it returns twice.

• It returns once in the original process (the Parent).

• It returns simultaneously in the new process (the Child).

Both processes are now at the exact same line of code. They are identical.

The only way the code knows “Who am I?” is by checking the return value.

if (pid == 0) {
    // I am the Child (The Clone)
} else {
    // I am the Parent (The Shell)
    // 'pid' holds the ID of my new child
}

execve()

At this exact moment, we have two copies of the Shell running. We don’t want two shells, we want our App.

This is where the child process performs the sacrifice. It calls execve() (or one of its wrappers like execvp).

// Inside the Child process
execve("/usr/bin/app", args, env);

This syscall is destructive.

1. Wipe - The kernel discards the entire memory address space of the process. The stack, heap, and old code are annihilated.

2. Load - The kernel maps the new executable (app) into memory.

3. Reset - The Instruction Pointer is reset to the entry point of the new binary.

The Process ID (PID) remains the same.

The file descriptors (mostly) remain open. But the “soul” of the process has been swapped out.

The code that called execve creates its own executioner, it never executes the line after execve because it no longer exists.

Windows vs. Linux

Why does Linux do this weird two-step dance instead of the Windows one-shot approach?

The Monolith (CreateProcess)

Windows treats process creation as a massive configuration task.

The CreateProcess API takes 10 parameters.

If you want to customize how the new process starts (e.g., redirecting Standard Output to a file), you have to fill out complex STARTUPINFO structures and pass them in.

It is efficient for the OS, but it creates a rigid API surface.

The Primitives (fork + exec)

By splitting creation (fork) from execution (exec), Unix gave us a magical “Interim State.”

Between the fork() and the exec(), the Child process is running, but it’s still running the shell’s code.

This allows the child to configure itself before it turns into the new program.

This is how Redirection works.

When you run ls > output.txt, the shell

1. Forks.

2. Inside the child (before exec): It closes stdout and opens output.txt in its place.

3. Execs ls.

The ls program doesn’t know about output.txt. It just writes to file descriptor 1. It works because the “Clone” adjusted its own plumbing before the brain transplant.

The “Tree” Consequence

Because every process must be spawned by an existing process, Linux has a strict Process Tree.

• Process A spawns B.

• B spawns C.

• Trace it all the way back, and you find PID 1 (systemd or init).

If you want to view the Process tree use pstree -c in your linux terminal.

In Windows, processes act more like independent objects. In Linux, they are a family line, traceable to a single ancestor.

Why we engineers should care?

You might be thinking, “I write Python/Node/Go. Why do I care about syscalls?”

Because this mechanism explains bugs that high-level abstractions cannot.

The Redis Latency Spike

Redis uses fork() to create background snapshots (RDB).

It relies on “Copy-on-Write” (CoW) so it doesn’t actually double RAM usage instantly.

However, fork() must copy the Page Tables.

If your Redis instance uses 60GB of RAM, the kernel pauses the main thread just to clone the page table mapping.

This causes the infamous “latency spike” in large Redis instances. It’s not the data copying; it’s the fork overhead itself.

The Thread Safety Trap

If your application is multi-threaded and you call fork(), the kernel only copies the thread that called fork.

The other threads vanish in the child.

The Horror, if a vanished thread was holding a mutex (lock) when fork() happened, that mutex is now locked forever in the child process, owned by a ghost thread that doesn’t exist.

If the child tries to acquire that lock (e.g., calling malloc), it will deadlock instantly.

The Zombie Leak

If you spawn a child process but never read its exit code (via waitpid), the kernel keeps the process entry alive in the process table so you can read the status later.

If you forget to do this, your server fills up with “Zombie” processes (defunct), eventually hitting the system-wide process limit and crashing.

Conclusion

High-level languages try to hide this from you.

Python’s subprocess.run or Node’s child_process.spawn are just wrappers.

Even modern POSIX introduced posix_spawn to mimic the Windows style for performance reasons (avoiding the overhead of copying page tables for a fork that immediately execs).

But underneath your abstractions, the machine is still doing the dance.

• Windows treats processes as Objects to be instantiated.

• Linux treats processes as Life to be evolved.

Next time you see a “Zombie Process” in top, remember, it’s not a bug. It’s a child waiting for its parent to acknowledge its death.

That’s just nature.

Last week, we talked about “Career Survival.” Survival isn’t just about writing code; it’s about leverage.

And right now, your leverage is bleeding out in the comments section of a Pull Request.

Here is a scenario I see in high-growth startups every day

1. A Senior Engineer opens a PR.

2. Another Senior Engineer stops their deep work, pays the cognitive tax of a context switch, and opens the diff.

3. They spend 15 minutes debating camelCase vs snake_case or pointing out a missing semicolon.

Let’s do the math.

A Principal Engineer’s fully loaded cost (salary, equity, benefits, overhead) is easily $200+ per hour.

Add the cost of the context switch, which research shows takes ~23 minutes to recover from, and that comment about a variable name just cost the company $100 to $500 in lost productivity.

If you are a human being and you are commenting on syntax, you are wasting the company’s money.

You are doing a job that a script does better, faster, and for free.

The Lint Barrier

I have a strict rule for my teams

If a machine can catch it, a human shouldn’t see it.

A Pull Request should never reach a human reviewer unless it has already passed a gauntlet of cold, unfeeling robots.

This is the Lint Barrier.

• ESLint/Pylint - Catches syntax errors and unused variables.

• Prettier/Black - Enforces formatting.

• Static Analysis - Catches potential type errors.

If a PR fails any of these, I don’t want a notification. I don’t want to see it.

The “Nitpick” Ban

“Nits” (minor, non-blocking suggestions) are poison.

They clutter the feedback loop. They make the author feel defensive.

Worst of all, they distract the reviewer from the structural flaws hiding in the logic.

When you flood a PR with 20 comments about indentation, you feel productive.

You aren’t. You’re just being a spellchecker.

The Hierarchy of Review

So, if you can’t complain about whitespace, what are you supposed to do?

You do the job we actually pay you for. You look for the things a computer cannot understand.

Your review should follow this hierarchy

1. Architecture and Design

• Does this fit our existing design patterns, or is it a bespoke snowflake?

• Is this over-engineered? (The “YAGNI” check).

• Does this introduce a circular dependency?

2. Data Integrity

• The Database Lock - Will this migration lock the users table for 10 minutes during peak traffic?

• Race Conditions - What happens if two requests hit this endpoint simultaneously?

• State Management - Are we mutating state that should be immutable?

3. Security

• Is there an IDOR (Insecure Direct Object Reference) vulnerability here?

• Are we sanitizing inputs?

• Are we logging PII (Personally Identifiable Information)?

4. Maintainability

• Will the next person understand this in 6 months?

• Is the business logic exposed, or is it buried in 5 layers of abstraction?

“LGTM” is not Lazy

There is a stigma that a fast “LGTM” (Looks Good To Me) implies laziness.

In a mature engineering organization, LGTM is a badge of honor.

It means the automated pipeline did the heavy lifting. It means the tests passed, the linter is happy, the style is compliant, and the architecture is sound.

It means the reviewer trusts the system.

The Bikeshedding Trap

Cyril Northcote Parkinson coined the “Law of Triviality” (Bikeshedding).

He observed that a committee would approve a $10 million nuclear power plant in 10 minutes but spend 45 minutes debating the color of the bike shed.

Why?

Because nuclear physics is hard, but everyone has an opinion on paint.

Engineering teams are the same.

We argue about tabs vs. spaces because understanding a distributed race condition is hard.

Syntax is easy. Do not fall into the trap of low-leverage debates.

Here what you should do

You want to stop wasting $500 on variable name discussions?

Implement this today

Agree Once, Then Never Again

• Sit the team down.

• Argue about the Linter Config for one hour.

• Pick a standard (Airbnb, Google, whatever).

• Commit that config file.

The discussion is now closed forever.

The Husky Hook

Add a pre-commit hook (using tools like Husky) that prevents a developer from even committing code that violates the style guide.

The “Style” Ban

Make it a team policy.

If a reviewer comments on code style in a PR, the reviewer is in the wrong, not the coder.

Update the linter instead.

Let’s start with a line of code I see in Pull Requests from developers who think they are “optimizing” memory usage.

const user = { 
    id: 1, 
    hugeData: new Array(1000000).fill('garbage') 
};

// "I'm freeing up RAM, boss!"
delete user.hugeData;

You think you just freed 8MB of RAM. You think the memory was released the moment that semicolon hit.

But, in reality, you didn’t free memory.

You just deleted a key from a hash map (and destroyed the Hidden Class optimization in the process, but that’s a rant for another day).

The massive array still exists in the Heap. It sits there, occupying space, waiting.

You are not the landlord of this memory. You are just a tenant. And the cleaning service, V8’s Garbage Collector (GC), doesn’t come on your command. It comes when it feels like it.

Mark and Sweep

Think of your application’s memory (the Heap) as a messy apartment.

You are the hoarders.

The Garbage Collector is your obsessive-compulsive roommate who cleans up after you.

V8 uses an algorithm primarily based on Mark and Sweep. Here is how your roommate decides what stays and what goes

The Roots (Entry Points)

The roommate starts at the front door. In V8 terms, these are the Roots.

• The global object (window or global).

• The current Call Stack (local variables in executing functions).

Reachability (The “Touch” Rule)

The roommate walks through the house touching everything they can reach.

1. They touch the Root (Global).

2. The Global object references a User object? They walk to the User object and touch it.

3. The User object references a Profile? They touch the Profile.

This is the concept of Reachability.

If there is a chain of references (a path) from the Root to an object, that object is “Reachable.” It is marked as “Alive.”

The Sweep (The Trash)

Once the roommate has traced every single path and marked every reachable item, they look at what’s left.

• That massive array you created inside a function that returned 5 minutes ago? No one is holding it.

• That object you set to null? The path is broken.

The roommate sweeps everything unmarked into the trash. Only now is the memory actually freed.

Generational Collection

If the roommate checked every single item in the house every time they cleaned, your app would be agonizingly slow.

To solve this, V8 divides the Heap into two neighborhoods, The Young Generation (Nursery) and the Old Generation.

Young Generation

This is where all new objects are born. It is small (usually 1 to 8 MB).

High turnover. Most objects die young (temporary variables, loop iterators).

The GC runs a “Scavenge” (Minor GC) here very frequently. It is fast because most things here are dead anyway. It flips the survivors into a clean space and wipes the rest.

The Old Generation

If an object survives two cycles of Scavenging in the Nursery, the GC says, “Okay, this thing is sticking around.” It gets promoted (moved) to the Old Generation.

This is the attic. It’s huge (can be gigabytes).

Cleanup here is expensive.

Stop-the-World Pauses

Here is the nightmare scenario.

You keep promoting garbage to the Old Generation. The attic gets full.

When the Old Generation is saturated, the Scavenger can’t handle it. V8 has to call in the heavy machinery: The Major GC (Mark-Compact).

To do this, V8 initiates a Stop-the-World pause.

It freezes your application. No click events. No server requests. No rendering.

The engine pauses execution to traverse the entire massive Old Heap, mark the living, and compact the memory to remove fragmentation.

If your user sees the UI stutter or your server drops a connection, it’s often because your roommate paused the world to clean up your mess.

The “Bad Tenant” Behavior

How do you saturate the Old Generation and cause these stutters? By being an unintentional hoarder.

The Forgotten Timer

function startTracking() {
    const heavyObject = new Array(1000000); // 8MB
    setInterval(() => {
        // This callback references heavyObject
        console.log(heavyObject.length); 
    }, 1000);
}

You ran startTracking() once. You forgot about it.

The setInterval references the callback. The callback references heavyObject (closure).

The result, a heavyObject is reachable from the Root (via the Timer). It survives the Nursery. It moves to Old Gen. It never dies.

The Detached DOM Node

You have a modal dialog.

let myModal = document.getElementById('modal');
document.body.removeChild(myModal);

You removed it from the DOM. The user can’t see it. The “Sweep” should pick it up, right?

Wrong.

The variable myModal still holds a reference to that DOM node in JavaScript memory. Even worse, if that DOM node has pointers to its children, you are keeping the entire tree alive in memory.

It is a “Detached Node”, a ghost haunting your heap.

So what to do?

Stop treating RAM like it’s infinite.

Use WeakMap and WeakRef

If you need to associate data with an object but don’t want that association to keep the object alive, use a WeakMap.

let user = { name: "Alice" };
const cache = new WeakMap();

cache.set(user, "Some Metadata");

user = null; // The reference is broken.

In a normal Map, cache would keep “Alice” alive.

In a WeakMap, the GC ignores the reference inside the map.

When user becomes unreachable elsewhere, the WeakMap entry is auto-deleted.

Inspect the Snapshots

Stop guessing. Open Chrome DevTools → Memory tab.

1. Take a Heap Snapshot.

2. Run your action (open/close the modal).

3. Force a GC (click the trash can icon).

4. Take a second Snapshot.

5. Compare.

If the Delta is positive and growing every time you repeat the action, you are leaking.

You are the bad tenant.

Clean up your room.

For the last ten years the industry has been obsessed with the myth of the Full Stack Engineer. Bootcamps sold the dream that you could learn React for three weeks and Node for three weeks and suddenly be a complete software engineer.

I am here to tell you that this era is ending.

If you describe yourself as a “Full Stack Developer” today you are putting a target on your back. You are positioning yourself as a generalist in a world that is rapidly automating general work. By trying to know a little bit of everything you are ensuring that you are not valuable at anything.

This is why the “Jack of All Trades” is about to become the “Master of None” and why specialization is the only safety net left.

The AI is becoming the Ultimate Junior Generalist

The primary reason the Full Stack role is dying is simple. Artificial Intelligence.

If your job consists of glueing a database to a frontend using standard frameworks you are in trouble. AI is starting to become exceptionally good at this. It can generate the boilerplate for a React form or a simple Express API in seconds.

The AI is the ultimate Full Stack Junior. It knows every framework. It knows every syntax. It can write the “glue code” faster than you ever will.

If your skillset is broad but shallow you are competing directly against a machine that works for free. The AI cannot yet debug a complex race condition in the Linux kernel. It cannot yet optimize a custom database engine for high frequency trading. But it can absolutely build a Todo App using the MERN stack. If that is your entire skillset you are a going to be replaced.

Abstraction is Career Debt

The rise of the Full Stack engineer coincided with the rise of massive abstraction. We have “Backend as a Service” and “Frontend as a Service” tools that hide all the complexity.

• You use Vercel so you do not need to understand servers.

• You use Supabase so you do not need to understand database administration.

• You use ORMs so you do not need to understand SQL.

Every layer of abstraction you use without understanding the layer below it is Technical Debt. But it is not debt for your job. It is debt for your career.

When you rely entirely on tools to do the hard work you are leasing your skills instead of owning them. When the abstraction leaks (and it always does) you are helpless. A specialist knows what happens when the request hits the metal. A generalist only knows how to call the API.

The T Shaped Lie

We often tell engineers to be T Shaped. This means having broad knowledge across many areas (the top of the T) and deep knowledge in one specific area (the vertical line of the T).

This is good advice but most people are lying to themselves.

Most “Full Stack” engineers are not T Shaped. They are Dashes. They have a thin layer of knowledge across the top but they have no vertical depth.

• They know how to use an Index but they do not know how a B Tree works.

• They know how to send an HTTP request but they do not know how a TCP handshake works.

Being a Dash is dangerous. In a tight job market companies do not hire Dashes. They hire experts who can solve the expensive burning problems that the generalists caused.

The Moat is Depth

So how do you survive.

You must pick a hard problem and go deep.

Stop trying to keep up with every new JavaScript framework released this week. That is a treadmill that goes nowhere. Instead pick a foundational technology and master it.

• Become a database internalist. Learn how storage engines write to disk.

• Become a networking specialist. Master the flow of packets and security protocols.

• Become a graphics engineer. Learn the math behind the pixels.

Depth cannot be prompted. AI relies on training data and the training data for deep complex edge cases is scarce. The generic code is abundant so the AI is good at it. The deep expert knowledge is rare so the AI is bad at it.

Your depth is your moat. Stop being Full Stack. Start being an Expert.

You see this code every day. You might have written it this morning.

const arr = [1, 2, 3];
// ... some logic happens ...
arr[100] = 4; // The crime

To a junior developer, this is the beauty of JavaScript, dynamic, flexible arrays that expand on demand. No malloc, no fixed sizes, no segmentation faults.

To a V8 engineer, this is a De-optimization Event.

By writing to index 100 of a 3-item array, you didn’t just add a number. You forced the engine to fundamentally alter the underlying storage structure of that object. You forced V8 to downgrade your high-performance memory block into a slow, lookup-heavy chain.

JavaScript arrays are not always arrays. Sometimes they are C-structs, sometimes they are Hash Maps. You control which one you get, often without realizing it.

Elements Kinds

V8 tracks the shape of arrays using a system called Elements Kinds. There are over 20 distinct kinds, but for performance tuning, we care about the war between PACKED and HOLEY.

The PACKED_SMI_ELEMENTS

When you initialize const arr = [1, 2, 3], V8 loves you. It detects that all elements are Small Integers (SMI) and the array is dense (no gaps).

Under the hood, this looks strikingly similar to a C++ std::vector or a raw C array.

• Memory - Contiguous.

• Access - Simple pointer arithmetic. Base Address + (Index * 4).

• Speed - O(1). Blazing fast.

This is PACKED_SMI_ELEMENTS. This is where you want to stay.

The Downgrade

When you execute arr[100] = 4, you create a “hole”, a massive gap from index 3 to 99 where no data exists. V8 cannot rely on pointer arithmetic anymore because reading memory at offset 50 would return garbage or uninitialized memory.

The engine must reclassify the array. It transitions from PACKED_SMI_ELEMENTS to HOLEY_SMI_ELEMENTS (or HOLEY_ELEMENTS if types are mixed).

This is the most critical concept to internalize: Elements Kinds transitions are generally irreversible.

Once an array becomes HOLEY, it stays HOLEY. Even if you backfill indices 3 through 99 later, V8 will not waste CPU cycles attempting to detect if the array is “healed” to switch it back to PACKED. You have permanently tainted the array’s performance profile for its entire lifecycle.

Why Holey is Slow?

Why does a hole matter? Why can’t the engine just return undefined for the gaps?

Because in JavaScript, undefined is a value. A hole is the absence of a property.

When you access arr[50] in a HOLEY array:

• V8 checks if index 50 exists on the array itself. It finds a hole.

• V8 cannot simply return undefined yet. It must walk up the Prototype Chain (Array.prototype, then Object.prototype) to ensure that index 50 isn’t defined as a getter or value somewhere higher up.

You turned a single memory fetch into a tree traversal.

DICTIONARY_MODE

The example above (arr[100] = 4) is bad, but manageable. However, if you do this

const arr = [1, 2, 3];
arr[100000] = 4; // Sparse Array

V8 looks at the memory waste required to allocate 100,000 empty slots for just two integers and decides to bail out entirely. It switches to DICTIONARY_MODE.

The array is no longer an array in memory. It becomes a Hash Map.

• Keys are converted to hashes.

• Access becomes significantly slower due to hash computation and collision handling.

• Iterating over this array is a performance disaster compared to a packed array.

The new Array()

I see this pattern in “optimized” codebases constantly

// I am pre-allocating memory for performance
const arr = new Array(100); 
arr[0] = 1;

Stop doing this.

new Array(100) creates an array with length: 100, but zero allocated slots. It is born HOLEY_SMI_ELEMENTS (or HOLEY_ELEMENTS). You have pre-allocated nothing but a promise of slowness.

How to fix that?

If you want a packed array, you must ensure it has no holes from birth.

// Use Literal (Best for small sets)
const arr = [1, 2, 3]; // PACKED_SMI_ELEMENTS

// Use Push (Best for Dynamic growth)
const arr = [];
for (let i = 0; i < 100; i++) {
  arr.push(i); // Remains PACKED as it grows
}

// Use Array.from (A cleaner syntax)
// Creates a PACKED array filled with initialized values
const arr = Array.from({ length: 100 }, (_, i) => i); 

Best Practices

As senior engineers, we optimize for the machine, not just the linter.

Never delete Array Elements

Using the delete operator on an array index creates a hole.

const arr = [1, 2, 3];
delete arr[1]; // Transitions to HOLEY_SMI_ELEMENTS
// arr is now [1, empty, 3]

Use .splice() if you need to remove it, or overwrite it with a value if you need to maintain the index.

undefined is better than a Hole

If you must “clear” a slot but keep the index, explicit undefined is superior to a hole.

const arr = [1, 2, 3];
arr[1] = undefined; 

This transitions the array to PACKED_ELEMENTS (generic packed). While generic packed is slower than PACKED_SMI (because it has to handle tagging/boxing), it is significantly faster than HOLEY because it skips the prototype chain walk.

Initialize Monomorphically

Don’t mix types if you can avoid it.

• [1, 2, 3] is PACKED_SMI (Fastest).

• [1, 2, 3.5] degrades to PACKED_DOUBLE (Fast, but requires conversion overhead).

• [1, 2, '3'] degrades to PACKED_ELEMENTS (Slower).

Treat your arrays like static memory blocks. Keep them contiguous, keep them typed, and never let V8 doubt the integrity of your indices.

Consider this line of code.

const point = { x: 1, y: 2 };

Ask a developer what this is and if they will tell you it is a dictionary or a hash map. They will tell you that x and y are string keys that map to values and looking them up requires a hash calculation.

They are wrong.

To the V8 engine (which powers Chrome and Node.js) treating this object as a hash map is a failure state.

A hash map is slow. It involves computing a hash handling collisions and chasing pointers.

V8 wants speed. It wants to access point.x with a single machine instruction just like a C++ struct.

To achieve this V8 relies on a hidden machinery called Shapes (often called Hidden Classes or Maps).

If you do not understand how Shapes work you are writing code that forces the engine to abandon its optimizations and fall back to the slow path.

This article will tear down the abstraction of the JavaScript Object and reveal the transition trees and inline caches that live underneath.

What is a Shape?

JavaScript is a dynamic language. You can add or remove properties at runtime.

C++ is a static language. The memory layout is fixed at compile time.

V8 is a bridge that tries to make the former behave like the latter.

When you create an object V8 does not just allocate a bucket for keys and values. It allocates a Shape.

A Shape is a metadata descriptor. It contains

1. The property names (e.g. “x”).

2. The attributes of those properties (writable enumerable etc.).

3. Crucially The Offset where the value is stored in memory.

The Transition Tree

V8 builds these Shapes dynamically as you mutate objects. Let us trace the creation of our point object.

const point = {};

V8 starts with the Root Shape (Empty).

point.x = 1;

V8 sees a new property x. It cannot fit this into the Root Shape. It creates a new Shape (Shape A) that says “Offset 0 contains x”. It links the Root Shape to Shape A via a Transition (If you are at Root and add ‘x’ go to Shape A).

point.y = 2;

V8 sees y. It transitions from Shape A to Shape B. Shape B says “Offset 0 is x, Offset 1 is y”.

Now when you access point.y V8 does not calculate the hash of the string “y”. It looks at the Shape sees that y is at Offset 1 and reads the memory directly.

This is near-native speed.

The Performance Killer (Polymorphism)

The system works perfectly as long as your objects share the same Shape. But developers often break this mechanism without realizing it.

Consider these two objects

// Path 1
const a = {};
a.x = 1;
a.y = 2;

// Path 2
const b = {};
b.y = 2;
b.x = 1;

To a developer a and b are identical. They both have x and y.

But to V8 they are completely different entities.

Object a followed the transition chain, Empty -> Add x -> Add y. It creates Shape ID 1.

Object b followed the transition chain, Empty -> Add y -> Add x. It creates Shape ID 2.

Because the properties were initialized in a different order the offsets are different. In a, x is at offset 0. In b, x is at offset 1.

The Inline Cache (IC) Fail

This matters because of the Inline Cache.

When V8 runs a function that reads obj.x it “learns” the Shape of the object passed to it.

function getX(obj) {
    return obj.x;
}

Monomorphic State (Fastest)

If you always pass objects with Shape ID 1 V8 compiles a hot path. It says “I know this shape. x is always at offset 0. Just load memory index 0.” It strips out all checks.

This is the Monomorphic state.

Polymorphic State (Slower)

If you pass a (Shape 1) and then b (Shape 2) V8 invalidates the optimized code. It switches to a Polymorphic state.

It generates code that looks like a switch statement

• “If Shape 1 load offset 0.”

• “If Shape 2 load offset 1.” This adds CPU cycles for branching and checking.

Megamorphic State (The Cliff)

If you pass objects with 5 or more different Shapes (e.g. created via random assignment loops) V8 gives up.

It marks the call site as Megamorphic.

It stops trying to optimize offsets.

It falls back to Dictionary Mode.

It effectively treats the object as a hash map and performs a slow lookup for every single access. Your performance just fell off a cliff.

Takeaway

Static Initialization is King.

You cannot control the V8 compiler but you can control the Shapes it generates.

Initialize Properties in the Same Order

Always write your keys in the same sequence, here is a bad example of the same

if (condition) { 
  obj.a = 1; 
  obj.b = 2; 
} else { 
  obj.b = 2; 
  obj.a = 1; 
}

The good way to ensuring the transition path is deterministic.

Use Object Literals or Constructor

Using

class Point { constructor(x,y) { this.x = x; this.y = y; } }

guarantees that every instance follows the exact same transition chain.

Piecemeal assignment (obj.a = ...) is risky.

Avoid delete

Using the delete keyword typically forces V8 to abandon the Shape optimization entirely and revert the object to Dictionary Mode because the transition tree does not support “removing” a step efficiently.

If you need to remove a value set it to null or undefined instead.

You are not writing JavaScript for a browser.

You are writing JavaScript for a JIT compiler.

Treat your objects like Structs and the engine will reward you with C++ performance.

Treat them like Hash Maps and you will pay the dynamic language tax.

We love UUIDs. They seduce us with a simple promise. You can generate a globally unique ID on a client device without talking to a database. You can merge records from ten different distributed systems without a single collision. They are stateless. They are easy.

But in the world of database design convenience usually comes with a tax.

When you make UUIDv4 your Primary Key you are making a decision that fights against the fundamental physics of how databases store data. You are choosing randomness. And in a B-Tree randomness is the enemy of locality.

This article will explain why your “lazy default” is causing your database to burn IOPS, trash its buffer pool and bloat your storage.

The B-Tree

To understand why UUID v4 is dangerous we have to look at the data structure that powers your database index the B-Tree (or B+Tree).

A B-Tree is designed for one specific goal Locality. It wants to store related data close together on the physical disk.

When you request Record 100 the database loads a “Page” (usually 8KB or 16KB) of data into memory. Because of locality that page likely also contains Record 99 and Record 101. This makes range scans and sequential reads blazing fast.

To maintain this locality the B-Tree must be Sorted.

The Sequential Insert

When you insert sequential IDs (like 1 2 3 or Timestamp) the database behaves like a logger. It simply adds new entries to the end of the file.

1. Locality - The database only needs to access the Rightmost page of the index. This page is “hot” (already in RAM).

2. Fill Factor - The database fills the page to 100% capacity, closes it, saves it to disk, and allocates a new empty page.

3. Result - Zero wasted space. Minimal disk movement.

This is the “Easy Mode” for any storage engine.

The Random Insert

Now imagine inserting a random UUID v4. a902... arrives.

This ID might have been generated on a user’s phone milliseconds ago or hours ago while offline. To the database time does not matter. Only the Value matters.

Even though this is a “new” record (inserted now) its Value (a902...) might be lexicographically smaller than records you created yesterday (e.g. b100...).

A B-Tree is strictly sorted by Value. The database cannot just append a902... to the end. It effectively says

“Oh based on alphabetical order this new book belongs on a specific shelf that I filled up and sealed three days ago.”

It forces the database to go back in time fetch that old “cold” page from the disk and try to shove the new data in. This triggers the mechanical disaster.

The Page Splitting Problem

The database fetches that old page from the disk into the Buffer Pool (memory). It tries to insert the new UUID.

The Problem, the page is full.

To make room for this random entry the database performs a Page Split.

1. It allocates a new empty page.

2. It takes half the records from the old page and moves them to the new page.

3. It inserts the new UUID into the middle.

4. It updates the parent pointer nodes to reflect the new path.

This has some consequences,

Write Amplification

You wanted to write 32 bytes of data. But to do it the database had to load a 16KB page create another 16KB page and write both back to disk.

You are burning CPU and I/O bandwidth just to move data around.

Buffer Pool Thrashing

In a sequential insert you only need the “latest” page in memory. In a random insert any page from the entire history of your database might be needed at any moment.

This defeats your caching strategy. Your Buffer Pool churns constantly evicting “hot” data to load “cold” pages just to insert a random ID.

Fragmentation and Disk Bloat

After a split you are left with two pages that are only 50% full. If you keep inserting randomly your pages effectively stay half empty.

This means your index takes up 2x more disk space than a sequential index. This is not just a storage cost issue.

It means your RAM can only hold half as many keys leading to even more disk reads.

UUID v4 vs BigInt vs UUID v7

UUIDv4 is the standard 128-bit random identifier. Its globally unique and required coordination.

But, destroys B-Tree performance. Fragments indexes. Clogs the WAL (Write Ahead Log) because page splits create massive log records.

So if UUID v4 is the villain who is the hero?

BigInt or Serial

The classic auto-incrementing integer.

Extremely fast sequential inserts. 64-bit size (half the size of a UUID).

But, it leaks business intelligence. If I create a user and get ID 105 and create another tomorrow and get 150 I know you only grew by 45 users.

It is also painful in active-active distributed systems where two servers might generate ID 106 at the same time.

UUID v7

Approved in 2024 this standard is designed specifically to solve the DB problem.

It embeds a 48-bit Timestamp at the start of the ID followed by random bits.

Because the start of the ID is time-based new records are generated in (mostly) increasing order.

To the B-Tree a UUID v7 looks like a sequential integer. It appends to the right. It fills pages completely. It avoids splitting. But it retains the distributed uniqueness and unguessability of a UUID.

I was once called in to consult for a logistics company. They had an event logging system that tracked packages. The system was simple. Every scan event got a UUID v4 Primary Key.

For the first year it was fine. Then they hit 50 Million Rows.

The Write Latency spiked from 2ms to 200ms. CPU usage on the RDS instance sat at 90%. They upgraded the instance class twice doubling their bill but the CPU didn’t drop.

After looking into the database memory I found, rhe dataset size had finally exceeded the available RAM (Buffer Pool).

When the table was small the entire B-Tree lived in memory. Random inserts were just RAM operations.

The moment the index grew larger than RAM every random insert became a Disk Read. To insert a record the DB had to fetch a random cold page from disk to memory.

For the fix, we couldn’t migrate the primary keys easily. Instead we introduced a composite sort key using a timestamp and rebuilt the clustered index to follow time rather than the UUID.

The UUID remained unique but it no longer dictated the physical storage order. Writes dropped back to 5ms. CPU dropped to 10%.

Conclusion

We are not guessing here. The benchmarks are clear. In tests inserting 100 million rows

• UUID v4 takes 2x to 3x longer than sequential IDs.

• UUID v4 indexes are 30% to 50% larger due to fragmentation.

• UUID v7 performance is nearly identical to BigInt sequential inserts.

So, stop using UUID v4 for database Primary Keys. It is a legacy default that ignores computer science fundamentals.

• If you need simple internal speed use BigInt.

• If you need distributed scale and public safety use UUID v7.

• Only use UUID v4 for ephemeral tokens or trace IDs that are never indexed.

Don’t let randomness kill your locality.