Writing code got Free but someone still has to Verify it

Generation got cheap. What stayed expensive was judgment, orchestration, context boundaries, and verification.

Aug 25, 2026

57 Bugs in AI-Generated Code: How I Verify Before Shipping | Chudi.dev

In a single 16-hour window in January 2026, seven security reports landed in Daniel Stenberg’s inbox.

Not one of them identified a real vulnerability.

Stenberg maintains cURL, the command-line tool that sits at the foundation of almost every connected software system on Earth. Since April 2019, curl’s bug bounty program on HackerOne had been a model for open-source security.

Over seven years, it paid out more than $100,000 to independent researchers across 87 verified vulnerabilities. Historically, better than 15% of incoming reports led to a confirmed fix.

By 2025, that confirmation rate collapsed below 5%.

In the first three weeks of 2026, cURL received 20 submissions. Zero were valid. Large language models had turned the labor of writing plausible-sounding security reports into a zero-cost operation. Stenberg called it “death by a thousand slops.”

His assessment was blunt

Not only the volume goes up, the quality goes down. So we spend more time than ever to get less out of it than ever.

On January 31, 2026, cURL officially shut down its HackerOne bug bounty.

Most commentators took this as a simple cautionary tale about AI ruining open source. They missed the real story. Three months later, the underlying math did something far more uncomfortable.

So what happened next?

In March 2026, just four weeks after shutting it down, Stenberg reopened cURL’s HackerOne bounty program.

It wasn’t because the submission traffic had died off. It hadn’t. What changed was the underlying model quality.

As reported by Cybernews in June 2026, the arrival of newer reasoning models suddenly shifted the nature of the incoming queue. The low-effort slop didn’t disappear, but it was joined by a wall of highly sophisticated, syntactically correct, deeply detailed vulnerability reports.

Stenberg watched the inbox fill up again

Something happened. The reports came back, and they did not slow down.

Here is the what actually happened, reopening the program didn’t fix Stenberg’s workload. It made it even heavier.

When the queue was filled with obvious garbage, a human maintainer could skim a report in 30 seconds and throw it out.

But when an AI generates a 20-page security teardown complete with reproduction scripts, edge-case memory maps, and pseudo-code fixes, you cannot dismiss it out of hand. A human senior engineer has to spend two hours stepping through gdb traces to verify whether the flaw is real or a hallucination.

The cost of producing the report fell to zero. The cost of auditing that report stayed exactly where it was in 1995.

This is the central asymmetry of the generative AI era. We collapsed the cost of creating technical artifacts, but we didn’t touch the cost of verifying them. The bottleneck didn’t vanish. It just shifted from generating the work to checking it.

What Industry reports say?

If this were isolated to open-source maintainers, it would be a manageable niche headache. It isn’t. The exact same gap is breaking application security at enterprise scale.

On August 1, 2026, security firm Veracode published its annual GenAI Code Security Report. They benchmarked over 100 frontier and open-weight models across four distinct testing cycles dating back to early 2025.

The findings put hard numbers on what every senior architect has felt intuitively

Syntax Pass Rate - Near 100%. Models generate code that compiles, runs, and passes basic unit tests without throwing errors.
Security Pass Rate - 56%.

That 56% figure is virtually unchanged from the 55% mark Veracode recorded in their initial 2025 benchmark.

Despite trillion-parameter scaling, massive context windows, and advanced reasoning loops, roughly 44% of AI-generated code tasks still introduce a known, exploitable vulnerability when generated without explicit security prompts.

Even the top-performing model in their Summer 2026 snapshot GPT-5.5 achieved only a 68% security pass rate. It still failed nearly one out of every three security tasks. For Java workloads, the mean security pass rate dropped to a dismal 30%.

Veracode’s report framed the problem directly

Models may be almost syntactically perfect, but they’re still failing on nearly half of all tasks where security is needed.

This structural blind spot is hitting production networks right now.

The 2026 Verizon Data Breach Investigations Report revealed that software vulnerabilities have officially passed stolen credentials as the number one entry point for enterprise data breaches, accounting for 31% of all confirmed security incidents.

Meanwhile, industry surveys (including data compiled across Veracode and GitClear) indicate that AI-generated pull requests carry roughly 1.7 times more defects than human-written code. Yet, only 18% of enterprises have instituted formal, mandatory AI code review policies.

We are shipping clean-looking code faster than ever before. We are checking it less than ever before.

How its affecting the Job Market?

When the cost of producing code drops while the cost of verifying it climbs, the job market shifts.

The broad media narrative claims that entry-level engineering is dead and AI has eliminated the talent shortage. The actual hiring data tells a completely different story.

According to ManpowerGroup’s 2026 Global Talent Shortage Survey which polled 39,063 employers across 41 countries roughly 72% of global tech employers still report severe difficulty filling technical roles.

For the first time in the survey’s history, traditional IT and software development skills dropped out of the top spot. They fell to seventh place globally (17% of employers reporting shortages).

What took their place at the absolute top of the global talent shortage list?

AI Model and Application Development (20%)
AI Literacy and Verification Capability (19%)

Jonas Prising, Chair and CEO of ManpowerGroup, noted

The rise of AI skills to the top of the shortage list reflects how quickly the talent landscape is evolving.

The market isn’t looking for people who can write code fast. Copilots do that for free. The market is desperately searching for engineers who possess the deep structural knowledge required to audit, verify, and architect systems around AI-generated code.

The hiring process itself is changing

Nowhere is the verification crisis more visible than in how companies hire.

In early 2025, a Columbia University student named Chungin Roy Lee built a lightweight tool called Interview Coder. The application sat hidden during live technical interviews, took real-time screen grabs of LeetCode style coding prompts, and fed back optimal solutions silently in the background.

Lee used the tool to clear technical loops and land job offers from Amazon, Meta, TikTok, and Capital One. Then he posted a video of himself using it live inside an Amazon interview loop.

Columbia suspended him. Amazon revoked his offer. Lee didn’t care. By April 2025, Interview Coder had crossed $3 million in annualized revenue, and Lee raised $5.3 million in venture funding to turn the engine into a broader stealth assistant named Cluely.

The standard technical interview, the algorithm puzzle on a shared screen was officially broken. Data from developer evaluation platform interviewing.io indicated that 81% of technical interviewers suspected candidates of using AI assistance during interviews, with roughly one-third catching a candidate outright.

Faced with a broken signal, the tech industry split into two distinct camps

Strict Interviews

Some legacy enterprises responded by doubling down on monitoring. They dragged candidates back into physical office rooms, enforced strict locked-down browser environments, and added grueling live follow-up interrogations.

They tried to preserve the old LeetCode signal by building bigger walls around it.

Different Verification Style

Other engineering organizations abandoned memory-based syntax tests entirely. Instead of asking a candidate to implement a binary tree inversion from scratch, they hand the candidate a messy, AI-generated pull request containing three subtle concurrency bugs and a memory leak.

The candidate’s score depends entirely on how fast they can spot the hallucinations, correct the underlying architecture, and guide the AI toward a secure implementation.

Initiatives like the open-source Hiring Without Whiteboards repository (now boasting over 50,000 stars on GitHub) reflect this reality. The industry is slowly realizing that testing whether a human can type code from memory is irrelevant when typing code is free.

The only metric that matters is whether the engineer can tell when the code is wrong.

Its all connected

If you have been following BinaryBox over the last few months, this pattern should feel intimately familiar. It is the underlying thread connecting our technical analyses across different domains

The AI Rehiring Panic - We established that AI easily automates the routine 94 percent of standard tasks, but exposes the remaining 6 percent of edge-case judgment as the true bottleneck holding up enterprise operations.

The Tokenpocalypse - We showed how unconstrained agentic execution loops multiply token consumption by 50x to 500x through context rot, proving that raw model generation is worthless without tight local orchestration and strict boundary controls.

Stop Treating MCPs as Plugins - We demonstrated that an AI assistant will output broken imports and duplicated code unless it is constrained by an NX monorepo pipeline that forces structural context into the session.

Every single one of these technical shifts stems from the exact same economic reality

Generation got cheap. What stayed expensive was judgment, orchestration, context boundaries, and verification.

Same mechanism. Five different costumes.

Conclusion

It is easy to look at the current landscape and assume senior engineers are immune to the changes sweeping through software development. That is too neat a conclusion.

Senior engineers who act purely as high-speed typists are just as exposed as junior developers. If your main value contribution is turning a Jira ticket into 200 lines of syntactically valid TypeScript, your margin is evaporating.

The cost of checking work was always present in software development. It was just invisible for forty years because checking code was tightly bound to the effort of writing it. You wrote a function line by line, thinking through the edge cases, memory allocations, and security boundaries as your fingers moved across the keyboard. Writing and checking happened simultaneously.

AI severed that link. It unbundled creation from verification.

It allowed anyone to generate 5,000 lines of syntactically flawless infrastructure-as-code in three seconds. But it left the cognitive burden of verifying that infrastructure entirely on human shoulders.

Daniel Stenberg didn’t shut down the curl bug bounty because AI couldn’t write security reports. He shut it down because AI wrote them so fast that human audit capacity collapsed under the weight.

Generation is now effectively free. Verification is exposed as the ultimate physical constraint of software engineering. Plan your career and your infrastructure accordingly.

Discussion about this post

Ready for more?