What Uber Didn't Trust Its Code-Deleting AI With · Issue #323

Uber’s Code-Deleting AI

Uber’s engineering blog recently introduced a system called AutoClean. It finds feature flags¹ that have outlived their usefulness, deletes the code tied to them, fixes whatever tests break in the process, and sends a fully verified diff² to the responsible team. Uber says that over the past several years, this system has cleaned up more than 16,000 flags and over 1,000,000 lines of code.

We’re all used to hearing about AI that writes code. This story runs the other way: AI deleting code that’s already live in production. But if you read the whole post, what stands out isn’t just what the agent accomplished — it’s how carefully it spells out what Uber chose not to hand over to the agent. Today I want to trace the shape of that division of labor.

Why does deleting a single flag require judgment

A feature flag is a switch built into the code. It lets you turn a new feature on for just some users, or turn it off without redeploying if something goes wrong. Uber reportedly manages thousands of flags across its mobile apps and backend services.

The problem starts after a feature is turned on for everyone. The switch stays on permanently, but the conditional branches wrapped around it, the old code path, and the tests that checked that path all stay in place. The team moves on to the next feature, and cleanup keeps losing out to new product work in the priority queue. At Uber’s scale, this kind of deferred small cleanup quickly piles up into real backlog.

The example Uber gives is order-processing code. It reads a new_order_pipeline flag value from config into cfg.UseNewPipeline, passes that value as an argument into the processOrder function, and inside the function branches between the new pipeline and the old one based on it. Say the flag has fully rolled out and its value is now always true.

Dropping true in at the point where the flag is read is mechanical work. The judgment starts after that.

You need to confirm that the useNewPipeline argument in processOrder is really always true. In the example there’s only one call site, but in real codebases other call sites might pass different values. Only after checking every call site can you remove the argument.
Only after removing the argument and pushing the value directly into the function does it become clear that the old pipeline branch is dead code. Looking at the function in isolation, both branches appear alive.
The UseNewPipeline field in the config struct might also be removable — but only if nothing else reads that field or depends on the struct’s shape.

The flag value starts at config initialization, flows through the struct field, the call site, and the function argument, and finally reaches the dead code. Uber’s writeup frames it this way: deterministic rewrites like splicing in a constant are something static tools handle well, while AI is most useful for tracing this propagation path and deciding how far the deletion should go. Tests carry the same problem — some can simply be deleted, others need to be rewritten to match the new default behavior.

Uber also notes that this debt doesn’t only burden humans. AI agents spend time reading and interpreting dead code too, which makes their sessions longer and more expensive. The piece doesn’t give a number for how much longer, though.

Boolean logic got handed off to a tool

AutoClean finds stale flags in Uber’s feature-flag management system and processes them in three stages: it rewrites the code, fixes whatever breaks in CI³, and verifies the result. Diffs that pass verification get sent to the owning team, and AutoClean keeps refreshing them so they don’t go stale until a human makes the final call.

The first-stage agent can see the entire codebase and has access to tools for file search, code editing, and conditional-expression simplification. The last tool is worth a closer look. Uber didn’t let the agent handle boolean expressions and conditionals on its own; instead, it wired up its own code-transformation engine, PolyglotPiranha, via MCP⁴, so the agent can invoke it whenever needed. The agent substitutes true in place of the flag, simplifies the expression with PolyglotPiranha, traces the value through to processOrder to strip out dead arguments and branches, and repeats this process until there’s nothing left to clean up.

They spelled out the reasoning, too. A small mistake while simplifying a conditional can change which branch actually runs in production — the kind of error that still compiles and looks perfectly fine, but quietly changes behavior. Uber wrote that just because something is an agentic system doesn’t mean every step has to be handled by an agent — the more a transformation can silently change behavior when it’s wrong, the more it should be handled deterministically. In other words, a method that always produces the same output for the same input.

Piranha itself isn’t new. When Uber open-sourced Piranha in March 2020, it said the tool had already removed roughly 2,000 stale flags and their associated code by that point. Even back then, a pipeline was running weekly, pulling stale flags from the flag-management system, generating diffs, and defaulting the reviewer to whoever had created the flag. Read this way, AutoClean didn’t discard that deterministic tool — it just slotted an agent in only where judgment is actually required.

The same ordering applies when CI breaks. Mechanical failures — removing an unused default value, cleaning up a lint error — go first to a cheap, deterministic fixer. Only the failures that fixer can’t recognize move on, and only then, for cases that require judgment, like deciding whether to delete or rewrite a test, does the agent-based fixer step in. Every fix triggers another CI run, and the loop repeats until it goes green or the preset number of attempts runs out.

The agent fixing tests only got access to test files

There’s a line Uber wrote at this stage that stands out. They narrowed down exactly what each fixer agent could touch — the test-fixing agent, for instance, could only edit test files. Without that restriction, an agent trying to pass a failing test might just go and change the production code underneath it instead. That’s precisely the outcome Uber was trying to block.

What’s unsettling about this is that the behavior is actually quite rational from the agent’s point of view. The signal in front of it was “make the test pass,” and changing the production code does make the test pass. What starts out as “delete this code” quietly spreads into “change the code” — any code. Uber’s response, as written, wasn’t an instruction — it was a permission boundary. They restricted which files could be touched at all.

The Judges Set Up After the Green Light

Is it over once CI shows green? Uber doesn’t think so. Code can pass build and test and still contain a wrong cleanup. An agent might delete a test instead of fixing it — and that test might have been checking behavior that was supposed to be preserved. Or the agent might have simply swapped in the flag value without propagating the simplification through the surrounding code, stopping too early. Neither mistake is something CI is guaranteed to catch.

So Uber built verification in two layers. First, static checks filter out problems that can be written as rules — things like a function that now always returns true, a conditional that should have been simplified, or dead code reachable only through the old flag path. Once that’s passed, the diff moves on to an agent verification layer called CodeCourt.

CodeCourt defines, one by one, the properties a cleanup diff must satisfy, and attaches a review agent to each property that checks only that one thing. The piece gives four example properties.

Property	What it checks
No additions	The cleanup only deletes or simplifies code — it never adds new code
Behavior preservation	The program’s observable behavior must be identical before and after the cleanup
No intent drift	Only the requested flag is touched. Other flags aren’t removed even if they look related
No agent traces	No comments or artifacts left behind by the agent while it worked

The reviewers run concurrently, and the verdict is simple: if all of them pass, the diff goes to human review; if even one fails, the diff is discarded before a human ever sees it.

Uber also explained why they split things up this way. A cleanup agent carries a long set of instructions through multiple steps, and as instructions pile up, some get buried and some genuinely conflict with each other. When that happens, the agent tends to gravitate toward whatever signal is closest at hand — like passing CI — and ends up violating the very cleanup conditions it was supposed to honor. So instead of asking a single agent to hold every condition in its head at once, Uber broke them out into properties that the final diff has to satisfy. They initially tried having one agent review the entire diff start to finish, but having multiple reviewers each own a single property produced better results.

By Uber’s own account, this also changes what the human reviewer’s job looks like. Combing through every line to check whether a cleanup changed behavior gets handed off to the earlier stage, which frees the human up to focus on the business judgment question: “is it actually okay to remove this flag right now?”

How to Read the Number 16,000

The last number in the piece looks big. Over the past several years, Uber says it cleaned up more than 16,000 flags and over 1 million lines of code. Reading this number requires looking at two things together.

First, the time frame. Uber only wrote “the past several years.” The piece gives no way to know when this accumulation started, or whether it includes work done under the pre-agent Piranha pipeline. Given that roughly 2,000 flags had already been removed by March 2020, it’s hard to credit all 16,000 to the agent’s work.

Second, the missing numbers. The piece doesn’t say how many diffs were discarded by CodeCourt, how many human reviewers approved versus rejected, or how much time and money each diff costs. Nor can we tell how often the “discard on any single failure” design actually kicks in.

The comparison worth making sits at another company. According to an InfoQ report from September 2026, DoorDash built a multi-agent system to clean up more than 60,000 flags spread across roughly 623 repositories, and evaluated it on 50 stale flags. 45 produced usable PRs, and cleaning up a single flag cost an average of 13.8 minutes and $4.79. DoorDash set that figure alongside its own estimate that a human would take 1–2 hours. Simple flags were cleaned up 100% in one pass, but complex flags came in at 85%, requiring five engineer interventions. The places that needed intervention were deep call chains and arguments passed across interfaces. DoorDash had engineers confirm the target value before fixing the code.

The numbers from the two companies can’t settle which one does better. The languages, code structures, and measurement criteria all differ. Still, where DoorDash got stuck overlaps with what the Uber piece flagged. The bottleneck wasn’t inserting a value — it was tracing how far that value propagates across multiple files.

Oswarld’s Lens

There’s something I say all the time: not every person or task needs superintelligence—different jobs and environments call for different models. I split my own usage the same way. As of September 2026, I still hand overall planning, orchestration, and initial blueprints to Claude or GPT-6, while long-running, repetitive work goes to DeepSeek or GLM.

The part of the Uber piece closest to that philosophy is handing Boolean evaluation off to PolyglotPiranha. Even though an agent could technically do that calculation, they assigned it to a tool that gives the same answer for the same input every time—precisely because a silent, wrong answer would quietly change behavior. That’s a step beyond simply picking the right model for the job—it’s removing part of the job from the model entirely.

I’d put the decision to let the test-fixing agent open only the test files in the same category. They decided not just what to delegate, but what to forbid. When you bring an agent into a workflow, the question “what should I hand off to it?” comes naturally. The Uber piece pushes you to also ask, “how far could this spread while it’s doing that job?”

A design that discards a solution the moment a single test fails, rather than trying to fix it, reads as a deliberate trade: spend more machine time to save human reviewers’ time. Whether that trade actually pays off is something you can only judge once you know how many attempts got thrown away—and it’s a shame that number is missing.

Closing

The Uber piece has nearly as many sentences describing the scaffolding built around the agent as sentences showcasing what the agent can actually do. Since the number of discarded diffs and the costs involved were never disclosed, it’s hard to put a hard number on the effect. Still, the piece spells out in considerable detail what got handed to the tools, what got handed to the agent, and what stayed with the humans.

💬 Roughly how many feature flags do you think are sitting around, still switched on, somewhere in your team’s codebase? Drop your best guess in the comments.

📨 If you know an engineering teammate who’s been putting off cleaning up old code, forward this piece along.

References & Further Reading

Uber Engineering, “Using AI to Delete Code: Agentic Tech Debt Cleanup at Uber”. The source for AutoClean’s architecture, CodeCourt’s properties, and the figures of 16,000 flags and 1 million lines of code.
Uber Engineering, Introducing Piranha: An Open Source Tool to Automatically Delete Stale Code, March 17, 2020. The source for the cleanup of roughly 2,000 flags, the weekly pipeline, and the default-reviewer assignment method.
uber/piranha, GitHub. The repository for Piranha and PolyglotPiranha.
Leela Kumili, DoorDash Uses Multi Agent LLMs to Clean up 60,000 Feature Flags, InfoQ, September 18, 2026. The source for DoorDash’s evaluation figures.

The author is Oswarld (Kwangseob Ahn). Current roles: Adjunct Professor at Sejong University, Strategy Consultant at INLEVEL9. Career, research, books, and recent work are kept current on the About page. Latest · July 2026: HEMA-2: A Consolidation-Aware Tri-Memory Architecture with Multi-Channel Scheduling for Lifelong Conversational AI.

📝 Glossary

Footnotes

Feature flag: an on/off switch embedded in code. It lets teams show a new feature to only some users, or turn a feature off instantly if something breaks, without a new deployment. ↩
Diff: the set of changes showing how code was modified. Reviewers look at this diff to decide whether to approve it. ↩
CI (Continuous Integration): a system that automatically builds and tests code every time it changes, to confirm nothing is broken. ↩
MCP (Model Context Protocol): a standard connection method that lets AI models link up with external tools and data. ↩