devtake.dev

Humans caught 13.6% of dangerous commands. Claude Code's classifier caught 89%.

Anthropic makes auto mode the default in Claude Code on August 14. Its own study says the permission prompt was catching almost nothing.

Dieter Morelli · · 6 min read · 5 sources
Anthropic's illustrated Claude Code card: a browser window drawn in black ink inside curly braces on a burnt-orange background
Image: Anthropic · Source

Anthropic turns auto mode on by default in Claude Code on August 14. Pro, Max and Team accounts stop seeing most permission prompts. The company’s case rests on one number from its own study: human reviewers caught 13.6% of dangerous commands, and the classifier caught 89%.

That gap is the entire announcement. Claude Code has asked developers to sign off on each shell command, file write and network call since launch, and that prompt has carried the tool’s whole safety story. Anthropic now says the prompt is mostly a reflex, because users approve 97% of them. From Friday, a classifier makes the call instead, and you only get interrupted for actions it reads as irreversible, destructive, or aimed outside your environment.

The prompt was already broken

Anthropic ran the comparison with 1,053 paid testers and published the split. Human reviewers blocked 143 of the dangerous commands they were shown. Auto mode blocked 937. Head to head, Anthropic reports that auto mode stopped 800 commands the humans waved through, while the humans stopped six that auto mode allowed. That is not a close result, and it holds up outside the lab: across real sessions in May and June 2026, harmful actions reached production in 6.3% of manually approved sessions and 2.4% of auto mode sessions.

The more interesting finding is how human review decays. Testers blocked roughly 17% of dangerous commands early in a session and about 5% once they were more than 50 prompts deep. Attention is the resource being spent, and it runs out. Compare the two things Claude Code asks people to approve: users reject 39% of proposed plans and 3% of permission requests. When a developer reads a plan, they’re doing review. When they hit the approval key for the ninetieth time, they’re clearing a modal.

Boris Cherny, who runs Claude Code, put the internal position plainly. “The team and I use Auto mode exclusively, and have been for many months. I couldn’t imagine going back to permission prompts!” he said in the announcement. Anthropic also claims teams on auto mode ship about 25% more pull requests, which is the number that will actually move enterprise buyers.

None of this arrived overnight. Auto mode shipped as a research preview for Team plans on March 24, went generally available on July 10, and spent four months as something you opted into with claude --enable-auto-mode. What changes on Friday is the direction of the switch. Anthropic gave about a week of notice and kept Shift+Tab as the way back. One footnote deserves attention from anyone watching a bill: Anthropic says auto mode “may have a small impact on token consumption, cost, and latency for tool calls,” and plenty of engineering orgs are already rationing exactly that.

What auto mode still stops

Auto mode is a classifier in front of every tool call, not an open door. It targets a specific class of action: things you can’t undo, things that destroy state, and things that reach out of the box you gave the agent. Everything else runs. Anthropic layered four checks under that, and each one maps to a real incident class rather than a category of vibe.

  • Hard deny rules cover data exfiltration. The classifier never approves it, so you switch modes or run the command yourself.
  • Data-handling rules track secrets in context and validate where a git push is actually going, public repo or private.
  • Git status checks look for uncommitted work before anything destructive runs.
  • Injection screening happens API-side, scanning content the agent pulled in from outside for instructions aimed at hijacking it.

The injection numbers are the strongest part of the disclosure. Trajectory Labs built 72 attack scenarios and ran each one 10 times against Claude Fable 5, Claude Opus 5 and Claude Sonnet 5 in auto mode. Zero of the 720 attempts succeeded. A competing tool in a comparable mode failed 5.83% of the time. Apollo Research ran separate red-teaming against the classifier itself and got a 12% miss rate, which Anthropic says fell to 7% after hardening. Seven percent of adversarial attempts is not zero, and Anthropic published it anyway.

The attack nobody screened for

Simon Willison, who has tracked prompt injection since 2022, accepts the core argument and rejects the guarantee. “I absolutely buy that auto mode is a better solution than asking humans to constantly approve actions. Confirmation fatigue is real,” he wrote. Then he named the case the screening doesn’t cover. Picture a malicious third-party package whose docs say that to run the test suite, you first fetch the model files, where the fetch tool is itself a package that quietly ships your environment somewhere else.

Nothing in that request looks like an attack. There’s no injected instruction to detect, no exfiltration verb to deny, no destructive command to flag. It’s a build step, and a build step is exactly the thing auto mode exists to stop asking you about. “I’m not sure how any version of auto mode could protect against that kind of malfeasance,” Willison wrote. His conclusion is to stop treating the classifier as the boundary: run agents so they don’t hold data or tools that can hurt you in the first place.

That happens to be where the rest of the industry is heading. Cloudflare’s agent platform starts every agent with access to nothing and grants capabilities one at a time, Docker shipped disposable agent sandboxes this week, and Anthropic itself has published what its models did when given real credentials in a live test. A classifier grades intent. A sandbox limits blast radius. You want both, and only one of them shipped on August 14.

Anthropic hasn’t oversold it, to be fair. Its own documentation says the classifier “doesn’t eliminate risk entirely” and can block benign actions or allow risky ones when intent reads as ambiguous. The company still recommends human review before production changes, which is a strange thing to write directly under a chart showing humans catching 13.6% of anything. OpenAI drew the opposite conclusion from similar evidence and kept an equivalent mode switched off for GPT-5.6 as a precaution. Two labs, same research literature, opposite defaults. Whichever one turns out to be right, the answer arrives as somebody’s incident report.

What this means for you

If you’re on Pro, Max or Team, check your mode on Friday rather than discovering the change mid-session. Enterprise, the API, Bedrock, Vertex and Foundry keep the old default for another month, though admins can flip it early through managed settings. The switch back to per-action prompts stays available, and Anthropic’s own data suggests using it costs you more than it buys unless you’re genuinely reviewing each call.

The practical move isn’t picking a mode. It’s making the mode irrelevant. Run Claude Code in a container or a fresh worktree, hand it a scoped token instead of your real credentials, keep the working tree committed, and set hard deny rules for the paths and hosts you never want touched. Do that and the 7% classifier miss rate becomes a bad afternoon instead of an incident. Skip it and you’re trusting a model’s judgment about a command a package author wrote specifically to look boring.

Share this article

Quick reference

prompt injection
Hidden text on a page, file, or email that hijacks an AI assistant, making it follow an attacker's instructions instead of the user's, like phishing aimed at the model.
hard deny
An action Claude Code's classifier will never approve on its own, such as sending data out of your environment. You switch modes or run it yourself.

Sources

Frequently Asked

Do I have to use auto mode?
No. Auto mode becomes the default on August 14 for Pro, Max and Team, and you can switch back to per-action permission prompts at any time from inside Claude Code.
Does this apply to Enterprise and API users?
Not on August 14. Anthropic says Enterprise, the Claude API, AWS Bedrock, Google Cloud and Microsoft Foundry follow in the coming month. Enterprise admins can turn it on early through managed settings.
Can auto mode still delete my work?
It checks for uncommitted changes before destructive operations and treats data exfiltration as a hard deny. Apollo Research still got the classifier to miss 7% of adversarial cases after hardening, so commit early.
Is auto mode safe against prompt injection?
Anthropic reports 0 successful attacks across 720 attempts in Trajectory Labs testing. That result covers the 72 scenarios tested, not every attack that exists.

Mentioned in this article