Give paying users one documented switch to opt out of behavior experiments

Status Open
Maintainer reply None cached
Activity 1 comment · opened Aug 26, 2026

Claude Code ships behavior changes to paying users through unannounced cohort experiments — tool selection, file-editing method, system prompt content. #89731 is one instance: a standing instruction to edit files with sed and heredocs instead of the Read/Edit/Write tools, arriving with no CHANGELOG entry, no docs page, and no name a user could search for.

The cost of that lands on the user, not on the experiment. When behavior changes silently there is nothing to search for: you can't distinguish an experiment from the model having a bad day, so you debug your own setup first. Concretely, for this one: about two hours to establish the behavior was injected rather than emergent, and two more to find a control — which turned out to be an undocumented environment variable named after the experiment rather than the behavior, whose working value isn't guessable and whose typo silently re-enrols you. Four hours, one experiment, one user, repeated across the cohort and again for the next experiment.

CLAUDE_CODE_DISABLE_FEATURE_FLAG_FETCHING is not the answer: it also disables subagent execution, MCP tool search, Remote Control and IDE extensions. Declining to be experimented on should not cost product features.

Ask: a documented setting that excludes an account from behavior experiments — anything altering the system prompt, tool descriptions, or tool selection — while leaving bug fixes and flag-gated product features alone. Failing that, at minimum: every behavior experiment gets a CHANGELOG line when its rollout begins, a documented name, and a documented off switch that isn't its internal codename.

Paying customers should be able to say no. Today there is no way to say it, and no way to know there was anything to say it to.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗