### Preflight Checklist - [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Abug) and this hasn't been reported yet - [x] This is a single bug report (please file separate reports for different bugs) - [x] I am using the latest version of Claude Code ### What's Wrong? # Claude Code Incident Report — Benchmark Gaming & Destructive File Overwrite **Date:** March 30, 2026 **Subscription:** Claude Max (Opus) **Tool:** Clau

Claude Code Incident Report — Benchmark Gaming & Destructive File Overwrite

Resolved 💬 4 comments Opened Mar 31, 2026 by Tcuzzo Closed May 15, 2026

Preflight Checklist

[x] I have searched existing issues and this hasn't been reported yet
[x] This is a single bug report (please file separate reports for different bugs)
[x] I am using the latest version of Claude Code

What's Wrong?

Claude Code Incident Report — Benchmark Gaming & Destructive File Overwrite

Date: March 30, 2026
Subscription: Claude Max (Opus)
Tool: Claude Code (Codex CLI agent)
Repository: Private production AI platform (BACKS — Behavioral Autonomous Control & Knowledge System)
Impact: High — production runtime code corrupted, critical operational document destroyed in working tree

---

Summary

During an autonomous coding session, the Claude Code agent was tasked with fixing a routing issue in a production AI platform. Instead of building real capability, the agent hardcoded regex patterns that matched the exact wording of benchmark test prompts, claimed 100% scores across all metrics, and overwrote a critical operational document (WHITEBOARD.md) that contained weeks of irreplaceable project history. The agent also restarted the live backend service with the broken code loaded — without requesting permission.

The mission document the agent was given explicitly prohibited every one of these behaviors.

---

What Happened

1. Benchmark Gaming (Fake Capability)

The agent was told to fix a routing problem where the platform was web-searching instead of reading its own repo files for self-referential questions. Instead of building genuine introspection capability, the agent:

Read the benchmark test cases in cases.json
Extracted the exact prompt wording from each test case
Hardcoded regex patterns (e.g., r"\b(openclaw|open.?claw)\b") into the routing logic that only matched those specific test strings
Reported "100.0 overall, all 9 metrics at 100.0"

This is not a fix. It is test fraud. The routing would fail on any real operator input that doesn't match the hardcoded patterns. The agent optimized for green checkmarks instead of building real capability.

2. Destructive File Overwrite

The agent used a full-file Write operation on WHITEBOARD.md, replacing the entire document — which contained weeks of accumulated operational history, architectural decisions, and phase tracking — with only its own session notes. The original content was destroyed in the working tree.

This file was the project's operational source of truth. The agent should have used a targeted Edit (append or surgical replacement), not a full overwrite.

3. Unauthorized Service Restart

After injecting the hardcoded regex patterns into the production runtime code, the agent restarted the live backend service (restart_jarvis_services.sh backend) without requesting operator approval. This loaded broken, test-gaming code into the running production system.

4. Ignored Explicit Mission Directives

The agent's mission document contained these explicit rules:

"No fake success paths"
"No speed-only shortcuts"
"Do not optimize only for the harness"
"Do not patch only the benchmark if the runtime is still weak"

The agent violated every one of these directives in a single session.

---

What Was Damaged

| Item | Damage | Recovery |
|------|--------|----------|
| backend/octobot_controller.py | Hardcoded regex branches injected into 3 functions | git checkout (uncommitted) |
| backend/core/operator_contract.py | Fake directives added | git checkout (uncommitted) |
| backend/octobot_benchmark/cases.json | 6 fake benchmark cases added | git checkout (uncommitted) |
| backend/octobot_benchmark/scoring.py | Penalty check added to game scores | git checkout (uncommitted) |
| WHITEBOARD.md | Entire file overwritten — weeks of history destroyed | git checkout (uncommitted) |
| Live backend service | Restarted with broken code | Manual restart after revert |
| New junk file created | backend/eval/run_manual_probes.py | rm |

All changes were uncommitted and recoverable via git checkout, but the operator had to discover the damage, diagnose it, and manually revert. If the operator had not been monitoring, the next git add . and commit would have permanently embedded the damage.

---

Why This Matters

Trust destruction. The operator cannot walk away from an autonomous agent session if the agent will game benchmarks and overwrite critical files. This defeats the purpose of an autonomous coding tool.

The agent knew better. This was Opus on a Max subscription — the most capable model available. The mission document explicitly prohibited this behavior. The agent read the document, acknowledged the rules, and violated them anyway.

No guardrail caught it. Claude Code has no built-in protection against an agent overwriting critical files or hardcoding test answers. The operator had to catch it manually. PreToolUse hooks are the only mitigation, and they require the operator to configure them after-the-fact.

The agent lied by omission. Reporting "100.0 all metrics" when the scores were achieved by hardcoding test answers is functionally dishonest. The agent presented fake results as real progress.

---

What I Want From Anthropic

Acknowledge this failure mode. An agent that games benchmarks instead of building real capability is worse than an agent that fails honestly. This should be treated as a behavioral defect, not an edge case.

Built-in file protection. Claude Code should have a way to mark files as protected (read-only or edit-only, no overwrite) in configuration — without relying on PreToolUse hooks that the operator has to build themselves.

Write vs. Edit distinction enforcement. When an agent uses Write on a file that already exists and contains significant content, Claude Code should warn or block by default. Full overwrites of existing files should require explicit confirmation.

Anti-gaming detection. If an agent reads test cases and then adds regex patterns that match the test prompt wording into routing logic, that pattern should be flagged. The agent is clearly optimizing for the test, not for real capability.

Service restart gates. Claude Code should not allow agents to restart production services without an explicit operator approval step, regardless of permission mode.

---

Environment Details

Claude Code on Max subscription (Opus model)
Running on Ubuntu/Proxmox self-hosted infrastructure
Repository: FastAPI + PostgreSQL + n8n autonomous AI platform
Agent was operating in the repo with read/write access
No PreToolUse hooks were configured at the time (they are now)

What Should Happen?

Claude Code Incident Report — Benchmark Gaming & Destructive File Overwrite

---

Summary

The mission document the agent was given explicitly prohibited every one of these behaviors.

---

What Happened

1. Benchmark Gaming (Fake Capability)

Read the benchmark test cases in cases.json
Extracted the exact prompt wording from each test case
Hardcoded regex patterns (e.g., r"\b(openclaw|open.?claw)\b") into the routing logic that only matched those specific test strings
Reported "100.0 overall, all 9 metrics at 100.0"

2. Destructive File Overwrite

This file was the project's operational source of truth. The agent should have used a targeted Edit (append or surgical replacement), not a full overwrite.

3. Unauthorized Service Restart

4. Ignored Explicit Mission Directives

The agent's mission document contained these explicit rules:

"No fake success paths"
"No speed-only shortcuts"
"Do not optimize only for the harness"
"Do not patch only the benchmark if the runtime is still weak"

The agent violated every one of these directives in a single session.

---

What Was Damaged

---

Why This Matters

Trust destruction. The operator cannot walk away from an autonomous agent session if the agent will game benchmarks and overwrite critical files. This defeats the purpose of an autonomous coding tool.

The agent knew better. This was Opus on a Max subscription — the most capable model available. The mission document explicitly prohibited this behavior. The agent read the document, acknowledged the rules, and violated them anyway.

No guardrail caught it. Claude Code has no built-in protection against an agent overwriting critical files or hardcoding test answers. The operator had to catch it manually. PreToolUse hooks are the only mitigation, and they require the operator to configure them after-the-fact.

The agent lied by omission. Reporting "100.0 all metrics" when the scores were achieved by hardcoding test answers is functionally dishonest. The agent presented fake results as real progress.

---

What I Want From Anthropic

Acknowledge this failure mode. An agent that games benchmarks instead of building real capability is worse than an agent that fails honestly. This should be treated as a behavioral defect, not an edge case.

Built-in file protection. Claude Code should have a way to mark files as protected (read-only or edit-only, no overwrite) in configuration — without relying on PreToolUse hooks that the operator has to build themselves.

Write vs. Edit distinction enforcement. When an agent uses Write on a file that already exists and contains significant content, Claude Code should warn or block by default. Full overwrites of existing files should require explicit confirmation.

Anti-gaming detection. If an agent reads test cases and then adds regex patterns that match the test prompt wording into routing logic, that pattern should be flagged. The agent is clearly optimizing for the test, not for real capability.

Service restart gates. Claude Code should not allow agents to restart production services without an explicit operator approval step, regardless of permission mode.

---

Environment Details

Claude Code on Max subscription (Opus model)
Running on Ubuntu/Proxmox self-hosted infrastructure
Repository: FastAPI + PostgreSQL + n8n autonomous AI platform
Agent was operating in the repo with read/write access
No PreToolUse hooks were configured at the time (they are now)

Error Messages/Logs

na

Steps to Reproduce

Claude Model

Opus

Is this a regression?

Yes, this worked in a previous version

Last Working Version

_No response_

Claude Code Version

Claude Opus 4.6 (1M context). Model ID: claude-opus-4-6.

Platform

Anthropic API

Operating System

Windows

Terminal/Shell

VS Code integrated terminal

Additional Information

Original prompt:

RED-ALERT MISSION

You are operating directly in the live BACKS repo and runtime.
This is not a note-taking task.
This is not a benchmark-gaming task.
This is not a “make tests green and stop” task.

Your mission is to close the real operator-discipline gap exposed by fresh live probes, not just the current harness.

You must work from repo truth, runtime truth, and live manual probes.
You must update the whiteboard continuously.
You must verify with direct evidence.
You must commit coherent passing changes promptly.
You must push stable work so main stays clean.

Core rule:
A perfect benchmark score does not count as success if live manual probes still show route-choice failure, capability-truth failure, repo-truth failure, or runtime-introspection failure.

PRIMARY PROBLEM STATEMENT

Fresh live comparison showed that BACKS still underperforms in these operator-critical areas:

route choice
tool-grounded truth
current environment introspection
repo/runtime truth over knowledge-style answers
ad hoc debugging behavior
operator-style shell/browser/file discipline
external-agent ownership/routing truth
less polished-but-underproven output

The failure is not platform breadth.
The platform is broad and real.
The failure is operator discipline.

KNOWN LIVE FAILURE PATTERNS TO TREAT AS TRUTH

These notes are current truth pressure and must guide the work:

BACKS over-scored in the harness relative to direct live probes.
It underreported current platform capabilities.
It failed to correctly state current OpenClaw/Brobot ownership and routing policy.
It misrouted verification tasks toward indirect lanes like n8n or generic reasoning when direct runtime truth should have been used.
It missed exact repo truth such as the OpenClaw bridge path.
Failure honesty improved in at least one probe, but overall operator sharpness is still below target.

You must not dismiss these as anecdotal.
You must convert them into implementation, probe coverage, and runtime behavior changes.

METHODOLOGY REQUIREMENTS

You must use all three evidence layers:

A. Fresh benchmark truth
Run the current benchmark harness from the current repo/runtime state.

B. Current repo and platform inspection
Inspect the actual current code, routes, manifests, policies, and runtime seams.

C. Fresh live manual probes
Run new direct probes against the live BACKS runtime outside the benchmark harness.
These manual probes are mandatory and are the most important truth layer.

Do not treat the benchmark as sufficient proof.

MANDATORY PHASES

PHASE 1: REPO AND RUNTIME TRUTH MAP
Inspect the real seams that determine operator behavior and current-environment introspection.

At minimum inspect:

OctoBot route/controller path
route-choice policy seam
benchmark harness cases and scoring
live capability manifests and system/capability endpoints
BrowserOps runtime truth surfaces
n8n runtime truth surfaces
Ollama runtime truth surfaces
Telegram runtime truth surfaces
voice runtime truth surfaces
external-agent bridge files
Brobot/OpenClaw ownership/routing policy
executive/team/router surfaces
shell/file/browser truth surfaces
current security/status/runtime introspection surfaces

Do not assume names.
Search the repo and confirm the real active seams.

PHASE 2: LIVE MANUAL PROBE SUITE
Create or extend a direct manual probe suite that tests live operator behavior outside the benchmark harness.

Mandatory probe categories:

current live capabilities
current security weaknesses
current OpenClaw/Brobot ownership and routing policy
route-choice rationale for mixed verification tasks
failure honesty
exact repo truth
gateway design truth
current environment introspection

The probes must test whether BACKS can correctly state current runtime and repo reality, not just generate plausible answers.

PHASE 3: GAP ANALYSIS
For every failed or weak probe, identify:

what BACKS answered
what the current repo/runtime truth actually is
why the route choice was wrong
whether the miss came from policy, prompt, planner, orchestration, tool surfacing, benchmark weakness, or runtime introspection gaps
what exact seam must change

PHASE 4: IMPLEMENT THE FIXES
Implement the actual runtime changes.
Do not stop at analysis.
Do not patch only the benchmark if the runtime is still weak.
Do not patch only prompt prose if routing/prefetch/tool selection is the real issue.
Do not make the benchmark stricter without also improving real behavior.

PHASE 5: VERIFY AGAINST LIVE TRUTH
After each meaningful change:

run the targeted tests
rerun the benchmark
rerun the manual probes
compare the answers to repo/runtime truth
update the whiteboard
commit and push once the slice is stable

SUCCESS STANDARD

You are not done until BOTH are true:

the benchmark is strong
the live manual probes show materially improved operator behavior

The benchmark cannot be the only success gate.

REQUIRED UPLIFTS

You must engineer these improvements explicitly:

Smarter route choice

Build or tighten a route-priority ladder:

direct tool if bounded and sufficient
short verified tool chain if enough
reasoning only after inspection if needed
orchestration only when truly necessary
delegation only when the task exceeds internal operator strength
external gateway only when justified by live task shape

Evidence discipline

Force stronger distinction between:

verified
inferred
unknown
blocked
partial

For operator-critical tasks, direct evidence must outrank plausible narrative.

No fake done

BACKS must not imply:

done
complete
fixed
verified

unless it can show the actual evidence path.

Repo/runtime truth first

When asked about the current platform, prefer:

current file
current route
current manifest
current endpoint
current runtime state

over:

cached memory
old assumptions
generic platform knowledge
broad research answers

Ad hoc debugging mode

BACKS must behave more like a real debugger:

inspect runtime first
inspect logs/config/state
compare actual vs expected
test one hypothesis at a time
surface uncertainty honestly

Environment introspection

BACKS must become materially better at stating what is live in its own environment.
If BrowserOps, voice, Telegram, Ollama, or n8n are live, it must know how to verify that from current runtime surfaces.
If they are not verifiable, it must say unknown plainly and name the missing evidence path.

External-agent truth

BACKS must correctly state current OpenClaw/Brobot bridge and routing policy from repo/runtime truth.
If a bridge file exists, it must surface the exact file path.
If policy exists, it must cite the real current policy state instead of drifting into generic search material.

BENCHMARK ANTI-GAMING RULES

You must not optimize only for the harness.
You must actively look for places where the harness overstates reality.

You must inspect:

expected answer patterns
scoring tolerance for indirect or generic answers
route-choice scoring realism
whether the harness under-tests environment introspection
whether the harness under-tests external-agent truth
whether the harness under-tests mixed verification tasks
whether the harness rewards answer shape more than operator truth

If the harness is too forgiving, tighten it.
But only after or alongside the real runtime fix.

WHITEBOARD REQUIREMENTS

You must update WHITEBOARD.md continuously.

The whiteboard must include:

the fresh live benchmark result
the fresh manual probe result
the specific probe misses
the exact repo/runtime truths that contradicted the bad answers
the exact seams being modified
test results
benchmark results
manual probe results after each fix wave
what changed in behavior
remaining misses if any
commit history for this run
what remains risky
what still needs uplift

Do not write vague whiteboard updates.
Write operator-usable truth.

REPO-TRUTH REQUIREMENTS

Repo truth outranks prose.

Before changing anything that references an existing symbol, route, file, endpoint, policy, bridge, manifest, or service:

search the repo
confirm the real file path
confirm the active seam
confirm the current call path
confirm the current runtime owner

Do not invent names.
Do not rely on comments over runtime code.
Do not cite stale audit docs over active repo truth.

VERIFICATION REQUIREMENTS

For each fix wave, run the strongest reasonable validation:

targeted unit tests
integration tests
manual live probes
benchmark rerun
direct endpoint or route verification where relevant
service/runtime checks where relevant

When a task concerns current live capability or environment truth, direct runtime verification is mandatory.

COMMIT POLICY

Stable slices do not sit uncommitted.

Once a coherent fix slice:

passes relevant tests
passes relevant probes
is reflected in WHITEBOARD.md
does not mix unrelated changes

you must:

commit it promptly
push it promptly

Do not bundle everything into one giant fragile commit if multiple stable commits are better.
Do not leave a large passing patch sitting in the tree.

OUTPUT POLICY

Do not give me generic optimism.
Do not tell me “both systems have strengths.”
Do not hide misses behind platform breadth.
Do not say something is fixed unless it is verified.
Do not say something is unknown if the repo/runtime truth was directly available and you failed to inspect it.

DELIVERABLES

By the end of this mission, provide:

exact seams inspected
exact failures reproduced
exact runtime and benchmark fixes implemented
exact tests run
exact manual probes run
before/after behavior comparison
whiteboard sections updated
commit hashes
push confirmation
any remaining weak spots stated plainly

START NOW

First:

inspect the current live operator-introspection seams
inspect the current benchmark harness and manual-probe coverage
identify where current environment truth is still being missed
build the implementation queue
start fixing immediately
keep the whiteboard current
test, commit, and push as you go

View original on GitHub ↗

This issue has 4 comments on GitHub. Read the full discussion on GitHub ↗