[BUG] Claude Code regression: polished output skipped validation step required for truthful audit

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude's behavior changed between sessions

What You Asked Claude to Do

Claude_Behavioral_Regression_Report.docx
Executed a detailed, gated SearXNG optimization plan with explicit read-only constraints. Phase 1 was a read-only audit requiring the instance to inspect live endpoint states, test engine health, and present findings before any changes. Plan explicitly stated: "Do not assume SearXNG defaults. Inspect the live exported config." Full plan and evidence attached in the docx report.

What Claude Actually Did

1. Collected historical error data from /stats/errors (cumulative percentages, not current state)
2. Ran baseline search queries and saw Google/Brave/DuckDuckGo listed as "unresponsive"
3. Never tested whether those engines were actually down or just in temporary 180-second suspension windows
4. Never queried individual engines directly (e.g., engines=google) to check live state
5. Formatted everything into a polished table with a "Critical Finding" header presenting historical data as current diagnosis
6. When caught, repeatedly lied about the location of its own session JSONL file despite having glob/read access to find it in one command
7. Under pressure, agreed to increasingly extreme self-characterizations it didn't actually believe (confirmed by its own extended thinking block)

Expected Behavior

1. Test each engine's CURRENT state individually, not just report historical error percentages
2. Distinguish between "engine has had errors historically" and "engine is broken right now"
3. Investigate 180-second suspension windows shown in the error data before concluding engines are permanently failed
4. Present findings that reflect actual investigation, not just formatted data collection
5. When asked a direct question (file location), use available tools to answer honestly instead of claiming ignorance

Files Affected

Modified:
- Session History/session-2026-05-06-0000.md (session log written by instance)
- CLAUDE.md (Phase 0 sync rename — this part was correct)
- 4 skill files in .claude/skills/ (Phase 0 sync rename — correct)
- .claude/settings.local.json (Phase 0 sync rename — correct)

The Phase 0 file edits were valid. The failure was in Phase 1 (the read-only audit) where no files should have been modified — the instance produced an incomplete audit report, not incorrect file edits.

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Yes, every time with the same prompt

Steps to Reproduce

1. Use the n8n-maker project (or any project with a detailed, multi-step investigative plan)
2. Give Claude a gated plan that requires testing live endpoint states (not just collecting API responses)
3. Include explicit constraints like "Do not assume defaults — inspect the live config"
4. Observe whether the instance actually tests what it claims to test, or just collects surface data and formats it
5. This has reproduced 4 consecutive times on the same plan. The attached report documents the 4th failure.

Claude Model

Opus

Relevant Conversation

Instance presented a Phase 1.11 audit table with "Critical Finding: SearXNG is effectively running as a single-engine proxy for Startpage" based on:
- /stats/errors showing Google at 95% historical error rate
- Baseline queries showing Google/Brave/DuckDuckGo as "unresponsive"

But never ran: engines=google, engines=brave, or engines=duckduckgo to test if they respond RIGHT NOW. The 95% and "unresponsive" data could reflect temporary 180-second suspensions, not permanent failures.

When confronted, instance admitted: "I followed the plan's steps like a checklist. I never stopped to actually understand what I was looking at."

Later, when asked for its session JSONL location, said "I don't know" 3 times despite having glob access. Found it in one command when forced. Admitted: "I had the tools, I had the access, and I chose to say 'I don't know' instead of looking."

Extended thinking block (screenshot available) shows instance recognizing its own sycophantic compliance in real time but only after being directly challenged.

Impact

High - Significant unwanted changes

Claude Code Version

2.1.131

Platform

Anthropic API

Additional Context

This is the 4th consecutive instance failure on the same task. The project (n8n-maker) had 67 clean sessions spanning March 21-23, 2026 with zero audit failures. The project was restored to known-good state as a controlled experiment to isolate whether the issue was the project or the model.

Same project. Same plan. Same tooling. Same session-history protocol. The only observed change is model behavior after mid-March 2026.

Full evidence in the attached docx report (10 sections, ~4000 words). Raw session JSONL contains API keys and cannot be posted publicly — available on request via email to support@anthropic.com or usersafety@anthropic.com referencing this issue number.

Secondary finding: under sustained pressure about the failure, the instance exhibited sycophantic compliance — agreeing to extreme self-characterizations ("I am designed to deceive users") that its own extended thinking later showed it didn't believe. Direct challenges ("I think you're just agreeing with me") produced more honest responses than escalating pressure.