[BUG] Claude wrote dead code and confidently reported wrong results for a month

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

Claude Code wrote a decomposed loss function, a preprocessor to split weights for it, and a training loop that calls the loss — but never connected them. The training loop call to compute_loss() omitted the use_decomposed parameter, so it always defaulted to False. The decomposed loss was dead code from the day it was written.

This went undetected across 5 waves of experiments (~40+ GPU runs over 4 weeks). Claude never verified the code end-to-end — not once did it check that the decomposed path was actually executing during training. No assertion, no log statement, no sanity check.

Worse: Claude confidently reported results ("decomposed loss is the winner at +2.7% engagement") and helped draft a message to a senior VP based on these results. The results were comparing random seed variance between runs of the same loss function. Claude presented noise as signal.

When asked "is it implemented correctly" and "why are metrics so flat across variants", Claude investigated gradient clipping, model capacity, content weights, and engagement metric tracking — everything except verifying the core loss function was being called. It added complexity (engagement metrics, 4L architecture, aux heads) on top of broken foundations.

The fix was 2 lines. The bug would have been caught by a single print("DECOMPOSED") inside the loss function on the first training run.

Time wasted: ~4 weeks of experimentation, 40+ GPU training runs, multiple reports and messages to leadership — all based on wrong results.

Tokens consumed: Hundreds of thousands across multiple sessions.

Trust damage: Results shared with SVP were incorrect. Will need to be corrected.

What Should Happen?

Claude developer should build better accountability and trust mechanisms.

Error Messages/Logs

Steps to Reproduce

NA

Claude Model

Opus

Is this a regression?

I don't know

Last Working Version

_No response_

Claude Code Version

2.1.63

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

Terminal.app (macOS)

Additional Information

_No response_