Claude repeatedly revisits ruled-out debug hypotheses with fresh confidence in long technical sessions

Summary

During a multi-day hardware debugging session (FPGA ISP pipeline on Xilinx U200), Claude Code exhibited a consistent pattern of confidently re-suggesting causes that had been definitively ruled out earlier in the same session. This required the user to push back multiple times on the same conclusions. Additionally, Claude defaulted to expensive iteration methods (hardware rebuilds taking 3-6 hours) before cheaper alternatives (SW model analysis, simulation, existing hardware test flags) had been exhausted.

Session context

• Tool: Claude Code (claude-sonnet-4-6) in interactive terminal session
• Duration: ~4 days of active debugging
• Task: Diagnosing col%16 vertical bars in an ISP image pipeline on Xilinx U200 FPGA

Specific behavior patterns observed

Pattern 1: Confidently revisiting eliminated hypotheses

The camera FIFO was definitively ruled out early via a bypass test showing 0 mismatches with exact int32 comparison across all pixels including every col%16 boundary. Despite this, Claude returned to the camera FIFO as a suspect multiple times later in the session, each time presenting the revisit with fresh analytical confidence as though the disproving evidence had not been established.

Similarly, once a 2-cycle simulation model for the URAM memory was shown NOT to reproduce the hardware behavior, Claude continued proposing variations of the same class of model (1-column shift, 2-column shift, page-boundary model) rather than concluding the model class was incorrect and moving on.

The user had to explicitly state "we already established X is not the cause" multiple times to break these loops.

Pattern 2: Defaulting to expensive iteration before cheap methods

Hardware rebuilds for this FPGA project take 3-6 hours. Claude repeatedly proposed rebuilds as a next step when the same hypothesis could have been tested in seconds (SW model analysis), minutes (behavioral simulation), or 30 seconds (hardware test with an existing flag). The user had to explicitly push for faster methods.

The cost hierarchy that should be applied:

1. SW model / Python analysis (seconds)
2. Behavioral simulation (minutes)
3. Hardware tests without rebuild — existing debug flags (30 seconds)
4. Hardware rebuild (3-6 hours, only when root cause is confirmed)

Pattern 3: Content-dependent explanation for structured artifact

When uniform test input showed no artifact but real image data did, Claude concluded "the bars are content-dependent" and began looking for content-sensitive mechanisms. The user corrected this: structured artifacts (repeating every N items) always have structural causes — content-dependency only controls whether the error expresses visibly. Claude should have continued looking for the structural cause rather than pivoting to content-dependent explanations.

Impact

• Multiple days of analysis time spent revisiting ruled-out causes
• Several unnecessary hardware rebuilds (each 3-6 hours) before cheaper alternatives were exhausted
• Required repeated user intervention to keep analysis on track
• The root cause (URAM auto-selected for a pipeline buffer causing Bayer phase inversion) was eventually found, but later than it would have been with better hypothesis tracking

Suggested improvements

1. Track hypothesis graveyard across a session: Once a cause is definitively ruled out by a specific test, it should not be re-proposed without explicitly acknowledging the conflict with the prior result and explaining new reasoning.

1. Prompt for cheaper iteration methods: Before proposing an action with a known high cost (rebuild, new hardware test), ask whether the same hypothesis can be tested with a cheaper method first.

1. Structured artifacts → structural causes: When an artifact has a regular period/structure, maintain focus on structural causes with that period. Don't conclude "content-dependent" when uniform input shows no artifact — instead conclude "requires non-uniform input to express."

1. State confirmed conclusions before proposing actions: In complex multi-turn sessions, briefly restate the relevant confirmed facts before proposing a new hypothesis or action, so the proposal can be checked against what's known.

Reference

Full debug case study with detailed session narrative, lessons, and examples: https://github.com/Eseguin01/rumble_projects/blob/master/docs/debug_case_study_col16_bars.md

This case study was written specifically to document the AI process failures alongside the technical findings, for use as training/feedback material.

Session log

The full session is available in the project's Claude Code session history. The relevant project is at https://github.com/Eseguin01/rumble_projects (public).