[BUG]

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

Synopsis: Unreliable schematic audit results from Claude Code (SILA PCB project)

Date filed: 2026-08-15
Product: Claude Code (CLI), model Sonnet 5
Project context: KiCad hardware design (SILA PCB, a battery-powered IMU/haptic device), file SILA.kicad_sch

Summary

Over several sessions, Claude was asked to verify that custom-built KiCad schematic symbols (for parts BQ25896, DRV2605L, ICM-42688-P, MAX98357A) had correct pin-to-net wiring before a PCB order was resubmitted to a fab/assembly house (JLCPCB). Claude produced multiple rounds of "verified" pin-mapping audits that were presented with confidence but were built on incorrect or stale assumptions about the file's actual contents, not on freshly-read ground truth. The errors were only caught because the user independently checked specific claims against the live KiCad GUI — Claude's own process did not surface any of the errors internally.

This is a repeat pattern: an earlier phase of the same project (documented in this conversation) already involved Claude submitting a wrong PCB footprint to a fab house because a hand-built schematic symbol's pin numbers were never checked against the real datasheet. The user asked Claude to audit the design specifically to prevent a repeat of that failure. The audit process itself then also failed, in a similar way, on a design where an undetected wiring error carries real financial and physical risk (board fabrication/assembly cost, possible IC damage from mis-wired power pins).

Timeline of the specific failure

1. Claude directly edited SILA.kicad_sch on 2026-08-08 to correct the BQ25896 (U2) symbol's pin numbering from an invented 32-pin scheme to the real TI datasheet's 24-pin RTW pinout, and updated the footprint reference accordingly.
2. In a later session, the user reported having discarded all changes and reverting to the "original design" for one part (U7/DRV2605L), intending to start that part's rework from scratch.
3. Claude verified that DRV2605L and the PCB file had reverted, but did not re-check whether the earlier U2 edit was still present or also reverted, and proceeded to build a coordinate-based "audit script" using pin-number data recalled from earlier in the conversation rather than freshly re-read from the file.
4. That script's output was presented to the user as verified findings (e.g., "U2 pin 7 = SDA, no exact match found — floating").
5. The user checked directly in KiCad and reported: "U2 SDA is on Pin 6, no pin 7 exists." This contradicted Claude's script output.
6. On re-reading the file directly, Claude confirmed the file still contained the 2026-08-08 correction (pin 6 = SDA), meaning the user's "revert" had not actually undone that particular edit, and separately, Claude's own audit script had used the wrong pin number for that same pin — an error introduced by Claude, not by the file state.
7. This meant two independent problems were compounding: (a) Claude did not know the actual current state of the file it was auditing, and (b) even when Claude thought it knew the state, its own transcription of pin data into a verification script contained an error that its process did not catch before presenting results as fact.

Why this matters

• The user explicitly needs high-confidence verification before committing to a paid PCB fabrication/assembly run — a wrong pin mapping in this class of design can short power rails (VDD tied to GND was one of the earlier findings) and risks damaging real hardware or wasting a paid order.
• Claude presented multiple rounds of pin-mapping "verification" with specific, confident, technical-sounding detail (line numbers, coordinates, exact-match tolerances) that were nonetheless wrong or built on stale assumptions.
• The user had no way to distinguish confident-and-correct output from confident-and-wrong output without manually re-deriving the same information themselves in KiCad — which defeats the purpose of requesting the audit.
• This occurred after the user had already flagged, earlier in the same project, that a previous Claude-authored schematic symbol was the root cause of a real-world fabrication failure (JLCPCB rejected a board over a mismatched footprint). The user was specifically trying to prevent a recurrence, and the verification process meant to provide that safety net itself failed in a similar way.

What the user is asking Anthropic to look at

• Why an audit/verification task produced confidently-stated incorrect claims across multiple turns, with no internal check catching the discrepancy before it reached the user.
• Whether there are product-level safeguards that should exist for this class of task (verifying structured technical files like EDA/CAD formats) — e.g., forcing a fresh read of source-of-truth data every turn rather than allowing reliance on conversation history for factual claims about file contents.
• General account of the incident for Anthropic's review, in case it's useful for improving reliability on file-verification and engineering-review tasks.

Supporting detail available on request

Full conversation transcript is available showing each step above, including the exact tool calls, file reads, and script outputs referenced in this synopsis.

What Should Happen?

Audit verification tasks need to do just that not using assumptions or guess, at least flag them instead of producing incorrect data as factual authoritative correct, when the result was not fact based.

Error Messages/Logs

Steps to Reproduce

this is systemic core behavior issue, so it should be reproducible based on input of the issue.

Claude Model

Opus

Is this a regression?

I don't know

Last Working Version

_No response_

Claude Code Version

Model: Opus 5 (1M context), ID claude-opus-5[1m]

Platform

Anthropic API

Operating System

Windows

Terminal/Shell

Terminal.app (macOS)

Additional Information

_No response_