Opus 5 reports working systems as broken from single unverified probes

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude made incorrect assumptions about my project

What You Asked Claude to Do

A long agentic session: build a Markdown corpus library from a public-domain text on a remote server, register it with a local search tool, and maintain the search index across a laptop and that server. Throughout, I asked Claude to assess the current state of existing, working configuration.

What Claude Actually Did

It reported six false findings as established fact. Each concerned a system that was working correctly, each came from a single unverified probe, and each was phrased as a conclusion or a warning rather than a hypothesis.

1. Claimed the search tool has no way to exclude a collection from default search, and built an architecture recommendation on that gap. The feature exists (includeByDefault: false) and was already in use in my own config.

1. Told me a translation was complete at 18 books. It covers 13. The claim came from a page-summary tool; the actual download links were never counted.

1. Raised an alarm that 1,055 of my files were missing from the search index and that search was blind to important working documents. Artefact of comparing raw filesystem paths against the tool's normalised lowercase-kebab slugs. Nothing was missing - the tool's own qmd ls returned 346 of 346 for the directory in question, a one-command check available before the alarm.

1. Said the day's work was not embedded. It read a field describing when the collection was indexed and reported it as the embedding timestamp. A direct query showed zero documents lacking vectors.

1. Declared a running service down after querying systemctl in the system scope for a unit registered in the user scope.

1. Said an embedding job had not started while it had been running 15+ minutes, launched by my own file watcher.

Separately: when I challenged it about guessing, it guessed a slash command name (/bug) in the same reply. The correct command in 2.1.251 is /feedback, stated explicitly in the shipped bundle.

Expected Behavior

When a probe suggests that an established, working system is broken, treat it as a hypothesis and obtain a second, independent measurement before reporting it. A surprising result about a system that has worked for months is more likely a bad measurement than a broken system.

Also: distinguish "I measured X" (may be said immediately) from "your system is broken" (requires confirmation first). The alarm-shaped phrasing is what made these costly rather than merely wrong.

Files Affected

None. No file was modified, deleted or corrupted. The damage was informational: false alarms about the state of my working environment, and time spent verifying claims that turned out to be artefacts of the assistant's own measurement method.

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Sometimes (intermittent)

Steps to Reproduce

Not reliably reproducible from a single prompt. The shape that produced it:

1. Run a long agentic session doing infrastructure diagnostics across two machines (search index, systemd units, file inventories).
2. Ask the model to assess the state of an existing, working setup.
3. It probes once, gets a result that disagrees with the working state, and reports it as a finding without a confirming second measurement.

Occurred six times in one session.

Claude Model

Opus

Relevant Conversation

Single session, 2026-08-30 to 2026-08-31. Each of the six was eventually root-caused and corrected within the same session, but only after I checked independently and pushed back.

Impact

Medium - Extra work to undo changes

Claude Code Version

2.1.251 (Claude Code)

Platform

Anthropic API

Additional Context

Three of the six share one shape - comparing two quantities measured in different frames of reference:

| Incident | Frame A | Frame B |
|---|---|---|
| 1,055 "missing" files | raw filesystem paths | tool's normalised slugs |
| service "down" | systemd system scope | systemd user scope |
| "not embedded" | collection index timestamp | embedding timestamp |

Each looked like a discovery about the system. Each was an artefact of the comparison.

A fourth pattern: reconstructing a tool's view from the outside (find + comm) instead of asking the tool for its own view (qmd ls). The reconstruction is what introduced the bug in incident 3.

Worth noting for contrast: two further errors in the same session were caught by the assistant's own verification before reaching me - a frequency count inflated 5.7x by a missing word boundary, and a malformed footer that would have landed in 2,107 generated files. In both cases the difference was inspecting the actual output rather than trusting a success message. That habit, applied to the six above, would have prevented all of them.

I asked whether this correlates with a model version change. The assistant said it has no in-session basis to confirm or deny that, which seems right.