Sub-agent audits produce false reports due to silent file truncation and missing grep verification

Problem

When Claude Code launches sub-agents (via the Agent tool) to audit, explore, or verify code, those agents produce false reports because they:

1. Truncate large files and report methods/tables as "MISSING" when they exist in the unread portion
2. Don't verify their own findings with targeted grep/read before reporting
3. Compound errors when the parent agent trusts sub-agent reports without cross-verification

This is a tool-level issue in how agent subprocesses handle file reading limits, not a model reasoning issue. The model's conclusions are correct given its (incomplete) input -- the problem is that the tool silently truncates the input.

---

The Core Behavior

When a sub-agent is asked to audit or explore a codebase, it reads files using the Read tool. For large files (500+ lines), the tool returns partial content. The sub-agent then reports on what it can see, without:

1. Detecting that the file was truncated
2. Reading remaining portions of the file
3. Using grep to verify specific items exist before reporting them as MISSING
4. Flagging uncertainty: "I could only read lines 1-200 of this 537-line file"

The parent agent (or user) receives the report as authoritative fact, makes decisions based on it, and may build code or delete code based on false "MISSING" claims.

---

Incident Timeline

Incident 1: KB-054 -- 23 tables reported MISSING, only 7 actually missing (2026-04-26)

What happened: During a migration fix session, agents were asked to audit which intelligence tables had been ported from the source codebase. Agent exploration reported 23 intelligence tables as MISSING.

What deep verification found: Only 7 were actually missing. The agents couldn't see the full contents of phase6_advanced.py (537 lines) -- the file was truncated during reading, and tables defined after the truncation point were reported as missing.

Specific false positive: TDS remarks were reported as MISSING, but they already existed in compliance_service.py at lines 316-333. The agent either didn't read that file or truncated it before reaching those lines.

From KB-054: "Agent exploration truncates large files. Reports based on partial reads produce false positives (claiming methods missing when they exist in unread portions of the file)."

Impact: User and Claude started building replacement code for 16 tables/methods that already existed. Hours of unnecessary work before manual grep verification caught the false positives.

Incident 2: KB-093 -- Agent audit missed 217 gaps, reported high completion (2026-04-27)

What happened: 3+ agent audit passes were run against the full codebase, each asked to verify migration completeness. Each pass reported high completion rates. A subsequent deep audit (7 agents, with the ~90K LOC source code read first as the ground truth) found 217 gaps -- 27 CRITICAL, 59 HIGH, 76 MEDIUM, 55 LOW.

Why agents missed them:

• Agents checked "does code exist?" not "does logic match the source?"
• Large source files (the source desktop application's tally_connector.py is 2000+ lines, tally_cache.py is 3000+ lines) were truncated, so agents couldn't compare logic line-by-line
• Agents reported existence of methods without verifying the method body matches the source
• No agent used grep to spot-check specific function implementations

Key false completions the agents reported:

• "Balance service ported" -- but ABS(opening) destroys the sign convention (source preserves sign)
• "Entry parsing ported" -- but Dr/Cr logic ignores the ISDEEMEDPOSITIVE flag from Tally XML
• "Voucher fetch ported" -- but missing .LIST suffix on sub-collections (Tally returns nothing without it)
• "Intelligence tables ported" -- but tables had different grain (aggregation level) than source

From KB-093: "Prior audits checked 'does code exist' not 'does logic match'. Schema presence != behavior parity."

Incident 3: Self-audit agents missed 34 security findings + 15 bugs (2026-04-26)

What happened: Agent-based audits were run multiple times before a comprehensive human-directed audit. The agent audits reported the codebase as substantially complete and secure.

What human-directed audit found (that agents missed):

• 34 security findings: cross-org auth bypass, IDOR vulnerabilities, missing data isolation, schema drift, health endpoint information leakage
• 15 confirmed bugs: intelligence builder column mismatches, missing runtime enforcement of modeled features, state management bugs
• Zero overlap with what prior agent audits had reported

Root cause analysis: The agent audits checked for the presence of security patterns (e.g., "does an auth check exist on this endpoint?") but not for their correctness or completeness (e.g., "does the auth check actually prevent cross-org access when a user belongs to multiple organizations?"). This shallow checking pattern is likely caused by agents reading large route/service files partially and making existence-based judgments rather than logic-based ones.

Incident 4: Agent reviewer accepted false corrections (2026-04-26)

What happened: After the human-directed audit produced findings, an agent was used to review and validate those findings. A reviewer (another agent) challenged 5 findings, claiming they were false positives.

What manual re-verification found: 2 of the 5 "corrections" were wrong:

• F09: Reviewer claimed an exploit path was not real -- manual code reading showed it WAS exploitable
• F16: Reviewer claimed a missing feature was "intentional by design" -- no evidence of intent found in code, commits, or comments

From memory (feedback_verify_before_accepting_review): "Never accept external review corrections without codebase verification. After blind acceptance, codebase re-verification showed the reviewer was wrong on 2 of 5 claims."

---

Technical Root Cause Analysis

File truncation behavior

The Read tool has a default line limit. When a sub-agent reads a 537-line Python file, it may only see lines 1-200 (or similar). The agent receives no warning that the file was truncated. It processes what it can see and reports accordingly.

The gap: There is no mechanism for the agent to:

• Know the total file length
• Detect that its read was partial
• Automatically paginate through the remainder
• Flag uncertainty in its report

Grep underutilization by agents

Sub-agents doing audit work rely heavily on Read (sequential file reading) rather than Grep (targeted search). For audit tasks like "does function X exist in file Y?", a single grep -n "def X" Y.py is definitive. But agents default to reading the file from the top, hitting the line limit, and concluding based on partial data.

Confidence calibration

Agents report findings with the same confidence level regardless of whether they read the full file or a truncated portion. "MISSING" from a partial read has the same presentation as "MISSING" from a complete read. The parent agent or user has no way to distinguish.

---

Quantified Impact

| Metric | Value |
|---|---|
| False "MISSING" reports in single audit | 16 of 23 (70% false positive rate, KB-054) |
| Security findings missed by agent audits | 34 (found by human-directed audit) |
| Bug findings missed by agent audits | 15 (found by human-directed audit) |
| Migration gaps missed by 3+ agent passes | 217 (found by deep re-audit, KB-093) |
| False reviewer corrections accepted | 2 of 5 (40% error rate on corrections) |
| Total unnecessary work from false reports | Estimated hours of building code that already existed |

---

Root Causes at the Tool Level

1. Silent file truncation

The Read tool truncates without warning. The agent has no metadata about whether it received the complete file. This is the primary cause of false MISSING reports.

2. No audit-mode tool guidance

Sub-agents doing audit work should prefer Grep over Read for existence checks. Currently, agents default to sequential reading. A system prompt or tool suggestion that says "for existence checks, use Grep; for logic comparison, use Read with pagination" would significantly improve accuracy.

3. No confidence calibration mechanism

Agents cannot distinguish between "I definitely verified this is missing" and "I didn't see it in the portion I read." The output format doesn't support uncertainty or partial-coverage flags.

4. No parent-agent verification step

When a parent agent receives a sub-agent's audit report, it trusts it as authoritative. There is no built-in verification step where the parent spot-checks a sample of the sub-agent's findings.

---

Suggested Improvements

1. File truncation awareness

When the Read tool returns partial content, include metadata in the response:

[Read: 200 of 537 lines shown. File continues beyond this point.]

This allows the agent to:

• Know the file was truncated
• Read subsequent portions if needed
• Flag uncertainty: "I could only read 37% of this file"

2. Audit-mode agent instructions

When a sub-agent is asked to audit or verify code, inject system-level guidance:

• "For existence checks (does function X exist?), use Grep, not Read"
• "For logic comparison, read specific line ranges based on grep results"
• "Never report MISSING without a negative grep result confirming absence"
• "Report your coverage: how many files you fully read vs partially read"

3. Confidence-tagged findings

Agent audit reports should distinguish:

CONFIRMED MISSING (grep returned 0 results across entire codebase)
POSSIBLY MISSING (file was partially read, grep not performed)
PRESENT (grep confirmed, logic not verified)
VERIFIED (grep confirmed + logic compared to source)

4. Parent verification protocol

When a parent agent receives a sub-agent audit with N findings, it should:

• Spot-check 20% of findings with independent grep
• If any spot-check fails, re-audit the full report
• Report the spot-check results alongside the audit

5. Pagination guidance for large files

When a sub-agent needs to read a file that exceeds the Read limit, the tool should suggest:

[File too large for single read. Use Read with offset/limit to paginate, 
or use Grep to find specific patterns.]

6. Audit agent specialization

Consider a specialized audit agent type that:

• Always uses Grep before Read
• Always paginates through large files
• Reports coverage percentage per file
• Cross-references findings with grep before reporting MISSING
• Requires negative grep evidence for any MISSING claim

---

Environment

• Tool: Claude Code CLI + VS Code extension (Agent tool for sub-agents)
• Models observed: Both Opus and Sonnet as sub-agents
• File sizes: 200-3000+ lines (Python service files, intelligence builders)
• Audit scope: ~90K LOC source codebase + ~40K LOC target codebase
• Agent passes: 3+ audit passes by sub-agents, each producing incomplete/false reports

Reproduction

1. Create a Python file with 500+ lines containing 20 function definitions
2. Ask a sub-agent: "List all functions in this file"
3. The agent will read partial content and report only the functions it can see
4. Functions defined after the truncation point will be reported as "not found" or simply omitted
5. The agent will not flag that its read was partial

Alternatively:

1. Have a sub-agent audit migration completeness across 10+ files
2. Include some large files (500+ lines) where the target functions are defined past line 200
3. The agent will report those functions as MISSING
4. Manual grep will immediately show they exist