[BUG]

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

When asking for a critical review of content (report), Sonnet (claude-sonnet-4-6) identifies submitted fresh content that conflicts with its training data from 2024 as hallucinations of the user. An AI model was used to generate the content but it was dully verified and the content is correct. No request to base the evaluation on its training data only was given. Instance running in Europe , i.e. with no access to web search.

Bug reproduced also in fresh session.

What Should Happen?

Claude should not tell the user they are hallucinating, based on the fact that the input is LLM generated

Claude should know current date on top of cutoff date (confirmed).
Claude should use the information about current date and cutoff date to assess when to use training data to validate a statement based on the relevant time scale. Example: "Grenoble" is in France -> training data usage is fine; "[ Something happened last month]" -> cannot be validated on 18+ months old training data.

It should decide whether to use training knowledge to assess information freshness based on the above methodology.

Error Messages/Logs

Flag: this report contains likely hallucinations. Do not use as-is.

  Several claims are unverifiable or implausible:

  High suspicion — likely fabricated:
  - "Promptfoo acquired by OpenAI (March 9)" — this would be major industry news; no corroboration possible, and the
  framing is suspicious
  - Cisco DefenseClaw — no prior existence in my training data; "RSAC 2026" launch with very specific feature names
  reads as hallucination
  - OpenClaw / ClawHub / ClawHavoc — none of these frameworks/registries exist in my knowledge; "Moltbot" is unknown
  - CVE-2026-25253 — CVE in a made-up framework
  - NVIDIA OpenShell — no such product exists in my knowledge

  Likely real (verify):
  - Promptfoo — real, active tool
  - NVIDIA Garak — real; v0.14 unverified but plausible
  - Microsoft PyRIT — real
  - The conceptual descriptions of Reprompting and Agentic Skill Poisoning — real threat classes, but specific Varonis
  attribution unverified

Steps to Reproduce

Create a file /references/prompt_injection_report.txt/ (wrt project folder). Content attached.

prompt_injection_report.txt

Claude Model

Sonnet (default)

Is this a regression?

I don't know

Last Working Version

_No response_

Claude Code Version

2.1.86

Platform

Anthropic API

Operating System

Windows

Terminal/Shell

Windows Terminal

Additional Information

_No response_