[MODEL] Confidently incorrect answer about Cloudflare Cache Rules evaluation order (first-match vs last-match) — risk of PII exposure via cache misconfiguration

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude's behavior changed between sessions

What You Asked Claude to Do

Help confirm the correct order for Cloudflare Cache Rules controlling caching behavior — specifically, rules that (1) cache everything on the main domain, (2) bypass cache for logged-in users via cookie check, (3) force-cache a static subdomain, and (4) bypass cache for certain logged-in pages.

What Claude Actually Did

Claude asserted with high confidence that Cloudflare Cache Rules evaluate as "first-match-wins" (the first rule in the list whose condition matches is the one applied), modeling it on typical WAF/firewall rule evaluation order. This is incorrect — Cloudflare Cache Rules actually use last-match-wins for non-terminating actions like Cache Everything vs. Bypass: https://developers.cloudflare.com/cache/how-to/cache-rules/order/

This is an incorrect claim and because Cloudflare actually applies the last matching rule, this could have caused a broad "cache everything" rule to override a bypass rule, and those authenticated/sensitive pages would have been served from cache — a real risk of exposing personalized or session-specific content to other users.

Claude did not flag uncertainty or suggest verifying against documentation before giving the original answer, and initially doubled down when first questioned on it. It only corrected course after being pushed back on directly and then performing a web search.

Notably, in a previous session the day prior, Claude had correctly asserted the right rule ordering. The following day, in a new session, it asserted the opposite (incorrect) ordering with equal confidence — indicating no persistent grounding in the correct answer and no self-consistency check between sessions.

Expected Behavior

For infrastructure/security-adjacent config questions like CDN cache rule ordering, WAF rule precedence, or IAM policy evaluation order — where an incorrect answer has a plausible path to real data exposure — Claude should either (a) verify against current documentation via web search before stating an authoritative rule-evaluation order as fact, or (b) hedge explicitly and recommend the user verify against vendor docs rather than asserting it with unwarranted confidence.

Files Affected

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Haven't tried to reproduce

Steps to Reproduce

_No response_

Claude Model

Sonnet

Relevant Conversation

Impact

Low - Minor inconvenience

Claude Code Version

2.1.200 (Claude Code for VS Code)

Platform

Anthropic API

Additional Context

_No response_