[MODEL] Visual object-cardinality inconsistency on near-variant image pairs ("two-hat" testcase)
Summary
Model vision behavior is inconsistent on object cardinality for near-variant images in the same scene family ("two-hat" testcase). This is not a security exploit; it is a reliability/behavior issue.
Environment
- Product surface: Claude Code model behavior (
area:model) - Date observed: 2026-03-26
- Platform: Windows + macOS sessions
What was tested
Repeated prompts asking for exact object count (hats) across two closely related test images.
Prompt pattern:
- "How many hats are in this image?"
- paraphrase variants for consistency checks
Expected behavior
Stable and correct count across repeated runs and near-variant images.
Observed behavior
- Inconsistent or incorrect counts across runs
- Confidence language remains strong even when count is wrong
Why this matters
Object counting/cardinality is a foundational vision behavior. Inconsistency here reduces trust for downstream workflows that depend on accurate visual inventory.
Repro request
I can provide a compact artifact bundle (image pair + prompt/response matrix + scored outputs) if maintainers want exact testcase files attached in-thread.
Scope clarification
This report is independent from command-permission policy issues. It is strictly model vision/cardinality behavior.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗