[MODEL] Visual object-cardinality inconsistency on near-variant image pairs ("two-hat" testcase)

Resolved 💬 3 comments Opened Mar 27, 2026 by Pro777 Closed Apr 27, 2026

Summary

Model vision behavior is inconsistent on object cardinality for near-variant images in the same scene family ("two-hat" testcase). This is not a security exploit; it is a reliability/behavior issue.

Environment

  • Product surface: Claude Code model behavior (area:model)
  • Date observed: 2026-03-26
  • Platform: Windows + macOS sessions

What was tested

Repeated prompts asking for exact object count (hats) across two closely related test images.

Prompt pattern:

  • "How many hats are in this image?"
  • paraphrase variants for consistency checks

Expected behavior

Stable and correct count across repeated runs and near-variant images.

Observed behavior

  • Inconsistent or incorrect counts across runs
  • Confidence language remains strong even when count is wrong

Why this matters

Object counting/cardinality is a foundational vision behavior. Inconsistency here reduces trust for downstream workflows that depend on accurate visual inventory.

Repro request

I can provide a compact artifact bundle (image pair + prompt/response matrix + scored outputs) if maintainers want exact testcase files attached in-thread.

Scope clarification

This report is independent from command-permission policy issues. It is strictly model vision/cardinality behavior.

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗