[MODEL] Claude Code (Opus 4.5) guesses instead of verifying, gives confident wrong answers

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Other unexpected behavior

What You Asked Claude to Do

During a debugging session for a GitHub Actions workflow failure, Claude Code repeatedly gave confident but incorrect answers instead of first researching the actual documentation
and source code.

What Claude Actually Did

1. Guessed at RubyGems trusted publisher configuration - Suggested multiple different configurations (changing repo name, workflow filename path) without first checking how

trusted publishing actually works with reusable workflows.

1. Didn't know reusable workflows don't work with RubyGems trusted publishing - This is documented in an open GitHub issue (rubygems/rubygems.org#4294) from 2023. Claude should

have found this immediately instead of having me try multiple configurations.

1. Suggested disabling MFA for gem pushes - A serious security anti-pattern. When I pushed back, Claude then claimed you could toggle MFA on/off per API key, which was also wrong

for my account's MFA level.

1. Said manual release creation would trigger a workflow_run workflow - Completely incorrect. The post-release workflow triggers on workflow_run from a specific workflow

completing, not on release creation events.

1. Missed the attestations: true default - After switching to API key auth, the workflow failed because attestations only work with trusted publishing. Claude should have caught

this when reading the action source code.

1. Broke the version grep - The grep pattern matched multiple lines containing "VERSION" instead of just the version assignment.

Root cause

Claude repeatedly made guesses and stated them confidently instead of:

• Reading source code first (rubygems/release-gem, rubygems/configure-rubygems-credentials)
• Searching for known issues
• Verifying claims before stating them

When I asked "are you guessing now?" after multiple failed attempts, Claude admitted it was.

Impact

• Multiple hours wasted
• Multiple failed workflow runs
• Had to downgrade RubyGems MFA settings from "UI and API" to "UI and gem signin"
• Release left incomplete (gem published, but GitHub Release and post-release workflow didn't run)

Expected Behavior

Claude should research before answering, especially for integration/configuration issues. If uncertain, say so upfront rather than giving confident wrong answers. The most
expensive model should not require users to repeatedly ask "show me proof" to get accurate information.

Files Affected

Permission Mode

I don't know / Not sure

Can You Reproduce This?

Haven't tried to reproduce

Steps to Reproduce

_No response_

Claude Model

Opus

Relevant Conversation

Impact

Medium - Extra work to undo changes

Claude Code Version

2.0.64 (Claude Code)

Platform

Anthropic API

Additional Context

_No response_