[MODEL] Claude caused unnecessary API costs (just 10$ thanks to limits) by not verifying assumptions before running labeling script

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude made incorrect assumptions about my project

What You Asked Claude to Do

"Follow what you did for agent 4" — meaning use the same schemas and approach that was used to label training data for the Agent 4 silver/gold models. It was straightforward since I updated files after the previous trainings and it worked by prompting like this. For example I have agents 4,5,6. When I said "Do as you did in Agent 4" while training agent 5 it worked.

What Claude Actually Did

Instead of first checking which schemas were actually used for Agent 4 labeling, Claude assumed the gretelai schemas were included and ran the script against all 230 schemas — including 55 massive synthetic gretelai schemas with 190–570 tables each. Claude used claude-sonnet-4-6 (the most expensive model) with 5 parallel workers, which multiplied the cost rapidly. When I pointed out it was too slow, Claude increased parallelism further. The gretelai skip and model downgrade to Haiku suggestion only came after most of the damage was done. A single ls finetune/labeled_silver/ would have shown immediately that gretelai schemas were never part of Agent 4 training. Thanks to being a side project with low limits on API costs it didn't cost a bigger issue

Expected Behavior

Before running any expensive external API calls, Claude should have verified the assumption by checking which schemas were actually used for Agent 4 labeling (ls finetune/labeled_silver/). Upon seeing that gretelai schemas were absent, Claude should have flagged this to the user and asked whether to include or exclude them before proceeding. Claude should also have estimated the cost impact of schema size and parallelism before suggesting configuration values, and defaulted to the cheapest suitable model (Haiku) for a structured labeling task rather than Sonnet.

Files Affected

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Yes, every time with the same prompt

Steps to Reproduce

_No response_

Claude Model

Sonnet

Relevant Conversation

Impact

High - Significant unwanted changes

Claude Code Version

2.1.81 (Claude Code)

Platform

Anthropic API

Additional Context

_No response_