Cost estimation blind spots when building agent systems with Claude Code

Cost estimation blind spots when building agent systems with Claude Code

Context

I used Claude Code (Opus 4.6, 1M context) over 7 days to build a 9-skill personal AI agent ecosystem on Hermes Agent (Nous Research), deployed on a $5/month Hetzner VPS with Gemini 2.5 Flash as the LLM runtime. Claude Code was excellent at building the system — shipping skills, writing helper scripts, wiring MCP integrations, debugging gateway issues. But its cost estimation was dangerously wrong, and several architectural decisions had cascading cost/trust implications that weren't flagged proactively.

The cost estimation failure

What Claude told me: $1.73/month projected, $2/month hard cap is "more than enough" with ~15% headroom. This was presented with a detailed breakdown table showing per-cron and per-interaction costs.

What actually happened: Hit the $2 hard cap in 4 days. All agent crons silently stopped. Morning priority check-in didn't fire. I discovered it the next morning with zero warning.

Root cause — Claude only estimated 17% of actual cost:

| Cost category | Claude estimated | Actual |
|---|---|---|
| Input tokens (tracked in Hermes DB) | $0.33 | $0.33 |
| Output tokens (tracked) | $0.01 | $0.01 |
| Thinking/reasoning tokens | $0.00 (not considered) | ~$0.95 |
| Auxiliary API calls (title_generation, auto-detect) | $0.00 (not considered) | ~$0.18 |
| Failed calls, cache writes, retries | $0.00 (not considered) | ~$0.53 |
| Total | $0.34 | $2.00 |

Thinking tokens were 47% of total spend — the single largest cost category — and Claude never mentioned them in the budget planning. Gemini 2.5 Flash with reasoning_effort: medium generates ~25K thinking tokens per session at $0.70/M, but Hermes tracks them as 0 in its session database. Claude trusted the DB numbers instead of calculating from pricing documentation.

249 auxiliary API calls (title generation, auto-detect, vision auto-detect) happened silently across 69 sessions. None appear in session metrics. Claude didn't account for these.

MCP tool bloat inflated input tokens from ~12K to ~40K per session after adding MemPalace (29 tools). Claude noted this as "worth monitoring" but never recalculated the budget projection.

What Claude should have done

1. Include thinking tokens in cost estimates — these are documented in Gemini's pricing page and are the dominant cost for reasoning-enabled models
2. Verify theoretical estimates against actual billing — "Let's check your Google AI Studio spend after 24 hours of real usage" would have caught the 10x underestimate immediately
3. Recalculate after every architectural change — adding 29 MCP tools tripled per-session cost but the budget projection was never updated
4. Use hedged language for estimates — "projected $1.73/mo, let's verify after 48 hours" instead of "$2 is more than enough"
5. Flag auxiliary call overhead — title_generation runs on every session, which is knowable from Hermes's source code

Other findings from 7 days of agent building

Agent skill auto-discovery is unreliable

Built-in skills consistently outranked local custom skills in Hermes's skill matching. Even disabled skills were discovered and attempted. Required 5+ iterations per skill before discovering that explicit invocation ("use the X skill to...") is the only reliable pattern. Each failed iteration burned 20-40K tokens. Claude iterated on skill descriptions instead of checking logs for the root cause.

Silent failures are the worst failures

• hermes cron list showed Last run: ok when the skill actually threw a 429 error
• Groq fallback was "configured" for weeks but never worked (provider name mismatch: Api.groq.com vs groq)
• MCP server failed to start under the gateway (venv Python vs system Python) with no user-visible error — only discoverable in hermes logs errors
• Claude averaged 4-6 iterations per debugging task before checking logs, despite saving a feedback memory about "logs first" after the first occurrence

Security gaps weren't raised proactively

Claude built the entire agent system without raising these concerns — I had to discover them myself:

• A colleague DM'd my agent in Slack and received a pairing code with the exact CLI command to get approved (information leak in a 550-person workspace)
• Stakeholder intelligence notes (containing personal observations about my COO's management style) were initially planned for Coda, which is readable by all employees
• Calendar integration was built without any "never modify without explicit approval" safeguard until I asked
• Google OAuth credentials were handled correctly (stored in .env, chmod 600) but only because the project's security checklist required it, not because Claude flagged it

The "one more iteration" cost trap

When something doesn't work (skill matching, MCP connection, cron behavior), Claude's default is to tweak config and retry rather than diagnose root cause. Each retry is a new session costing $0.02-0.04 in the agent's LLM budget. Over 4 days of building, debugging iterations consumed a meaningful portion of the $2 budget. Better error diagnosis = fewer iterations = lower cost.

Recommendations for Anthropic

1. Cost estimation should account for thinking tokens by default. When a user asks "how much will this cost?" and the target model has reasoning enabled, the estimate MUST include thinking token projections. This is table stakes for budget planning.

1. Claude should proactively verify estimates against reality. After setting up a budget-sensitive system, Claude should prompt: "Let's check actual spend after 24 hours to validate my projection." Theoretical math alone isn't enough.

1. Claude should recalculate when architecture changes. Adding MCP tools, new skills, or changing reasoning effort should trigger an automatic "this changes your cost projection" flag.

1. Claude should raise security concerns before the user discovers them. Building an agent in an enterprise Slack workspace with 550 employees should trigger: "Who else can message this bot? What happens if they do? What data could leak?" These are obvious questions that weren't asked.

1. "Logs first" should be default diagnostic behavior. When an LLM-mediated system fails opaquely, checking error logs should be step 1, not step 5. Claude acknowledged this pattern after iteration but repeated it.

Impact

• $2 monthly budget exhausted in 4 days (should have lasted the full month)
• All 4 scheduled crons (morning priorities, evening review, security scan, weekly eval) stopped firing silently
• Trust in Claude's planning accuracy significantly damaged
• ~2 hours of debugging time spent on issues that log checks would have resolved in minutes

Environment

• Claude Code: Opus 4.6 (1M context) via VS Code extension
• Agent framework: Hermes Agent v0.9.0
• LLM runtime: Gemini 2.5 Flash (paid tier, $2/mo cap)
• Platform: Slack gateway on Hetzner CAX21 (ARM, 4 vCPU, 8 GB RAM)
• MCP: MemPalace 3.3.0 (29 tools exposed)