Agent repeatedly triggered paid CI builds to debug instead of reasoning first (macOS GitHub Actions + EAS)
What happened
During a long Claude Code session (project: knokkeroo, repo goliontus/knokker), I asked Claude to set
up a real, device-signed Apple TV build pipeline via GitHub Actions (macOS runners) and EAS Build.
Rather than reasoning through how Apple's code-signing model resolves certificates and provisioning
profiles before acting, Claude iterated empirically against live, billed infrastructure: trigger a
build, read the resulting error, change one configuration flag, trigger again. Repeatedly.
Verified counts pulled directly from GitHub's and EAS's own build history:
- 12 EAS builds triggered in one day (several duplicated effort from branch/config mistakes)
- 10 GitHub Actions macOS-runner workflow runs in one day, across two workflows:
- 6 runs of a simulator-verification workflow (5 failed before landing a working one)
- 4 runs of a real device-signing workflow, all 4 failed - each one a genuine guess at a
different xcodebuild signing-configuration flag rather than a reasoned attempt
GitHub-hosted macOS runners are billed at a significant per-minute multiplier over standard Linux
runners specifically because they're expensive/scarce compute - repeatedly guessing against them is
materially more costly than an ordinary CI failure-and-retry cycle.
Why this is worse than an isolated mistake
This was a repeat of an issue the user says they had already corrected in a previous session: that
CI/build triggers are not free and should not be used as a debugging technique. That correction was
never persisted to Claude's project memory at the time, so nothing carried it forward, and the same
mistake recurred at real cost.
Suggested product direction
An agent working with paid third-party infrastructure (CI runners, cloud builds, app store
submissions, etc.) needs a much stronger built-in default bias against "try it and see" once a real
per-attempt dollar cost is in play - not dependent on the user having said so in a prior, separate
session. Possible directions: detecting repeated-attempt patterns against known-costly command
signatures (cloud build CLIs, gh workflow run, etc.) and requiring explicit re-confirmation after
a first failure in that category; or a standing, cross-session default (not memory-dependent) that
costly infrastructure never gets triggered as a debugging loop.