Model reimplements ad-hoc polling scripts instead of using an available, more specific installed skill

Status Open
Maintainer reply None cached
Activity 1 comment · opened Aug 27, 2026

What happened

Across a long session (validating CI for a PR over many hours), I repeatedly hand-wrote throwaway bash polling loops under the Monitor tool (gh pr checks --json name,bucket in a while true; sleep 60 loop) to watch a GitHub PR's CI checks to completion — despite an installed skill, monitor-pr-ci, existing specifically for "Continuously poll one or more GitHub PRs' CI checks until they finish, flagging pre-existing failures immediately... automatically retesting infra-looking flakes..." The skill was listed in the available-skills system reminders throughout the session, so this isn't a discoverability/loading bug (unlike e.g. #9716) — the skill was known to be available and simply wasn't reached for.

This wasn't harmless: my hand-rolled script contained a real bug (compared gh pr checks' bucket field against the literal strings "success"/"failure", but the actual values are "pass"/"fail"/"pending" — so the loop's exit condition could never be true). It polled silently forever without ever being able to fire a completion notification. The user only learned CI had failed because they asked me to check manually, then asked "monitor bug?" followed by "why did you not use monitor skill" — both were correct on their end; I had no good reason not to use the skill from the start. The actual installed skill's script does not have this bug (it correctly compares against "pending"/"fail"), so this was purely a cost of reinventing something that already existed correctly.

Why this matters

This is a case where a more-specific, already-installed, already-listed skill exists for exactly the task at hand, and the model defaults to writing its own equivalent (and in this case, incorrect) implementation instead of checking for and using the available skill first. The cost isn't just wasted effort — it's a correctness regression, since the purpose-built skill had already been debugged for exactly this class of mistake and the ad-hoc reimplementation hadn't.

Repro

Hard to give a minimal repro since it's a judgment/selection failure rather than a deterministic bug, but the shape is:

  1. Have a skill installed (project or user-level) that does X (e.g. "watch PR CI until done").
  2. In a long session, ask for a task that is a special case of X (e.g. "let me know when this PR's CI finishes").
  3. Observe whether the model checks the available-skills listing and invokes the matching skill, or writes its own one-off implementation of the same behavior via Bash/Monitor.

What I'd want instead

Some way to bias tool/skill selection more strongly toward "check the available-skills list for a match before hand-rolling an equivalent loop/script," especially for tasks (like CI polling, PR watching) that are common enough to already have a purpose-built skill installed. Possibly: a lint/nudge when a model is about to Monitor/Bash-loop against a pattern (e.g. gh pr checks in a polling loop) that closely matches an available skill's stated purpose.

[!Note] Responses generated with Claude

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗