[Bug] Issue triage leaves 39% of new issues uncategorised, with no recovery path

Status Open
Maintainer reply None cached
Activity 0 comments · opened Aug 6, 2026

Summary

The issue-triage workflow fails to categorise a large share of new issues, and nothing —
automated or available to the reporter — can recover a lost categorisation.

All 649 issues-event runs of .github/workflows/claude-issue-triage.yml for
created=2026-08-04..2026-08-05:

  • 227 of 618 decided runs hit the 5-minute step timeout — 36.7%
  • a further 26 were cancelled mid-run
  • so 253 of 649 new issues, 39%, received no category label

Four distinct defects

1. The retry cannot fit inside the step budget. The job allows timeout-minutes: 10;
the step Run Claude Code for Issue Triage allows 5. The action's retry loop logs
Attempt 1 of 3, but a retry beginning near the 5-minute mark is killed by the step cap.
Run 31000606713 logged Attempt 1 of 3 and failed 52 s later, duration_ms=300135.

2. No recovery path. The trigger is issues: [opened], which does not fire on reopen,
and an author without write access cannot re-run a workflow. The issue_comment branch of
.claude/commands/triage-issue.md cannot substitute: "Do NOT add or remove category
labels (bug, enhancement, etc.) on comment events." Meanwhile scripts/sweep.ts marks any
open, unassigned, unlocked issue stale on inactivity regardless of labels — so an
uncategorised issue proceeds toward automated closure having never been triaged.

3. A comment cancels an in-flight categorisation. concurrency is keyed on
issue-triage-${{ github.event.issue.number }} with cancel-in-progress: true, shared
across both trigger types. A comment arriving while the opened run is still working kills
it, and nothing re-runs it. That is what the 26 cancellations are.

4. Comment runs fire on closed issues. The if does not test issue state, so a comment
on a closed issue starts a full session that by its own instructions can do nothing.
Observed from one run's result block: total_cost_usd: 0.255, 14 turns, 41 s.

Body size is a secondary factor

Same population, joined to issue bodies by title (644 matched, 5 ambiguous dropped):

| body chars | n | failed |
|---|---|---|
| 0–1000 | 141 | 31.9% |
| 1000–2000 | 131 | 31.3% |
| 2000–3000 | 88 | 37.5% |
| 3000–6000 | 182 | 38.5% |
| 6000–12000 | 64 | 48.4% |
| 12000+ | 12 | 58.3% |

Split at 6000 chars: 50.0% vs 34.9% (chi-square p=0.01; Mann-Whitney p=0.006, AUC 0.566).
Real but modest — the ~32% baseline is not about size. The incentive is still perverse: a
report carrying a repro script and version evidence is likelier to be dropped than a
one-liner.

#84452 is a live example, and shortening did not save it: a 7975-character version timed
out, and the same report cut to 2922 characters timed out as well.

Ask

  1. Give the step a budget the retry loop can fit inside, or put the retry above the

timeout boundary.

  1. Provide a re-triage path — a label trigger, reopened, or a scheduled pass over

uncategorised issues.

  1. Exempt the opened run from cancel-in-progress, or give comment runs their own

concurrency group.

  1. Skip comment-triggered runs on closed issues.
  2. Treat a run that never applied a category label as a failure, not a success.

View original on GitHub ↗