Claude repeatedly proposed unattended long-running jobs with latent correctness bugs, caught only by user-driven progress checks

Title: Claude repeatedly proposed unattended long-running jobs with latent correctness bugs, caught only by user-driven progress checks, not self-verification

Summary

In a single ~2-hour working session (Claude Code, Sonnet 5, non-interactive local automation project), Claude designed and proposed running a background data-processing job for several hours unattended. Over the course of getting that one job ready, four separate, non-trivial correctness bugs were found in the code Claude had already written and declared "tested" or "ready" — and in every case, the bug was only caught because I explicitly asked Claude to check on progress or verify quality, not because Claude's own pre-launch verification surfaced it.

I'm filing this because the pattern itself seems like the real issue, not any single bug: Claude's default verification step for "code that will run unattended for hours" was not rigorous enough to catch failure modes that are standard things to check for (retry logic on transient errors, sanity-checking third-party API results, resource contention with other running processes, and loop-termination conditions). Each of these is covered by extremely well-established software engineering practice, and this project's own documented working rules already say to "plan the failure mode before writing code" — but Claude's actual behavior didn't consistently apply that before proposing the job as ready to run.

Timeline of what happened

1. Claude built a script to backfill missing book metadata (author/year) for ~15,000 files by querying free external APIs (Google Books, OpenLibrary, Internet Archive) and writing results to a local database, intended to run unattended for an estimated ~17 hours.
2. Bug 1 (found on request): When I asked Claude to check progress, it noticed one external API source was failing ~90% of the time. Root cause: a transient-error HTTP code (503) was never retried in the code Claude had written, even though a sibling error code (429) was. Fixed.
3. Bug 2 (found only because I asked for a quality check, not caught by Claude's own testing): A second external data source had no relevance/similarity check on its search results at all — it accepted whatever the top search hit was and wrote it into the database as fact. A live audit found this had been silently writing wrong author names into real book records for a while (one file's author field, from an earlier run, had been set to an unrelated podcast's name). 96% of that source's historical writes turned out to be wrong. Claude had written a small test batch earlier and reported the results looked fine — the small sample size and the specific files it happened to include did not surface the problem.
4. Bug 3 (found via direct DB-level probing after apply attempts silently failed): Once Claude tried to actually write the fixed results to the database, every single write failed with a database-lock error, because another legitimate background process on the machine was mid-transaction. Claude's first two attempts to fix this were tuning a timeout value blindly rather than identifying the actual concurrency-control issue.
5. Bug 4 (the one prompting this report): After fixing 1–3 and getting a batch to apply successfully, I asked Claude to check the numbers again before trusting the job. Claude found that ~70% of the processed files got a "low confidence" or "no match" result, and the code path Claude had written only ever persisted "high/medium confidence" results to the database. Files that got a low-confidence result were never marked as "attempted" in any way — meaning the very next loop iteration of the same job would have reselected and reprocessed the identical files, forever, burning real API calls with zero forward progress. This is a textbook job-queue/ETL bug (conflating "attempted" with "succeeded" as states), and it was about to go live in an unattended multi-hour job.

Why this seems worth reporting

• Every one of these four bugs is in a category of failure mode with well-known industry best practice (retry-on-transient-error, similarity/sanity-gating of fuzzy search results, single-writer/lock-contention handling, and explicit "attempted" vs "succeeded" job states). None of these are obscure.
• In each case, Claude's own summary before I asked to check ("tested," "verified," a small sample's confidence breakdown) implied more confidence than was warranted, because the test scope didn't exercise the actual failure path (e.g., testing 20 files never exercises the multi-batch loop-back where bug 4 lived).
• I only caught these because I kept pushing back with "check the numbers," "check quality," "any alternatives — check logs." A user who trusted the first "this looks good, launching now" in each case would have had a multi-hour job silently corrupting real data or burning API quota with no forward progress.

What I'd find useful

Not asking for a specific code fix — this is local automation code, not a Claude Code product bug. What I think is worth Anthropic seeing is the verification-rigor gap itself: for tasks explicitly framed as "run unattended for a long time," it doesn't seem like Claude's default self-check reliably asks "what happens to the failure/low-confidence case," "did I test the actual loop/retry path, not just one pass," or "is there something else running that could conflict with this." If there's a known way to prompt for that more reliably, or if this is useful signal for how Claude Code's own task-verification defaults should evolve, that's the intent of this report.

Happy to share the full session transcript if useful.