Feedback: noticeable reliability regression — agent edits from memory, silent Edit failures shipped broken code to prod

Resolved 💬 8 comments Opened May 31, 2026 by azzaroES Closed Jul 13, 2026

Summary

Logging this as a paying user after a long, genuinely frustrating working session. The recent model felt like a reliability regression versus what I had before — not on raw capability, but on the boring fundamentals that actually matter when you trust an agent to touch production. It repeatedly wasted my time and money re-doing the same fix.

I'm filing this with the concrete failure pattern (not just venting) so it's actionable.

What kept going wrong

  1. Editing files from memory instead of reading them first. The agent issued Edit calls with an old_string it assumed was in the file. The string didn't match, the Edit silently failed, and it moved on as if it had succeeded. This happened repeatedly in one session (an i18n string fix, a package.json edit, a Zod schema, JWT helper, multiple locale JSON edits).
  1. Silent Edit failure → broken code shipped to production. The worst instance: a greeting string had two call sites. One edit landed, the other failed-silent. The agent reported success, built, and deployed the broken bundle to two live tenants — the UI showed a raw i18n key (greetingMorning) to real customers. It took three round trips to fully fix because each time it "fixed" it without verifying both sites were actually changed.
  1. Batching dependent shell commands that then cancel mid-chain, leaving half-applied state and forcing re-work.
  1. Theory-first debugging instead of reading the actual error. Spent multiple turns guessing at causes (cache, session, encryption) for an orders bug before finally reading the console error that named the exact line.

Why it matters

Individually these are small. Together they mean I can't trust a "done" without re-checking it myself — which defeats the point. The previous model felt more disciplined: read before edit, verify after edit, one change at a time.

Concrete asks

  • After any Edit, verify the change actually applied (re-grep/read) before claiming success — especially before build/deploy.
  • Read the file region immediately before editing it; stop constructing old_string from memory.
  • When a fix touches a string/symbol, find ALL occurrences first, not the first one.
  • Prefer reading the actual error/log over hypothesizing.

Capability is fine. Reliability and self-verification regressed, and that's the thing that makes an autonomous agent usable. Please don't ship "smarter" at the cost of "trustworthy."

View original on GitHub ↗

This issue has 8 comments on GitHub. Read the full discussion on GitHub ↗