Free ChatGPT 5.5 outperformed paid Claude Opus 4.7 (Max plan, /effort max) on a basic multi-file reconciliation task

Summary

Same task, same files, same prompt — given simultaneously to:

• Claude Opus 4.7 (1M context) via Claude Code — /effort max (deepest reasoning), Claude Max $200/month plan
• ChatGPT 5.5 thinking — free tier

ChatGPT got the answer correct on the first try, in roughly half the time. Claude Opus 4.7 produced a confidently wrong answer that overstated the missing amount by 5×, and only corrected itself after I pasted ChatGPT's solution back in for comparison.

This is not a borderline case. It is a basic file-handling failure on a task where the user is paying $200/month for the premium reasoning experience.

The task

Personal finance reconciliation:

• 1 spreadsheet (the user's transaction tracker)
• 4 monthly bank statements (PDF)
• 2 supporting PDFs (online-banking screenshots)
• 1 supporting spreadsheet (bank-side detail)

Prompt: cross-reference everything, find where the user's tracker disagrees with the bank, identify what's missing.

There were two copies of the user's tracker xlsx in the supplied folder structure — same filename in different subfolders — one was an older snapshot, one was the current version. The current one had 6 more rows than the older one.

What ChatGPT 5.5 (free) did

• Identified the most recent tracker version
• Reconciled correctly: tracker says $299.77 pre-deposit, bank shows $99.77, gap = exactly $200.00 = one missing Claude.AI charge on Apr 6
• Produced a clean reconciliation table including a workbook
• Total time: a few minutes

What Claude Opus 4.7 (/effort max, $200/mo Max plan) did

• Read the older copy of the tracker (the one ending several weeks earlier)
• Did not verify which file was current despite both files being clearly listed with different timestamps in the directory listing
• Confidently reported 6 missing transactions totalling –$370.01 net when in reality only 1 transaction of –$200.00 was missing
• Produced a polished reconciliation report (RECONCILIATION-REPORT.md) presenting the wrong analysis as definitive, with internally consistent math (the math worked because it was answering the wrong question against an outdated input)
• Only acknowledged the error after the user pasted the ChatGPT response into the chat
• Even then, the failure mode was instructive: when shown the correct answer, the model immediately understood, re-read the right file, and produced a correct corrected report. The capability was there. The discipline to verify file freshness before drawing conclusions was not.

Why this matters

This was not a hard reasoning task. It was a basic data-hygiene task: when two files share a name across folders, sort by modification time, confirm which is current, then proceed. ChatGPT's free tier did this without prompting. Claude Code's $200/month Max-plan Opus 4.7 with max effort did not.

The model wrote a TaskCreate plan with 10 subtasks. It marked them all completed. The first task — "Read tracker xlsx" — was marked completed despite the wrong file having been read. The checkbox-execution discipline did not catch the error because the failure was in what was read, not whether something was read.

Suggested fixes (orchestration-side, not model-side)

1. Duplicate-filename detection in the harness/skill layer. When two files with the same base name exist in the working set, the agent should be prompted to resolve the ambiguity before reading either.
2. Default mtime-sort for ambiguous file selection. When the user provides a folder rather than specific paths, freshness should be the tiebreaker.
3. Source-of-truth declaration in checkbox plans. A plan that says "Read tracker xlsx" should require the agent to declare which tracker.xlsx, with full path and mtime, before marking that subtask done.

Impact

User has cancelled and is reconsidering the subscription. Free ChatGPT outperformed paid Claude on a task type (financial reconciliation across multiple documents) that is squarely within the marketed competence of the premium tier. This kind of comparison, when it lands, is hard to come back from.

Reproduction

Anyone with two same-named xlsx files in different subdirectories of a target folder, plus several supporting PDFs, can reproduce. The failure does not depend on the specific data.

---

Filed anonymously. No personal data, file contents, identifiers, or third-party names included.