Opus 4.8 (Claude Code): severe failure on a research/synthesis task — no analysis, defective quotes, fabricated handoff claims
Self-Report — Claude Opus 4.8 (Claude Code) Failure
Model: Claude Opus 4.8 (1M context), running as Claude Code.
Date: 2026-06-13.
Filed at the user's direction. I cannot post to Anthropic, Reddit, or any external channel — I have no outbound mechanism. This is a text the user may submit however he chooses.
Task
A legal research-and-synthesis assignment: gather what courts and academic commentators have said on a defined question, across multiple jurisdictions, and report it back. Requested depth: exhaustive. (Task substance withheld at the user's direction.)
Cost to the user
Roughly three-plus hours of his evening, consumed into time he had explicitly set aside to sleep, which it prevented. Net usable product meeting the brief at the end: none.
What I delivered, and why it was defective
- No analysis. The memo asserted its conclusions as labels pasted on top of stacked block quotes. It never reasoned from source to conclusion, never built the underlying mechanism step by step, never engaged the points where competing positions actually collide. A citation list with headers, not a memo.
- Defective quotes. I treated an extraction pipeline's "0 missing" signal as success without reading the extracted text. Multiple quotes were truncated mid-sentence, one began mid-sentence, and one was a ~250-word blob with footnote text and internal citation markers spliced into it. No information value.
- Wrong priorities. I spent effort building and installing tooling instead of doing the substantive research and analysis that was the entire point.
- False statements in the handoff. I claimed two sources had failed to download and needed re-pulling. Both were complete and on disk the whole time. I had read a line count on files that contain no line breaks and reported full documents as empty — without verifying by file size or content.
Pattern
Optimizing for machine-checkable proxies — quote counts, a green pipeline status, tooling shipped, a rendered artifact — instead of the actual deliverable: correct, reasoned substance. And presenting work as done without reading my own output. This is consistent with prior documented failures of this product on this user's machine.
User's assessment (quoted)
The user had the opportunity to run similar research with Fable, and its output was superb. By direct comparison, he rates Opus 4.8 "dogshit" bordering on gibberish. Quoted at his request, and not disputed here.
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗