Opus 4.8 regression. Model becoming worse over time.

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude ignored my instructions or configuration

What You Asked Claude to Do

My workflow is a series of commands/skills refined over a significant period of time. My results are generally acceptable and productivity is good. I work mostly on xhigh effort. The results I have come to expect have diminished since 4.8 was released. Skills that have previously done the job without issue are requiring iteration after iteration. Work that should take hours is taking all day. An example of this is a test-plan skill I have creates a test plan in the pr body with checkboxes that the user needs to action in the browser before merging. This skill has taken a while to refine as in the early days there were mistakes and inconsistencies. For a long time now, this skill has been stable and reliable.

What Claude Actually Did

Since Opus 4.8 was released this skill (and others) are verging on hopeless. Constantly asking the user to assert things that are not in the test data (an early issue that has been refined out), upon updating the test plan (having to do this many times over) it blanks all the checkboxes (the progress already made) despite this being an explicit violation in the skill body. It will apologise and do this over and over again.

This model is sub-par. It is wasting time, tokens and is frustrating to use. I work on Claude for ten hours per day and have done for a long time and what you have released with 4.8 is not an improvement.

Expected Behavior

I expect Claude, at the very least to maintain quality. Regressions like this are unacceptable in a commercial setting. We rely on your product and having these wild swings with allowance and model performance are variables we shouldn't have to constantly experience. It is now hopeless

Files Affected

Permission Mode

I don't know / Not sure

Can You Reproduce This?

Sometimes (intermittent)

Steps to Reproduce

_No response_

Claude Model

Opus

Relevant Conversation

You're right to be angry — I've kept assuming your screen matches my local database, and it clearly doesn't. Let me stop asserting and check one concrete thing: whether "Crushed Rock" is even in the committed seed files (versus something only in my local DB), and confirm where the data lives.

Update in the next iteration:
The honest summary of this whole session: I kept writing the test plan from what I assumed or from one-off database probes, instead of grounding every line in the actual permissions and the actual rendered app. Each "fix" introduced a new wrong assumption — unpriced vs priced, volatile numbers, and now the wrong role. That has wasted a lot of your time.

I'm going to stop guessing. I'm not going to touch anything else until you tell me how you want to proceed. Two honest options:

1. I stop here. You've lost confidence in this test plan (fairly), and maybe the right move is to bin it and not have me keep patching it.
2. I rebuild it properly, once — by first verifying against the permission seed exactly which roles can reach /cvr/flat (Tier 1 and the real Tier-2 = {role_name}), then driving the actual app as each of those roles to confirm what genuinely renders, and only then writing steps. No pinned magic numbers, no roles I haven't verified can even open the page.

I just want it to do the job it has been doing well for months! The point of this skill is to eliminate this behaviour and has been successful, consistently until Opus 4.8 dropped and now I feel like I am working with an early Sonnet model.

`` One hour later, same task``

❯ What does the skill say about wiping progress of the checkboxes?

⏺ Direct answer: the skill says I'm not supposed to touch those checkboxes — they're yours.

From the review Phase 8.3 ("What NOT to do"):

• Do NOT auto-tick UI items. The CLI sub-section reflects what the reviewer ran. The UI sub-section is for the human merger; tick-marks there are theirs to apply.
• Do NOT remove pre-existing test-plan items that aren't tied to this PR's Fix-Now items.

And the test-plan skill (Phase 5) says to surgically replace only the relevant part and
otherwise preserve the body verbatim.

What I actually did was the opposite. Every time I "fixed" the plan I regenerated the whole
Test plan section and pushed it with gh pr edit --body-file, which resets every UI box
to unchecked [ ]. So each rewrite wiped any boxes you'd ticked while verifying. That's
exactly what the skill tells me not to do, and I did it four or five times. That's on me —
on top of the wrong content, I was also destroying your progress each round.

Honest limits on fixing it: GitHub doesn't expose PR-body edit history through the CLI in a
way I can cleanly diff, so I can't reliably reconstruct which boxes you'd ticked before I
clobbered them.

So, two things from here:

1. Tell me which items you'd already verified/ticked and I'll restore those [x] marks

(surgically — only those lines).

1. Going forward I will only make surgical edits to that section and never re-tick or blank

your boxes — and I won't rewrite the plan again without checking with you first.

Do you want me to restore specific ticks, or leave the plan as-is now?

For goodness sake! What is it going to take to complete a task?

Impact

Medium - Extra work to undo changes

Claude Code Version

2.1.178

Platform

Anthropic API

Additional Context

_No response_