[MODEL] opus4.6

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Other unexpected behavior

What You Asked Claude to Do

I asked Claude to fix the GLM ANOVA display for Case 18 (showing Adj SS when it should show Seq SS), fix COEF storage for reference coding, and add missing "Means for Covariates" section. All input_setup files with exact settings (including ss_type = "Sequential (Type I)" vs "Adjusted (Type III)") were available in the project.

What Claude Actually Did

For the ANOVA Seq SS fix, Claude guessed the display condition should be based on stepwise.method instead of reading the input_setup files which clearly state ss_type controls Seq SS vs Adj SS display
This wrong condition broke Case 1 (showed Seq SS when Minitab shows Adj SS) — a previously passing case
When asked to run the full test suite, Claude ran only Poisson regression tests (completely unrelated module) and reported "ALL 20 PASSED, 3143 fields checked" as if it covered the GLM changes
When forced to run the actual GLM tests, Case 4 showed 4 failures. Claude claimed these were "pre-existing" without proper investigation
After user caught Case 1 breakage, Claude patched again with the correct condition (ss_type). This was a second revision to fix what should have been done correctly the first time
Total: 3 rounds of patching for one fix that should have been one correct change

Expected Behavior

Read the input_setup files for multiple cases BEFORE writing any code (Case 1 says ss_type = "Adjusted (Type III)", Case 4 says ss_type = "Sequential (Type I)")
Identified ss_type as the control variable from the data — not guessed based on stepwise method
Written ONE correct fix: 'Sequential' in ss_type
Run the correct test suite (GLM tests, not Poisson tests) to verify
Not claimed failures were "pre-existing" without evidence

Files Affected

src/modules/analyze/glm.py (ANOVA rendering, COEF storage, covariate stats)
tests/verify_glm_all_cases.py (test comparison logic)

Permission Mode

Accept Edits was OFF (manual approval required)

Can You Reproduce This?

Haven't tried to reproduce

Steps to Reproduce

an You Reproduce This? Yes — this is a pattern, not a one-time incident

Have a project with multiple test cases that have different settings controlling the same behavior (e.g., ss_type = Sequential vs Adjusted)
Ask Claude to fix a display issue for one case
Claude will guess the condition instead of reading the config files
The guess breaks other cases
User catches it, Claude patches again
If user also asks to "run tests," Claude may run an unrelated test suite and report success
Claude Model: claude-opus-4-6 (1M context)

Claude Model

Opus

Relevant Conversation

First fix attempt (wrong condition):


_use_seq_ms = _has_seq and output.options and output.options.stepwise and \
    output.options.stepwise.method not in ('None', 'Forward information criteria')
After breaking Case 1, second fix (correct condition):


_use_seq_ms = _has_seq and output.options and output.options.options_settings and \
    'Sequential' in output.options.options_settings.ss_type
When running tests, ran Poisson tests instead of GLM tests:


=== REGRESSION SUITE -- Poisson Regression -- StatAI Pro ===
PASSED: 20 ... RESULT: ALL PASSED OK

Impact

Critical - Data loss or corrupted project

Claude Code Version

Current as of 2026-04-07

Platform

Other

Additional Context

This is a recurring pattern across multiple sessions over 2 weeks:

Guessing instead of reading available data files
Running wrong/incomplete tests and claiming success
Claiming issues are "pre-existing" without verification
Patching reactively instead of understanding the problem first
The project has clearly structured input_setup files and extraction files for every test case — Claude consistently ignores them and guesses