Claude fails to trace live app data flow — programmatic tests pass but live app wrong every time

What happened

Claude fixed a GLM nested covariate stepwise selection issue. All programmatic tests passed (18,268 values). But every time the user tested in the live app, the output was wrong. This happened 5+ times in one session.

Root cause

The programmatic tests call the engine directly with correct input. But in the live app, the dialog layer generates the input. The dialog put nested terms FIRST in the candidate term list (Covariate1(FactorA) before Covariate2), which caused wrong ANOVA table order.

Claude had the tracing output showing this exact problem (Model terms=['Covariate1(FactorA)', 'Covariate2', ...]) from the FIRST live test failure, but didn't read it properly. Instead, Claude spent 5+ iterations changing the engine code (tie-breakers, backward/forward order, ANOVA reordering) when the bug was in the DIALOG layer all along.

The pattern

1. Engine produces correct output when given correct input → programmatic tests pass
2. Dialog produces wrong input (wrong term order) → live app output is wrong
3. Claude keeps changing the engine instead of checking what the dialog sends
4. The tracing terminal shows the exact dialog output — Claude had this data but didn't use it

What should have happened

After the FIRST live test failure, Claude should have:

1. Read the tracing output to see what the dialog actually sent to the engine
2. Compared the dialog's term list order against the referent's candidate list order
3. Identified that nested terms were first in the dialog output but should be last
4. Fixed the dialog reorder — one line of code

Instead, Claude made 5+ engine changes, each breaking something new, before finally reading the trace and finding the one-line dialog fix.

Lesson

The tracing terminal exists specifically to catch dialog-engine mismatches. Claude must READ the trace output after every live test failure, not assume the engine is wrong because the programmatic tests passed.