Model fails to trace full execution paths when making multi-phase code changes
Problem
When making code changes that span multiple phases within the same function, the model correctly modifies the first phase (e.g. data collection loop) but fails to trace through to dependent phases (e.g. assembly/evaluation of collected data). The result is silently broken code that passes superficial review.
Concrete example from this session
File: backend/app/services/v5/maut_pdf_ingestor.py
Function: _extract_pdf_text_with_routing()
The function has two structurally separate phases:
- Iteration phase — classify each PDF page, collect into
vision_rendersandpage_decisions - Assembly phase — build output text from
all_page_nos = sorted({pd.page_no for pd in page_decisions})
The model was asked to add a page_hint path (force Vision OCR for specific pages). It correctly modified the iteration phase — hint pages were added to vision_renders with continue. It did not read the assembly phase 20 lines below, which is driven exclusively by page_decisions. Because hint pages bypassed page_decisions, they were silently dropped from the output even after successful OCR.
The model only caught this when the user explicitly pointed it out. After the fix was applied, the model explained why it happened — which is the wrong order.
Pattern
This is not a one-off. Within the same session:
- Fix applied to diesel ingestor correctly (simpler structure: plain list assembly)
- Same fix copied to maut ingestor incorrectly (different structure: PageRoutingDecision-driven assembly)
- Model did not verify structural equivalence before writing
User impact
The user (a paying customer doing serious production work) had to review my output, identify the bug, and report it back to me — multiple times across multiple fixes in the same session. Each time I explained the mistake after being told. That is not acceptable quality for production code assistance.
What should change
Before completing a code change that modifies one phase of a multi-phase function, the model should:
- Read the entire function, not just the section being changed
- Identify all downstream consumers of modified data structures
- Verify structural equivalence when copying a pattern from one implementation to another
This is a systematic gap, not a one-time error. The user explicitly asked me to report it rather than apologize.