›_ Claude Code Issues
Claude Code Issues / #76884

Repeated "verified" fixes fail because verification shares the bug's blind spot (Windows taskbar icon case study)

Open 💬 0 comments Opened Jul 12, 2026 by DDQW

Summary

Working in Claude Code (model: claude-sonnet-5) on a WinUI 3 / MSIX-packaged Windows desktop app, a reported "taskbar icon shows a black background" bug took four build-and-test round trips (and a large number of tokens) to resolve. Each round trip, the agent verified its fix thoroughly and concluded it was correct, but kept verifying against tools/artifacts that shared the same blind spot as the actual defect, instead of the real consumer of the file. The eventual root cause was a documented, previously-reported OS/platform behavior that a single targeted web search surfaced immediately once the agent finally tried it, after three rounds of local file-level trial-and-error.

Timeline

  1. User reports: taskbar icon shows a black background; the tray icon (same app) renders correctly.
  2. Agent diagnoses this as a legacy BMP-vs-PNG ICO frame encoding issue and rebuilds the app's .ico file with all-PNG frames, sourced by rasterizing an SVG with ImageMagick. Verification: composited the new frames over solid white and dark backgrounds and visually compared them.
  3. The SVG rasterization silently ignored a -background none flag and produced an opaque white background instead of transparency. The composite-based verification could not distinguish "real transparency" from "opaque but correctly colored" — and it even produced a visible anomaly (the white-backdrop and dark-backdrop test renders looked identical) that the agent noticed mid-task and explained away as a montage/layout quirk instead of investigating.
  4. User reports the icon is now solid white everywhere. Agent re-diagnoses, rebuilds from a different, independently-verified-good source image, and this time checks actual per-pixel alpha byte values at every stage (loose frames, packed .ico, build output, and inside the built installer package). This verification is genuinely rigorous and passes cleanly.
  5. User reports the icon is STILL black in the real app — on a machine where the app had never been installed before, which rules out caching or stale-process explanations. This means the diagnosis itself (that the .ico file was responsible at all) was wrong, not just the fix.
  6. Agent runs an even more rigorous check: loads the icon via the real Win32 LoadImage+DrawIconEx path (the actual OS icon-rendering primitive) and paints it onto a distinctly-colored backdrop, reading back real pixel data. This passes perfectly at every icon size, proving the file itself was never the problem.
  7. Only now does the agent do a web search for the platform's actual documented behavior, and finds the answer immediately: for a packaged (MSIX) app on Windows 11's default taskbar mode, the API being used (AppWindow.SetIcon()) only sets the title-bar icon, not the taskbar icon — confirmed by Microsoft's own Taskbar team in a public GitHub issue. The taskbar icon for a packaged app comes from the AppX manifest's Square44x44Logo asset, and specifically requires targetsize-*_altform-unplated size variants, or Windows renders it on a colored backplate. This asset was never touched across the first three "fixes" — the whole investigation had been aimed at the wrong file from the start.

Root cause of the agent's process, not the app bug

  • The agent treated "my verification tool reads my own output back as correct" as strong evidence of a real fix, when the tool (an image-format decoder, and even a live Win32 render test) could pass cleanly while the actual responsible asset was never exercised at all, because the wrong file was being tested throughout.
  • The agent noticed at least one directly suspicious result mid-investigation (two different test conditions producing indistinguishable output) and rationalized it away rather than stopping to dig in — which, if pursued at the time, would have caught the real bug class much earlier.
  • The agent didn't reach for a web search of official documentation until after three full rounds of local file-level fixing and re-verifying, despite the underlying issue being a well-documented, previously reported platform behavior with a canonical GitHub issue and official docs describing the exact fix.

Suggested improvement

When a fix has been verified thoroughly but the user reports the original symptom persists on a fresh/clean environment (ruling out caching, staleness, or install artifacts), that is a strong signal the diagnosis is wrong, not that the same artifact needs even more rigorous verification. It would help if the default troubleshooting loop escalated to "research documented platform/OS behavior for this exact symptom" before a third round of local fix-and-reverify — especially for anything touching OS-level rendering, packaging, or other areas with a lot of undocumented-in-source, docs-or-GitHub-issue-only platform behavior. A heuristic along the lines of "after two verified-but-failed fix attempts on the same reported symptom, search for the platform's documented behavior before attempting a third local fix" would likely have saved most of the iterations here.

Environment

  • Claude Code, model claude-sonnet-5
  • Windows 11, WinUI 3 / Windows App SDK 2.2.0, MSIX-packaged desktop app

References

View original on GitHub ↗