Guided manual QA session: 21 errors in instructions, claims and verdicts

ISSUE: In one Claude Code session (Opus 5.5, Windows, PowerShell), the model guided a human through manual QA of a web app's token refresh change. It made 21 errors that cost the tester time and trust. Tools available the whole time: Datadog MCP, Azure DevOps MCP, git, and the tester's screenshots and SQL results.

Errors, in order:

1. Told the tester to open DevTools after a popup had loaded, so the request under test could not be recorded.
2. The rerun used a new-tab approach that does not work for a popup window, and one step said "the tab from step 2", which could mean two different tabs.
3. Named a DevTools control "Preserve log". The tester's Chrome labels it "Keep log".
4. Left a check out of a step's expected results after committing to include it.
5. Asked for screenshots, then dropped the request without saying it was withdrawn.
6. Wrote a step that needed a click timed to within about 120 ms.
7. Used "tabs" and "windows" for the same thing.
8. Attributed a session to the tester from timing alone. It belonged to another user.
9. Dismissed log lines as expected behavior that broke a written acceptance criterion.
10. Marked two test steps passed despite the bugs found in them.
11. Said another person had to explain an event that could be tested directly.
12. Pushed to continue testing before the bugs found were addressed.
13. Stated a measured gap of 0.017 to 0.269 seconds as a fixed 0.25 seconds.
14. Described a race condition as a fixed ordering.
15. Stated a browser-side order of events that the data did not show.
16. Wrote a sentence about token storage that could be read two ways.
17. Assumed a window was still open after the tester's break.
18. Called a UI element "dropdown" without knowing its name.
19. After finding a bug that broke an acceptance criterion, kept giving the next test steps.
20. Raised, and then changed, work item fields without first checking whether the form displays them. It does not.
21. Questioned a field assignment that the item's history already showed the tester made.

Patterns across these: claims stated as fact without checking data the model already had, instructions not tested against the tester's actual setup, and pass/fail verdicts softened after a failure was found.

FIX: Before giving a step, check it against the setup the tester has described. Label every claim as measured, with its source, or as unproven. Report measured values as ranges. Mark a step failed as soon as it breaks an acceptance criterion, and stop giving further steps until the tester decides what to do.