Opus 4.6: Poor code generation quality — basic errors in Playwright test output

Summary

During a session generating Playwright E2E tests from BDD feature files, Claude Opus 4.6 produced code with multiple basic quality failures that required repeated user intervention to fix. The earlier BDD authoring work in the same session was careful and well-researched, but the code generation phase was rushed and error-prone.

Specific failures

1. Incoherent dynamic import() expressions alongside static imports from the same package

Generated import('@playwright/test').BrowserContext as inline type annotations in function parameters — in the SAME FILE that already had import { test, expect } from '@playwright/test' at the top. This appeared across all three generated test files, multiple times per file. The fix was trivially adding import type { BrowserContext } from '@playwright/test' next to the existing import. This is not an edge case — it's a fundamental TypeScript pattern that the model got wrong repeatedly while simultaneously getting it right for value imports in the same file.

2. Ignored project infrastructure (Docker Compose)

Provided run instructions assuming local npx playwright test when the repository is configured for Docker Compose (docker-compose.yml with postgres, backend, frontend, and playwright services). The .env file and Docker setup were clearly present but not consulted before writing instructions.

3. Concurrency bug — shared mutable state across parallel test describes

Used let threatId: string at the outer test.describe scope, shared across nested describes that run in parallel with fullyParallel: true. This is a textbook Playwright concurrency bug.

4. Dead code and unused parameters

• SEVERITY_LABELS constant defined but never referenced
• _targetThreatRating parameter accepted but ignored — function always hardcoded intent=4, capability=4 regardless of input

5. Overly broad assertions

• page.getByRole('dialog').getByText('High') matching ANY "High" text in the dialog instead of targeting the specific likelihood/risk-rating field
• page.getByText(new RegExp(expected, 'i')) as an .or() fallback that would match button labels, not just the derived rating

6. Unscoped selectors

• page.getByRole('button', { name: /menu|more/i }).first() grabbing the first menu button on the entire page instead of scoping to the selected threat item

Pattern

The model appeared to rush the code generation phase, using "GENERATED: verify selector" markers as a crutch rather than thinking through the code. When confronted with the import issue, it fixed it but didn't proactively audit the rest of the output. The user had to explicitly ask for a full review, which then uncovered all the other issues listed above.

Environment

• Model: Claude Opus 4.6 (claude-opus-4-6[1m])
• Tool: Claude Code CLI
• Task: Converting BDD Gherkin scenarios to Playwright E2E test files
• Platform: macOS (Darwin 25.3.0)

Expected behavior

Generated code should be self-consistent (don't use dynamic imports when static imports from the same package already exist). Basic patterns (reading project config before writing instructions, avoiding shared mutable state in parallel test frameworks) should not require user correction.