computer use: add window-scoped screenshot (capture one app window, not the full screen)
Summary
The computer-use screenshot tool only captures the whole screen. Please add an option to capture a single application window (window-scoped capture).
Why the full-screen-only behavior is a problem
While guiding a user through a settings dialog in a native Windows app (NVIDIA App), screenshot returned an image of the Claude Code desktop window itself, not the target app. The target app was simply behind it. Nothing failed — a valid image came back, so there was no way to detect the mistake from the tool result alone.
Three distinct problems, all caused by capturing the screen rather than the window:
- Wrong window captured. Anything in front of the target is what you get. The target being occluded is invisible to the model.
- Coordinate scaling error. On this machine the screen is 3840×2160 and the returned frame is 1456×819 — a ~2.64× conversion sits between what the model sees and where it must click. The target window itself is 1298×942, which would have fit in the budget at native resolution with no scaling at all. The docs already note that DPI scaling is a primary cause of click misses; window-scoped capture removes the scaling entirely for the common single-app case.
- Privacy. The full screen includes every other window — mail, chat, file managers, credential folders. Masking non-allowlisted apps helps, but allowlisted-but-irrelevant windows and the desktop still go over the wire. Window scope sends strictly what the task needs.
Workaround (works well, suggesting it as the reference behavior)
On Windows, PrintWindow(hWnd, hdc, PW_RENDERFULLCONTENT /* 2 */) renders the window's own content into a bitmap:
- It works while the window is behind others — no need to raise it, so the user's focus, mouse and keyboard are never stolen. This matters a lot for background agent work.
- The bitmap is the window's native size, so no downscaling and no coordinate conversion.
flags = 2is required; with0, CEF/Chromium-based and WPF windows come back empty.
Two caveats worth building in:
- DirectX-direct-rendered and protected-content windows return an all-black bitmap, and
PrintWindowstill returnstrue— the return value cannot be used to detect this. Sampling pixels for uniformity is a reliable check. - Falling back to raising the window + screen capture should be opt-in, and should restore the previously focused window afterwards.
macOS has an equivalent path via CGWindowListCreateImage with a window ID.
Proposed API
screenshot(scope: "screen" | "window", window: { app?: string, title?: string, pid?: number, index?: number })
Returning the window's screen-space origin alongside the image would let the model convert image coordinates back to screen coordinates for clicking:
{ width, height, screenX, screenY, method: "window" | "screen" }
A list_windows tool (app name, title, pid, size, position) would make the window selectable without a screenshot first.
Environment
- Claude Code desktop app on Windows 11 Pro 26200, display 3840×2160
- Reproduced 2026-09-22 with a CEF-based native app (NVIDIA App, window 1298×942)