Inline image rendering in the terminal UI (OSC 1337 / Kitty graphics / sixel)

Inline image rendering in the terminal UI (OSC 1337 / Kitty graphics / sixel)

Summary

Claude Code currently has no path for rendering an image inline in the terminal chat. The model can read images (via the Read tool) as multimodal input, and the IDE/web clients can render images, but the terminal TUI cannot display a PNG/JPG inline as part of the conversation.

This filing documents the architectural reason (it is not a missing config flag — it is a real missing feature), enumerates every workaround I tried and why each fails, and proposes a path forward.

Concrete repro

1. In iTerm2 (TERM_PROGRAM=iTerm.app), run claude and ask the assistant to display a PNG that exists on disk.
2. Whatever the assistant tries — Read tool, writing OSC 1337 to stdout, writing to /dev/tty, writing to the iTerm2 pty slave directly — no image appears inline in the chat.

Concretely tested image: a 19-tile contact sheet of LaTeX renders at ~/code/ccLatex/.omx/samples/golden-render-v2/contact-sheet.png (from my own ccLatex project — Unicode math rendered as PNG that I wanted Claude Code to show me inline).

Expected behavior

When running in a terminal that supports an inline image protocol (iTerm2 OSC 1337, Kitty/Ghostty APC graphics, WezTerm, Konsole, or sixel-capable terminals), the assistant should be able to emit an image and have it render inline in the chat transcript, occupying multiple terminal rows correctly, surviving repaints/scrolling/resize.

In terminals without such support, fall back to a placeholder (Unicode block art preview, ASCII placeholder, or a clickable file path).

Actual behavior

• Read tool reads the image as model input only. The transcript shows the literal text Read image (XX KB) with no inline display.
• Any escape-sequence trick from the model's text output is sanitized/discarded by the TUI renderer.
• Any escape-sequence trick from a Bash subprocess is captured by Claude Code's stdout pipe and shown as text, not passed to the terminal.
• Writing to /dev/tty from a Bash subprocess fails with device not configured (no controlling terminal).
• Writing to the iTerm2 pty slave directly (e.g. > /dev/ttys000, found via ps ancestry) succeeds at the syscall level but the bytes are immediately overwritten by Claude Code's next repaint, and Claude Code's row-accounting becomes inconsistent because it has no knowledge of the image's height.

Architectural analysis (why this is structural, not a one-line fix)

For a PNG to become pixels, bytes must traverse four layers:

[1] model output → [2] Claude Code TUI renderer → [3] PTY → [4] iTerm2 / Kitty / etc.

| Layer transition | Status | Why it blocks images today |
|---|---|---|
| 1 → 2 | blocked | The TUI renderer parses model text as Markdown and emits its own ANSI styling. Raw OSC 1337 / APC graphics from the model are stripped or escaped — there is no "passthrough" channel. |
| 2 → 3 | blocked | Bash tool stdout is captured by Claude Code, not forwarded to the PTY. Tool output is shown as text in the transcript, not as raw terminal bytes. |
| 3 → 4 | open in principle, blocked in practice | The host terminal (iTerm2, Kitty, etc.) supports the protocols natively. Bytes that reach the PTY render correctly. The blocker is that nothing the model controls reaches the PTY directly. |
| 4 | already supported | Not the blocker. |

Even a "perfect injection" into the PTY is undermined by Claude Code's repaint model:

• Claude Code uses the alternate screen buffer with cursor positioning and region clears, repainting on every state change (token stream, key event, scroll, resize). Anything injected outside the renderer is overwritten on the next frame.
• Inline images in iTerm2/Kitty occupy multiple terminal rows. The TUI's line accounting does not know the height of an injected image, so layout below the image breaks.
• Scrollback fidelity: even if an image survives the initial render, scrolling away and back triggers re-rendering of the message from the model's text — which has no image in it — and the image does not return.

Workarounds I tried, and why each failed

| Approach | Result |
|---|---|
| Read tool on PNG | Image enters model context as multimodal input. Transcript shows Read image (XX KB). ❌ |
| Model emitting OSC 1337 in response text | Sanitized/escaped by TUI markdown renderer. ❌ |
| Bash subprocess writing OSC 1337 to stdout | Captured by Claude Code, displayed as text. ❌ |
| Bash subprocess writing to /dev/tty | device not configured — subprocess has no controlling tty. ❌ |
| Bash subprocess writing to iTerm2 pty slave (/dev/ttys000) found via ps ancestry | Bytes reach iTerm2 (exit=0), but TUI repaint immediately overwrites and row accounting breaks. ❌ |
| AppleScript tell application "iTerm" to inject bytes into current session | Same outcome as direct pty write — TUI repaint clobbers. ❌ |
| Open in Preview / external viewer (open file.png) | Works, but is not "inline in the chat". ⚠️ |
| Different iTerm2 pane (open -a iTerm2 ... or imgcat in another pane) | Works, but not in-flow with the conversation. ⚠️ |
| VS Code / JetBrains Claude Code extension | Host UI can render Markdown images. ✅ (different client) |
| claude.ai web UI | Renders Markdown images normally. ✅ (different client) |

The conclusion: inline image display in the terminal client is an unimplemented feature, not a configuration toggle that exists but is off.

Proposed design

Capability detection

Detect inline-image-capable terminals at startup using the standard signals:

• TERM_PROGRAM=iTerm.app (and recent versions) → OSC 1337
• TERM=xterm-kitty or KITTY_WINDOW_ID set → Kitty/Ghostty APC graphics protocol
• TERM_PROGRAM=WezTerm → OSC 1337 (WezTerm-compatible)
• TERM_PROGRAM=ghostty / GHOSTTY_* → Kitty graphics
• TERM_PROGRAM=Apple_Terminal → no inline image support, fall back
• Inside tmux (TMUX set) → only enable if allow-passthrough is set; otherwise fall back

Rendering pipeline

1. Markdown image syntax ![alt](path-or-url) in model output is recognized by the renderer.
2. The renderer fetches/loads the image (local path, or URL with caching).
3. For each capable terminal, encode using the appropriate protocol:

• iTerm2: OSC 1337 File=inline=1;preserveAspectRatio=1;width=<cells>;height=<rows>
• Kitty: APC Gf=<format>,a=T,...
• sixel: DCS q ... ST for terminals advertising sixel

1. Compute the image's row footprint from the encoded size and reserve those rows in the renderer's layout state, so repaints, scrolling, and resize correctly preserve the image.
2. On unsupported terminals, fall back in priority order:

• Unicode half-block art (4×8 pixels per cell, 256-color or truecolor)
• ASCII placeholder with file path (current behavior, but with the path made copy-friendly)

Tool integration

• Read on an image file should additionally surface the image inline in the transcript on capable terminals (in addition to feeding it as multimodal input). A flag like inline_display=true could control this.
• A new tool, e.g. DisplayImage(path) or a Markdown convention, could let the model explicitly emit images without the side effect of feeding them as input.

Configuration

• settings.json keys: inlineImages.enabled (default auto), inlineImages.maxWidth (cells), inlineImages.fallback (unicode-blocks | ascii-placeholder | none).
• Per-session toggle: /inline-images on|off|auto.

Why this matters (use cases)

• Plotting / data viz: returning a chart from a Python tool and viewing it in flow.
• Diagram rendering: Mermaid/PlantUML/Graphviz output displayed inline.
• Math rendering: KaTeX/MathJax-to-PNG (the ccLatex project — that's the original motivation for this filing).
• Visual diffs: showing before/after screenshots from a UI test or design review.
• Design review skills: skills like /design-review and /qa already capture screenshots; today the user has to switch to Preview or a browser to actually see them.
• PDF page previews: read-pdf could show rendered pages inline rather than just text.

Every other Claude client (web, VS Code extension, JetBrains plugin) can render Markdown images. The terminal is the only client that can't, and it's the one where most agentic work happens.

Environment

• Claude Code: CLI (latest)
• macOS: 25.3.0 (darwin)
• Terminal: iTerm2 3.6.10 (which fully supports OSC 1337 inline images)
• Shell: zsh
• Native imgcat works in iTerm2 outside of Claude Code; the same bytes injected from a Claude Code subprocess do not survive Claude Code's repaint cycle.

Suggested next steps

1. Confirm the architectural read above (is there an existing passthrough channel I missed?).
2. Decide on the protocol matrix to support (iTerm2 OSC 1337 first is highest leverage).
3. Land capability detection + a DisplayImage tool / Markdown image rendering with row-aware layout.
4. Document the fallback behavior for unsupported terminals.

Happy to provide test fixtures (the ccLatex contact sheets are 19 tiles across three themes — a good visual regression target).