feature request: ability to log/trace the full llm span (query and response)

Status Closed — not planned
Maintainer reply None cached
Activity 10 comments · opened Jun 14, 2025 · closed Jan 23, 2026

as far as i can tell from spending half a day trying to figure this out, OTEL_LOG_USER_PROMPTS will log the prompt, but there is no way to log the response from claude.

!Image

is this true?

this seems crazy to me, is there a reason why everything but the response itself can be logged with otel? i don't see why it would be a privacy/security issue, since the response is returned to the user.

maybe there is something i'm missing, why is this a "privacy issue"?

View original on GitHub ↗

10 Comments

rawwerks · 1 year ago

playing around with this a little bit more, what i would specifically like to request is a OTEL_LOG_API_RESPONSES variable, which would default to false just like OTEL_LOG_USER_PROMPTS. (feel free to name differently)

functionally, this would contain the claude api responses in the same OTEL system as all of the other OTEL logs.

as a user, i can access all of the information separately (for example, see my workaround below) - so again i don't really understand why including the responses in the OTEL logging presents a user privacy or security issue.

---

for others who might also be frustrated by this limitation, right now i'm basically doing a workaround by combining the claude -p output with console otel logs.

unfortunately this becomes extremely messy if you want to try the interactive REPL (not -p ), so my workaround is probably not immediately relevant to the average user.

i make a script like this, then run it with ./script "my prompt"

#!/usr/bin/env zsh
set -euo pipefail

# ─────────────────────────────────────────────────────────────────────────────
# 1) Optional: Check for API key
# ─────────────────────────────────────────────────────────────────────────────
# if [[ -z "${ANTHROPIC_API_KEY:-}" ]]; then
#   echo "ERROR: Please export ANTHROPIC_API_KEY before running." >&2
#   exit 1
# fi

# ─────────────────────────────────────────────────────────────────────────────
# 2) Collect prompt
# ─────────────────────────────────────────────────────────────────────────────
if (( $# == 0 )); then
  echo "Usage: $0 \"Your coding prompt…\"" >&2
  exit 1
fi
PROMPT="$*"

# ─────────────────────────────────────────────────────────────────────────────
# 3) Prepare logging
# ─────────────────────────────────────────────────────────────────────────────
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
SESSION_ID=$(uuidgen)
LOG_DIR="${LOG_DIR:-$PWD/claude_logs}"
mkdir -p "$LOG_DIR"

JSON_LOG="${LOG_DIR}/${SESSION_ID}_${TIMESTAMP}.jsonl"

# ─────────────────────────────────────────────────────────────────────────────
# 4) Enable OpenTelemetry (metrics+events)  [oai_citation:0‡docs.anthropic.com](https://docs.anthropic.com/en/docs/claude-code/monitoring-usage)
# ─────────────────────────────────────────────────────────────────────────────
export CLAUDE_CODE_ENABLE_TELEMETRY=1
export OTEL_METRICS_EXPORTER=console       # prints metrics to console
export OTEL_LOGS_EXPORTER=console          # print events to console
export OTEL_LOG_USER_PROMPTS=1            # include prompt text in events
export OTEL_METRIC_EXPORT_INTERVAL=10000  # 10s for faster debug
export OTEL_LOGS_EXPORT_INTERVAL=1000     # 1s for faster event flush
export OTEL_METRICS_INCLUDE_VERSION=1

# ─────────────────────────────────────────────────────────────────────────────
# 5) Run Claude Code in non-interactive “print” mode,
#    with verbose, structured JSON output, and limited turns  [oai_citation:1‡docs.anthropic.com](https://docs.anthropic.com/en/docs/claude-code/cli-reference)
# ─────────────────────────────────────────────────────────────────────────────
claude -p \
  --max-turns 5 \
  --output-format stream-json \
  --verbose \
  "${PROMPT}" \
  >"$JSON_LOG"

echo "✅ Logged JSON events to $JSON_LOG"
rawwerks · 1 year ago

additional breadcrumbs for users in interactive mode: most of what you would want to trace is already being stored in ~/.claude/projects/[project]/[hash].jsonl.

again re: claude's statement that this is a "privacy" issue, the logs are already on my machine and saved by default. it just seems annoying to have to write a script to put this into an observability tool, and potentially manually merge this information with the current OTEL implementation.

iwinux · 1 year ago

Yeah. My OpenTelemetry collector received the logs but it's completely useless:

{
  "cache_creation_tokens": "1171",
  "cache_read_tokens": "13929",
  "cost_usd": "0.011956950000000001",
  "duration_ms": "11145",
  "event.timestamp": "2025-07-25T11:05:28.952Z",
  "input_tokens": "4",
  "model": "claude-sonnet-4-20250514",
  "output_tokens": "225",
  "session.id": "<REDACTED>",
  "terminal.type": "zed",
  "user.id": "<REDACTED>"
}

With fields like cost_usd and output_tokens, the log is definitely reported at response phase, yet it omits the actual response. What's the point!?

fabiodouek · 10 months ago

+1 - any update on this?

rawwerks · 10 months ago
github-actions[bot] · 8 months ago

This issue has been inactive for 30 days. If the issue is still occurring, please comment to let us know. Otherwise, this issue will be automatically closed in 30 days for housekeeping purposes.

knksmith57 · 8 months ago

Still relevant 🙏

github-actions[bot] · 7 months ago

This issue has been automatically closed due to 60 days of inactivity. If you're still experiencing this issue, please open a new issue with updated information.

marcindulak · 7 months ago

This issue was closed incorrectly despite recent human comments. This behavior of the bot is reported at https://github.com/anthropics/claude-code/issues/16497. Please upvote that issue, so maybe it gets noticed.

github-actions[bot] · 7 months ago

This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.