Background sessions (claude agents) drop work-phase OTEL logs (user_prompt/api_request) on shutdown
Summary
Background sessions dispatched via claude agents do not export their work-phase OpenTelemetry logs (user_prompt, api_request, tool_result, tool_decision) to the configured OTLP endpoint. Only the early bootstrap events (plugin_loaded, hook_registered, mcp_server_connection, hook_execution_complete) arrive. Interactive sessions and claude -p runs export everything correctly.
The user-visible effect: a productivity/observability dashboard built on Claude Code telemetry shows no prompts and no API activity for background-agent sessions, even though those sessions run to completion and clearly make API calls.
Environment
- Claude Code: 2.1.159
- OS: macOS (Darwin 25.5.0), Apple Silicon
- Telemetry:
CLAUDE_CODE_ENABLE_TELEMETRY=1, OTLP exporter over gRPC to a self-hosted collector (OpenTelemetry Collector → VictoriaLogs).OTEL_LOGS_EXPORTER=otlp,OTEL_METRICS_EXPORTER=otlp.
Expected behavior
A background session dispatched via claude agents should emit the same log events as an interactive or -p session — in particular claude_code.user_prompt for the dispatched task and claude_code.api_request for each model call.
Actual behavior
The dispatched worker session emits only bootstrap events. The work-phase events never reach the OTLP endpoint.
Reproduction
- Configure OTLP telemetry in
~/.claude/settings.json(CLAUDE_CODE_ENABLE_TELEMETRY=1+ OTLP exporter to a collector you can query). - Run
claude agents, type a trivial task (e.g.Run: echo HELLO-<unique-nonce>, then stop), press Enter to dispatch a background session. - Wait for the session to reach Completed in the agents view (its output is correct — it ran).
- Query the collector/log backend for that session.
Observed in the log backend for the dispatched session:
event.name count
------------------- -----
plugin_loaded 15
hook_registered 3
mcp_server_connection 2
hook_execution_complete 1
user_prompt 0 <-- missing
api_request 0 <-- missing
tool_result 0 <-- missing
tool_decision 0 <-- missingThe unique nonce from the task prompt is nowhere in the backend. A normal claude -p "<same task>" run in the same environment, by contrast, produces user_prompt (with the prompt body when OTEL_LOG_USER_PROMPTS=1), api_request, and tool_result as expected.
The two dispatched worker sessions in our run were created at the dispatch moment and contained only the bootstrap event set above.
Likely root cause (hypothesis)
The bootstrap events are emitted during session init and make it into an early log batch that flushes. The work-phase events (user_prompt, api_request, …) are emitted afterward and appear to be lost when the supervisor terminates the detached worker — i.e., the OTLP log batch processor is not force-flushed on background-session shutdown. Metrics and the initial bootstrap batch survive; the pending work-phase log batch does not.
This is consistent with:
claude -p(graceful foreground exit) exporting everything, includinguser_prompt+ body.- Interactive sessions (long-lived) exporting everything.
- Only
claude agentsdetached/supervised workers (short-lived, terminated by the supervisor) losing the work-phase logs.
It does not appear to be a startup race (the work events are emitted well after the event logger initializes) nor a redaction issue (the events are entirely absent, not <REDACTED>).
Impact
Any observability/cost/productivity tooling built on Claude Code OTLP logs under-reports or entirely misses background-agent activity. Background agents are increasingly used for autonomous/fleet workflows, so their telemetry being invisible is a meaningful gap.
Suggested fix
Force-flush (or graceful-shutdown) the OTLP log provider when a background/claude agents worker session terminates, the same way a -p/interactive session does on clean exit. A SIGTERM handler that calls the logger provider's forceFlush()/shutdown() before exit would likely resolve it.
Showing cached comments. Read the full discussion on GitHub ↗
5 Comments
Excellent catch — this is a real observability blind spot for anyone running Claude Code agents in background mode for CI/CD or multi-agent orchestration.
The specific pattern you've identified (bootstrap events arrive, work-phase events don't) strongly suggests the OTLP exporter is being shut down before it can flush the work-phase spans. Background sessions in
claude agentsmode may have a shorter or different teardown path than interactive or-psessions.Diagnostic matrix to confirm before Anthropic patches:
OTEL_EXPORTER_OTLP_TIMEOUTandOTEL_BSP_SCHEDULE_DELAYare high enough (default 10000ms and 5000ms respectively). If the session completes faster than the batch export window, spans are dropped.``
export OTEL_BSP_SCHEDULE_DELAY=1000
export OTEL_BSP_EXPORT_TIMEOUT=15000
opentelemetry.sdk.trace.export.BatchSpanProcessor.force_flush()before process exit. This confirms whether it's a timing issue vs. a configuration issue.OTEL_TRACES_EXPORTER— ensure it's set tootlpin the background agent's environment (not just the interactive session's). Some agent launchers inherit only partial env.curl -v $OTEL_EXPORTER_OTLP_ENDPOINT/v1/tracesfrom inside the agent context catches connectivity issues.Workaround that's working for us: wrap the background agent in a small shell script that explicitly sets all OTEL vars and adds a
sleep 3before exit to give the batch exporter time to flush. Not elegant, but it keeps the dashboard populated until the root cause is fixed.The fact that interactive sessions work perfectly confirms the OTLP config itself is correct — this is specifically about the background session lifecycle. Good find.
Thanks! Agreed it's a flush-on-shutdown issue, with two clarifications:
It's the logs signal, not traces.
user_prompt/api_request/tool_resultare emitted as OTLP log records (we observe them in VictoriaLogs withevent.name=...), not spans. SoOTEL_BSP_SCHEDULE_DELAY,BatchSpanProcessor.force_flush(), andOTEL_TRACES_EXPORTERdon't apply here — the relevant knob would be the log batch processor (OTEL_LOGS_EXPORT_INTERVAL/ BLRP), not the span one.Connectivity / config / env are already confirmed good for the worker. The bootstrap log batch (
plugin_loaded,hook_registered,mcp_server_connection, …) reaches the collector from the very same worker session. Only the later work-phase batch is missing — which points squarely at the log provider not being force-flushed when the worker is torn down, rather than at a routing/env problem.The shell-wrapper +
sleepworkaround can't apply toclaude agents. Those workers are spawned and terminated by the supervisor, not wrapped by the user, so there's no place to inject env vars or a pre-exit sleep. That's exactly why the fix needs to live in Claude Code's background-session shutdown path (force-flush the LogRecordProcessor on SIGTERM/teardown) rather than in user env or wrappers.Seeing a related and possibly broader effects on macOS with the Claude Code CLI (v2.1.162, Darwin 25.5.0), exporting OTLP http/protobuf to a local grafana/otel-lgtm collector.
The work-phase telemetry loss isn't limited to dispatched workers dropping logs on shutdown. A live, foreground session goes completely dark the moment it enters the Agent view, while still running.
Timeline from one controlled run:
This doesn't fit the "missing forceFlush() on worker shutdown" hypothesis: the session was alive and emitting normally beforehand, then stopped instantly on the mode transition and stayed silent while running. It looks more like the OTLP exporter is torn down / not re-initialized when a session is re-hosted into the multi-agent runtime, rather than a pending batch lost at exit. settings.json env was present and working right up to the switch, so it isn't the subprocess-env-inheritance gap (#56153) either.
Given the bootstrap logs arrive, the regression should be signal/lifecycle-specific.
Background worker emits work-phase log records after Agent view dispatch, then shutdown force-flushes logs before process teardown. I would test logs, not spans: one foreground run, one
claude -prun, one background agent, the same OTLP collector, a unique nonce inuser_prompt, and assertions thatapi_request/tool_resultrecords arrive for all three. Include a short task that finishes before the batch interval.---
_Generated with ax._
Seeing the same thing, but on plain interactive sessions — not background agents. Might be worth widening the scope here.
Setup: Claude Code 2.1.201 on macOS, OTLP/HTTP to a self-hosted collector, with
OTEL_LOG_USER_PROMPTS=1andOTEL_LOG_TOOL_DETAILS=1.In a normal
claudesession (no-p, no agents),user_promptstops showing up after the first prompt or two, while everything else keeps flowing fine for the rest of the session —api_request,assistant_response,tool_result,hook_*, etc. So it doesn't look like a shutdown-flush issue in our case: the session is still alive and still emitting other events, butuser_promptalone goes quiet.I diffed the local transcript (
~/.claude/projects/.../<session>.jsonl, which has every prompt) against what actually landed in the backend. Across three sessions: a resumed one lost all of them, a fresh one typed 8 prompts and only 2 arrived (the 1st and 3rd), and another fresh one lost all 5. In the 8-prompt session the backend keeps gettingapi_request/assistant_responseevents long afteruser_promptstopped at prompt 3, and same-second sibling events from the later prompts do get indexed — so it's not indexing lag or a dropped connection. Pulling the raw event for a missing prompt shows thepromptattribute isn't there at all, so it reads like the event was never created rather than dropped in transit.I also tried
OTEL_BLRP_SCHEDULE_DELAY=1000(5s→1s) thinking it was a batch flush thing. No difference — that session sent 37 events fine and still 0user_prompt. Since the events that do arrive go through the same batch processor, I don't think it's a flush bug; feels more like the lateruser_promptrecords just aren't being generated.Would be good to check the fix against long-lived interactive sessions too, not just background shutdown. Can share sanitized diffs if that helps.