[FEATURE] Structured session journal: a configurable event stream with repo and OTLP sinks, detail levels, and privacy profiles

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

Claude Code produces a lot of work and almost no durable record of it. After a session ends, three questions are effectively unanswerable, and each is unanswerable for a different reason.

Related issues, none of which covers this: #38235 (conversation export, plus a request for hooks to reach message content — stale), #15222 (DECISIONS.md), #44200 (handoff.md for cross-session continuity), #40439 (context export), #40634 (permission audit: whether a tool ran by rule or by human approval), #32500 (session metadata for agents), #18645 (session export/import). OpenTelemetry covers a different problem — see Alternatives.

1. "What happened, and where?"

A change lands. Two weeks later someone asks why. The commit shows what changed; nothing shows what was asked, what was considered, what was tried and abandoned, or what a subagent decided on its own. Subagent work is the worst gap: the most consequential exploration often happens in a sidechain that leaves no trace outside the live session. /rewind makes it worse — work that was done and undone vanishes, so the record, if reconstructed by hand, describes a path nobody took.

2. "Where does the process stall?"

There is no session-level aggregate that would show it. OTel reports claude_code.active_time and per-tool events, but not the numbers that describe how a session actually went: how many iterations before an edit was accepted, how often proposals were rejected and for which tools, how many rewinds happened, how many times the same files were re-read across a session, how work split between inline edits and subagent fan-out. Those are the figures that tell a team its prompts are too vague, its context is thrashing, or its fan-outs are not paying for themselves. None of them is derivable from what is exported today.

3. "Can this be recorded without surveilling the person?"

Today the answer is: only by building it yourself. OTel's model is "we emit, you redact downstream" — user.email is included and organisations are told to filter it at the collector. Every event carries a precise timestamp. So the choice on offer is full timeline visibility or nothing, and identity and chronology cannot be separated. They are independent axes: a team can legitimately need to know who changed a subsystem without needing a minute-by-minute record of when someone worked. Nothing supports that today.

What I built, and where it stops

I run a Go hook on nine events that writes a semantic log to the repo: the prompt verbatim, subagent start/stop, a per-turn summary (model, effort, duration, subagent count, tool breakdown), the touched files annotated via git status --porcelain and git diff -U0 with create/modify/delete markers, line counts and exact changed ranges (+[50-61,200-205] -[120-124]), the final report, compactions, errors. Privacy choices are deliberate: no timestamps at all (append order carries sequence), no token counts, agent ids never logged — only readable agent types, one file per session keyed by session id with merge=union so parallel sessions never conflict.

It is a workaround, and it hits a hard ceiling:

• It parses an internal format. Per-turn stats come from reading the transcript JSONL for isSidechain, promptSource: "typed", effort, and the shape of message.content. That format is undocumented and does change — #18645 documents session files breaking between 2.1.5 and 2.1.9 as validation and fields shifted. My log will break silently on some future release.
• Subagents are opaque. I can log that an agent of type X started and stopped. The task it was given, what it concluded, and which files it touched are all in the sidechain, which is excluded from the turn's accounting.
• Rejected actions leave nothing. What Claude proposed and the user declined is invisible — often the most informative part of a session, and the subject of #40634.
• No link to git history. The log records changed line ranges but cannot tie an entry to the commit that eventually carried it, so ~ src/api.go +[50-61] decays into noise once the file moves on.
• Reads are unattributed. Read×5 says five files entered context, not which — so "what did it know when it decided that" is unanswerable.
• Rewinds are not recorded, so undone work reads as done.
• The redaction I want has to be reimplemented from scratch, and my version is crude: dropping timestamps entirely is the only lever I have, when what I actually want is coarse ones.

Every one of these is information the client holds and does not expose. No amount of effort in a hook reaches them.

Proposed Solution

One mechanism: a structured stream of session events, emitted by the client, with configurable detail, configurable sinks, and configurable privacy. The journal and the aggregates are two renderings of that stream, not two features.

1. A documented event schema

Semantic events at the level people reason about, not raw telemetry:

session.start / session.end
prompt                    (text, gated by detail level)
decision                  (what was proposed, what was chosen, why — when the model states it)
tool.use                  (name, target, outcome, approval source: rule | user | denied)
subagent.start            (type, task given)
subagent.end              (outcome, files touched, summary returned)
files.changed             (path, create/modify/delete, ± counts, changed ranges, commit sha when known)
rewind                    (anchor, what was discarded)
compaction                (trigger)
error                     (type, detail)
turn.summary              (model, effort, duration, tool breakdown, subagent count)
session.summary           (the aggregates below)

files.changed is the load-bearing one: paths and ranges are what make a record reconstructible rather than anecdotal, and the client already knows them from the edit tools.

2. Detail levels

{ "journal": { "detail": "reconstruct" } }

• audit — who, what, where, outcome. No prompt text, no reasoning, no code. Small enough to keep indefinitely.
• reconstruct — adds prompts, stated decisions, subagent tasks and conclusions, rejected proposals. Enough to answer "why did this happen" without storing code.
• full — adds assistant reasoning and tool content, for debugging a specific incident. Expected to be enabled temporarily.

Levels are cumulative and the schema is the same at each — a consumer written against audit keeps working at full.

3. Sinks

{ "journal": { "sinks": ["repo", "otlp"] } }

• repo — a human-readable file in the project, versioned with the code, reviewable in a PR. This is what is missing entirely today: the only destinations are ~/.claude internals and an OTLP endpoint.
• otlp — the same events into an existing collector, for teams that already run one.

Default layout: one journal per project with per-author attribution, so "what happened in this subsystem" is answerable in one place. A perSession option (one file per session id, keyed by author and short session id) suits repos that prefer to avoid merge pressure; with merge=union in .gitattributes either layout is workable.

4. Privacy profiles — identity and chronology as independent axes

The core of this request:

{
  "journal": {
    "identity": "author",        // author | anonymous | pseudonymous
    "timestamps": "coarse",      // full | coarse (day/hour) | none (order only)
    "content": "prompts"         // none | prompts | full
  }
}

Three independent dials, because the questions they answer are independent. identity: author, timestamps: none — attribution without a work-hours record. identity: anonymous, timestamps: full — process analysis without naming anyone. pseudonymous — stable per-person id, so sessions can be correlated without identifying whom.

This is the piece no downstream redaction can provide: a collector filter can drop a field, but it cannot turn precise timestamps into coarse ones without the client having emitted them that way, and by then the precise values have already crossed the wire.

5. Session-level aggregates in the same schema

session.summary carries the figures that describe how a session went and that nothing exports today:

iterations_to_accept      per accepted edit
proposals_rejected        by tool, with approval source (see #40634)
rewinds                   count, and work discarded
files_reread              same path read N times in one session
inline_vs_delegated       edits made directly vs via subagents
fanout_size               agents spawned per delegation
context_pressure          compactions, and how much preceded each

Content-free by construction, so they are safe at audit level and safe to keep long-term. They exist to diagnose the process — vague prompts show up as high iterations-to-accept, context thrashing as re-reads, unprofitable delegation as fan-out with low yield. Deliberately not normalised for comparing individuals: the same numbers vary more with task difficulty than with anything else, and the schema should not invite a use it cannot support.

6. Stable access for hooks

Expose the same events to hooks — this subsumes #38235's request for message content and removes the need to parse the transcript JSONL. A stable event contract is what makes a user-space implementation like mine unnecessary rather than merely awkward.

7. Constraints

Off by default; nothing is written until configured. No new API calls — every field comes from state the client already holds. repo sink writes append-only, one line-block per event, designed to merge. Backward compatible: a repo with no journal config behaves exactly as today.

8. Minimal viable version

Ship audit detail, repo sink, and the three privacy dials, with files.changed including commit sha and subagent.end including files touched. That alone makes "what happened and where, for each person" answerable, and it deletes the JSONL-parsing workaround people are building today. Aggregates, full detail and hook exposure can follow.

Alternative Solutions

OpenTelemetry. Solves a different problem well. Its audience is a security team streaming to a SIEM; its granularity is events, not turns; its privacy model delegates redaction to the collector, which cannot coarsen timestamps that were already emitted precisely. Raw file contents and code snippets are excluded from metrics and events by design, so files.changed with ranges and commit linkage is not derivable from it at any configuration. And there is no local sink at all — a journal that lives in the repo and merges with the code is outside what OTel is for.

Compliance API. Enterprise-only, and scoped to organisation-level events — logins, permission changes, administration. It does not describe what happened inside a coding session.

Transcripts in ~/.claude/projects. Undocumented internal format that changes between releases (#18645). Parsing it is what my hook does, and it is the part I expect to break.

Hooks alone. Hooks see tool events, not conversation content — the gap #38235 asks to close. Everything my hook knows about a turn comes from parsing the transcript behind the hook's back.

My own implementation. Detailed in the problem statement. It reaches roughly half of audit level and none of reconstruct, and the missing half is precisely the part only the client can supply.

Adjacent requests. #15222 (DECISIONS.md) and #44200 (handoff.md) both want a durable record, each shaped for one purpose and each requiring the model to author it as prose. A structured event stream produces both as renderings, without asking the model to remember to write a document. #40634 (permission audit: rule vs human approval) is a single field in the tool.use event here. #40439 and #18645 concern moving context between sessions and machines, not recording what happened.

Doing nothing. The current equilibrium: each team writes its own hook, each parses the same undocumented JSONL, each reimplements redaction badly, and each breaks on a different release.

Priority

Medium - Would be very helpful

Feature Category

Configuration and settings

Use Case Example

Reconstructing an incident

A config loader change breaks staging. The commit is three weeks old and its author has moved on to other work. Today: the commit message, a diff, and nobody's memory. With the journal at reconstruct: the prompt that requested the change, the alternative the model raised and the user declined, the subagent that was asked to audit callers and reported back that two were missed, and the exact ranges touched — tied to the commit sha. The question "why was it done this way" has an answer instead of a guess.

Handover between people

A developer goes on leave mid-refactor. The next person reads the project journal filtered to that subsystem: what was tried, what was abandoned and why, what was left half-done. This is #44200's handoff file, produced as a by-product instead of written by hand and forgotten.

Diagnosing the process

Aggregated across a month, iterations_to_accept is high on one part of the codebase and normal elsewhere. Reading a few journals shows why: that area has no CLAUDE.md context, so every session re-derives the same conventions. The fix is a documentation change, and it was found from data rather than from someone complaining.

Seeing whether delegation pays

inline_vs_delegated alongside fanout_size shows a pattern of eight-agent fan-outs producing two useful results. That is a habit worth changing, and it is invisible without per-session aggregates.

A developer reviewing their own sessions

Reading back one's own journal shows where a vague opening prompt cost four iterations that a specific one would not have. This is the most valuable use of the detailed data and the one that needs no coordination with anyone.

Regulated environments

At audit detail with identity: author, timestamps: coarse, content: none, the journal records that a person's session modified a given subsystem on a given day, with no prompt content and no work-hours trail. That is what compliance actually asks for, and today it can only be approximated by writing the tool yourself.

Additional Context

I have a working partial implementation — a cross-platform Go hook on nine events that writes a semantic per-session log with git-derived file annotations, deliberately timestamp-free. Happy to share it as a reference for what people are reaching for; the point of this request is that its ceiling is structural. Subagent internals, rejected proposals, rewinds, read attribution and commit linkage are all held by the client and reachable from nowhere else, and the privacy control I want (coarse timestamps rather than none) cannot be implemented downstream of an emitter that has already sent precise ones.

On scope: this is one mechanism, not two requests. The journal and the session aggregates are the same event stream at different detail levels, and separating them would mean specifying the same schema twice.