Session Evidence Ledger

Session Evidence Ledger

Reference for the session evidence ledger (#1131, part of #1129) — the per-project, append-only record of what tooling sessions actually used: skills invoked, agents spawned, commands run. It is the observational substrate consumed by gap detection, demotion evidence, and orchestrator feedback. The ledger itself is deliberately dumb: no analysis logic lives in it.

Storage

Property	Value
Location	<project-root>/.claude/session-ledger.jsonl
Format	JSON Lines — one event per line, append-only
Schema	schemas/session-ledger.schema.json (validates a single line)
Git status	Ignored — per-project machine state, like .claude/loop-notes.json
Owner (single source of truth)	scripts/evidence/ledger.js append — the only writer; the collector hook and the transcript backfill both write through it

CI validates the committed sample fixture (schemas/fixtures/session-ledger.sample.jsonl) plus any live ledger present in the repo (check-schemas.js, block 8f).

Event schema

Each line is one event:

Field	Type	Meaning
v	integer	Event format version (currently 1)
ts	string	ISO 8601 timestamp the event was observed
project	string	Basename of the project root the session ran in
machine	string	Fleet machine id (hostname-matched against machines.json, falling back to bare hostname)
harness	string	The agent harness that produced the event (claude-code today)
event_type	enum	invocation (an item was used) · improvised (the session hand-built something an item should cover) · no-trigger (a task ran with no skill/agent triggering). Collectors emit invocation; improvised/no-trigger are gap observations recorded via gap-analysis.js observe (#1134)
item_type	enum	skill · agent · command · task · cli (a routed bob command, named as scripts/bob/commands.json names it — bob warp status, #3923)
item_name	string	Skill/command slug or agent type; a short task label for no-trigger/improvised
session_id	string	Harness session id, when known (may be empty)
source	enum	hook (live collector) · backfill (one-time transcript bootstrap) · observation (a deliberate gap observation, gap-analysis.js observe — #1134) · control (a bob command BoB Control typed into a project’s terminal on this machine — #3923)
detail	object?	Optional collector extras (e.g. the raw tool name). Never required by consumers

The schema is harness-agnostic by contract: Claude Code is the first collector; a future harness adds its own collector writing the same format with its own harness value.

Matching convention: consumers joining ledger events against provisioned items should match by item_name, not item_type — the same registry name may surface as a skill or a command depending on harness wiring.

Collector (live mechanism)

hooks/session-ledger-append.sh — a PostToolUse guard on the Skill|Task|Agent and Bash matchers, run by hooks/dispatch.sh from hooks/dispatch.tsv:

• Skill calls append as item_type: skill (tool_input.skill)
• Task / Agent calls append as item_type: agent (tool_input.subagent_type, defaulting to general-purpose)
• Bash calls append one item_type: cli event per routed bob command the line runs (#3923), named as the routing table names it: bob doctor && bob warp status is bob doctor and bob warp status. A segment counts when its first word, after any VAR=value, is bob or a path ending in /bob. Quoted text and heredoc bodies are text, not runs, and an unrouted noun, bob help and a bare bob record nothing (scripts/evidence/cli-invocations.js). A Bash payload that never says bob returns before node starts.
• All other tools are ignored

BoB Control appends a cli event with source control for each bob command it types into a project’s terminal on this machine. These events are the evidence for Control’s never used tag (bob-control.md).

Fail-open everywhere. Missing node, missing script, unparseable stdin, unwritable disk — every path exits 0 and appends nothing. Evidence collection must never block or slow a session. Overhead is one short-lived node process per Skill/Task call; machine identity deliberately avoids the full fleet resolver (which may shell out to tailscale with a 3s timeout) in favor of a hostname match against machines.json.

Backfill (one-time bootstrap)

node ~/.claude/scripts/evidence/backfill-transcripts.js              # dry-run
node ~/.claude/scripts/evidence/backfill-transcripts.js --apply     # write
node ~/.claude/scripts/evidence/backfill-transcripts.js --project X # scope to one project
node ~/.claude/scripts/evidence/backfill-transcripts.js --transcripts DIR  # override ~/.claude/projects
node ~/.claude/scripts/evidence/backfill-transcripts.js --item-type cli     # one item type only
node ~/.claude/scripts/evidence/backfill-transcripts.js --cli-once         # the once-per-machine bob command fill

Mines existing Claude Code transcripts (~/.claude/projects/*/) for historical Skill/Task/Agent invocations and appends them marked source: "backfill". This is a one-time bootstrap, not the ongoing mechanism — run it once when adopting the ledger so consumers start with history. Properties:

• Dry-run by default — reports per-project counts; --apply writes.
• Idempotent — an event whose (ts, session_id, item_type, item_name) already exists with source backfill is never appended twice; re-running is safe.
• Non-fatal gaps — a transcript whose cwd no longer exists on this machine is counted and skipped.

The cli fill (#3923). Ledgers started before cli events existed hold none, so every bob command would read as never used. --cli-once fills them once per machine. It covers every ledger on the machine, because they are all mined from the same transcripts, and records that it ran in $BOB_HOME/.ledger-cli-backfill.json. A call the live hook already recorded (same session and command within ten minutes) is skipped, so nothing is counted twice. The fleet snapshot (audit.js snapshot) runs it before it rolls up the ledgers; BOB_FLEET_LEDGER_BACKFILL=0 skips it, and the test sandbox sets that so a suite never fills a real ledger.

Query helpers

node ~/.claude/scripts/evidence/ledger.js query  [--cwd DIR] [--days N] [--json]
node ~/.claude/scripts/evidence/ledger.js unused [--cwd DIR] [--days N] [--universal] [--json]

• query — “what did project X invoke in the last N days”: per-item count, item types, first/last seen, distinct sessions, sources. Sorted by count.
• unused — “which provisioned items have zero ledger events”: joins the ledger against the project’s provision manifest (provisions/<project>.json, _default.json fallback — the same resolution bob provision applies). --universal widens the declared set to the always-present universal skills/commands/agents. This is the demotion-evidence question (#1132) and gap-detection input (#1134): gap-detect.js divergence prints it as its unused section (#2078).

Fleet aggregation

scripts/fleet/audit.js snapshot (#415) attaches a compact ledger block to each audited project:

{ "days": 30, "events": 57, "items": 12, "last_ts": "…", "top": [{ "name": "Explore", "count": 19 }],
  "usage": [{ "name": "Explore", "types": ["agent"], "count": 212, "first_ts": "…", "last_ts": "…",
              "sessions": 40, "sources": { "hook": 190, "backfill": 22 } }] }

null means the project has no ledger at all (distinct from a ledger with an empty window). days, events, items, last_ts and top cover the 30-day window. usage (#2899) is all time: one row per item with its event count, first and last use, distinct sessions, and a count per source (hook = observed live, backfill = mined from transcripts). It carries names and counts only — no session ids, no detail. Snapshots travel cross-machine over the existing #415 mechanism, so BOB_HOME-level decisions can weigh evidence from every machine a project runs on.

Fleet-wide usage: fleet-usage.js (#2899)

A ledger only ever sees its own checkout, so an item used daily on a laptop reads as unused on the farm. scripts/evidence/fleet-usage.js joins every usage row across every project on every machine in the fleet manifest, from the published snapshots only (no SSH, no forge call):

node ~/.claude/scripts/evidence/fleet-usage.js                 # machines, missing evidence, every item
node ~/.claude/scripts/evidence/fleet-usage.js --item commit   # where and when one item was used
node ~/.claude/scripts/evidence/fleet-usage.js --live --json   # rebuild this machine's snapshot first; bob-fleet-usage/1

Each item carries its fleet total, machines, projects, first/last use, per-source counts and where (one row per machine and project). A machine whose evidence is incomplete is listed under gaps, never counted as zero:

gaps[].reason	Meaning	Counted?
missing	No snapshot from that machine here — its usage is unknown	no
no-usage	The snapshot predates #2899 (no usage rows) — re-run audit.js snapshot there	no
stale	Older than BOB_FLEET_STALE_SECONDS (default 24 h) — may miss recent use	yes

Exit 0 with a document, 2 on a usage error or an unreadable fleet manifest. bob tour --fleet grades from this document — see Tour What Is Provisioned. Tests: tests/test-fleet-usage.js.

Consumers: gap detection (#1134)

Two engines read the ledger (the ledger itself stays analysis-free):

• Provisioning gaps — scripts/orchestrator/gap-detect.js match --task "<text>": does an unprovisioned registry item cover the task at hand? Conservative frontmatter match, #412-aware dedupe (project manifest + this machine’s resolved _bob-home chain), emits the bob provision add <kind>/<name> --now fix with the automation-level action (L1 ask / L2 confirm / L3+ auto). Exit 0 = no gap, 3 = gap, 2 = usage. Its divergence verb adds the reverse direction from this ledger — an unused section of provisioned items with zero events, each with a why: line from frontmatter and a try: command; skipped when the project has no ledger events, --no-unused opts out (#2078).
• Authoring gaps — scripts/evidence/gap-analysis.js: observe appends improvised / no-trigger events (source observation); analyze applies the recurrence threshold (_workflow.gap_recurrence_count default 3 across ≥2 sessions) with cross-project routing (≥2 projects → registry candidate, else project-docs); recommend [--apply] files the qualifying batch as authoring-gap issues routed toward skill-creator; dismiss suppresses a declined recommendation until new evidence accrues (memory: .claude/.gap-recommendations.json). Hosted by /trace-mining §5b.

Tests: make test-gap-detect, make test-gap-analysis.

Library API

const l = require('~/.claude/scripts/evidence/ledger');
l.append(root, fields)             // -> event | null; fills envelope, never throws
l.eventFromHook(hookInput)         // -> partial fields | null (unmapped tool)
l.readEvents(root, {days})         // -> events, corrupt lines skipped
l.query(root, {days})              // -> per-item aggregation
l.unused(root, {days, universal})  // -> declared items with zero events
l.summary(root, {days})            // -> fleet-snapshot roll-up (incl. all-time usage) | null
l.usage(root)                      // -> all-time per-item usage with per-source counts

const fu = require('~/.claude/scripts/evidence/fleet-usage');
fu.aggregateUsage(snapshotsById, {fleet, staleSeconds, now})  // -> the join, pure
fu.fleetUsage({live})                                           // -> bob-fleet-usage/1

Tests: make test-session-ledger (tests/test-session-ledger.js) — schema validation, append (library/CLI/hook mapping), query/unused, backfill marking + idempotency, and the snapshot summary.