Token Monitoring

Token Monitoring & Budget System

Warp-drive tracks token usage per chunk, per session, and across sessions. It estimates costs, enforces budgets via a circuit breaker, and generates optimization insights. This document covers the full pipeline: capture, storage, enforcement, and reporting.

---

Architecture

Claude Code session transcript (~/.claude/projects/<slug>/*.jsonl)
       │
       ▼
token-snapshot.js  ── reads transcript, computes token delta ──►  JSON snapshot
       │
       ▼
state-machine.js   ── stores snapshots in .warp-drive-state.json
       │                  chunk_snapshots[] (per chunk)
       │                  session_total    (at session end)
       │
       ├──► checkBudgets()    ── enforces hard limits, triggers circuit breaker
       │
       └──► persistTokenUsage() ── appends records to ~/.claude/token-usage.jsonl
                                        at session completion
       │
       ▼
token-report.js   ── reads JSONL, generates markdown/JSON reports
       │                  /token-report skill wraps this
       ▼
Optimization insights, cost estimates, trend analysis

Key files

File	Purpose
scripts/warp-drive/token-snapshot.js	Reads Claude Code transcript, computes token totals or deltas, and stamps each snapshot with its provenance (#1806)
scripts/warp-drive/provenance.js	Derives model / effort / deployed-BoB provenance for a session window from its transcript — the lookup token-snapshot.js calls; derive CLI (#1806)
scripts/warp-drive/token-report.js	Aggregates JSONL records into reports with cost estimates
scripts/warp-drive/token-recount.js	Re-prices ledger records written before #4204 from their transcripts, where a transcript reproduces the record exactly
scripts/warp-drive/state-machine.js	Orchestrates capture, stores state, enforces budgets, persists data
registry/skills/token-report/SKILL.md	User-invocable /token-report skill definition
~/.claude/token-usage.jsonl	Persistent append-only log of all session and chunk records
.claude/.warp-drive-state.json	Per-project state file (transient, deleted at session end)

---

Data Flow

When tokens are captured

Event	What happens	State field
chunks_defined	Mark chunk start timestamp	token_usage.current_chunk_started_at
next_chunk	Capture delta since chunk started, push snapshot	token_usage.chunk_snapshots[]
requirement_done	Capture final chunk delta, push snapshot	token_usage.chunk_snapshots[]
session_ended	Capture full-session total (no --since filter)	token_usage.session_total
session_ended	Persist all records to ~/.claude/token-usage.jsonl	— (written to disk)

How snapshots work

token-snapshot.js reads the Claude Code session transcript — the .jsonl file that Claude Code writes to ~/.claude/projects/<slug>/. Each line is a JSON record with a message.usage object containing token counts.

• Full snapshot (no --since): sums the usage of every API response in the transcript.
• Delta snapshot (--since <ISO>): sums only responses with a line where timestamp >= since. Used for per-chunk deltas.

Each API response is counted once (#4204). Claude Code writes one transcript line per content block of a response (thinking, text, each tool_use). Every one of those lines repeats the response’s message.id and the same usage. Lines are therefore grouped by message.id, else by requestId, else each line stands alone, and one usage is taken per group. Subagent transcripts are grouped the same way. Every snapshot says so with counting: "per-response", and the ledger records carry the field. Before this fix usage was summed once per line, which read about 1.5–2× high. See Over-counted records.

The state machine calls captureTokenSnapshot(projectRoot, sinceTimestamp, sessionId) which spawns the script and parses its JSON output. sessionId is the run’s recorded owner (session.session_id, set from CLAUDE_CODE_SESSION_ID), passed as --session so the snapshot reads that session’s transcript rather than the newest file in the project directory (two sessions side by side would otherwise cross-read). If the script fails or doesn’t exist, it returns null and the session continues without token data.

Every snapshot also carries provenance — model, effort and a provenance object — derived from the same transcript over the same window (see Provenance).

---

Budget System

The budget system prevents runaway sessions. Its constraints are checked before every state transition by checkBudgets() — the table below covers the time, count and spend limits; the same function also enforces the no-progress stall guard (max_no_progress), the in-session QA loop cap (max_qa_cycles) and the decision budget (max_auto_decisions), listed with the other trigger reasons in the state reference.

Constraints

Constraint	Config key	Default	Counter	Enforcement
Phase timeout	max_phase_minutes	30	Elapsed time in current phase	Configurable: warn, block, or abort
Retry limit	max_retries_per_chunk	5	state.budgets.retry_count	Hard (enforced)
Coding cycles	max_coding_cycles	3	state.budgets.coding_cycles	Hard (enforced)
Total chunks	max_total_chunks	20	state.metrics.chunks_completed	Hard (enforced)
Session duration	max_session_minutes	480	Elapsed since session.started_at_epoch	Hard (enforced)
Session tokens	max_session_tokens	0 (disabled)	token_usage.session_total (else chunk-snapshot sum)	Hard (enforced) when > 0
Session cost (optional)	max_session_usd	0 (disabled)	estimated via token-report’s cost model	Hard (enforced) when > 0 and the estimate is qualified — an unqualified estimate (records carry no priced model, #1805) is advisory only

The cost ceiling (#587) emits a cost_budget_exceeded issue and routes to budget_exceeded like the other hard limits; its reason reports actual spend vs ceiling (e.g. cost_budget_exceeded: 1500 tokens exceeded 1000 tokens). Both ceilings default to 0 (disabled). Token count is the reliable signal; max_session_usd is optional/secondary.

Unqualified estimates never trip the breaker (#1805). The dollar figure comes from token-report.js’s model-keyed rate table, selected by record.model. A record with no model (every record until #1806 stamps one) or an unknown model is priced as an upper bound at the priciest known rate and marked qualified: false. The breaker enforces max_session_usd only on a qualified estimate; when an unqualified bound exceeds the ceiling it emits an advisory cost_budget_unqualified issue (enforced: false, carrying the bound, the ceiling and the reason) that is surfaced in the transition’s warnings and the viz burn-down (est spend … (upper bound — not enforced)), but the run continues — a guessed rate can never abort a session. The token ceiling is unaffected.

Phase timeout enforcement mode is set via phase_timeout_enforcement in config:

• warn (default): advisory only, included in response but doesn’t block
• block: rejects the transition
• abort: auto-transitions to budget_exceeded

Circuit breaker

When a hard limit is exceeded, the state machine doesn’t crash or silently continue. It transitions to the budget_exceeded phase — a first-class state in the state machine.

any phase ──[hard limit exceeded]──► budget_exceeded
                                          │
                                  ┌───────┴───────┐
                                  ▼               ▼
                           budget_continue    budget_abort
                                  │               │
                                  ▼               ▼
                       interrupted phase      aborted
                          (budgets reset)   (session ends)

The circuit breaker:

1. Runs checkBudgets() before every transition.
2. Filters for enforced issues (hard limits, not advisory warnings).
3. If enforced issues exist and the event is not on the bypass list, transitions to budget_exceeded.
4. Stores diagnostic info: exceeded_reasons, exceeded_at, exceeded_from_phase.
5. Presents the user with a choice: continue (extend budget) or abort.

Bypass events — these skip the circuit breaker check to avoid deadlocks: abort, abort_resolved, session_ended, budget_continue, budget_abort

Human checkpoint — budget_exceeded requires human approval below Level 4. At Level 4 the engine’s opt-in bounded auto-continue policy (state-machine.js budget-decision, #873) may continue it a capped number of times (max_budget_auto_continues); the hard token/dollar ceiling and the decision-budget digest always halt for a human.

Budget recovery

When the user chooses budget_continue:

• retry_count and coding_cycles reset to 0
• The chunk cap gains 10 chunks of headroom (max_total_chunks_extension, #1149) and the session-time cap enough minutes to clear the overrun plus an hour (max_session_minutes_extension, #1538)
• budget_extensions counter increments (tracks how many times the user extended)
• Execution returns to the phase the breaker interrupted (exceeded_from_phase, #1183); coding is only the fallback for older state files

Per-chunk budget resets

After each successful chunk (next_chunk and requirement_done events), retry_count and coding_cycles reset to 0. This means per-chunk limits apply fresh to each chunk, while max_total_chunks and max_session_minutes apply across the entire session.

---

Reasoning Budget

The reasoning budget controls Claude’s thinking effort per phase. High-reasoning phases get deeper analysis; standard phases get efficient execution. This is the reasoning sandwich pattern.

Defaults

One row per phase — all 23, the same key set as PHASES. This table is pinned to DEFAULT_REASONING_BUDGET in scripts/warp-drive/state-machine.js through tests/fixtures/phase-contract.json (make check-phase-contract, #2042); a phase with no explicit override is standard, the same value the lookup falls back to.

Phase	Level	Rationale
idle	standard	No run in flight
prerequisites	standard	Mechanical setup
preflight	standard	Engine-graded gate — no judgment to spend
blocked	standard	Holding state; the todo carries the fix
discovering	high	Work discovery needs judgment
planning	high	Architecture decisions need depth
chunking	high	Decomposition affects everything downstream
coding	standard	Implementation follows the plan
updating_docs	standard	Straightforward documentation
testing	high	Test interpretation needs careful analysis
committing	standard	Mechanical commit creation
doc_drift_check	standard	Mechanical drift verdict
reporting	standard	Structured output
chunk_complete	standard	Status check
requirement_complete	high	Final verification — last chance to catch issues
qa	high	Exercising the running app and judging the evidence
merging	standard	Mechanical merge/PR
promoting	standard	Mechanical deploy to the ceiling
awaiting_continue	standard	Segment/continue decision is computed by the engine
session_ending	standard	Reporting
budget_exceeded	standard	Decision presentation
aborted	standard	Cleanup
completed	standard	Terminal — nothing left to reason about

Configuration

Override per-phase reasoning in .claude/settings.local.json:

{
  "_workflow": {
    "reasoning_budget": {
      "coding": "high",
      "testing": "standard"
    }
  }
}

Config overrides take precedence over defaults. The level is injected into the state machine’s systemMessage for each phase transition, where it instructs Claude to adjust its reasoning effort.

---

State File Structure

During a session, token and budget data lives in .claude/.warp-drive-state.json:

{
  "token_usage": {
    "session_total": null,
    "chunk_snapshots": [
      {
        "chunk_index": 0,
        "acs": ["AC-01", "AC-02"],
        "input_tokens": 61250,
        "output_tokens": 22250,
        "cache_read_tokens": 18400,
        "cache_creation_tokens": 3200,
        "total_tokens": 83500,
        "message_count": 11,
        "timestamp": "2026-04-12T14:30:00Z"
      }
    ],
    "current_chunk_started_at": "2026-04-12T15:10:00Z"
  },
  "budgets": {
    "phase_started_at": "2026-04-12T15:10:00Z",
    "retry_count": 0,
    "coding_cycles": 0,
    "merge_retries": 0,
    "push_retries": 0,
    "budget_extensions": 0,
    "exceeded_reasons": null,
    "exceeded_at": null,
    "exceeded_from_phase": null,
    "aborted_at": null,
    "aborted_from_phase": null
  },
  "metrics": {
    "commits": 0,
    "reports_filed": 0,
    "tests_run": 0,
    "chunks_completed": 0,
    "session_duration_minutes": 0
  }
}

session_total is null during the session and populated by a full (unfiltered) snapshot on entry to session_ending (#560, so the session summary can read it), refreshed at session_ended. chunk_snapshots accumulates one entry per completed chunk. current_chunk_started_at is the ISO timestamp used as the --since argument for the next chunk delta.

---

Persistent Storage

token-usage.jsonl

At session completion, persistTokenUsage() appends records to ~/.claude/token-usage.jsonl. Each line is a self-contained JSON object.

Other runs (#4101). warp-drive is not the only writer. scripts/lib/token-ledger.js appends one type: session record, in the same shape, for every other kind of run the Costs page names, each measured by token-snapshot.js over the transcript window the run ran in:

kind	Written by	Window
decompose, do	machine-engine.js, when the run reaches a terminal phase (and bob do stop)	the driving session (CLAUDE_CODE_SESSION_ID) since the run started
groom	scripts/groom/groom.sh, after its claude -p call(s)	the session id(s) groom.sh passed as --session-id
interactive	the token-ledger-session-end.sh SessionEnd hook	the whole session, less every session record already in the ledger for that session id
tick	warp-drive’s session_ended for an unattended run	as for warp-drive

A run is never counted twice: a session started by bob groom carries BOB_RUN_KIND=groom and its SessionEnd writes nothing, a session ending while a warp-drive run is live in the checkout writes nothing, and an interactive record names the runs it left out in less_runs. A record of a kind exists only once such a run has finished; until then the Costs page reads that kind as not measured. Every writer is fail-open: no transcript, no record.

Reconcile (#4421)

The SessionEnd hook fires when a session ends normally. It does not fire for a session killed with its tmux window, a crash or a machine that slept, and the interactive sessions before 2026-10-11 were never recorded at all — on one Mac over 30 days, 123 transcripts and 30 records for one project. COGS is cited from this ledger, so each of those is a session that cost something and is priced at nothing.

bob labor reconcile closes the gap and keeps it closed. It reads every transcript under ~/.claude/projects/<slug>/ written since the window opened (--since, default 30 days) and classifies each by its session id against the ledger’s session records (session_id, or a member of a combined record’s session_ids):

State	Meaning	Listed	Recorded by --apply
recorded	a session record names it, whichever writer wrote it	no	never
unrecorded	no record, and at least one priced API response	yes — project, session id, last event, response count	yes, once
settling	no record, but written inside --settle minutes (default 60): the session may be live, or its hook may still be measuring	yes, under its own heading	no — the next pass sees it again
empty	no record and no priced response (a session that only opened)	no	never

Without --apply it is a report: exit 1 when an unrecorded transcript exists, 0 when none does. That exit is the health signal: bob status --scope project and bob doctor print the count for the last 30 days (zero is healthy; a non-zero count names bob labor reconcile --apply as the fix), the fleet snapshot carries it per project as ledger.unrecorded, and --json is one bob-labor-reconcile/1 document.

The writers. --apply appends, per unrecorded transcript, one record of the shape every other writer uses — token-ledger.js sessionRecord: type: session, kind: interactive, the per-response usage (#4204), by_model, model, effort and provenance (#1806), measured by token-snapshot.js over the whole transcript, subagents included — with three fields of its own:

Field	Value
source	reconcile — the pass wrote it, not the hook
timestamp	the transcript’s last event, so the day series prices the session on the day it happened, not the day it was found
recorded_at	when the pass wrote it

started_at is the transcript’s first event. The record is the list-price estimate from the rate table, like every other; whether that is a cash cost is the margin slice’s question.

The duplicate rule. One record per session id, whichever writer comes second loses and says so. The reconcile refuses a session any record names (it is recorded, never listed), and a second --apply appends nothing because the record it wrote names the session. The SessionEnd hook, running after the reconcile recorded the session, refuses too: recordInteractive returns written: false with a reason naming bob labor reconcile as the earlier writer (and duplicate_of: { source: "reconcile", timestamp }); a hook running second after another hook names the hook. The trade-off is explicit: a session that idled past the settle window and was then resumed is recorded by the reconcile from the transcript as it stood, and the hook’s later whole-session record is the refused duplicate — --settle is the lever, and the default of 60 minutes keeps that case rare.

The timer. The pass runs unattended so the gap never reopens: audit.js snapshot (what the fleet-snapshot timer runs) runs reconcile --apply before it reads the projects (BOB_FLEET_LABOR_RECONCILE=0 opts out; the test sandboxes set it), so the snapshot it writes carries the post-reconcile count — per project as token_ledger: {unrecorded, settling, recorded, written, transcripts, since, fix} (a project’s transcripts are the ones whose checkout resolves to it, worktrees included) and per machine as summary.token_ledger: {transcripts, recorded, unrecorded, settling, written, failed, since, projects_with_gaps, fix}, null when the pass did not run and {error} when it could not. bob tick runs the same step (report only on a dry run) and its digest’s labor_reconcile is the bob-labor-reconcile/1 document — counts.written is the number of records the tick wrote, and the markdown digest’s Ledger reconcile row says so. By hand, bob labor reconcile --apply is the fix the health lines name.

The health lines. bob status --scope project adds a graded ledger section (the project’s slice of the document: ok at zero, drift with the fix otherwise) and bob doctor one never-graded line — INFO token ledger: every transcript since <date> has a session record (N recorded) or WARN token ledger: N transcript(s) since <date> have no session record — unpriced COGS (project N, …) — Fix: bob labor reconcile --apply. Both read in quick mode: a transcript the ledger already names is read only as far as its first line (whose it is), and one it does not only as far as its first priced response — a few kilobytes each instead of the machine’s whole transcript store, the same classification, and never --apply.

Forward-only by decision on #3135: a one-off --since 2026-01-01 --apply over the transcripts that survive is how past COGS is recovered if wanted; a pruned transcript is simply not there, never estimated.

What the test pins. tests/test-labor-reconcile.js (make test-labor-reconcile) drives the real bob labor reconcile, the real SessionEnd hook, bob status --scope project, the doctor’s reading and audit.js snapshot over a scratch HOME holding a captured priced transcript with its subagent (killed: no record), one the hook recorded first, one that only opened, and one two hours old: the settle rule, the dry listing and its exit codes, the single append and its record (the usage token-snapshot.js itself reads), the idempotent second run, the refusal in both directions naming the earlier writer, the status section’s drift → ok, the doctor’s WARN / INFO lines, the snapshot’s record and token_ledger keys (and their null under the opt-out), and the usage exits. tests/test-bob-tick.sh pins the tick’s labor_reconcile on a dry and a live tick.

Chunk record

{
  "type": "chunk",
  "session_id": "abc123",
  "project": "paulirv/bodmail",
  "requirement": "#42",
  "branch": "feat/42-email-templates",
  "level": 2,
  "started_at": "2026-04-12T14:00:00Z",
  "chunk_index": 0,
  "acs": ["AC-01", "AC-02"],
  "input_tokens": 61250,
  "output_tokens": 22250,
  "cache_read_tokens": 18400,
  "cache_creation_tokens": 3200,
  "total_tokens": 83500,
  "message_count": 11,
  "timestamp": "2026-04-12T14:30:00Z",
  "model": "claude-fable-5-1",
  "effort": "high",
  "provenance": {
    "schema": "chunk-provenance/1",
    "source": "transcript",
    "session_id": "abc123",
    "transcript": "/home/paul/.claude/projects/-Users-paul-projects-bodmail/abc123.jsonl",
    "window": { "since": "2026-04-12T14:00:00Z", "until": "2026-04-12T14:30:00Z" },
    "messages": 11,
    "models": { "claude-fable-5-1": 11 },
    "efforts": { "high": 11 },
    "effort_source": "transcript",
    "claude_code_version": "2.1.273",
    "bob_sha": "4cddd31e44bec2cd37d010a413b535be1b91bfbc",
    "bob_version": "1.273.0",
    "bob_channel": "checkout",
    "derived_at": "2026-04-12T14:30:00Z",
    "reason": null
  }
}

model, effort and provenance are the #1806 fields — see Provenance. Records written before #1806 have none of the three; a record whose snapshot could not derive carries "model": null, "effort": null and a provenance whose source is null and whose reason says why.

Session record

{
  "type": "session",
  "session_id": "abc123",
  "project": "paulirv/bodmail",
  "requirement": "#42",
  "branch": "feat/42-email-templates",
  "level": 2,
  "started_at": "2026-04-12T14:00:00Z",
  "chunks_completed": 3,
  "commits": 3,
  "input_tokens": 245000,
  "output_tokens": 89000,
  "cache_read_tokens": 72000,
  "cache_creation_tokens": 12800,
  "total_tokens": 334000,
  "message_count": 42,
  "timestamp": "2026-04-12T16:45:00Z"
}

Both record types share a base set of fields (session_id, project, requirement, branch, level, started_at) for filtering and grouping. The session record carries the same model / effort / provenance fields as a chunk record, derived over the whole transcript (window.since: null).

---

Provenance (#1806)

A token record says what a chunk cost and how long it ran; provenance says what produced it — without it cost is uncomputable (#1805 needs record.model as its key), “did quality change?” is unanswerable, and trace-mining cannot control for the most obvious confounder.

Derived, never self-reported. There is no CLAUDE_MODEL environment variable, and an agent asked to stamp its own model is unreliable — models are poor at reporting their own identity, and the value drifts silently when the model is switched mid-session. A wrong stamp is worse than no stamp: it corrupts exactly the analysis the field exists to enable. So scripts/warp-drive/provenance.js reads the session transcript at ~/.claude/projects/<slug>/<session_id>.jsonl, where every assistant message carries "model" (and "effort", "version"), keyed by the session_id the state machine already records. The model is a lookup, not a claim.

Field	Where it comes from	When absent
model	The one model on every assistant message in the record’s window	null — no transcript, an empty window, or more than one model (see below)
effort	The one effort on every assistant message in the window; else CLAUDE_EFFORT from the environment (older transcripts carry no effort field)	null
provenance.models / provenance.efforts	The distribution — { "<id>": <assistant message count> }	null when the window is empty
provenance.effort_source	transcript or env	null
provenance.claude_code_version	The version on the transcript’s assistant messages	null
provenance.bob_sha / bob_version / bob_channel	$BOB_HOME/.bob-version (bob-version/1, written by every non-dry deploy, #2046) — the deployed commit the runtime reads, which can lag the source checkout	null when no deploy record exists
provenance.source	transcript when the window had assistant messages	null, with reason (no session id, transcript not found (pruned or never on this machine), no assistant messages in window, transcript unreadable: …)
provenance.window	{ since, until } — the chunk’s start (current_chunk_started_at) to the snapshot timestamp; the session record’s since is null	—

Provenance is not a scalar. One warp session spans many chunks, the model can change mid-run, and subagents carry model overrides. Each record is derived over its own window; a window that saw more than one model records the distribution in provenance.models and leaves model null, so the cost model prices it as an explicit upper bound (qualified: false) rather than at one collapsed rate. A record that also carries by_model (#2685) is priced per model instead, and is qualified when every model in it is in the rate table. The same rule applies to effort / provenance.efforts.

Best-effort by contract. Derivation never fails a chunk: a missing or pruned transcript, an unreadable file or a missing .bob-version yields null fields with a reason, and the token counts stand. Readers tolerate records with no provenance — the records written before #1806 carry none of the three fields, and token-report.js prices them exactly as before (the upper-bound path, #1805).

Where it goes — and where it never goes. The token log only: per chunk, machine-readable, keyed by session_id, so warp viz can render from the log. Provenance is never written into issue bodies (telemetry in prose is noise) and never into commit metadata — attribution trailers remain prohibited, and the commit hook still rejects them.

Inspect a session by hand:

node ~/.claude/scripts/warp-drive/provenance.js derive --session "$CLAUDE_CODE_SESSION_ID" --project-root "$(pwd)" [--since <iso>] [--until <iso>] [--json]

Backfill (#1806 AC-07)

The token log predates provenance, and the backfill window is closing: transcripts age out of ~/.claude/projects/, and a record whose transcript is gone can never be stamped. provenance.js backfill joins every record that lacks provenance by session_id to a surviving transcript and stamps it over its own window — a session record over the whole transcript, a chunk record from the previous chunk’s timestamp in the same session (else the session’s started_at) to its own. What was deployed at the time is unknowable, so a backfilled stamp carries bob_sha: null and backfilled: true, and effort comes only from the transcript (never the current shell’s CLAUDE_EFFORT).

node ~/.claude/scripts/warp-drive/provenance.js backfill            # dry run: what would be recovered, what cannot be
node ~/.claude/scripts/warp-drive/provenance.js backfill --apply    # rewrite ~/.claude/token-usage.jsonl (backup: token-usage.jsonl.bak-<stamp>)
node ~/.claude/scripts/warp-drive/provenance.js backfill --json     # provenance-backfill/1 for scripts

Outcome	What happens	Where it shows
Recoverable	The transcript is on this machine and the window has assistant messages: stamped with the model / effort distribution	recovered[] per session with the model tally; records_recovered
Empty window	The transcript is on this machine but the window has no assistant messages — a chunk from an earlier segment of a resumed run, whose record carries the last segment’s session id	Stamped source: null, reason: no assistant messages in window; records_empty_window, counted apart from recovered
Unrecoverable	No transcript on this machine (pruned), or the record has no session id	Marked (source: null, the reason, backfilled: true) so it is never re-examined, and reported in unrecoverable[] — never silently skipped; exit 1
Already stamped	The record carries a provenance object	Untouched; already_stamped

The rewrite is atomic (temp file + rename) and preserves every other line verbatim; a second --apply is a no-op, so the command is idempotent. Exit codes: 0 nothing unrecoverable, 1 unrecoverable records remain (reported), 2 usage or no log.

Per machine. Each machine’s token log joins its own surviving transcripts, so the backfill runs once per machine, not once per fleet. The unattended run on farm-01 backfilled farm-01’s log as part of #1806; the laptop backfill is the one command Paul runs afterwards on each laptop:

node ~/.claude/scripts/warp-drive/provenance.js backfill && node ~/.claude/scripts/warp-drive/provenance.js backfill --apply

Run the dry run first and read the unrecoverable list — those sessions are gone for good, and the report is the record of that. A missing transcript is an honest null, never a guess.

---

CLI Tools

token-snapshot.js

Reads a Claude Code session transcript and computes token usage.

node ~/.claude/scripts/warp-drive/token-snapshot.js <project-root> [--since <ISO-timestamp>] [--session <id>]

Argument	Required	Description
<project-root>	Yes	Absolute path to the project directory
--since <ISO>	No	Only count tokens from messages after this timestamp
--session <id>	No	Read this session’s transcript when it exists in the project directory (the state machine passes the run’s recorded owner, #1806); otherwise the newest transcript as before

Output: JSON to stdout with timestamp, session_id, project, project_root, input_tokens, output_tokens, cache_read_tokens, cache_creation_tokens, total_tokens, message_count, counting (per-response, #4204), plus model, effort and provenance (#1806 — derived from the same transcript over the same window; null fields when it cannot derive, never a failed snapshot). The counts include the session’s Task subagents (#2685): subagent is their share as a separate bucket (the same token fields plus message_count and transcripts, the number of subagent transcripts read), and by_model splits all usage (main loop and subagents) per message model, { "<model>": { input_tokens, output_tokens, cache_read_tokens, cache_creation_tokens, total_tokens, message_count } }. The state machine writes both into the token-usage.jsonl records.

Exit codes: 0 = success, 1 = missing argument, 2 = no transcript found.

How it finds the transcript: Slugifies the project path (/Users/paul/projects/bodmail becomes -Users-paul-projects-bodmail), looks in ~/.claude/projects/<slug>/ for <session>.jsonl when --session was given and it exists, else the most recent .jsonl file (excluding subagent transcripts). It then adds every <session>/subagents/agent-*.jsonl beside that transcript (the .meta.json sidecars are skipped), counted over the same --since window.

token-report.js

Aggregates ~/.claude/token-usage.jsonl into human-readable reports.

Rate table (#1805). PRICING is keyed by model id (Fable 5.1 / 5, Opus 5.5 / 5 / 4.8 / 4.7 / 4.6, Sonnet 5.5 / 5 / 4.6, Haiku 5.5 / 4.5 — the 5.5 rows verified 2026-10-06; Haiku 5.5’s row is its rate for prompts up to 100K tokens), USD per million tokens, cache write at 1.25× input and cache read at 0.1× input unless the model publishes its own. Adding a model is one data row. The table is time-sensitive: PRICING_AS_OF records the date the rates were last verified against the published price list, every markdown report prints Rates as of <date>, and --json carries pricing.as_of and pricing.unknown_policy — refresh the table (and the date) when it drifts from the current price list. estimateCostDetailed(record) returns { usd, qualified, model, model_key, rates, as_of, reason }; estimateCost(record) is the numeric view; costQualification(records) counts how many records were priced on an upper bound, and the report annotates the total accordingly. A record carrying by_model (#2685) is priced bucket by bucket at each model’s row; an unpriced model’s bucket takes the upper bound and the reason names it. The Totals block adds an Unstamped line (--json: unstamped: { count, total, mostly_unstamped }; a window more than half unstamped is flagged as unfit for a before/after comparison) and a Subagent tokens line (--json: totals.subagent). unstampedCount(records) and subagentTotals(records) are exported.

Provenance readers (#1806). The Session History table carries a Model column — the record’s model, mixed(N) when its window spanned N models (the distribution is on provenance.models), or - when the record carries no provenance — and the Totals block ends with a Provenance line: how many of the selected sessions are stamped, the per-model tally, the mixed-model count and how many carry nothing. --json adds provenance: { total, stamped, unstamped, mixed, models, efforts }. provenanceLabel(record), provenanceSummary(records) and formatProvenanceLine(summary) are exported for other readers. Every reader tolerates records with none of the three fields — the pre-#1806 history — exactly as it did before they existed.

node ~/.claude/scripts/warp-drive/token-report.js [options]

Flag	Default	Description
--last <N>	5	Show last N sessions
--all	—	Show all sessions
--project <name>	—	Filter by project name (partial match)
--since <iso>	—	Only sessions recorded at or after this instant; every one in the window unless --last is given (#2055)
--issue <N>	—	Only sessions whose record names requirement N (#N accepted); every one unless --last is given. --json carries issue. The join bob-series/1 prices a merged requirement on (#2072)
--json	—	Output as JSON instead of markdown
--insights	—	Include optimization insights section
--budget <N>	5.00	Budget threshold in dollars (used by insights)
--decide-log <file>	$DECIDE_LOG, else $BOB_HOME/logs/decide.jsonl	The decision seam’s per-call ledger, read for the decision-shaped split (#2983)
--callers <file>	scripts/lib/decide-callers.json	The caller registry whose decision_agent_types name the classification subagents (#2983)

Standard output sections:

1. By Project — aggregated tokens and cost per project
2. Session History — per-session table: date, project, requirement, chunks, model, input / output / cache-read tokens, messages, estimated cost
3. Totals — sum across displayed sessions
4. Decision-shaped spend — the share of spend that went to decisions, by backend (see below, #2983)
5. Averages — per-session averages (shown when 2+ sessions)

Insights output (with --insights):

1. Top Token Consumers — projects ranked by estimated cost
2. Cost Efficiency per AC — cost and messages per acceptance criterion
3. Sessions Over Budget — sessions exceeding the threshold, with overage amount
4. Optimization Suggestions — automated analysis:
   • Cache reuse ratio (read/create) — flags low reuse (<5x)
   • Messages per AC — flags high back-and-forth (>50)
   • Output-to-input ratio — flags verbose sessions
   • Cost trend — compares recent 3 sessions against earlier average

Decision-shaped spend (#2983)

Every tiering decision in BoB assumes cheap models should make the classifications and frontier models should do the reasoning. This section measures that assumption: what share of spend went to decisions a cheap model could have made, and how much of it a frontier model served.

What counts as decision-shaped. A call is decision-shaped only on recorded evidence, never on a reading of its prompt:

• A decide() call. The decision seam (scripts/lib/decide.js) appends one decide-call/1 record per call a backend answered to $BOB_HOME/logs/decide.jsonl. DECIDE_LOG overrides the path, off disables it, and under BOB_TEST it is off unless set. The record holds ts, caller, backend (jev, deterministic, or haiku for haiku-forced-tool), model, decided, reason when undecided, stakes, session_id (from CLAUDE_CODE_SESSION_ID), and tokens. tokens is { input_tokens, output_tokens, total_tokens } when the backend reported usage, else the string not measured, never 0. A deterministic rule calls no model, so its zero is a measurement. State and question text are never written.
• A decision made outside the seam. A frontier or haiku call that answers a declared caller’s question is recorded with decide.js record --caller <c> --backend <jev|deterministic|haiku|frontier> [--model <id>] [--input <n> --output <n>] [--session <id>]. The caller’s entry in decide-callers.json is its recorded purpose. An undeclared caller or an unknown backend is refused with exit 2. Use it only for calls outside the session transcripts. A subagent is already counted through its session.
• A classification subagent. token-snapshot.js splits the subagent bucket by agent type (subagent.by_agent). The type comes from each subagent’s agent-*.meta.json (agentType), or is unknown when that file is missing, and each type has its own by_model split. A type listed in decide-callers.json → decision_agent_types.types is decision-shaped. It counts as haiku or frontier by the model that served it. The list is empty today because no agent exists to classify.

Everything else in a session record is reasoning-shaped.

The arithmetic, which reconciles with the totals. The split starts from the report’s own session totals (input + output tokens, and USD priced as above). Seam calls run outside the Claude transcripts, since Jev is its own API, so their measured spend is added. A classification subagent’s spend is already inside its session and is moved from reasoning to decision. So decision + reasoning = session totals + seam spend, in tokens and in USD. Per backend the report shows calls, tokens, USD and the count of calls with nothing measured. A backend whose calls measured nothing shows not measured. Jev tokens count, but Jev’s own model has no row in the rate table, so its USD is not measured. An unpriced haiku or frontier model is priced as the upper bound and flagged.

Window and sessions. A seam call joins a session by session_id. The window takes every call in its span: from --since, or else from the first session’s started_at to the last session’s end. That includes calls from no Claude session, such as a bob tick grooming pass. Each session also gets its own split from its own subagents and joined calls, with shares against its own spend.

The headline is the frontier share of decision-shaped spend, in USD and tokens. Tiering exists to drive it down.

Output. The markdown report prints the headline, the decision and reasoning totals, a backend table, and one row per session that made decisions. --json carries the whole split under decision_split with schema: "decision-split/1": window, decision_agent_types, totals (tokens, usd, session_*, seam_*), decision (tokens, usd, share_tokens, share_usd, by_backend.{jev,deterministic,haiku,frontier}), reasoning, headline (frontier_share_of_decision_tokens, frontier_share_of_decision_usd), and sessions[] with the same shape per session. decisionSplit(sessions, calls, opts), readDecideCalls(file) and decisionAgentTypes(file) are exported.

bob viz. The bob-series/1 document (#2072) carries the same split on each project’s usd_per_day series as decision_split. It is built with sessionsOnly: only calls that joined one of that project’s sessions are counted, because an unjoined call belongs to no project.

Over-counted records (#4204)

A ledger record without counting: "per-response" was written before #4204, when every response was counted once per content block. Such a record is over-counted. Readers never mix it silently with corrected records:

• estimateCostDetailed(record) returns counting (per-response or per-line) and over_counted.
• The markdown Totals block adds an Over-counted line when any selected session is per-line.
• --json carries counting: { total, per_response, over_counted, recounted, note }, and totals.over_counted gives the count.
• The Costs page marks each run over_counted. Its token-ledger row gives the count in its reason and in over_counted.

countingOf(record) and countingSummary(records) are exported.

Re-pricing from transcripts. token-recount.js corrects the records whose transcripts still exist:

node ~/.claude/scripts/warp-drive/token-recount.js            # dry run: what would change
node ~/.claude/scripts/warp-drive/token-recount.js --apply    # rewrite the ledger; a dated .bak copy first

A record is re-priced only when its transcript reproduces it exactly. Some window ending at the record’s timestamp, counted per line the old way, must give the record’s four token figures to the token. The scope is the main transcript, plus its subagents when the record carries a subagent share. That window is then counted once per response. The record takes the new figures and counting: "per-response", and a recounted block keeps since, scope and the old per_line figures. Nothing is estimated. A record with no transcript, or one that is a difference of runs (an interactive record), stays as it was and stays marked over-counted. The recount is idempotent. Lines appended to the ledger while it runs are kept. Flags: --ledger <file>, --projects-dir <dir>, --json.

/token-report skill

The /token-report skill (provisioned from registry/skills/token-report/) wraps token-report.js for interactive use. It runs the script and presents the output as formatted markdown.

---

Cost Estimation

Costs are estimated per record at its model’s rates (#1805): token-report.js’s PRICING table is keyed by model id and selected by record.model — the field provenance derives from the transcript. A record with no model (every record written before #1806, or a chunk whose window spanned more than one model) or a model the table does not know is priced as an explicit upper bound at the priciest known rate and marked qualified: false; nothing that gates behaviour on dollars (the session cost breaker, the A4 ceiling) acts on an unqualified estimate. The table is dated (PRICING_AS_OF) and every report prints the date, so a stale table is visible. Cost estimates appear in both the standard report and insights output.

---

Configuration Reference

All budget and reasoning settings live in .claude/settings.local.json under the _workflow key:

{
  "_workflow": {
    "max_phase_minutes": 30,
    "max_retries_per_chunk": 5,
    "max_coding_cycles": 3,
    "max_total_chunks": 20,
    "max_session_minutes": 480,
    "phase_timeout_enforcement": "warn",
    "reasoning_budget": {
      "discovering": "high",
      "planning": "high",
      "coding": "standard",
      "testing": "high"
    }
  }
}

Key	Type	Default	Description
max_phase_minutes	number	30	Minutes before phase timeout triggers
max_retries_per_chunk	number	5	Hard limit on retries per chunk
max_coding_cycles	number	3	Hard limit on code/test cycles per chunk
max_total_chunks	number	20	Hard limit on total chunks per session
max_session_minutes	number	480	Hard limit on total session duration (8 hours)
max_session_tokens	number	0	Hard limit on cumulative session tokens; 0 disables
max_session_usd	number	0	Optional estimated-dollar ceiling; 0 disables
phase_timeout_enforcement	string	"warn"	"warn", "block", or "abort"
reasoning_budget	object	{}	Per-phase reasoning level overrides
reasoning_budget.<phase>	string	varies	"standard" or "high"

---

Session Summary Integration

Warp-drive session summaries (filed as GitHub Issues with label session-summary) include a Token Usage section with input, output, cache read, and total token counts read from state.token_usage.session_total. This makes cost visible in the project’s issue history without needing to run a separate report.

---

Troubleshooting

No data in token report

• Complete at least one warp-drive session. Data is only persisted at session end (session_ended transition).
• Check ~/.claude/token-usage.jsonl exists and has content.

Token counts are zero or null

• token-snapshot.js couldn’t find the session transcript. Verify ~/.claude/projects/<slug>/ contains .jsonl files.
• The slug is computed by replacing / and . with - in the project path.

Budget exceeded unexpectedly

• Check state.budgets in .claude/.warp-drive-state.json for current counter values.
• max_total_chunks defaults to 20 in state-machine.js (plus 10 per budget_continue) unless overridden in settings.local.json.
• Run node ~/.claude/scripts/warp-drive/state-machine.js status "$(pwd)" to see current budget state.

Cache reuse ratio is low

• Short sessions with few chunks create cache entries that are never reused. Longer sessions with more chunks per session improve the ratio.
• Context-busting tool calls (large file reads, many parallel agents) force cache recreation.

Pricing is outdated

• Update the PRICING table in token-report.js — and its PRICING_AS_OF date — when API pricing changes.