Consolidate Traces into the Wiki

Consolidate Traces into the Wiki

Every warp-drive run leaves traces: loop notes, decisions you made, harness defects, ledger records of work no skill covered, lessons and session reports. /trace-mining mines them for patterns. Its first step is the wiki maintainer, which turns those traces into pattern pages: one page per thing that goes wrong or works, each citing the traces it came from. The skill proposer reads those pages later to decide which skill to change (#2387).

The maintainer never commits to master. Each run opens one pull request on BOB_SOURCE, and you merge it.

1. See what a run would read

From the project whose traces you want consolidated:

bob wiki maintain --dry-run

You should see one line per trace store, then the packet:

wiki maintain: dry-run — project bigbrain (dry run — no agent call, no watermark, no branch)
  loop-notes       new 9, read 0, refused 0
  decision-log     new 0, read 0, refused 0
  harness-defects  new 0, read 0, refused 0
  ledger           new 0, read 0, refused 0
  lessons          new 0, read 0, refused 0
  session-reports  new 75, read 8, refused 0
  packet: 8 record(s), 0 refused, 76 left for the next run

Here the 8 oldest new records are session reports, so the loop notes wait for a later run.

Column	Meaning
new	records added to that store since the last run
read	records in this run’s packet
refused	records the client-data scrub refused; they are counted, never written
left for the next run	records past the packet cap; the next run takes them

A dry run calls no agent and writes nothing. If it prints no wiki store, the BOB_SOURCE checkout it found predates the wiki (#2388): pass --source <dir>.

2. Run it

bob wiki maintain

You should see wiki maintain: opened and a PR: line. Open the PR. Its body lists, per store, what was read and refused, the pages created and updated, any edit the script refused, and every record in the packet by its ref. Review the pages like any other change, then merge.

When nothing is new you see wiki maintain: noop — … (nothing new since the last run), and no PR is opened.

If it exits 1, the run failed: the agent, the wiki lint, the push or the PR. It consumed nothing, so the next run retries the same records. The last line names the failing step. If the lint refused a page, the message names the rule and the line.

What it reads

Store	Where	What counts
loop-notes	.claude/loop-notes.json	each note, by its signature; a note that recurs (newer last_seen_at) is read again
decision-log	.claude/.decisions.jsonl	each decision
harness-defects	harness_defects[] in .claude/.warp-drive-state.json	each hand-fired event while a run is live
ledger	.claude/session-ledger.jsonl	no-trigger and improvised records only
lessons	open lesson issues	each issue
session-reports	session-report issues	each issue

The packet is the oldest records first, at most 8 (--max-records), each cut to 15,000 characters (--max-chars). Those are the bounds the WikiSkill paper used. A store the maintainer has never read starts 30 days back (--since YYYY-MM-DD changes that), so a first run does not open on months of history.

Client records

In bigbrain, a record is cited directly (loop-note:<signature>, #N). In any other project, records are scrubbed before the agent sees them:

• A record that contains a tenant literal from your operator profile is refused.
• A record that matches a credential pattern is refused, in any project.
• With no operator profile resolved, every record from another project is refused. The scrub cannot know who to keep out, so it fails closed.

What survives is cited only by an opaque ref (ext:<kind>:<hash>), so no client name or path reaches a page. The wiki lint grades the pages before anything is committed, and refuses a run whose page names a tenant.

Why a second run does nothing

The run records a watermark: the last record it consumed per store. It lives in the PR (.maintain-state.json in the wiki store) and in $BOB_HOME/.wiki-maintain-state.json on this machine. The machine copy means a run whose PR is still open counts here. After the merge, the committed copy counts on every machine. A PR you close without merging keeps its records consumed on the machine that ran it.

3. Propose a skill change

Once the wiki has pages, the skill proposer turns them into one change to one skill, command or agent. /trace-mining offers it as its last step. See which item it would pick first:

bob wiki propose --auto --dry-run

You should see skill propose: dry-run — <item>, the item’s file, the pages about it, and a candidates list: each item’s new evidence, all its evidence, and whether it has an eval suite. Only an item with a suite can be proposed, because the gate needs a score. If you see noop with “no item with an eval suite has new wiki evidence”, there is nothing to propose yet. The line names any item that has evidence but no suite; to make it eligible, author a suite for it (Skill evals).

Then run it:

bob wiki propose --auto

You should see skill propose: opened, a gate: line and a PR: line. The proposer runs on a branch, runs the eval gate on its change, and opens one PR labelled skill-proposal. The PR body shows the diff, the pages it cites, the traces behind them, and the gate’s scores. A change the gate rejects still opens, as a draft, so the attempt is on record. Either way the skill-impact row is on the branch. Merge the PR if you agree with it, or close it. The proposer never merges.

If you see exit 1, the run failed: the agent answered outside its contract, the eval did not run, or the push or PR failed. Unless the error says the branch was pushed, nothing is left behind. To run it for one item, name the item: bob wiki propose commit.

Loop notes and lessons

A loop note or lesson the maintainer absorbs is cited in its page’s evidence. That replaces promoting a recurring loop note by hand (#2063). A lesson’s “Tooling Potential” section is read as part of the lesson, so the tooling change it suggests reaches the skill proposer as wiki evidence.

Every night: bob tick

bob tick runs the maintainer once per tick, after the fleet snapshot. A real run happens only when the tick’s project is at automation Level 3 or 4; at Level 1 or 2, and on bob tick --dry-run, it runs --dry-run. The tick digest has a Wiki row with the PR link.

When the tick’s project is bigbrain itself, it then runs bob wiki propose --auto: for real at Level 3 or 4, and as --dry-run at Level 1 or 2 and on a dry tick. In any other project it is skipped. The digest’s Skill proposal row links the PR. See Farm Tick.

Do not read the pages in a session

Only the skill proposer reads the wiki. An agent with the wiki in its context solves tasks from it, so its traces stop showing what the skills lack. No hook, command or manifest may load it, and make check-wiki enforces that. Review the pages in the PR, not by pointing a session at them.

Related

• Wiki Store: the page format and the run contract
• Skill evolution: why the maintainer exists
• Farm Tick: the nightly cycle that runs it
• bob CLI reference: every flag and exit code
• Skill evolution: the proposer: what the proposer may change and why a person merges