Verification System

Verification & Testing System

TL;DR. make check validates the system (deps, schemas, symlinks, hooks, provisions); make test runs the suites; make ci runs both — what GitHub Actions runs. One command to verify everything.

Bob’s infrastructure-as-code verification layer. One command to validate the entire system.

make check    # verify everything
make test     # run all tests
make ci       # both (what GitHub Actions runs)

Quick Start

cd ~/projects/bigbrain    # or wherever BOB_SOURCE is
make help                 # see all targets
make check-deps           # am I set up correctly?
make check                # is everything healthy?
make test                 # do all tests pass?

Targets

Verification (make check)

make check runs all five checks in order. Stops at first failure.

Target	What It Checks	Speed
check-deps	node >= 18, jq, git, make installed	instant
check-schemas	All JSON configs valid against schemas	instant
check-manifest-meta	Every provisions/<project>.json carries its own _meta.project — never the _default template’s (#1869)	instant
check-kits	registry/kits.json validates against schemas/kits.schema.json, no two kits share a name, no kit is empty, every runbook a kit names exists in registry/runbooks/ (#2239), and every skill/command/agent a kit names exists in registry/<type>/ or the universal <type>/ directory (#2225). ARGS=--coverage lists registry items in the review/security/process/governance categories that belong to no kit (report-only, exit 0). See Session Kits	instant
check-rituals	registry/rituals.json validates against schemas/rituals.schema.json and describes every ritual in the fixed vocabulary exactly once, every kit a ritual applies is declared in registry/kits.json, every farm-eligible ritual names at least one evidence collector from the fixed set (#2233), and every provisions/<project>.json _meta.rituals is a valid declaration — a known ritual, a valid cadence (P3M, P1M, before-stable-release), a drill_kind on drills only, no ritual twice (#2230). See Rituals	instant
check-secrets-inventory	secrets-inventory.json (the credential inventory — names and references, never values) validates against schemas/secrets-inventory.schema.json; every string in it is refused when shaped like a secret value (known token prefixes, 32+ hex digests, 20+ character high-entropy letters-and-digits runs — the path is named, the string never printed); ids are unique, consumer projects/machines exist in projects.json / machines.json, per-machine copies name a machine, every location ref is in its kind’s notation, every runbook path exists. A null runbook or interval is a warning, not a failure (#2237). See Credential Inventory	instant
check-retired-items	Every item path git history has seen under hooks/, commands/, agents/, skills/ or their registry/ twins that is now absent has a record in scripts/lib/retired-items.json — schema-valid, no stale or duplicate records, a live replacement on every superseded-by. An archive cannot land without its record (#2314). See Retired Items	~1s
check-dispositions	registry/dispositions.json validates against schemas/dispositions.schema.json; one record per item; every record names an item in registry/<type>/ or the universal <type>/; a certified record names its model and eval baseline and rests on an improved exam with a positive with-vs-without delta for that model (no-signal never certifies); a retire record is completed by its scripts/lib/retired-items.json entry. Items with no record are counted as ungraded, never failed (#2901). See Dispositions	instant
check-item-collisions	No item name exists in both the universal directory and its registry/ twin — commands/, agents/, skills/, runbooks/. A name on both sides resolves differently per project (bob init links the universal copy, bob provision links the registry one and the registry link shadows it), so attended and unattended runs can execute different instructions under one name — which is what /requirement did until #2735. The fix is never “keep both in sync”: decide which side owns the name, delete the other, and record the retirement (#2735, decision #2741)	instant
check-symlinks	Symlinks in the current repo resolve (--all / make check-fleet for every project)	~1s
check-hooks	All hooks in settings.json exist + executable	instant
check-provisions	The current repo’s symlinks match its manifest (--all for the fleet). An item the manifest declares that was promoted out of the registry into the universal <type>/ has no registry source, and is graded as a stale declaration — a warning naming bob provision remove <type> <name> — not a failure, matching what universalItemPath() in the provisioning engine already does. An item in neither the registry nor the universal set is still a hard failure (#2735)	~1s
check-scenarios-drift	Every command/flag/path/count named by the workday scenarios page — and the BoB in One Day tutorial rendered from it — still ships; every bob command, verb and flag is a scripts/bob/commands.json row; the committed tutorial equals a fresh render (#843, #2076)	instant
check-abs-paths	No machine-absolute path (/Users/<name>/, /home/<name>/) in any synced or ledgered file, and no absolute-target symlink in a synced tree (#1924); no operator identity literal from scripts/checks/operator-literals.json in any synced, ledgered, test or app file (#2268); no path joined directly to an operator data file (machines.json, provisions/, ports.json, mirrors.json, cloudflare-accounts.json, apps/intake/config) — the one way is operatorPath() (#2270). One allow-list with a why per entry (until names the requirement that retires it); --apply migrates paths	instant
check-decide-lint	Every decision call goes through the decide() seam: no require(...jev.js) or jev.evaluate( / jev.batch( outside scripts/lib/decide.js, no decide() state key outside its caller’s allow-list in decide-callers.json, no undeclared caller (#2982). A state it cannot resolve is counted as unverified, never passed silently. Allow-list scripts/checks/decide-allow.json, a why per entry	instant
check-legacy-names	No legacy entrypoint name (cdprov, cdfork, dev-up, warp, …) is invoked from scripts/, hooks/, commands/, skills/, bin/, docs/, the Makefile or the root docs — callers use bob <noun> (#2091). Allow-list with a why per entry; --apply rewrites	~1s
check-checkpoint-dirt	The segmentation checkpoint’s dirt rule is written once: docs/reference/checkpoint-dirt-rules.md and tests/fixtures/checkpoint-dirt.json agree on class names + outcomes, and every class has a fixture case (#2037)	instant
check-phase-contract	The warp-drive phase/transition contract is pinned: PHASES, TRANSITIONS and DEFAULT_REASONING_BUDGET in scripts/warp-drive/state-machine.js equal tests/fixtures/phase-contract.json (exact diff on any addition, removal or retarget), and the transition table in docs/reference/warp-drive-state-reference.md + the reasoning defaults table in docs/reference/token-monitoring.md list exactly the fixture’s rows. Frozen until 2026-10-11 — a change needs the fixture edited in the same commit and a linked decision issue (#2042)	instant
check-workflow-keys	The _workflow config table is declared once: every _workflow.<key> read in hooks/scripts/bin/commands is in WORKFLOW_DEFAULTS, every table key is read somewhere, every key has a comment, nothing reads the _workflow block outside the two resolvers — scripts/lib/workflow-config.sh for shell (jq-leak) and scripts/lib/workflow-key.js for Node (js-leak, #2719) — and both generated artefacts, docs/reference/workflow-config.md and the _workflow block of schemas/manifest.schema.json (#2703), are what --write renders from the table (#2041)	instant
check-workflow-runs-on	Every shipped GitHub Actions workflow — .github/workflows/*.yml and every workflow template under templates/ — resolves each job’s runner from the CI locus seam runs-on: ${{ vars.CI_RUNNER_LABEL || 'ubuntu-latest' }}, or pins it explicitly with # pinned: <reason> on the runs-on line or the line directly above. A hardwired runner, an empty pin reason or a variant seam fails, naming file, line, job and fix (#1729). See Workflow Authoring	instant
check-report-labels	No warp-drive report carries a knowledge label: no issue holds chunk-report/session-report and lesson/decision, and no report-shaped issue carries lesson/decision with no report label (#2062). Fails open on a degraded forge and is skipped when $CI is set — it grades the tracker, not the checkout	~2s
check-gh-contract	Every gh call in the tree is inside the recorded adapter contract (scripts/forge/gh-contract.json): a new command, flag or --json field fails naming its call sites, every entry carries its behavior, output and portability, and the generated block of the gh Adapter Contract page is current (#1753)	<1s
check-howto-coverage	Every row of scripts/bob/commands.json is reachable from a how-to: some page under docs/how-to/ or docs/tutorials/ (section index.md excluded) invokes bob <noun> or bob <noun> <verb>, word-bounded — or the row declares "reference_only": "<why>" in the table itself, declared never inferred. Fails uncovered, reference-only-empty and reference-only-stale (a declared row a how-to now runs), each naming the fix (#2553). --json is howto-coverage/1	instant
check-troubleshooting	Every entry of docs/how-to/troubleshooting.md (each ### under ## Errors, by message) declares the file that prints its message in an <!-- emitted-by: <file> :: <literal> --> marker, and every literal still appears verbatim in its file. A reworded message fails its entry as stale; an entry with no marker is undeclared, a missing file is missing-file. Fenced examples are ignored (#2562)	instant
check-cli-table	The CLI table is generated once from scripts/bob/commands.json: every <!-- cli-table:start --> block (CLAUDE.md, README.md, docs/reference/glossary.md) is what the generator renders, and every bin/ entrypoint is a table row or a declared not-a-noun (#2061). ARGS=--write re-splices	instant
check-doc-counts	Every component count is generated once: docs/reference/generated/counts.json equals a fresh count by gen-reference.js, and every <!-- count:<key> --> marker in README.md, CLAUDE.md and the handbook equals counts.json (#2087). Also a docs-check prerequisite. ARGS=--write regenerates and re-splices	instant
check-host	This machine’s BOB_HOME matches source on every deploy-managed surface, every BOB_HOME symlink resolves here, toolchain preconditions hold, settings hooks converged (#1921). The project’s .claude/ links are graded too, but as a separate project_symlinks section that is reported without changing the machine verdict — so bob doctor and bob doctor --project X always agree on whether this machine is converged (#1953). Skipped under $CI — it grades the host, not the checkout. Alias make doctor / bob doctor	~0.2s
check-fleet	check-symlinks + check-provisions across all registered projects (#294)	~2s

Testing (make test)

make test runs every tests/test-*.sh and tests/test-*.js through one runner, scripts/run-tests.sh (#2043). There are no per-test Makefile targets: dropping a file into tests/ is the registration, so a new test can never be silently left out of the suite by a forgotten Makefile edit.

Command	What it does
make test	Discover every tests/test-*.{sh,js} (byte-ordered), run each in its own process from the repo root, list every failure with the last 20 lines of its output, exit non-zero if any failed
make test-<name>	Run one file — tests/test-<name>.sh or .js — streaming its output live (a pattern rule, test-%, not a per-test target)
scripts/run-tests.sh --list	Print the discovered test names
scripts/run-tests.sh <name>...	Run a subset by name; the runner is the same entry point make calls

The runner keys on filenames, not on the old target names: make test-fleet is now make test-fleet-machine, make test-warp-drive still works because tests/test-warp-drive.sh exists, and an unknown name exits 2 listing the nearest files. A suite that lives elsewhere — the state-machine suite beside its engine (scripts/warp-drive/test-state-machine.js), tests/orchestrator/, the intake Worker under apps/intake — is on the glob through a thin tests/test-<name>.sh wrapper, never a special case in the runner. Nested directories under tests/ are not globbed.

Knob	Default	Effect
TEST_TIMEOUT	300	Seconds per test; a test past it is killed (SIGTERM, then SIGKILL 5s later) and reported TIMEOUT. 0 = no limit. Uses timeout/gtimeout when present, a watchdog subshell otherwise, so macOS without coreutils behaves the same
# test-timeout: N (in the test file)	—	A test that legitimately needs longer declares it in its own header (first 20 lines; // test-timeout: N in a .js). Only ever lengthens: the larger of the declaration and TEST_TIMEOUT applies. tests/test-warp-drive.sh declares 900 — the state-machine suite runs ~280s alone
TEST_TAIL_LINES	20	Lines of a failing test’s captured output shown in the FAILURES block

Operator sandbox (#2679). Since #2270 a manifest or ports.json write lands in the resolved operator repo, so a test that sandboxed BOB_SOURCE but not the operator wrote its fixtures into the real one — farm-01’s CI runner did, on every run. The runner now exports BOB_OPERATOR as a throwaway copy of tests/fixtures/operator/sample for every test, and guards the operator repo that would otherwise resolve (the caller’s environment, and with BOB_OPERATOR unset) plus this checkout’s provisions/: each is hashed before the run and after it, and any file created, changed or deleted fails the run with OPERATOR REPO CHANGED DURING THE RUN, naming the file, even when every test passed. A suite that writes manifests still owns its sandbox, so it is safe run directly too: tests/lib/operator-sandbox.sh / .js give it one (operator_sandbox / operatorSandbox()), make a fake BOB_SOURCE its own operator (operator_self_contained / selfContained()), or give a check an environment with no operator at all (noOperatorEnv()). A generated product page’s drift check needs none of them: its owner: comes from the page’s own frontmatter (#2689), so it points BOB_OPERATOR at a foreign profile instead and asserts the render is unchanged.

Shell and tmux sandbox (#3165). Suites that type into tmux panes (send-keys) or start an interactive shell left those commands in the operator’s ~/.bash_history and ~/.zsh_history. A make test started inside tmux also sent every bare tmux call to the operator’s own server, where a new window takes the server’s environment, not the test’s. The runner now exports HISTFILE under its work dir and a private TMUX_TMPDIR for every test, unsets TMUX, and kills any server left on that socket at the end. It also guards both history files: each file’s size is recorded before the run and checked after it. Growth fails the run with SHELL HISTORY CHANGED DURING THE RUN and the appended lines, even when every test passed. A shell of your own that exits and saves its history during the run shows there too. An rc file that assigns HISTFILE itself outranks the exported value, and that is the case the guard catches. A tmux-driving suite is safe run directly too: tests/lib/shell-sandbox.sh gives it the same sandbox (shell_sandbox).

Each test runs exactly once — the runner has no rerun-on-failure. Exit codes: 0 all passed, 1 at least one failed or timed out, a guarded operator repo changed during the run, or a shell history file grew, 2 usage error or unknown test name. The run ends with All tests passed. on success, the sentinel local-delivery checks for.

CI (make ci)

Runs check then test. This is the target GitHub Actions calls on every push and PR to master.

---

Check Details

What a lint grades: tracked files only (#2772)

Every check that walks a tree grades the paths git tracks. The rule is declared once, in scripts/lib/tracked-scope.js, and read by check-abs-paths and check-legacy-names — never restated per lint:

A lint grades a path git tracks. Outside a git work tree — a tarball, a fixture checkout, git unavailable — it grades everything it walks.

Verification is about what the repo carries. A gitignored, locally-generated file is not the repo’s state, so grading it makes make check red on a developer machine for content git does not carry, while CI — a fresh clone with no build output — stays green. That asymmetry is the expensive part, not the individual hit: the same redness kept being re-filed (#2509, #2701) and three loop notes accumulated telling agents to ignore three gates. A gate that is red for reasons the repo does not carry stops being a signal.

Measured on farm-01: all 32 check-legacy-names hits were in the docs-site build output (docs/reference/generated/cli.md, commands.md — mirrors of the tracked cli.md, whose old→new table is already exempt), and two of check-abs-paths’ three were in a machine-local .mcp.json.

• An ignored path is never a blanket exemption. The filter is on tracked-ness, not on a path pattern, so a hit planted in a tracked file still fails — asserted in both directions by tests/test-tracked-scope.sh.
• The narrowing is reported, never silent. Both gates print what they graded and how many paths they skipped, and carry files_untracked / scoped_to_tracked in --json. A PASS never hides a shrunken scope.
• No git means no notion of “ignored”, so the full walk is the only defined behaviour — which is also what keeps the mktemp-fixture suites meaningful.
• Not git check-ignore. Tracked-ness is one cheap call for the whole tree, and a file that is both tracked and ignored (added before the ignore rule) is still the repo’s state and must still be graded.
• Trade-off: a brand-new file is not gated until it is git add-ed. CI runs on a clean checkout where every file is tracked, so the gate keeps its full reach exactly where the merge decision is made.

A check that grades a named file rather than walking a tree is outside the walker rule — it opted into that path deliberately, so the scope question is a decision about that check. check-schemas validating the live .claude/.decisions.jsonl (and .claude/session-ledger.jsonl) is the one such case, and it now answers the question the same way (#2113, #2787), through the shared predicate rather than a second copy of the rule:

const { gradedAsRepoContent } = require('../lib/tracked-scope');

• Tracked — graded as repo content, exactly as before: a malformed record fails the gate.
• Untracked (the normal case; both paths are gitignored) — advisory. The non-conforming record is still printed, with its line and the reason, but as a WARN: it is this machine’s runtime residue, not the repo’s content, and grading it as content made make check red on one machine and green on another with no commit able to clear it.

The complement is that the residue should not become malformed in the first place. A session-state file a gate validates must be produced by a writer that validates it, so scripts/decision-log/log-decision.sh refuses a record it knows cannot conform — an out-of-enum source, or a context that is not an object — reading the enum from the schema so the two cannot drift. It stays fail-open: a refusal exits 0, appends nothing, and explains itself on stderr.

check-deps

Verifies the four required tools are installed:

  PASS  node 25.5.0 (>= 18)
  PASS  jq 1.7.1
  PASS  git 2.48.1
  PASS  make (GNU Make 3.81)

If something fails, the output includes a fix command:

  FAIL  node 16.20.0 (need >= 18)
        Fix: Install Node.js 18+ via nvm or brew

check-schemas

Validates every JSON configuration file against its JSON Schema definition:

File	Schema
projects.json	schemas/projects.schema.json
settings.json	schemas/settings.schema.json
provisions/*.json (17 files)	schemas/manifest.schema.json
schemas/fixtures/operator.sample.json, templates/operator/operator.json — and schemas/fixtures/operator.rejected.json, which must fail (a secrets[] entry carrying a value, #2267)	schemas/operator.schema.json
templates/dev.json, templates/operator/operator.json — validated as the copy they render into, {{BOB_UPSTREAM_*}} tokens filled (#2441)	their schema
every schemas/*.schema.json $id — must name its own file at BoB’s declared upstream (scripts/bob/upstream.json), <web>/schemas/<file> or <raw>/schemas/<file> (#2441)	—

If something fails, the output shows the exact path and error:

  FAIL  provisions/bodmail.json
        $._meta.project: expected type string, got number
        $.skills[3]: duplicate item "webapp-testing"

check-manifest-meta

Guards manifest identity (#1869). cdprov --init --auto once copied provisions/_default.json verbatim when no stack was detected, leaving _meta.project: "_default" and _meta.path: null — a name that collides across projects in audit.js / cdprov status and a path nothing can resolve. The generator now stamps both; this gate keeps the corpus honest:

• FAIL — _meta.project missing, "_default", or not equal to the manifest’s basename.
• WARN (non-fatal) — _meta.path is null; set it ("$HOME/projects/<name>" convention) or re-run cdprov --init --auto from the project.

Exempt: _default*.json (templates), _bob-home*.json (the BOB_HOME surface, #411) and ports.json (the port-slot registry, #197).

To fix: Edit the JSON file to match the schema. The error path ($._meta.project) tells you exactly where the problem is.

check-projects

projects.json — the registry nine fleet readers consume (check-symlinks / check-provisions / check-hooks --all, repair-fleet.sh, check-schemas.js, the dashboard, the launchpad links index, gap analysis, audit.js, deploy.sh) — is derived, never hand-edited (#2066). scripts/fleet/projects-derive.js projects every manifest’s _meta:

Field	Source
name	_meta.project
path	_meta.path
group	_meta.group, else derived from the path: Sites/peabod* → Peabod, other Sites/* → Other Sites, a top-level $HOME dir → Personal, else Other
active	_meta.active !== false

provisions/_bob-home.json contributes the BoB entry ($HOME/.claude); templates (_default*), per-machine global variants and ports.json contribute nothing. make check-projects fails when the committed file differs from the derivation — an added, removed or re-pathed project, a group or active change, or order. _meta.updated moves only when the registry does, so a re-run is a byte no-op.

active is declared, not measured. Every reader treats active: false as “archived — skip it” and measures checkout presence itself, and bob doctor byte-compares the deployed copy against source, so a flag measured on one machine would hide every project checked out only on another. Presence is still reported:

projects: 43 registered from 43 manifest(s); presence measured on farm-01
  absent on this machine: 26 (--verbose lists them)
  local repos with no manifest (listed, not registered): $HOME/Sites/loclook (≈ loclook), $HOME/projects/bodfeed
  PASS  projects.json matches the manifests

Anything ambiguous is listed, never registered or deleted: a manifest with no _meta.path, and a git checkout beside a declared one that no manifest names (≈ name when it shares a registered project’s basename — usually the same repo at a machine-local path). Declaring a project is cdprov --init’s job.

To fix drift: change the manifest (_meta.path, _meta.group, _meta.active: false to archive), then regenerate and commit both:

make check-projects ARGS=--write     # or: node scripts/fleet/projects-derive.js --write
node scripts/fleet/projects-derive.js --json    # the projects-derive/1 document

check-symlinks

Scans every project listed in projects.json, finds all symlinks in each project’s .claude/ directory, and verifies each one resolves to a real file.

  FAIL  /Users/you/Sites/myproject/.claude/commands/sprint.md -> /Users/you/.claude/registry/commands/sprint.md (dangling)

To fix dangling symlinks:

# Option 1: Re-provision the project (recommended)
cdprov --refresh /path/to/project

# Option 2: Remove the stale symlink manually
rm /path/to/.claude/commands/sprint.md

Projects that don’t exist on disk or lack a .claude/ directory are silently skipped.

check-hooks

Reads settings.json, extracts every hook command path (from SessionStart, PreToolUse, PostToolUse, Stop), and verifies:

1. The script file exists
2. The script is executable (chmod +x)

Also checks the statusLine.command if configured.

  PASS  /Users/you/.claude/hooks/temporal-context.sh
  FAIL  /Users/you/.claude/hooks/missing-hook.sh
        Fix: File not found — remove from settings.json or create the script

To fix:

# If the hook should exist:
chmod +x ~/.claude/hooks/the-hook.sh

# If the hook was removed:
# Edit settings.json and remove the hook entry

check-provisions

For each project in projects.json, loads its manifest (provisions/<project>.json or provisions/_default.json) and verifies that every declared skill, command, and agent is:

1. Present in the registry (registry/skills/, registry/commands/, registry/agents/)
2. Symlinked into the project’s .claude/ directory
3. Not dangling

  PASS  bodmail (bodmail.json)
  FAIL  myproject: commands/sprint declared in _default.json but not symlinked
  FAIL  myproject: agents/code-expert symlink is dangling

Failure types and fixes:

Failure	Meaning	Fix
declared but missing from registry	Manifest references something that doesn’t exist	Remove from manifest or add to registry
declared but not symlinked	Item exists in registry but wasn’t linked to project	Run cdprov --refresh /path/to/project
symlink is dangling	Symlink exists but target was moved/deleted	bob provision --refresh — it retargets or prunes what the retirement record and the deployed tree explain, and reports the rest (#2314); bob provision --prune for the explicit repair (see below)

A bare checkout is expected to fail it (#2635). Without --all the target grades the current repo, so a fresh clone or a detached git worktree — which has no .claude/ links until bob init + bob provision refresh (#1479) — reports every _default.json item as declared but not symlinked. That is the target working: “make check is green on master” therefore means green on a provisioned checkout (what CI asserts), and two targets grade the machine rather than the tree — this one, and check-host, which compares BOB_HOME against source and is red whenever source has moved ahead of the last deploy (it is skipped under $CI for exactly that reason).

check-abs-paths

State that deploy.sh syncs to every machine, or that a ledger records, rots the moment it carries one machine’s home directory. The two recorded shapes are a symlink inside a synced tree whose target is absolute — rsync -l copies the link verbatim, so a runbook link written on the Mac pointed at /Users/paulirving/... on the farm (#1914) — and a manifest whose _meta.path exists on one machine only (#1919). The #1475 LINK_ROOT rule (see the handbook) removes the cause for registry links at write time; cdi’s universal links and cdprov’s universal retargets follow it too (every link BoB writes targets $BOB_HOME, never the source checkout), and check-abs-paths catches everything else at commit time.

check-abs-paths runs scripts/checks/abs-path-lint.js:

• Scope — skills/, registry/, runbooks/, templates/, provisions/, scripts/, hooks/, bin/ and every *.json at the repo root. docs/ is deliberately excluded: prose quotes real paths as examples. Only paths git tracks are graded — see What a lint grades.
• Files fail on any literal /Users/<name>/ or /home/<name>/. Symlinks fail on any absolute target — a synced link cannot be derived from $HOME on the receiving machine, so it must be relative.
• Allow-list — scripts/checks/abs-path-allow.json. Each entry is { "path", "pattern"?, "why" }; path is a file or directory prefix, pattern an optional substring the offending line must contain. An entry without a why is a config error (exit 2) — this is a list of documentation examples, not a suppression list, and there is no implicit comment exemption (a comment is where a copy-pasted real path hides best).
• Migration — --apply (dry-run by default; the report always shows what it would do) rewrites a JSON string value that is an absolute home path to the $HOME/... form the hand-authored manifests use and provision.sh manifest_project_path writes (#1869), and rewrites a symlink whose absolute target lies inside the repo to the equivalent relative target. Each rewrite is reported; a hit in a script, or a link pointing outside the repo, is listed as unmigratable for a human. Idempotent.
• Contract — --json emits abs-path-lint/1: {rules[], hits[] (each with its rule), hits_by_rule, allowed, allowed_by_rule, allow_entries, burn_down, unused_allow[], rewrites[], unmigratable[], ok}. --root DIR lints another tree (fixtures); --literals FILE swaps the literal set. Exit 0 clean · 1 hit(s) remain · 2 usage/config error.

abs-path-lint — /Users/.../bigbrain
  scope: skills registry runbooks templates provisions scripts hooks bin *.json   allow-list: scripts/checks/abs-path-allow.json (244 allowed hit(s))
  operator literals: 4 declared in scripts/checks/operator-literals.json   scope: + tests apps .codex root non-JSON (no *.md)
  operator-literal allow-list: 41 entries carrying 243 hit(s)   burn-down by retiring requirement: #2270 ×12 · #2271 ×1 · #2441 ×18 · #2442 ×8 · deliberate ×2
  would rewrite 1 ledgered path(s) (dry run — pass --apply):
    [would-rewrite] provisions/seebod.json _meta.path: /home/farm/Sites/seebod -> $HOME/Sites/seebod
  FAIL  2 hit(s) — 1 machine-absolute path(s), 1 operator literal(s):
    provisions/seebod.json:4  [abs-path] /home/farm/  "path": "/home/farm/Sites/seebod",
    scripts/warp-drive/new-gate.js:12  [operator-literal] paulirv  '--assignee', 'paulirv',

Where a path is legitimately machine-specific, derive it at use time — $BOB_HOME for deployed tooling, $HOME for project roots (the port ledger’s contractHome, projects.json), $BOB_SOURCE for the checkout — or record a project id and resolve it through projects.json on the current machine.

The operator-literal rule (#2268)

BoB ships the same to every operator (#2214, decision #2266), so a literal naming one operator — the forge login, e-mail, worker subdomain or display name — in a script, hook, template, registry item, test or app is a value that belongs in the operator profile (#2267), not in the product. The same engine grades it as a second rule:

• Literal set — declared once in scripts/checks/operator-literals.json (operator-literals/1), never inlined in the lint. A fixture root that carries its own set is linted with that one; a missing or empty set is a config error (exit 2) — the rule is never silently off. Matching is a case-sensitive substring per line.
• Scope — the abs-path scope plus tests/, apps/, .codex/ and every root-level non-JSON file, minus every *.md. docs/ and journal/ stay outside both rules. .service, .timer and .plist files are graded like any other text file (a synced systemd unit is exactly where a home path or an upstream URL rots).
• Hits — every hit carries its rule (abs-path | operator-literal | operator-path) in the report line and in --json (hits_by_rule, allowed_by_rule). An operator-literal hit is never rewritten by --apply; it is listed as unmigratable, because the fix is to read the value from the profile.
• Allow-list — the same file, with two optional fields per entry: rule scopes an entry to one rule (absent = both), until names the requirement that retires it (#NNNN or owner/repo#NNNN; absent = deliberate for good, the #1831 transcript fixtures whose real transcript paths are the evidence). Both are validated as config errors like why.
• Burn-down — the report prints the number of operator-literal entries, the hits they carry and a count per until, so each follow-on requirement’s progress is observable (allow_entries, burn_down in --json). An entry that matches nothing is listed [unused] (INFO, not a failure) — delete it; a retired entry must not accumulate.

Baseline (2026-09-15). #2268 carried every then-current hit rather than cleaning up in one pass: 56 entries over 74 files, 266 lines. By retiring requirement: #2269 (people, doc owners, settings owner, BoB-upstream fallback) ×44, #2270 (intake config, secrets inventory, machine-absolute Codex hooks) ×10, #2271 (the remote-decision-bridge route) ×1, deliberate ×1 (tests/fixtures/hooks/). Product files carry a pattern so a new literal in the same file still fails; test files are carried per file. The tracked registry/skills/skill-creator/scripts/__pycache__/*.pyc was deleted (__pycache__/ is gitignored) and three comment-only examples were rewritten to neutral names.

Burn-down so far. #2269 (same day) retired 16 entries — the todo assignees, the doc-owner defaults, the settings owner, the trace-mining upstream fallback and the test fixtures tied to them — and re-pointed the rest: the product’s own BoB-upstream references (schema and template URLs, BOB_REPO defaults, REMOTE_URL, upstream issue links in the generated references) to #2441, the forge-owner-shaped sample data in tests to #2442, the gh-projects template owner to #2270. #2441 retired all 18 of its entries: the product’s upstream is declared once in scripts/bob/upstream.json, and every product-side reference renders from it or is checked against it (schema $ids by make check-schemas). That declaration is the one deliberate entry the change added. After #2441 the list stands at 29 entries carrying 307 hits: #2270 ×12 · #2442 ×10 · #2271 ×1 · deliberate ×16. The deliberate entries are captured fixtures, IWPI’s own profile fixture, the upstream declaration and similar. The lint’s own report is the live figure.

The operator-path rule (#2270)

The operator’s data — machines.json, mirrors.json, provisions/ and every provisions/<x>.json, ports.json, config/cloudflare-accounts.json, apps/intake/config — lives in the operator repo (decision #2266), and for one release also at its legacy BOB_SOURCE location. The one way to reach any of them is operatorPath(<name>) in scripts/lib/operator-resolve.js (bob_operator_path in the shell twin), which resolves the operator repo first and the legacy location second, with a dated deprecation warning. The same engine grades that as a third rule, so a consumer cannot quietly keep joining its own path:

• What fires — a path.join / path.resolve whose first argument is a BoB root identifier (BOB_SOURCE, BOB_DIR, bobDir, bobSource, ctx.bobDir, scriptDir, __dirname, root, src, …) and a later argument names a data file — path.join(bobDir, 'provisions', f) — or a $ROOT/<data> interpolation in a shell script or template string — "$BOB_SOURCE"/machines.json, ${S.BOB_SOURCE}/provisions, and the deployed-copy spellings $HOME/.claude/… and ~/.claude/…. A registry or schema path from the same root is not a hit; neither is a message string that merely says provisions/.
• Scope — skills/ registry/ templates/ scripts/ hooks/ bin/ apps/ .codex/ and root-level non-JSON files, code only: no *.md, no *.json (a JSON file cannot join a path) and not tests/ — a test builds fixture BOB_SOURCE trees, and $ROOT/provisions there is the fixture. The AC that no test reads this checkout’s machines.json or a real manifest is held by the tests reading a fixture operator repo, not by this rule. provisions/ and runbooks/ are data, not code, and are outside it.
• Exemption — the resolver’s own two files, scripts/lib/operator-resolve.js and .sh, are never scanned by this rule: they are the path.
• Allow-list — the same file and fields. A check that grades the checkout’s own copy (check-schemas over provisions/_default.json, check-kits, check-rituals, check-manifest-meta, check-provisions; check-operator.js, whose root is the operator repo it was asked to validate) is a deliberate entry with that why. Every other entry carries until: "#2270" and is deleted as the consumer migrates; the report prints the operator-path entry count and the hits carried, and the entries count in the same per-until burn-down. An operator-path hit is never rewritten by --apply — the fix is a call site.

Baseline (2026-09-15, #2270 chunk 1). The rule landed carrying every then-current direct path: 32 entries over 32 files, 53 hits — 25 consumers waiting to migrate under until: "#2270", 7 deliberate (the six checkout checks and the lint’s own header).

Burn-down so far. Chunk 2 (same day) migrated the fleet-side consumers — machine.js, mirror.js, port-ledger.js, role-kits.js, the evidence ledger, the dashboard server, gap-detect.js, ritual.js, audit.js, port-fleet-migrate.js, the Launchpad, distribute-secret.sh and the deploy-cloudflare skill — and deleted their 12 entries. Chunk 3 migrated the provisioning-side readers — provision.js (reads per file through manifestFile(), writes into the resolved provisions/), projects-derive.js, settings-hooks-reconcile.js, host-drift.js, ready.js, provision-preflight.js, plan-scope.js, the gh-projects manifest lib, cdi, cdb and cdr.sh — and deleted 11 more, adding one deliberate entry for the reconcile’s no-resolver fallback. Chunk 4 moved the manifest writers: interview.sh reads through the resolver (entry deleted) and config-branch.sh’s $root is the operator repo _cb_root resolved (entry kept as deliberate). Standing at 9 entries, all deliberate — the operator-path burn-down for #2270 is complete; the rule now guards new code. The lint’s own report is the live figure.

check-legacy-names

The #2060 alias shims keep the old entrypoint names working for one release cycle; nothing inside BoB may keep calling them, or the shims could never retire. check-legacy-names runs scripts/bob/rewrite-callers.sh --check (#2091):

• Scope — scripts/, hooks/, commands/, skills/, bin/, docs/, the Makefile, CLAUDE.md, README.md and AGENTS.md. The mapping’s own sources (scripts/bob/commands.json, gen-shims.sh, help.js), the generated shims and the rendered old→new reference docs/reference/cli.md are never scanned, and a generated cli-table block is skipped — its Was column is the record. Only paths git tracks are graded — see What a lint grades.
• What counts as a call — a legacy name in command position: a code span that opens with it, a word after a shell operator (&&, ||, ;, |, $(, exec), command -v, a line that opens with it (Markdown: inside a fenced block only), or a bin/<name> path run as a command. Prose — “a swarm worker”, “the dev-up lifecycle” — and longer tokens such as warp-drive or /dev-up are not calls.
• Manual hits — a bin/<name> path used as data (assigned to a variable, listed for an existence check, named in prose) and every path.join(…, 'bin', '<name>') cannot be rewritten mechanically: the verb has to move into argv. The check reports them; --apply leaves them for a hand fix.
• Fix — scripts/bob/rewrite-callers.sh --apply rewrites every mechanical hit to its bob <noun> spelling, taken from scripts/bob/gen-shims.sh --list so a shim and its rewrite never disagree. A reviewed false positive goes in scripts/bob/rewrite-callers-allow.json as { "path", "pattern"?, "why" } — an entry without a why exits 2.

check-host (bob doctor)

make check grades the checkout; nothing graded the deployed runtime a machine actually runs (#1914). A laptop can sit months behind source on scripts/, carry registry symlinks that point at another user’s home, or run the real gh ahead of the forge shim — and every other check stays green. check-host runs scripts/checks/host-drift.js against this machine’s BOB_HOME (#1921). Same entry point from bin/: bob doctor [--json] [--project <dir>] [--section a,b].

Four sections, each graded independently and each naming its own remediation:

Section	What it grades	Fix it names
surfaces	Every deploy-managed directory and file, compared with the commit the deploy recorded (.bob-version source_sha), not the source working tree (#3160) — the list is derived from deploy.sh’s own sync_dir / sync_file lines at that commit, so the check and the deploy cannot disagree about what is managed. Reports missing / stale / extra (extras count: deploy.sh syncs with --delete). Exclusions are an explicit list (.DS_Store, __pycache__, node_modules, .git), never a heuristic. Paths another owner writes inside a managed surface — skills/synced/, Claude Code’s sync of the account’s claude.ai skills — are declared once in scripts/lib/harness-owned-paths.json: neither missing nor extra here, and deploy.sh’s sync_dir derives an rsync --exclude from the same file so --delete leaves them alone (#2558). Anything else undeclared under the surface is still extra	scripts/deploy.sh, scripts/lib/harness-owned-paths.json
symlinks	Every symlink under BOB_HOME (runtime dirs like projects/, cache/ skipped) and under the current project’s .claude/ resolves on this machine. A dangling target, or a target under a foreign home (/Users/someone-else/…, /home/… on a Mac), fails and names the link	scripts/deploy.sh (BOB_HOME) · cdprov refresh (project links)
toolchain	node, jq, gh on PATH; BOB_HOME/bin/gh resolves ahead of the real gh (the forge shim must win, #1748); the one guarded bob-shell.zsh line is in ~/.zshrc	install the tool · scripts/bob-install.sh
settings_hooks	settings.json hook convergence — hook-drift.js (#1442/#1918) included as one section, not re-implemented — against the hooks the deployed commit declares (#3160). On the working-tree baseline, with --in-flight-base <ref> on a self-hosted run whose branch rewired settings.json / hooks/dispatch.tsv, graded against the source at <ref> and reported as self-hosting when it matches (#2203)	scripts/deploy.sh (reconciles .hooks)

Exit code is the answer: 0 converged, 1 drift, 2 error/usage. --json emits a versioned bob-host-drift/1 document (verdict, drift_count, sections[], version, operator), which the fleet snapshot carries per machine as summary.host_drift and audit.js view rolls up as a “BOB_HOME drift” table — so a machine behind source is visible from any other machine, with the fix named.

What it is measured against (#3160). BOB_HOME is compared with the commit deploy.sh recorded in .bob-version (source_sha): each deployed file’s git blob id against the one that commit holds for the path. The source working tree is not read, so a dirty tree, a feature-branch checkout or new commits in the source repo are never drift — they used to fail every other project’s gate on the machine. The document names the baseline and how far the checkout has moved since:

Key	Meaning
baseline	{kind, sha, dirty, reason} — deployed when .bob-version names a commit the checkout has; working-tree (with the reason) when there is no marker, no source_sha, or the commit is not in the checkout
behind	{deployed_sha, head, branch, commits, diverged, files, sample, fix} — commits on the checkout’s HEAD since the deploy and the deploy-managed files that differ. Information, never drift: rendered as INFO behind source: …
usable	false only when a hard finding exists. ok, verdict and drift_count still count every finding
hard_count / advisory_count	findings by severity; each section and each item carries its own severity

Severity: a finding is hard when it makes BOB_HOME unusable — a surface that was never deployed, node / jq / gh missing, a section that errored, or BOB_HOME differing from the commit it records (FAIL). Everything else is advisory and printed WARN with its fix: a BOB_HOME link that does not resolve, the forge shim or shell guard, and any difference measured on the working-tree baseline. A deploy from a tree with uncommitted changes to a deploy-managed path records source_dirty: true in the marker; after it a difference from the commit is advisory too, because what was synced is not what the commit holds. A file in BOB_HOME that the commit lacks but the source tree has (an untracked or ignored file the deploy synced) is listed under surfaces.untracked and is not drift.

Where “source” is (#2315). The engine resolves BOB_SOURCE through the #1220 chain — scripts/lib/bob-source-resolve.js, the JS twin of bob-source-resolve.sh: an explicit $BOB_SOURCE → the engine’s own tree when it is a checkout (.git + provisions/, never BOB_HOME) → ~/projects/bigbrain → BOB_HOME itself, flagged. It is never the engine’s own directory: the deployed copy under ~/.claude used to resolve __dirname and so compared BOB_HOME with itself — a vacuous PASS surfaces on every run from the router, and WARN contract: unreadable because tests/ is not a deploy surface. The document names the outcome as bob_source_resolution (rung: env / self / default / fallback, fallback, warning). On the fallback rung — no checkout anywhere, or a BOB_SOURCE pointing at the deploy target — the sections that compare against source do not pass: surfaces and settings_hooks are returned not-measured (na: true, ok: true so the exit code and every consumer keep their contract, the reason and the fix named), the contract line is INFO contract: not measured, the document lists them under unmeasured, and the text form closes with No drift measured — … Not a converged verdict. instead of Converged. The fleet snapshot’s compact form carries host_drift.source (rung, fallback) and the per-section na, so a machine that measures nothing is visible from any other. node scripts/lib/bob-source-resolve.js --json prints the resolution (exit 3 on the fallback rung).

The operator line (#2267). bob doctor also names the operator repo this machine resolves — scripts/lib/operator-resolve.js’s rungs, $BOB_OPERATOR → the $BOB_HOME/.bob-operator pointer → ~/projects/bob-operator (decision #2266) — as one line: INFO operator: <id> — <label> (<rung>: <root>) when the profile validates, WARN operator: … invalid profile naming the first error and bob operator show as the fix, or INFO operator: no operator declared with the two ways to declare one. The bob-host-drift/1 document carries it as operator (resolved, source, root, id, label, valid, errors), and the fleet snapshot’s host_drift.operator keeps the compact form. It is reported, never graded: an unresolved operator is legal until the legacy in-repo fallback is retired. See Operators.

The version line (#2046). Every non-dry deploy.sh run ends by writing $BOB_HOME/.bob-version — a bob-version/1 document: version (the checkout’s package.json), channel (stable when a vX.Y.Z tag sits at HEAD, nightly for a nightly-YYYY-MM-DD tag, checkout for an untagged HEAD), tag, source_sha, source_dirty (#3160: a deploy-managed path had uncommitted or untracked changes), bob_source and deployed_at. bob doctor reads it back as an INFO version: line — which source this machine runs — and adds source_head / source_moved so a checkout that moved on after the last deploy is named. It is provenance, never graded: a missing or malformed marker is a WARN naming deploy.sh, not drift (the surfaces section already grades content). bob update reads the same marker before moving the checkout, and the fleet snapshot carries host_drift.version so a machine’s deployed tag is visible from any other machine.

A machine whose checkout has moved on since the deploy is converged, and says how far behind it is:

BOB_HOME drift — /Users/you/.claude vs /Users/you/projects/bigbrain
  PASS  surfaces: 23 deploy-managed surface(s) match the deployed commit 79d7d59
  PASS  symlinks: 134 BOB_HOME link(s) resolve on this machine
  PASS  toolchain: node, jq, gh present; forge shim first on PATH; shell guard installed
  PASS  settings_hooks: 25 declared hook(s) wired

  INFO  version: 1.273.0 (v1.273.0) [stable] deployed 2026-09-11T06:02:11Z from 79d7d59 — source HEAD is now eea9316: deployed before the checkout moved
  INFO  behind source: 9 commit(s) behind source HEAD eea9316 on master — 18 deploy-managed file(s) changed since the deploy; scripts/deploy.sh brings them (#3160)
Converged — this machine's BOB_HOME matches the deployed commit 79d7d59.

A BOB_HOME that is not what its recorded commit produced is a hard finding:

BOB_HOME drift — /Users/you/.claude vs /Users/you/projects/bigbrain
  FAIL  surfaces: 2/23 surface(s) differ from the deployed commit 79d7d59 (3 file delta)
        scripts/                 missing 1  stale 1  extra 0
        CLAUDE.md                missing 0  stale 1  extra 0
          missing  scripts/fleet/settings-hooks-reconcile.js — in the deployed commit 79d7d59, not in BOB_HOME
          stale    scripts/deploy.sh — content differs from the deployed commit 79d7d59
        Fix: scripts/deploy.sh
  …
Drift: 3 — scripts/deploy.sh, then re-run make check-host.

Not graded in CI ($CI set → SKIP), for the same reason as check-hooks scope 3 (#1708): the verdict would depend on when the runner host last ran deploy.sh, not on the commit under test. The signal is not lost — the fleet audit owns per-machine drift.

check-scenarios-drift

Keeps the narrative workday scenarios page honest: it re-grounds every BoB surface the page names against the actually-shipped inventory, so the page can’t quietly rot into fiction against BoB’s daily churn (#843). The shipped command/skill/agent inventory is reused from the same harvester the docs-site reference generator uses (scripts/docs-site/lib/gen-reference.js) — the check and the reference can never disagree about what ships. It is deterministic and side-effect-free (pure filesystem reads, no network, no deploy), consistent with the “audits stay cheap” rule.

What it verifies — backticked syntactic classes, the bob commands in fenced command blocks, plus prose counts:

Class	Example	Grounded against
Slash command	`/warp-drive`	a shipped command or skill name
Repo file/dir path	`scripts/fleet/audit.js`	the file/dir exists on disk
CLI flag	`--from-issues`	the literal appears in a shipped command/script source
bob command	`bob warp viz --tailnet`, or a bob … line in a ```bash block	a scripts/bob/commands.json row for the noun (and verb, when the noun takes one); every flag on the line declared by that row’s usage or flags (#2076)
Surface count	“19 universal commands”	the live gen-reference inventory count

Bare words (cap, todo, cdprov) are not checked — in prose they are indistinguishable from domain nouns, and repo aliases like cdprov resolve through shell config outside the repo. In-repo CLI entrypoints are still covered, via their scripts/… / bin/… paths.

  STALE  docs/explanation/workday-scenarios.md:84  /nonexistent-command   (no such command or skill)

FAIL — 1 stale reference(s) in docs/explanation/workday-scenarios.md.

How to fix a failure:

• Renamed/removed surface — update the reference in the page to the surface’s current name (or drop it).
• Deliberate forward-reference — when the page intentionally names a surface a sibling requirement will add (e.g. /update-journey, built by #846), declare it in an HTML comment in the page so the allowance is explicit and visible:

  <!-- drift-allow: /update-journey -->

  Remove the allowance once the surface ships.

Command blocks and the tutorial (#2076). Inside a fenced block labelled bash / sh / shell / zsh / console / text, a line (or && / ; / | segment) starting bob gets the bob command check and a line starting / gets the slash and flag checks. Other lines — git, gh, claude — are not BoB surfaces and are left alone; an unlabelled fence (the seam map) is prose. The default run also checks every tutorial the registry declares (templates/tutorials/registry.json, #2412), one entry at a time. It scans each committed tutorial, such as BoB in One Day, with the same rules. It fails when a committed tutorial differs from a fresh scripts/render-scenarios.js --tutorial render, and names the first differing line and the re-render command. It also fails when the render errors: a frame slot with no page block, a page block that no slot places, a missing page or frame, or a broken learning path. A stale entry is reported on its own, so it never hides another. --page alone checks just that page. --tutorial <path> and --frame <path> compare one fixture render, and --registry <path> points at a fixture registry.

Run it directly with make check-scenarios-drift (it is part of make check).

check-cli-table

Keeps every copy of the shell CLI inventory equal to the routing table it is generated from (#2061). Three docs used to carry the table by hand and drifted independently — README was missing every bob noun added since the router landed. Now one renderer owns the body and several docs receive it.

The pipeline — one table, one renderer, several destinations:

1. scripts/bob/commands.json declares every bob command (#2044).
2. renderCliTable in scripts/docs-site/lib/gen-reference.js renders it, reading the table through the help engine that produces bob help --json (bob-help/1) rather than by hand. The rendered table carries no timestamp — one would make drift ungateable, since every regeneration would differ.
3. scripts/doc-keeper/sync.js --blocks-only --apply splices that body between the <!-- cli-table:start --> / <!-- cli-table:end --> markers in each host. sync writes what the generator produced; it never re-derives it.
4. check-cli-table gates the committed copies against what the renderer produces today.

The host list lives once, in gen-reference.js (CLI_TABLE_HOSTS), so the splicer and the gate cannot disagree about which files carry a block.

What it fails on:

Problem	Meaning	Fix
block-drift	a host’s block is not what the generator renders	make check-cli-table ARGS=--write
block-missing	a declared host has no marker pair	add the markers where the block belongs — placement is a human call, never invented
host-missing	a declared host file does not exist	fix CLI_TABLE_HOSTS
missing-entrypoint	a bin/ script is neither a row’s was nor a declared not-a-noun, so the generated table would omit a runnable command	add the row (#2059), or declare it not-a-noun in scripts/bob/help.js
render-failed	the routing table or help engine is unreadable (exit 2)	fix the table

--json emits cli-table-check/1 (source, command count, per-host verdicts, unmapped entrypoints, problems, ok). Exit 0 clean · 1 problems · 2 unreadable. Coverage: make test-cli-table. What is genuinely not a bob command — the claude shell wrapper, the cdr alias — stays hand-written outside the block in each host, so the generator never has to model a non-command.

Run it directly with make check-cli-table (it is part of make check).

check-workflow-runs-on

Holds every GitHub Actions workflow BoB ships or scaffolds to the CI locus seam (#1729). A project’s locus — farm or hosted — is one declared setting projected into the repo variable CI_RUNNER_LABEL; a workflow that does not read it sits outside the locus, and on a billing-blocked repo a hardwired ubuntu-latest job fails in seconds with no steps while the gate on the seam stays green. That is how the release cut and the docs deploy silently died in August 2026 (#1782); this gate is what stops a workflow added later from re-creating it.

What it scans: every tracked *.yml / *.yaml with a jobs: block — .github/workflows/ and every template under templates/ (tests/ excluded, comment lines skipped). Each job’s runs-on is graded:

Problem	Meaning	Fix
hardwired	a runner that is neither the seam nor pinned	runs-on: ${{ vars.CI_RUNNER_LABEL || 'ubuntu-latest' }}, or # pinned: <reason> directly above
pin-no-reason	# pinned: with nothing after it	write why the job cannot run on the farm
seam-drift	a vars.CI_RUNNER_LABEL expression that is not the canonical seam (another default, no default)	use the exact seam — a non-farm need is a pin, never a second seam

A pinned job passes and is reported on every run, so a standing hosted dependency stays visible. --json emits workflow-runs-on-check/1 (seam, scanned, jobs, on_seam, pinned[], workflows[], problems[], ok). Exit 0 clean · 1 problems · 2 unreadable. Coverage: make test-workflow-runs-on (fixtures for every verdict) and tests/test-fleet-runner.sh (the six shipped files by name). The full convention — seam, pin form, toolchain assertions, GitHub-held secrets — is Workflow Authoring.

Run it directly with make check-workflow-runs-on (it is part of make check).

check-doc-counts

Keeps every component count a doc states equal to one counter (#2087). Two mechanisms used to compete — the reference generator harvested the surfaces while doc-keeper/sync.js recounted directories into prose — and README, CLAUDE.md and AGENTS.md still disagreed. Now the generator counts, one file holds the result, and the docs receive it.

The pipeline — one counter, one file, several destinations:

1. countsJson in scripts/docs-site/lib/gen-reference.js counts universal and registry skills, commands and agents (the same harvest the reference pages list), runbooks (excluding the section index), hook scripts with their wired/unwired split (read through hook-drift.js, the reader behind the unwired-hooks block in hooks-reference.md), and bin/ entrypoints.
2. gen-reference.js --counts --write commits the result as docs/reference/generated/counts.json (bob-counts/1, no timestamp). It is the one file in that directory tracked by git; a docs-site build rewrites it identically.
3. sync.js --blocks-only --apply rewrites every <!-- count:<key> -->N<!-- /count --> marker in the declared hosts from that file. sync reads the file; it counts nothing.
4. check-doc-counts gates both hops.

The host list (COUNT_HOSTS) and the marker shape (COUNT_MARKER_RE) live once, in gen-reference.js: README.md, CLAUDE.md and the handbook. AGENTS.md and handbook §15.1 carry no architecture tree of their own — they link to CLAUDE.md’s, so the one copy left is the generated one (#2089).

What it fails on:

Problem	Meaning	Fix
counts-stale	counts.json is not what a fresh count produces — a surface was added or removed without regenerating; the message names each changed key	make check-doc-counts ARGS=--write
counts-missing	counts.json is absent or unreadable	make check-doc-counts ARGS=--write
marker-drift	a marker’s value differs from counts.json (a hand edit, or a regeneration that was not spliced)	make check-doc-counts ARGS=--write
unknown-key	a marker names a key counts.json does not carry	use an existing key, or add the count to countsJson
markers-missing	a declared host carries no marker	mark its counts, or drop it from COUNT_HOSTS
host-missing	a declared host file does not exist	fix COUNT_HOSTS
render-failed	the generator could not count (exit 2)	fix the error

--json emits doc-counts-check/1 (source, counts, per-host marker verdicts, problems, ok). Exit 0 clean · 1 problems · 2 unreadable. Numbers in prose that are not marked are still reported by the auditor’s component-count-* rules, now as fix_kind: judgment — sync no longer rewrites them. Coverage: make test-doc-counts.

Run it directly with make check-doc-counts (it is part of make check, and a prerequisite of make docs-check).

---

Repairing drift (prune & repair-fleet)

check-fleet reports drift; these repair it (#303).

bob provision --prune — per-project

Removes dangling symlinks under a project’s .claude/ that point into BoB (*/.claude/*, $BOB_SOURCE, $BOB_HOME). It is dry-run by default; pass --yes (or --apply) to delete. Each candidate is classified:

• removed-tooling — target under this machine’s BoB home/source but gone (e.g. the retired PM-file / sprint / requirements / risk items).
• cross-machine — an absolute target under a different home’s .claude (e.g. a repo provisioned on another box: /home/farm/.claude/...).
• other — any other dangling BoB symlink.
• now-universal — a commands/, agents/, hooks/ or skills/ link whose item is live in the universal set of this machine’s BOB_HOME — promoted from the registry (#1872), or a link written on another machine. This is moved tooling, not removed tooling: the link is retargeted to the deployed universal copy (reported as would retarget / [retargeted]), never deleted — deleting it would silently drop the command until someone re-ran bob init (#2009).
• declared-registry — a demoted item (universal → registry/) that the project’s manifest or session overlay declares: retargeted to the deployed registry copy (#2314).
• undeclared-registry — a demoted item the project does not declare: pruned, with the hint provision it with: bob provision add <type> <name> (#2314).
• retired — an item the retirement record (scripts/lib/retired-items.json) marks removed or superseded-by: pruned, naming the disposition, the replacement and the retiring issue. A moved-* record whose copy is not deployed on this machine is [pending] (run deploy.sh), untouched.

Two forms (#2314). The explicit repair above classifies an unknown BoB dangler as removed-tooling / cross-machine / other and removes it on --yes — the operator asked. The convergence form, --converge, is what bob provision refresh runs at the end of every refresh and what the bob ready links step runs before it grades: it acts only on what is known (a live universal or declared registry item, a retired item) and reports an unknown dangling link as [unknown] — left in place, never silently deleted, with the rm / add-a-record remedy named. Both forms are idempotent: a second run acts on nothing.

Safety guarantees (covered by make test-prune): prune never removes a resolving symlink, a real (non-symlink) file, a foreign symlink (one whose target is not a BoB path), or a link whose item is live in BOB_HOME. It is idempotent.

bob provision --prune                    # dry run — preview what would be removed, by class
bob provision --prune --yes              # apply (operator repair: unknown BoB danglers removed too)
bob provision --prune --converge         # what refresh / bob ready would do — unknown reported only
bob provision --prune --converge --yes   # apply the convergence form by hand

make repair-fleet — every project

Walks projects.json and, per live project, runs bob provision --prune then (on apply) bob provision --refresh to re-link currently-declared items. Dry-run unless ARGS=--yes, and best-effort — a failure in one project warns and the run continues.

make repair-fleet            # dry run across the whole fleet (prune preview)
make repair-fleet ARGS=--yes # apply: prune + refresh every registered project
make check-fleet             # confirm remaining drift afterwards

The prune→refresh order matters: prune clears stale/cross-machine links first, then refresh recreates the items the manifest still declares.

---

JSON Schemas

Located in schemas/. Each defines the exact structure a config file must follow.

manifest.schema.json

For provisions/*.json files:

{
  "_meta": {
    "project": "string (required)",
    "path": "string | null",
    "stack": ["string"]
  },
  "skills": ["string (unique)"],
  "commands": ["string (unique)"],
  "agents": ["string (unique)"]
}

projects.schema.json

For projects.json:

{
  "_meta": {
    "description": "string (required)",
    "updated": "YYYY-MM-DD"
  },
  "projects": [{
    "name": "string (required)",
    "path": "string (required)",
    "group": "string (required)",
    "active": "boolean (required)"
  }]
}

settings.schema.json

For settings.json:

• permissions.allow / permissions.deny — string arrays
• hooks — keyed by event name (SessionStart, PreToolUse, PostToolUse, Stop, Notification, SubagentStop)
• Each hook entry has type: "command", command: string, optional timeout: integer
• statusLine — type + command
• enabledPlugins — map of string: boolean

---

File Layout

~/.claude/
├── Makefile                          # Entry point — run `make help`
├── schemas/
│   ├── manifest.schema.json          # Provisions manifest schema
│   ├── projects.schema.json          # Project registry schema
│   └── settings.schema.json          # Settings schema
├── scripts/checks/
│   ├── check-deps.sh                 # Dependency verification
│   ├── check-hooks.sh                # Hook script verification
│   ├── check-manifest-meta.sh        # Manifest identity (_meta.project / path, #1869)
│   ├── check-provisions.sh           # Manifest-vs-reality verification
│   ├── check-schemas.js              # JSON Schema validation runner
│   └── schema-validator.js           # Shared validator module (zero deps)
├── scripts/run-tests.sh              # The one test runner — globs tests/test-*.{sh,js} (#2043)
├── tests/
│   ├── test-*.sh, test-*.js          # Every file here IS a test; no Makefile registration
│   ├── test-checks.sh                # Integration tests for the check scripts
│   └── test-schemas.js               # Schema validator unit tests
└── .github/workflows/
    └── ci.yml                        # GitHub Actions — runs on push/PR

---

CI

.github/workflows/ci.yml runs on pushes and PRs to master:

1. Checkout repo
2. Setup Node.js 22 (the repo is ESM-first and relies on Node’s ES-module auto-detection)
3. Verify the runner toolchain (jq, and since #2043 tmux — the cdfork / segment-spawn / warp-reap suites drive a real tmux server; a hosted runner installs it, a self-hosted one must carry it) and install a pinned PyYAML for the versioning tests
4. make check-deps check-schemas check-hooks check-intake (skip symlinks/provisions — no projects on a CI runner)
5. make test — every tests/test-* file through scripts/run-tests.sh; each test runs exactly once, there is no rerun-on-failure in CI (a flake fails the run and is re-run by a human or by the warp-drive quarantine, #1979)
6. make docs-check

CI skips check-symlinks and check-provisions because they validate local provisioning state — the cdi-created symlinks in a working copy — which doesn’t exist in a bare CI checkout (.claude/ symlinks are gitignored). Since #294 these checks default to the current repo (so make check in any working copy is no longer failed by drift in other registered projects); the cross-project sweep moved behind --all / make check-fleet. They remain local/dev gates by design.

Two lanes under one required check (#1693, #3717)

ci.yml runs on every pull request and every push to master, with no path filter, and its last job, ci-passed, is the one check master requires. GitHub leaves a required check that was skipped pending, so the required check must report on every change. Inside the workflow the work is split into two lanes. A lanes job asks scripts/ci/lanes.js which lanes the change needs, and a lane that does not apply is skipped:

Lane	Jobs	Applies when the change touches	Runs
Code	check-and-test, skillspector	anything except only docs/**, runbooks/**, README.md, CLAUDE.md (DOCS_ONLY)	the checks, the full test suite, docs-check
Docs	docs (docs-deploy.yml, called as a reusable workflow)	those doc paths, the docs-site tooling, or the CI wiring (DOCS_LANE)	site build, flat-link verify, docs-check, PR previews, the master deploy

ci-passed runs lanes.js verdict. It passes only when the lanes job succeeded, every job of a lane that applies succeeded, and the jobs of a lane that does not apply were skipped. When the changed files cannot be read (a new branch, a force push), both lanes run.

A docs-only change therefore runs the docs lane alone rather than dragging the full test suite behind it, and make docs-check is what makes that safe: the doc naming, filename-convention, and anchor gates are prerequisites of that target, not separate CI steps, so they fire from either lane. Both lanes also carry a per-ref concurrency group with cancel-in-progress, so a burst of pushes bills one run rather than one per commit.

Two consequences worth knowing:

• The two path lists live in one file. DOCS_ONLY and DOCS_LANE are in scripts/ci/lanes.js. Every DOCS_ONLY path must be in DOCS_LANE, or a docs-only change is gated by nothing. make test-ci-lane-split asserts that, drives lanes.js on real commits, and checks the workflow structure.
• Require ci-passed, never a lane job. .github/required-checks.json and the manifest’s _meta.branch_protection name it. In bigbrain, docs-deploy.yml has only workflow_call, so it never runs twice. The template other projects get keeps its own triggers.
• A few test targets read the real docs tree, and those do not run on a docs-only push — they run on the next push that touches source. Nothing ships in between: auto-release.yml no longer runs per push (#2045) — a stable release is cut weekly or on dispatch, and a nightly tag only lands when ci.yml’s latest run for that SHA is green. The site-breaking risks (build failure, orphan, drift) are all covered by the docs lane, which does run.

The release bot’s own commit is excluded from both lanes: its subject carries [skip ci] (see Versioning), since a version bump plus a regenerated changelog gives either lane nothing to decide.

---

Common Workflows

After cloning Bob on a new machine

cd ~/projects/bigbrain    # BOB_SOURCE
make check-deps           # verify tools are installed
make check-schemas        # verify configs are valid
make check-host           # this machine's BOB_HOME matches source (or: bob doctor)

After changing settings.json or a hook

make check-hooks check-schemas

After changing a provision manifest

make check-schemas check-provisions

After adding a new project

# Add to projects.json, then:
make check-schemas      # validates projects.json structure
make check-symlinks     # checks new project's symlinks
make check-provisions   # checks new project matches manifest

Before pushing changes to Bob

make ci     # runs everything

Investigating a project that feels broken

make check-symlinks     # find dangling symlinks
make check-provisions   # find manifest mismatches
cdprov --refresh /path/to/project   # fix it

---

Rendering the scenarios page (two-render split, #844)

The narrative workday scenarios page carries an introspection layer — the “friction margin” that is honest about where BoB’s seams are clumsy. That honesty makes a poor outsider-facing brochure, so the friction content is wrapped in HTML-comment markers and stripped or kept at render time, from a single source (no forked copies to keep in sync):

<!-- friction:TYPE -->   ...introspection-only content...   <!-- /friction -->

TYPE is one of pain-point / automation-opportunity / candidate-feature / section. Produce either render with scripts/render-scenarios.js:

# Outsider-facing: friction blocks removed, clean narrative.
node scripts/render-scenarios.js --mode showcase       [--out showcase.md]

# Maintainer-facing: friction content kept (markers stripped).
node scripts/render-scenarios.js --mode introspection  [--out introspection.md]

The transform is pure and deterministic — same input always yields the same output, and re-running it is idempotent. HTML comments (the friction fences, the authoring note, the drift-allow marker) are internal metadata and are stripped from both renders; only the friction content differs between them. The test-render-scenarios target proves showcase-strips / introspection-keeps / idempotent.

The tutorial render (#2076)

The third render is a tutorial, not a variant of the page. A source page marks each command block the tutorial needs:

<!-- tutorial:ID -->   ```bash … ```   <!-- /tutorial -->

and a prose frame such as templates/tutorials/bob-in-one-day.frame.md places each one with a {{tutorial:ID}} slot.

The tutorial registry (#2412). Every generated tutorial is declared once in templates/tutorials/registry.json as a (source page, frame, output) triple. make check-schemas validates the file against schemas/tutorial-registry.schema.json:

{ "page": "docs/explanation/workday-scenarios.md",
  "frame": "templates/tutorials/bob-in-one-day.frame.md",
  "output": "docs/tutorials/bob-in-one-day.md" }

A source page can be any explanation page. Block IDs are unique per page, so two pages can both use setup. Render every registered tutorial, or one of them:

node scripts/render-scenarios.js --tutorial                                          # every registry entry → its output
node scripts/render-scenarios.js --tutorial --out docs/tutorials/bob-in-one-day.md   # one entry (its page + frame)

With --page / --frame and no matching registry entry, the render is an authoring preview of that pair: it prints to stdout unless --out is given, and it has no “Next tutorial” line. The exit codes are unchanged: 0 rendered, 1 render error, 2 usage.

Commands therefore live in exactly one place, the page, and the prose in exactly one other, the frame. A slot with no block, a block with no slot, or a duplicated ID exits 1 and writes nothing. Every such error names the page, so a command removed from the page cannot survive in a tutorial. The frames live under templates/, outside the doc-keeper scan, because they are sources rather than published docs. check-scenarios-drift fails when any committed tutorial is stale.

Learning path and “Next tutorial”. A frame’s frontmatter carries the tutorial’s order: and prerequisites:, which are the fields the documentation standard defines for every tutorial. Each render ends with a **Next tutorial:** line pointing at the next step on the learning path, or saying the tutorial is the last stop. The line comes from learningPath() in scripts/docs-site/lib/section-index.js, the same function that orders docs/tutorials/index.md, so the pointer cannot drift from the index. The frame’s frontmatter takes precedence over the committed copy, so a render after an order: change, or the first render of a new tutorial, is right in one pass.

Adding a generated tutorial:

1. Mark the source page’s command blocks with <!-- tutorial:ID --> fences.
2. Write templates/tutorials/<slug>.frame.md with title, description, order and prerequisites in its frontmatter and one slot per block.
3. Add the (page, frame, output) entry to templates/tutorials/registry.json.
4. Run node scripts/render-scenarios.js --tutorial, then make docs-sync so the tutorials index picks up the new row.

Design Principles

• Zero dependencies — only node, jq, git, make (all pre-installed or one brew install)
• Every check exits 0 (pass) or 1 (fail) — no ambiguity
• Actionable output — every failure includes a fix instruction
• Idempotent — safe to run any target as many times as you want
• Fast — full make check completes in under 5 seconds
• Composable — run individual targets or combine them: make check-deps check-hooks