Verification & Testing System

TL;DR. make check validates the system (deps, schemas, symlinks, hooks, provisions); make test runs the suites; make ci runs both — what GitHub Actions runs. One command to verify everything.

Bob’s infrastructure-as-code verification layer. One command to validate the entire system.

make check    # verify everything
make test     # run all tests
make ci       # both (what GitHub Actions runs)

Quick Start

cd ~/projects/bigbrain    # or wherever BOB_SOURCE is
make help                 # see all targets
make check-deps           # am I set up correctly?
make check                # is everything healthy?
make test                 # do all tests pass?

Targets

Verification (make check)

make check runs all five checks in order. Stops at first failure.

Target	What It Checks	Speed
check-deps	node >= 18, jq, git, make installed	instant
check-schemas	All JSON configs valid against schemas	instant
check-symlinks	Symlinks in the current repo resolve (--all / make check-fleet for every project)	~1s
check-hooks	All hooks in settings.json exist + executable	instant
check-provisions	The current repo’s symlinks match its manifest (--all for the fleet)	~1s
check-fleet	check-symlinks + check-provisions across all registered projects (#294)	~2s

Testing (make test)

Target	What It Tests	Count
test-schemas	Schema validator unit tests (type, enum, pattern, etc.)	34
test-warp-drive	Warp-drive state machine transitions	99
test-checks	Integration tests for all check scripts	13

CI (make ci)

Runs check then test. This is the target GitHub Actions calls on every push and PR to master.

---

Check Details

check-deps

Verifies the four required tools are installed:

  PASS  node 25.5.0 (>= 18)
  PASS  jq 1.7.1
  PASS  git 2.48.1
  PASS  make (GNU Make 3.81)

If something fails, the output includes a fix command:

  FAIL  node 16.20.0 (need >= 18)
        Fix: Install Node.js 18+ via nvm or brew

check-schemas

Validates every JSON configuration file against its JSON Schema definition:

File	Schema
projects.json	schemas/projects.schema.json
settings.json	schemas/settings.schema.json
provisions/*.json (17 files)	schemas/manifest.schema.json

If something fails, the output shows the exact path and error:

  FAIL  provisions/bodmail.json
        $._meta.project: expected type string, got number
        $.skills[3]: duplicate item "webapp-testing"

To fix: Edit the JSON file to match the schema. The error path ($._meta.project) tells you exactly where the problem is.

check-symlinks

Scans every project listed in projects.json, finds all symlinks in each project’s .claude/ directory, and verifies each one resolves to a real file.

  FAIL  /Users/you/Sites/myproject/.claude/commands/sprint.md -> /Users/you/.claude/registry/commands/sprint.md (dangling)

To fix dangling symlinks:

# Option 1: Re-provision the project (recommended)
cdprov --refresh /path/to/project

# Option 2: Remove the stale symlink manually
rm /path/to/.claude/commands/sprint.md

Projects that don’t exist on disk or lack a .claude/ directory are silently skipped.

check-hooks

Reads settings.json, extracts every hook command path (from SessionStart, PreToolUse, PostToolUse, Stop), and verifies:

1. The script file exists
2. The script is executable (chmod +x)

Also checks the statusLine.command if configured.

  PASS  /Users/you/.claude/hooks/temporal-context.sh
  FAIL  /Users/you/.claude/hooks/missing-hook.sh
        Fix: File not found — remove from settings.json or create the script

To fix:

# If the hook should exist:
chmod +x ~/.claude/hooks/the-hook.sh

# If the hook was removed:
# Edit settings.json and remove the hook entry

check-provisions

For each project in projects.json, loads its manifest (provisions/<project>.json or provisions/_default.json) and verifies that every declared skill, command, and agent is:

1. Present in the registry (registry/skills/, registry/commands/, registry/agents/)
2. Symlinked into the project’s .claude/ directory
3. Not dangling

  PASS  bodmail (bodmail.json)
  FAIL  myproject: commands/sprint declared in _default.json but not symlinked
  FAIL  myproject: agents/code-expert symlink is dangling

Failure types and fixes:

Failure	Meaning	Fix
declared but missing from registry	Manifest references something that doesn’t exist	Remove from manifest or add to registry
declared but not symlinked	Item exists in registry but wasn’t linked to project	Run cdprov --refresh /path/to/project
symlink is dangling	Symlink exists but target was moved/deleted	cdprov --refresh if the target still exists; cdprov --prune if it’s gone/cross-machine (see below)

---

Repairing drift (prune & repair-fleet)

check-fleet reports drift; these repair it (#303).

cdprov --prune — per-project

Removes dangling symlinks under a project’s .claude/ that point into BoB (*/.claude/*, $BOB_SOURCE, $BOB_HOME). It is dry-run by default; pass --yes (or --apply) to delete. Each candidate is classified:

• removed-tooling — target under this machine’s BoB home/source but gone (e.g. the retired PM-file / sprint / requirements / risk items).
• cross-machine — an absolute target under a different home’s .claude (e.g. a repo provisioned on another box: /home/farm/.claude/...).
• other — any other dangling BoB symlink.

Safety guarantees (covered by make test-prune): prune never removes a resolving symlink, a real (non-symlink) file, or a foreign symlink (one whose target is not a BoB path). It is idempotent.

cdprov --prune            # dry run — preview what would be removed, by class
cdprov --prune --yes      # apply

make repair-fleet — every project

Walks projects.json and, per live project, runs cdprov --prune then (on apply) cdprov --refresh to re-link currently-declared items. Dry-run unless ARGS=--yes, and best-effort — a failure in one project warns and the run continues.

make repair-fleet            # dry run across the whole fleet (prune preview)
make repair-fleet ARGS=--yes # apply: prune + refresh every registered project
make check-fleet             # confirm remaining drift afterwards

The prune→refresh order matters: prune clears stale/cross-machine links first, then refresh recreates the items the manifest still declares.

---

JSON Schemas

Located in schemas/. Each defines the exact structure a config file must follow.

manifest.schema.json

For provisions/*.json files:

{
  "_meta": {
    "project": "string (required)",
    "path": "string | null",
    "stack": ["string"]
  },
  "skills": ["string (unique)"],
  "commands": ["string (unique)"],
  "agents": ["string (unique)"]
}

projects.schema.json

For projects.json:

{
  "_meta": {
    "description": "string (required)",
    "updated": "YYYY-MM-DD"
  },
  "projects": [{
    "name": "string (required)",
    "path": "string (required)",
    "group": "string (required)",
    "active": "boolean (required)"
  }]
}

settings.schema.json

For settings.json:

• permissions.allow / permissions.deny — string arrays
• hooks — keyed by event name (SessionStart, PreToolUse, PostToolUse, Stop, Notification, SubagentStop)
• Each hook entry has type: "command", command: string, optional timeout: integer
• statusLine — type + command
• enabledPlugins — map of string: boolean

---

File Layout

~/.claude/
├── Makefile                          # Entry point — run `make help`
├── schemas/
│   ├── manifest.schema.json          # Provisions manifest schema
│   ├── projects.schema.json          # Project registry schema
│   └── settings.schema.json          # Settings schema
├── scripts/checks/
│   ├── check-deps.sh                 # Dependency verification
│   ├── check-hooks.sh                # Hook script verification
│   ├── check-provisions.sh           # Manifest-vs-reality verification
│   ├── check-schemas.js              # JSON Schema validation runner
│   └── schema-validator.js           # Shared validator module (zero deps)
├── tests/
│   ├── test-checks.sh                # Integration tests (13 tests)
│   └── test-schemas.js               # Schema validator unit tests (34 tests)
└── .github/workflows/
    └── ci.yml                        # GitHub Actions — runs on push/PR

---

CI

GitHub Actions runs on every push and PR to master:

1. Checkout repo
2. Setup Node.js 18
3. Install jq
4. make check-deps check-schemas check-hooks (skip symlinks/provisions — no projects on CI runner)
5. make test

CI skips check-symlinks and check-provisions because they validate local provisioning state — the cdi-created symlinks in a working copy — which doesn’t exist in a bare CI checkout (.claude/ symlinks are gitignored). Since #294 these checks default to the current repo (so make check in any working copy is no longer failed by drift in other registered projects); the cross-project sweep moved behind --all / make check-fleet. They remain local/dev gates by design.

---

Common Workflows

After cloning Bob on a new machine

cd ~/projects/bigbrain    # BOB_SOURCE
make check-deps           # verify tools are installed
make check-schemas        # verify configs are valid
make check-hooks          # verify hooks are wired up

After changing settings.json or a hook

make check-hooks check-schemas

After changing a provision manifest

make check-schemas check-provisions

After adding a new project

# Add to projects.json, then:
make check-schemas      # validates projects.json structure
make check-symlinks     # checks new project's symlinks
make check-provisions   # checks new project matches manifest

Before pushing changes to Bob

make ci     # runs everything

Investigating a project that feels broken

make check-symlinks     # find dangling symlinks
make check-provisions   # find manifest mismatches
cdprov --refresh /path/to/project   # fix it

---

Design Principles

• Zero dependencies — only node, jq, git, make (all pre-installed or one brew install)
• Every check exits 0 (pass) or 1 (fail) — no ambiguity
• Actionable output — every failure includes a fix instruction
• Idempotent — safe to run any target as many times as you want
• Fast — full make check completes in under 5 seconds
• Composable — run individual targets or combine them: make check-deps check-hooks