Skip to content
BoB — Big ol' Brain

Dispositions

Dispositions

Section titled “Dispositions”

registry/dispositions.json records what Bob University (#2898) decided about each BoB item and why. The record is declared state about product items, so it lives in BOB_SOURCE next to them. make check-dispositions, which is part of make check, keeps it honest. The schema is schemas/dispositions.schema.json.

Disposition Meaning
certified An exam shows the item improves outcomes on the named model.
needs-work Worth keeping, but not yet earning its place.
not-applicable The right item for a stack or portfolio BoB is not shipping. This is not a quality verdict.
retire The item should stop existing. It is completed by an entry in scripts/lib/retired-items.json in the same change. It is never a second retirement mechanism.

An item with no record is ungraded. That is an honest state: it is counted and reported, never failed.

A record

Section titled “A record”
{
  "type": "skills",
  "name": "tdd-flow",
  "disposition": "certified",
  "date": "2026-09-23",
  "evidence": {
    "applicability": "unscoped",
    "usage": { "events": 14, "machines": ["farm-01", "macbook-air"], "base_events": 412 },
    "exam": {
      "schema": "skill-eval/1",
      "ref": "eval-2026-09-23-tdd-flow",
      "model": "claude-opus-5-5",
      "date": "2026-09-22",
      "with": 0.82, "without": 0.61, "delta": 0.21,
      "verdict": "improved"
    }
  },
  "rationale": "Used weekly across the fleet; the with/without exam shows a clear gain.",
  "model": "claude-opus-5-5",
  "baseline": "registry/skills/tdd-flow/evals/baseline.json"
}
Field Required Meaning
type yes skills, commands, agents or runbooks, the vocabulary of registry/kits.json and retired-items.json. Two non-registry kinds that bob tour also grades are allowed too (#2902): cli, a bob noun or its legacy alias in scripts/bob/commands.json, and scripts, an engine under scripts/<name>/ or scripts/<name>.js|.sh. They are recorded but never counted in the ungraded backlog, and cannot be retire until retired-items.json records that kind.
name yes The item name as a manifest names it: the skill directory, or the basename without .md. type + name is the key.
disposition yes One of the four above.
date yes When it was decided, YYYY-MM-DD.
evidence.applicability no The item’s bucket from bob tour --applicability at decision time.
evidence.usage no events (fleet-wide invocations, #2899), machines, and base_events (the evidence base those counts rest on).
evidence.exam certified A reference to an exam result from #2390’s claude plugin eval suite: schema (skill-eval/1), ref (the run id or result path; transcripts stay out of git), model, date (the day the exam ran, YYYY-MM-DD; required on a certified record), with, without, delta, and verdict (improved / no-signal / regressed).
rationale yes Why, in one or two sentences a reviewer can check against the evidence (at least 10 characters).
model certified only The model the certification holds for.
baseline certified only The eval baseline it was graded against (#2390’s baseline.json, the one score of record).
retired_by retire only, optional The issue or commit. When present, it must agree with the retired-items.json record.
history no The records this one replaced, newest first. Each entry keeps disposition, date, evidence, rationale and, where they were set, model, baseline and retired_by. bob u dispose fills it; nothing is overwritten.

Rules

Section titled “Rules”

scripts/checks/check-dispositions.js fails make check on each of these:

Rule Fails when
schema The file does not validate against the schema.
duplicate Two records share type + name.
unknown-item A record other than retire names an item that exists in neither registry/<type>/ nor the universal <type>/. For cli, the name is in no commands.json row; for scripts, there is nothing under scripts/.
certified-no-model A certified record has no model or no baseline.
certified-no-exam A certified record has no evidence.exam. Applicability and usage alone cannot certify.
certified-no-signal The exam’s verdict is not improved, or its delta (recorded, or with − without) is not positive. A no-signal exam can never be recorded as certified.
certified-model-mismatch The exam ran against a different model than the one the certification names.
certified-no-exam-date A certified record’s exam has no date. A certification holds for the model it was examined against, as of the day it was examined.
exam-delta-mismatch A recorded delta disagrees with with − without.
history-order A history entry is newer than the record that replaced it, or the history is not newest first.
retire-unrecorded A retire record has no retired-items.json entry with the same type + name.
retire-mismatch retired_by disagrees with that entry’s.
misplaced-field model or baseline appears on a non-certified record, or retired_by on a non-retire one.

A retire record is matched against the retirement record, not the tree. The retired item is gone by design, and check-retired-items already fails a record whose path is still present. retired-items.json does not yet know runbooks, so a runbook cannot be dispositioned retire until it does.

The first cull

Section titled “The first cull”

The first graduating class (#2902) was the 89 items bob tour graded unknown or parked in bigbrain on 2026-09-23. The fixed baseline is posted on #2902. The fleet evidence base then was 60 events from 1 of 3 machines, and 88 of the 89 items had none. Insufficient evidence is not grounds for retire, and certification needs an exam, so every one of them is needs-work. Each rationale ends in the exam question that would decide it. A later pass revisits them once every fleet machine publishes its usage.

Running it

Section titled “Running it”
Terminal window
make check-dispositions                                   # PASS — … N/M item(s) ungraded (counted, not failed).
node scripts/checks/check-dispositions.js --json          # bob-dispositions-check/1
node scripts/checks/check-dispositions.js --root <dir>    # grade another checkout

The --json document is {contract, ok, error, violations: [{rule, type, name, message}], coverage: {counts, items, ungraded, by_type: {<type>: {items, graded, ungraded, ungraded_names}}}}. The ungraded count is the backlog Bob University works down.

Exit Meaning
0 Clean. Ungraded items do not affect this.
1 At least one violation.
2 Unreadable record, schema or retirement file, or a usage error.

Working the record with bob u

Section titled “Working the record with bob u”

bob u (alias bob learn, #2903) is the command that reads and writes this record. The how-to walks through it; every flag is in the CLI reference.

Command Document (--json) What it does
bob u status bob-u-status/1 Shows counts per disposition, each undisposed item with its applicability bucket and fleet usage, the certifications due for re-examination, and this gate’s violations. Exits 1 when the gate fails.
bob u exam <item> bob-u-exam/1 Runs the item’s #2390 suite under a kit overlay and prints the skill-eval/1 result with the disposition it supports: certified only for an improved verdict with a positive delta.
bob u dispose <item> <disposition> --reason "…" bob-u-dispose/1 Builds the record from the evidence and runs this gate’s rules on it. It refuses a record that would fail (exit 1) and writes only with --apply. retire needs --retired-by and prints the retired-items.json draft.

Re-examination

Section titled “Re-examination”

A certification holds for the model the item was examined against. It is not tied to a calendar date, so what re-opens it is a model change (#2904). bob u status resolves the model each certified item runs on now:

Item Runs on runs_on_source
An agent with model: <alias or id> that model agent-frontmatter
An agent with model: inherit, or a skill, command or runbook the session default: $ANTHROPIC_MODEL, else the model key of BOB_HOME/settings.json (a [1m]-style suffix is dropped) session-default:ANTHROPIC_MODEL / session-default:settings.json
An agent with no model:, or no session default declared nothing unresolved, never guessed

An alias (opus, sonnet, haiku) is translated to its API id through the model-assignment policy table in registry/skills/llm-tiering/SKILL.md (#2680), so opus compares as claude-opus-5. The resolved model is then compared with the certified one:

State reason When
current — The same id, or the same id at the precision the policy names: claude-opus-5 covers a certification on claude-opus-5-5.
current family-only An alias the policy does not translate, in the certified family.
re-examine model-change The item now runs on another family, a model it was not certified against.
re-examine model-replaced The same family, but a new id replaced the certified one (claude-opus-5-1 → claude-opus-5-5).
unresolved — Nothing to compare with (above).

Each certification in the bob-u-status/1 document carries certified_model, examined (the exam date), runs_on, runs_on_id, runs_on_source, state, reason and a message naming both models. reexamine is the re-examine subset. The human report prints one line per item:

RE-EXAMINE (1)
  agents:qa-strategist  model-change: certified on claude-opus-5-5 (examined 2026-09-22), now runs on haiku (claude-haiku-4-5) — a model it was not certified against [agent-frontmatter]

The same list reaches the attended rituals: the usage-gaps evidence collector reads bob u status --json and summarises each item as re-examine: type:name (certified → runs-on) (Rituals).

To re-examine, run bob u exam <item> --model <runs-on model> and record the result with bob u dispose <item> <disposition> --exam <result.json>. The new record names the new model and exam date. The certification it replaces moves into history, so both exams stay in the record.

Section titled “Related”