Skip to content

Runbook: Local Delivery Fallback

Purpose: Deliver without GitHub Actions — run the CI gate, cut the release, and publish the docs site from a workstation, using the same entry points the workflows use When to use: Actions is unavailable (billing cap hit, spending limit, outage — e.g. #1648) and delivery must not stall Prerequisites: A clean checkout of the repo on its main branch; node >= 22, npm, jq, make, and gh authenticated (gh auth status); Cloudflare credentials for the docs deploy only Estimated time: 10–20 minutes for all three

The three Actions workflows and their local equivalents. Each local path runs the same underlying entry point as the workflow — there is no parallel logic to drift:

Workflow What it does Local equivalent
ci.yml checks + tests + doc-drift gate make ci (step 1)
auto-release.yml stable release: version bump, CHANGELOG, tag, GitHub Release — weekly as promote --auto from the newest nightly (#2057), or stable / promote on dispatch (#2045) bob release promote [--auto] / bob release --channel stable (step 2)
— (farm bob tick, #2054) nightly: lightweight nightly-<date> tag on a green, moved master bob-release --channel nightly (step 2)
docs-deploy.yml build + verify + publish the docs site npm run docs:deploy (step 3)

Run them in this order: gate first, then release, then docs.

make ci is exactly what Actions runs (checks + full test suite + docs gates):

Terminal window
BOB_ALLOW_NONPROJECT=1 make ci
  • BOB_ALLOW_NONPROJECT=1 is required: test-prune builds fixtures in temp dirs that the #707 project guard otherwise refuses (CI sets the same flag).
  • The full suite takes several minutes; test-warp-drive (the state-machine suite) alone runs 2–3 min.
  • make test is one runner over every tests/test-* file (#2043): a failing test is listed at the end with the last 20 lines of its output; re-run one with make test-<name> (the filename under tests/), and raise the per-test limit with TEST_TIMEOUT=<seconds> if a slow machine trips the 300s default.
  • Do not trust a piped exit code (make ci | tail reports the pipe’s status, not make’s) — run it bare, or capture to a file and check $? directly.

Verify: the run ends with All tests passed. and exit code 0. A red suite blocks the release cut — fix first, exactly as a red CI check would block a merge.

Known flaky tests (pass on isolated re-run — re-run before treating as real): test-warp-drive (execSync timeout under load), test-checks check-fleet grep (#1621), test-dev-up-docs (port race under suite load).

Terminal window
bob-release --channel stable # or: make release-local (stable is the default channel)
bob-release --channel nightly # or: make release-nightly
bob release promote # stable FROM the newest nightly (#2057); --auto adds the 24 h gate

promote cuts stable at the newest nightly-* tag’s SHA. It refuses while an open p1-critical + harness-defect issue exists (named), and with --auto while the nightly may be younger than 24 h (counted from the end of its UTC day). If master moved past the nightly, the release commit is built on the nightly and merged into master ([skip ci]); an already-promoted nightly is a no-op. Gates and the merge route: versioning — Promoting a nightly. Every new stable GitHub Release is announced once with bob notify (the CHANGELOG section and bob update --channel stable).

Both channels are one engine, scripts/release/release.sh — the same file auto-release.yml runs (#2045), so there is no parallel logic to drift.

stable is auto-release.yml verbatim: npm run release (commit-and-tag-version) bumps from Conventional Commits, rewrites CHANGELOG.md, commits chore(release): X.Y.Z, tags vX.Y.Z, pushes commit + tag atomically (3-attempt rebase retry), and publishes a GitHub Release from the new changelog section. Since #2045 it is not cut on every push, and since #3171 there is no weekly schedule: a stable release is cut when the operator decides (dispatch, bob-release, or bob release promote).

nightly puts a lightweight nightly-YYYY-MM-DD tag on HEAD — no bump, no commit, no CHANGELOG entry, no GitHub Release — and only when master moved since the last nightly-* tag and the latest ci.yml run for that SHA is green (gh run list). It is idempotent for the day: a second run, or a run on an unmoved master, is a stated no-op. A HEAD that ci.yml legitimately skipped (a [skip ci] release commit, a docs-only change) is judged by its nearest ancestor that did run CI; the tag still lands on HEAD. A [skip ci] commit that changes anything but release files is refused, not judged by an ancestor (#3630). The farm’s bob tick invokes this channel (#2054).

  • Preview first with bob-release --channel <c> --dry-run; force a stable bump type with --release-as patch|minor|major (parity with the workflow’s manual dispatch).
  • It refuses on: tracked working-tree changes (stash first — untracked files only warn), a branch other than main, or HEAD != origin/<main>.
  • It is idempotent: re-running when up to date is a no-op, and re-running after a partial failure (bump landed, push or Release didn’t) resumes the unfinished half instead of double-bumping. A nightly never blocks a stable: stable measures “pending” from the last vX.Y.Z tag, ignoring nightly-*.

Verify:

Terminal window
git describe --tags --exclude 'nightly-*' # stable: the new vX.Y.Z, on HEAD
gh release view "$(git describe --tags --abbrev=0 --exclude 'nightly-*')" --json tagName,url
git tag -l 'nightly-*' | tail -1 # nightly: today's tag, on HEAD

Both stable lines must show the new version. If the Release is missing, re-run bob-release (the resume path publishes it).

Terminal window
export CLOUDFLARE_API_TOKEN=... # token with Pages:Edit
export CLOUDFLARE_ACCOUNT_ID=... # account UUID
npm run docs:deploy # primary engine (vitepress) -> bigbrain-docs

Same engine the workflow drives: builds via scripts/docs-site/build.sh, then wrangler pages deploy, then fetches the published site back (#1446) — a green upload alone is not evidence of publication. A comparison engine deploys with npm run docs:deploy -- --engine starlight (lands at <project>-starlight). An interactive wrangler login session works in place of the env vars.

Verify: the deploy script’s own post-deploy fetch must pass (it fails the run if the site doesn’t answer). Spot-check a page that this delivery changed — “done” for docs means live and verified, not uploaded.

A laptop declared bob_channel: stable in the operator’s machines.json runs only what a stable release holds (#3171). Cutting the release in step 2 is what makes it available; each laptop takes it with one command, from a plain terminal with no Claude Code session open (the live-session guard refuses otherwise):

Terminal window
bob update # fetches tags, deploys the newest vX.Y.Z from its own worktree, runs bob doctor

The laptop’s ~/projects/bigbrain checkout is not moved: keep developing on master there. A git pull followed by scripts/deploy.sh still deploys the tag, not master. To run master on a laptop deliberately, use scripts/deploy.sh --channel checkout.

Verify:

Terminal window
bob doctor | grep 'version:' # … [stable] … — channel: stable vX.Y.Z (newest)

A laptop not yet on a release with scripts/fleet/channel.js (older than the release that carries #3171) takes the first one with a plain bob update: it fast-forwards the checkout to the tag and deploys it. From then on the channel applies.

  • If Actions is down for billing (#1648-class), leave the tracking todo open — this runbook is the bridge, not the fix.
  • Nothing else to reconcile: the release commit is loop-guarded (chore(release) prefix), so when Actions comes back the next push resumes auto-release cleanly on top of whatever was cut locally.
  • Versioning how-to — how auto-release works and how it’s provisioned
  • .github/workflows/auto-release.yml, ci.yml, docs-deploy.yml — the workflows this substitutes for
  • Incident response runbook — when something is broken rather than unavailable
  • Self-hosted Actions runner — the structural fix for minute caps: CI on fleet hardware bills zero minutes, so this fallback is needed less often