Docs-Site Lifecycle

Docs-Site Lifecycle (#153, #689)

Provision and operate the engine-pluggable docs site. Extracted from the dev lifecycle reference.

Docs Site Lifecycle

Every BoB-provisioned project can host a docs site that reads its own docs/, README.md, and CLAUDE.md. The lifecycle is orthogonal to the dev environment lifecycle but follows the same declarative pattern.

The tooling is engine-pluggable (#689). The engine field in docs-site.json selects the static-site generator:

Engine	Value	Notes
VitePress (default)	`vitepress`	Flat `.html` with native `.md` link rewriting — correct under flat Cloudflare-Pages serving with no post-build step. Folder-autogenerated Diátaxis sidebar (`vitepress-sidebar`), built-in local (MiniSearch) search.
Astro Starlight (alternative)	`starlight`	`build.format: 'file'` + `trailingSlash: 'never'` for the same flat serving. Folder-autogenerated nav, Pagefind search, richest theming.

Both render the same docs/ taxonomy and resolve cross-directory links correctly under flat serving with no post-build link rewriting. Quartz — and its fix-links.js hack — was retired in #689 (building on the #487 SSG research).

Accepted regression: Quartz’s graph view and automatic backlinks have no equivalent in either engine; [[wikilinks]] need a plugin or a one-time conversion.

Node requirement: the Starlight engine (Astro 7) needs Node ≥ 22.12; VitePress is fine on Node ≥ 18. CI pins Node 22 in docs-deploy.yml for this reason.

Every sub-command runs two ways — as a slash command in a Claude Code session, or from a terminal by invoking its script directly. The table pairs them; the examples further down show the slash form, but each has the terminal twin listed here.

Slash command	Terminal	Purpose
`/docs-site init`	`~/.claude/scripts/docs-site/init.sh`	Scaffold `.docs-site/<engine>/` for the resolved engine + a default `docs-site.json` (if missing). Idempotent on re-run. `--engine` forces an engine.
`/docs-site build`	`~/.claude/scripts/docs-site/build.sh`	Generate reference pages from command/skill/agent frontmatter (see below), then sync the `docs/` taxonomy + run the engine build. Flat output at `.docs-site/public/`. `--engine` overrides.
`/docs-site dev`	`~/.claude/scripts/docs-site/dev.sh`	Sync content + engine dev server with hot reload. Binds the project’s docs port from the port ledger (`base+3`), so it never collides with the app or another project. `--port` overrides; `--engine` overrides the engine; `--print-port` resolves and prints the port without starting a server.
`/docs-site deploy`	`~/.claude/scripts/docs-site/deploy.sh`	Build, then publish to the configured target (`cloudflare-pages`, `static`, `custom`). The primary engine deploys to `deploy.project`; a comparison engine deploys to `<deploy.project>-<engine>`.
`/docs-site draft`	`~/.claude/scripts/docs-site/draft.sh`	Build + publish a PR-independent preview from any branch into an isolated URL namespace (needs `draft.enabled` — see the draft/preview section below).
`/docs-site clean`	`~/.claude/scripts/docs-site/clean.sh`	Drop build artifacts (preserves installed deps). `--hard` removes the whole `.docs-site/` workspace.

Per-project config (docs-site.json at project root):

{
  "engine": "vitepress",
  "title": "Bodmail Docs",
  "description": "Documentation for the Bodmail email platform",
  "base_url": "https://docs.bodmail.app",
  "sources": ["docs/", "README.md", "CLAUDE.md"],
  "exclude": ["docs/archive/**", "docs/audits/**"],
  "deploy": { "target": "cloudflare-pages", "project": "bodmail-docs" }
}

base_url drives the serve path. The engine base is the path component of base_url, normalised to /…/ — / for a root-served Pages project (the BoB default, empty/host-only base_url), or e.g. /docs/ only if the site is genuinely served under that subpath. A base that doesn’t match where the site is actually served makes every asset 404 (an unstyled page). The flat-link gate (make docs-verify) runs at this same derived base.

How each declared source lands (#1261). docs/ is special-cased: its contents sync to the content root, so the Diátaxis taxonomy sits at the top (docs/how-to/x.md → /how-to/x). Every other declared directory publishes as a named section keeping its own directory (runbooks/x.md → /runbooks/x), and a declared file publishes at the root under its basename (CLAUDE.md → /CLAUDE). README.md is the home page when the content provides no index.md of its own.

Cross-root relative links are fixed at sync time (#1468). Because docs/ loses a level and a named section does not, a relative link between the two resolves on GitHub but would land at the wrong site path. The sync’s rewrite pass (rewrite-escapes.js) detects in-scope links whose synced location shifted and rewrites them on the synced copy to the site-correct relative path — so author every link for the repo (docs/index.md → ../runbooks/index.md, docs/** → ../../CLAUDE.md, a runbook → ../docs/how-to/x.md) and the site gets the corrected form. As a backstop, verify-links.js --flattened docs fails any surviving docs/-first-segment link as an off-by-one instead of skipping it as external.

runbooks/ is published (#1447). BoB declares runbooks/ as a source, so the universal runbooks are a first-class site section rather than repo-only files. The reasoning, since it is the kind of thing that gets re-litigated: a runbook is a reader-facing procedure (Purpose / When to use / Prerequisites / Steps), /runbook-gen exists to produce more of them, and a generated surface nobody can read is a coverage gap. Three consequences worth knowing:

Promotion is publication. runbooks/.drafts/ is git-ignored and excluded from the site; promote.js moving a draft into runbooks/ is what puts it on the web. See Generate a Runbook.
Membership is derived. runbooks/index.md is a managed section index (scripts/docs-site/lib/section-index.js), so a promoted runbook joins the index — and stops being an orphan — by existing. runbooks/**/*.md is in the doc-keeper scan_paths, so runbooks are governed like any other published doc.
A runbook edit republishes. runbooks/** is a path trigger on .github/workflows/docs-deploy.yml.

A project that does not want its runbooks published simply omits runbooks/ from sources — the directory still works as agent-loadable scoped context.

Docs-currency policy (publish_on, draft) — #759. Two optional keys declare when docs publish:

publish_on (merge | production-deploy, default merge). merge publishes on push to the main branch — the existing behavior, so omitting the key is a no-op. production-deploy holds the production publish until the app’s production deploy has succeeded, for projects where merge ≠ production: deploy.sh gates the publish on a prod-deploy-ok signal (--prod-deploy-ok or DOCS_SITE_PROD_DEPLOY_OK=true) and otherwise skips the publish (a hold, not a failure).
draft ({ enabled, path_prefix }, default { enabled: false, path_prefix: "drafts" }). Declares the draft/preview mechanism the PR-independent preview workflow consumes — when enabled, a build from any branch can publish a preview into an isolated URL namespace (/<path_prefix>/<branch>/) without touching the live site.

Both are validated by schemas/docs-site.schema.json, and this repo’s docs-site.json is checked against it in make check (check-schemas).

Adopting in a project:

cd /path/to/project && /docs-site init — scaffolds .docs-site/<engine>/ and a default docs-site.json.
Edit docs-site.json — set the engine, title, base URL, and deploy target.
/docs-site dev — preview at the printed URL (the project’s ledger docs port; run /docs-site dev --print-port to see it).
/docs-site deploy when ready.

Trying the other engine, ad hoc (no config change):

Slash command:

/docs-site build --engine starlight
/docs-site dev   --engine starlight

Terminal:

~/.claude/scripts/docs-site/build.sh --engine starlight
~/.claude/scripts/docs-site/dev.sh   --engine starlight

Flat-serving link gate (make docs-verify): builds both engines against the real docs/ and asserts every in-docs cross-directory link resolves under flat Cloudflare serving with no rewriting (#689 AC-04; scripts/docs-site/verify-engines.sh + verify-links.js). Links pointing outside the published docs (repo source) are skipped, not failed.

Out-of-scope link rewrite (#1393): a relative link that resolves on disk but escapes the declared content roots (registry/…, apps/…, skills/…) is valid on GitHub yet 404s on the published site. Every sync runs scripts/docs-site/lib/rewrite-escapes.js over the synced copy — such links become <repo>/blob|tree/<branch>/<path> GitHub URLs (remote/branch from origin; anchors preserved; fenced and inline code untouched), so sources stay GitHub-idiomatic while the published site never links into content it doesn’t carry. The companion guard (rewrite-escapes.js check, wired into make docs-check) fails only on an escaping link whose target doesn’t exist on disk — that one 404s everywhere and needs a hand-fix.

Cross-root link rewrite (#1468): the same pass also fixes in-scope links whose relative depth changes when the sources sync (docs/ flattens to the root; named sections and root files do not): the link is rewritten on the synced copy to the site-correct relative path, using the inverse of the same source maps. This runs even with no GitHub remote — it is a relative-path fix, not a GitHub URL rewrite. The flat-serving gate passes --flattened docs to verify-links.js, so a leaked repo-rooted path (docs/… first segment) fails as an off-by-one rather than being skipped as external.

Universal portability: the scripts read --root, $PROJECT_ROOT, or cwd — they never hardcode BOB_SOURCE. The .docs-site/<engine>/ workspace is entirely generated from templates/docs-site/<engine>/ + the synced docs/, so the whole directory is gitignored.

Auto-deploy on doc changes (#155). A GitHub Actions workflow at .github/workflows/docs-deploy.yml rebuilds, verifies, and publishes the site whenever a published doc source changes (docs/**, README.md, CLAUDE.md, docs-site.json, the docs-site scripts, or the engine templates). On master push it deploys both engines for a live A/B; on PRs it deploys previews and posts the URLs back as a sticky comment.

The pipeline is:

checkout → cache .docs-site/{vitepress,starlight}/node_modules
        → make docs-check                 (doc-governance job — reports, does NOT block deploy)
        → verify-engines.sh               (build + flat-link-verify BOTH engines)
        → build vitepress → wrangler pages deploy → bigbrain-docs            (primary)
        → build starlight → wrangler pages deploy → bigbrain-docs-starlight  (comparison, best-effort)
        → verify-deploy.sh                (fetch the published root + changed pages)
        → comment preview URLs on the PR

Post-deploy verification (#1446). A green wrangler pages deploy says the upload succeeded, not that the site publishes — the build can emit nothing, the output directory can be wrong, a routing change can 404 every page, and none of that turns the run red on its own. So after a production deploy the workflow fetches the site back:

~/.claude/scripts/docs-site/verify-deploy.sh --project bigbrain-docs \
  --doc docs/how-to/foo.md          # routes are derived from changed doc sources

Checks the site root plus the pages the pushed docs publish to (docs/how-to/foo.md → /how-to/foo, index.md → its directory, README.md → the home page). A source that publishes no page is skipped, not failed.
Follows redirects — Pages legitimately 308s clean URLs, so only the final status is the verdict.
Retries with backoff to absorb propagation delay, bounded by --attempts (6) and --max-wait (180s), so a dead deploy fails in bounded time.
A failure fails the run, which is what makes notify-failure fire.
Skipped when DOCS_SITE_DEPLOY_ENABLED != 'true' — there is nothing published to verify, and the whole deploy job is gated on that flag.
Works unchanged for a comparison engine: it verifies whatever project name that matrix leg resolved (<project>-<engine>). Only the primary engine is verified at a configured base_url.

deploy.sh runs the same check after a manual publish; pass --skip-verify to publish without it.

Blast radius: the two gates are not equal (#1444). A doc that nothing links to and a site that will not build are different failures, and only one of them should stop publishing:

Job	Catches	Deploy needs it?	A red run on master means
`gate`	Build failure, flat-serving links that don’t resolve	Yes	Nothing publishes. The last-known-good deployment stays served and goes stale until fixed.
`doc-governance`	`make docs-check` — orphan / root-placement / freshness	No	The site still republished. The offending page is live but hard to find; the failure is scoped to it rather than freezing every page.

Both turn the workflow red, both file (or bump) the docs-deploy-failure issue, and on a PR both block the merge via branch protection.

The alert clears itself (#1445). A green master run closes every open docs-deploy-failure issue, commenting the run URL and commit SHA. Green means the gate passed, doc governance passed, and deploy either published or was legitimately skipped (publishing disabled — the pipeline is healthy, it just isn’t publishing). A green run with no open issue is a silent no-op. So an open docs-deploy-failure issue means the pipeline is red right now — before this, one stayed lit for twelve days of successful runs, which is exactly how an alert gets ignored. What changed is that a findability finding no longer holds every other page hostage — a single orphan used to freeze the entire site’s republishing until a human noticed. The primary defence against that class is now local and earlier anyway: hooks/doc-orphan-gate.sh refuses the commit that would land the orphan, naming the index to edit.

Required repo configuration:

Secret CLOUDFLARE_API_TOKEN — Cloudflare API token with Pages:Edit scope.
Secret CLOUDFLARE_ACCOUNT_ID — the Cloudflare account UUID.
Variable DOCS_SITE_DEPLOY_ENABLED=true — feature flag that gates the wrangler steps. Until set, the workflow still builds, verifies, and runs the drift gate (catching breakage) but skips the deploy.
A one-time bigbrain-docs-starlight Cloudflare Pages project for the comparison build — the Starlight deploy is best-effort, so a missing project never blocks the primary production deploy. See #187 for the Cloudflare connection step.
Secret BOB_CHECKOUT_TOKEN — on non-BoB projects only: a fine-grained PAT (Contents: Read-only on the BoB repo) so CI can check out the private BoB tooling. Without it the gate degrades gracefully — the BoB checkout is non-fatal and engine verification skips with a loud ::warning naming the secret; the deploy job stays strict (#1265). docs-site init provisions the secret (or files a todo), and ~/.claude/scripts/fleet/distribute-secret.sh status|distribute|reconcile handles the whole fleet in one pass — see the docs-site skill for details.

NPM script aliases live in the root package.json so users can run npm run docs:build / docs:dev / docs:deploy from anywhere in the repo.

Draft / preview, PR-independent (#760)

/docs-site draft builds and publishes a shareable preview from any branch without opening a PR — the counterpart to /docs-site dev (local hot-reload). It is keyed to the current git branch (--branch overrides), and the mechanism is read from the draft config block (#759), so it must be enabled per project:

Slash command:

/docs-site draft

Terminal:

~/.claude/scripts/docs-site/draft.sh

{ "draft": { "enabled": true, "path_prefix": "drafts" } }

Cloudflare Pages → a per-branch preview alias (wrangler pages deploy --branch <slug>), reachable independent of any PR.
Static target → the build is staged at .docs-site/<path_prefix>/<branch>/, leaving the production output (.docs-site/public) untouched.

docs-site dev (local hot-reload) is unchanged and remains the inner-loop tool; draft is the shareable preview. PR-preview deploys from docs-deploy.yml stay a best-effort bonus when a PR happens to exist — never the mechanism, so previewing WIP docs never requires a PR.

Generated reference (#758)

build.sh runs scripts/docs-site/lib/gen-reference.js before the engine sync, harvesting the frontmatter of every command, skill, and agent surface (universal and provisioned registry/ equivalents) into per-category reference pages under docs/reference/generated/ — commands.md, skills.md, agents.md, and an index.md. For a tooling project these commands are the API surface, so no separate API harvest is needed.

The generated pages are a build artifact, not hand-authored content:

Reproducible / idempotent — same sources produce byte-identical output; the output dir is wiped each run, so a removed or renamed surface drops out with no orphaned page left behind.
Machine-generated — each page carries generated: true in its frontmatter and a MACHINE-GENERATED banner; the path is git-ignored and excluded from the manual-edit doc-drift checks (scripts/doc-keeper/rules.json), so it never trips drift gates.
100% coverage — every command/skill/agent surface appears on a generated page. gen-reference.js --check-coverage prints the coverage JSON and exits non-zero if any surface is unlisted; make docs-reference-check runs it as a gate.

Projects with no command/skill/agent directories skip generation entirely.

Drift report on the site (#156)

Every /doc-audit run writes two artifacts to docs/audits/:

doc-audit-YYYY-MM-DD.{md,json} — date-stamped historical record
latest.{md,json} — canonical aliases the docs site reads

The site (when built via /docs-site build) consumes docs/audits/latest.md for a “Docs last verified” badge on the landing page and an “Audits” entry in the nav. latest.json is exposed as a downstream API endpoint at /audits/latest.json. See docs/audits/ for the canonical-path contract and history convention.

The nav + landing-badge component wiring lands as a follow-up — the data plumbing (latest aliases + history) is in place.