Farm Tick (the night shift)

Farm Tick — the night shift

The farm tick is BoB maintaining itself while nobody is at a keyboard: farm-01 runs one bounded maintenance cycle on a schedule, merges only what its own CI proves green, and tells you what it did in one message. This page is the one coherent explanation of that night shift — what runs, what it may take, how it stops, how you see it. It is authored with the eligibility rule (#2052) and refreshed by the last requirement of the capability (#2033); the parts that have not landed yet say so, by issue.

What runs

One cycle is bob tick (#2054): under a lock so two ticks never overlap, it grooms, builds a plan, runs one warp-drive session, reconciles the plan, and files a digest. The timer is a shipped systemd user unit (#2056) — declared in machines.json (bob_workload) and thefarm’s workload registry, never a crontab someone typed — and the farm runtime is pinned to the last nightly release, deployed only between ticks.

bob tick --dry-run                    # what tonight's tick WOULD do — files nothing
bob tick                              # one live cycle
bob tick --pause | --resume           # the night-shift switch
bob tick --install-timer --dry-run    # the units the declaration renders — writes nothing
bob tick --install-timer              # install + enable the timer (idempotent)
bob tick --uninstall-timer            # disable it and remove both units

The timer is a declaration

farm-01’s entry in machines.json carries the workload (#2056), validated by schemas/machines.schema.json:

"bob_workload": {
  "source": "~/projects/bigbrain",
  "bob_home_channel": "nightly",
  "tick": { "schedule": "daily 02:30 America/Los_Angeles", "memory_max": "3G", "dry_run": true }
}

bob tick --install-timer renders it through the shared unit renderer (templates/systemd/render.js, #2067) into two systemd user units under ~/.config/systemd/user/, then runs systemctl --user daemon-reload and enable --now bob-tick.timer:

Unit	From the declaration
`bob-tick.timer`	`OnCalendar=--* 02:30:00 America/Los_Angeles` — `schedule` is `daily HH:MM [Area/Zone]`; no zone means the host’s local time
`bob-tick.service`	`ExecStart=%h/.claude/bin/bob tick --cwd %h/projects/bigbrain` — the deployed `bob`, run in `source`; `--dry-run` is appended while `tick.dry_run` is `true`; `MemoryMax=` from `memory_max` (absent = `infinity`); exit 3 (lock held, paused) counts as success

The same bob_workload may also declare brief — the portfolio briefing’s cadence, quiet hours and retention (#2246). bob brief --install-timer renders it into bob-brief.service + bob-brief.timer the same way; see Schedule and retention.

Every path in the units is %h-relative, so the same declaration renders the same bytes on any account. A unit already on disk with the rendered content is reported unchanged and not rewritten; change the declaration and re-run the install to converge. The engine is scripts/tick/tick-timer.js (install | uninstall | status | render, --json = tick-timer/1; exit 0 ok, 1 a systemctl step failed, 2 usage, no valid bob_workload, or not Linux). On macOS the install is refused, naming launchd as the scheduled path there. The report reads Linger from loginctl — a user timer fires with nobody logged in only when lingering is on (loginctl enable-linger $USER).

Check it on the host:

systemctl --user list-timers bob-tick.timer   # next elapse
node ~/.claude/scripts/tick/tick-timer.js status   # units current vs the declaration, timer active

tests/test-tick-timer.sh renders both units from a fixture declaration and diffs them against tests/fixtures/tick-timer/systemd/ (UPDATE_GOLDEN=1 rewrites), then drives install / status / uninstall with a fake systemctl. Registering the workload in thefarm’s workloads.json is a separate repo and stays a human step.

In order, a live tick:

takes flock on $BOB_HOME/.bob-tick.lock — a second tick exits 3 immediately — and exits 3 paused, running nothing, while $BOB_HOME/.bob-tick.pause exists (see how it stops). It also exits 3 archived, running nothing, when the project it was pointed at is decided archive (active: false in projects.json, #3609);
bob groom --all --apply — a groom that reports blockers is an outcome, not a failed tick;
bob plan build --from farm --max-items <tick_max_issues> — only work the rule below admits, and at most that many items (see budgets);
converges the host on BOB_SOURCE (#2472). The previous tick’s run merged scripts, commands or hooks, and the update in step 8 comes after this tick’s run, so without this the machine is still behind source when the run starts. When the gate names host:behind (or a host:surfaces finding) and nothing else failed (#3160), the checkout is the clean main branch and no Claude Code session is live, it runs scripts/deploy.sh and bob doctor. A refusal is an outcome, and its reason goes in the digest’s host_converged and its Host row. Only a deploy or doctor that fails fails the tick;
marks the project _workflow.unattended: true and runs bob warp start --flightplan N --level 3 --rdb (plus --max-tokens when a token budget is set), then waits for its tmux session to end. Past _workflow.tick_wall_clock_minutes (default 300) it stops the run with bob warp stop and the tick exits 4 with budget: wall-clock;
flightplan.js reconcile --apply --plan N, then audit.js snapshot;
on BOB_SOURCE: bob release --channel nightly (the engine’s own gates decide whether master moved and is green) and bob release promote --auto (#2057);
bob update --channel nightly — its own live-session guard decides: a live Claude Code session refuses it (exit 4, queued, recorded as bob tick) only when the nightly would rewrite the settings.json hooks (#2746);
writes the digest, comments it on the plan issue and sends it with bob notify; 9b. reconciles the briefs — bob brief --reconcile --apply --json in the project (#3100) ticks every brief action whose named issues have all closed, closes the finished briefs, and rebuilds and publishes the index. It is never a step: its outcome is the digest’s brief_reconcile and its Briefs reconciled row, and a forge that answers nothing (exit 3) is recorded as skipped. A dry run reports without --apply. See Reconcile; 9c. tests the to-dos a machine can test — bob asks verify --apply --json (#3986) runs every open to-do’s Done-check:. A to-do whose check passes gets completed and is closed with the evidence, a new failure reason is written into its body, and a failed CI run is re-run once. It is never a step and never a strike; a dry run reports without --apply. See Done-check; 9d. reconciles the attention budget — bob asks --reconcile --apply --json (#3805) labels each ask beyond the budget ask-waiting and releases each one that is now inside it. It is never a step and never a strike; a dry run reports without --apply. See Asks that wait; 9e. re-checks the to-dos nobody has touched — bob asks recheck --json (#3808) counts the to-dos untouched for longer than the re-check window: how many are moot, still needed and unchecked. It is report only in every mode: the tick prints the counts and never closes a to-do, which is bob asks recheck --apply. It is never a step and never a strike. See Re-checking;
makes the portfolio briefing — bob brief --send --scheduled in the project (#2246) — unless this machine’s declared bob_workload.brief.quiet_hours hold. It is never a step: its outcome is the digest’s briefing, and a briefing that could not be sent never fails or strikes the tick. A dry run makes none.

A failed step is exit 1 and is named in the digest; 2 is a usage error. --dry-run runs the same pipeline read-only. The groom runs without --apply, the farm queue (discover-queue.sh --farm) stands in for the plan build, the host converge and warp start run as --dry-run. There is no reconcile or snapshot, and release, promote and update run with --dry-run. It creates no issue, branch, tag, PR or commit.

tests/test-bob-tick.sh exercises both modes against stubbed engines, a fake gh, a fake tmux and real git checkouts:

the live order above, including the wait on the run’s tmux session;
the unattended marker, the plan comment and the digest fields;
the exit codes 1, 3 and 4, each budget, and the two-failures pause;
a dry run that leaves git and the forge untouched.

The pieces a tick composes can each be driven by hand:

bob plan build --from farm            # the night shift's plan — farm-eligible work only
bob plan remaining <N> --json         # what it would take next, in order
~/.claude/scripts/warp-drive/discover-queue.sh --farm   # the same membership as a queue
node ~/.claude/scripts/warp-drive/eligibility.js check <N>   # why one issue is in or out

What it may take

The night shift must not take work you have not delegated. One predicate decides — scripts/warp-drive/eligibility.js — and both consumers apply it: flightplan.js build --from farm (so bob plan build --from farm) and discover-queue.sh --farm. Nothing else re-derives it. The rule, verbatim from the module (eligibility.js rule prints it):

ELIGIBLE  — `req` + `approved` carrying the `farm` label
          — `bug` + `approved` carrying the `farm` label, when the bug is a fingerprinted `harness-defect` (body marker `<!-- harness-defect: <fp> -->`) OR names a reproducing / regression test, a concrete test (a `tests/` path, `make test-<name>`, or `test-<name>.{sh,js}`), or a `## Reproduce` section in its body
          — with `_workflow.farm_eligible: true` on the project, the `farm` label is not required: every approved req (and every verified bug as above) is eligible
NEVER     — a `cap` (or anything that is not a req/bug)
          — `serial-only`
          — `blocked`
          — `needs-clarification`
          — narrative work: a `baseline-doc` issue, or a title that begins `Vision` / `CLAUDE.md` or names docs/vision.md (human-authored prose)
          — a bug with no harness-defect fingerprint, no named test and no reproduce section
          — anything not `approved` — `--allow-unapproved` never applies to the farm scope

Delegating work. Add the farm label to an approved req (or a verified bug). The label is in the standard set bootstrap-labels.sh ships, and it is applied by a human — or by /groom when you ask it to — never by tooling on its own. A project that wants the farm to take everything approved sets _workflow.farm_eligible: true in .claude/settings.local.json (declared in the _workflow table, default false).

Seeing the verdict. A farm plan build prints every exclusion with its reason — [dropped] #N title — no farm label — …, serial-only, cap, needs-clarification, … — on stderr, and carries them as dropped in --json. discover-queue.sh --farm prints the same as [excluded] lines. eligibility.js check <N> answers for one issue (exit 0 in, 3 out). tests/test-eligibility.js covers every line of the rule with a fixture issue.

How it merges

An unattended run never pushes to master directly. It opens a PR, waits for the runner’s make ci, and fast-forwards only when it is green (#2053) — a broken master never reaches the laptops.

Before it pushes, the engine runs the runner-availability preflight (#1731) on entry to merging: a fleet runner that is offline parks jobs rather than failing them, so the run asks ci-locus.js preflight whether the next job will run, falls back to GitHub-hosted within one job when the farm is down and hosted is viable (restoring the declared locus once the runner is back), and stops with a named cause — never a merge-on-green timeout — when neither can run. The verdict is on state.ci_preflight and in the session summary; see CI Locus.

The switch is one _workflow key, unattended: true (declared in workflow-config.md; the tick sets it on the project it runs). With it on, promotion.js decide resolves merge: pr at every level — the ceiling still governs the deploy axis — and the merging phase pushes the branch, opens the PR and hands it to the merge engine:

bash ~/.claude/scripts/promotion/merge-on-green.sh <pr> [--timeout <min>] [--json]

The base branch says which checks are required, never the PR (#3625). The PR’s head decides which workflows run on it, so the checks it reports say nothing about the checks it owes. The names come from branch protection’s required status checks when the base has them, otherwise from .github/required-checks.json as it stands on the base branch:
```
{ "required": ["ci-passed"] }
```
bigbrain requires ci-passed, the last job of ci.yml, which reports on every pull request whatever it changes (#3717).

With neither, the verdict is no-required-checks and nothing is merged. Only a 404 on the protection read means the branch has no protection. Any other failed read — a 403 from a token that may not read it, a 5xx, a network error — is exit 4 and nothing is merged (#3630). Use the check’s name as the PR page shows it — for an Actions job, the job name.
It polls gh pr view --json statusCheckRollup until every required check is a completed success. A required check that never reports is waited for and then refused; skipped or neutral is not a pass. A PR that removes or skips the required workflow is therefore never green.
Under the declaration the names are a floor: any other check the PR reports must still finish without failing. Under branch protection a check the base does not name is informational, as on GitHub itself.
A PR that changes what grades it is never merged unattended. The diff from the base to the exact head SHA is read, and again if the head moves. A change under .github/workflows/ or .github/actions/, or to .github/required-checks.json, is verdict workflow-change: the PR is left open with a comment that names the paths, for a person to review and merge. A PR of 300 files or more cannot be read in full and is left open the same way (diff-too-large).
A draft PR is never merged, and the draft flag is read on every poll: a PR turned back into a draft while CI ran is verdict draft, exit 3 (#3630).
Green → the base ref is fast-forwarded to the PR head through the git refs API (force: false). The SHA that lands is the SHA CI tested: never a squash, never a rebase, never a merge commit. Exit 0.
Red, timeout, no required check that ran (no-checks), no required check declared, a workflow change, conflicts, or a base that moved since CI ran → the PR is left open with a comment naming the failed check (or the reason) and the exact re-run command. Exit 3. The run records {pr, ci_verdict, failed_check} on state.merge_on_green, the session summary reports it, and the fix is a human’s (or the next tick’s after a rebase) — commits are gated outside the committing phase, so the loop never patches a red PR from merging.
Re-running it on a merged PR is a no-op (already-merged, exit 0) — the tick can retry a timeout without re-deciding anything.
--json is the merge-on-green/1 document: verdict, checks, failed checks, polls, waited seconds, whether a comment was posted. --dry-run reports the verdict without moving the ref or commenting.

A PR that triggers no CI at all (nothing under a path CI watches) is never merged — the no-checks verdict hands it to a human rather than shipping unproven work. tests/test-merge-on-green.sh covers green / red / timeout / already-merged / no-checks / required-subset against a stubbed gh.

What stops a run from reaching master by itself

Two things stand between an unattended run and master, and a third is declared but not yet applied.

1. The promotion decision. With unattended: true, promotion.js decide answers merge: pr. The run is told to open a PR and hand it to merge-on-green.

2. The promotion guard (#3624). A PreToolUse hook, promotion-ceiling-guard.sh, reads every Bash command the run is about to make and blocks one that would move master. In an unattended run it blocks:

Route	Example
A merge while on the main branch	`git merge feature/x`
A push whose destination is the main branch, in any spelling	`git push origin master`, `x:master`, `HEAD:refs/heads/master`, `+x:master`, `--all`, `--mirror`
A push with no refspec while on the main branch	`git push`
A pull-request merge	`gh pr merge 12`, `gh api -X PUT repos/<owner>/<repo>/pulls/12/merge`
A write to the main branch’s ref	`gh api -X PATCH repos/<owner>/<repo>/git/refs/heads/master -f sha=…`

Each command is judged in the directory it runs in, so cd <dir> && git push and git -C <dir> push are read against that directory’s project and branch. The run’s own work is left alone: a push of its feature branch, gh pr create --base master, a read of the refs API.

In an unattended run the guard fails closed. If the promotion decision cannot be read, or the guard cannot read the command (its detector is missing from the deploy), the command is blocked and the message says which. In an attended session it fails open, as it always has, and landing a PR (gh pr merge) is left to you.

merge-on-green is not blocked: it is a script the run starts, and the guard reads the command the run types, not what the script does. It moves master only to the commit whose checks are green.

The guard reads commands as written. It does not see a destination held in a variable (git push origin "$BRANCH") or a merge written as a GraphQL mutation inside a quoted query. The detector is scripts/promotion/main-advance-detect.js; its header lists the routes.

3. GitHub itself: declared, not yet applied. A branch protection rule on the forge refuses the push whatever command made it. It is a declared project setting, _meta.branch_protection in the project manifest: bob provision status reports it against what GitHub holds, and branch-protection.js reconcile --apply applies it. See Branch Protection.

The farm uses the operator’s admin token, so only a rule enforced for admins stops it, and that rule binds every attended session and the stable release too. Before anything is applied, the same command checks that each route to master still gets there under the declared rule, and refuses to apply while one would be cut off. master on bigbrain has no rule today: its CI is split by path, so no single check reports on every pull request, and the attended direct merge and the release both push commits no check has run on.

How it stops

Budgets per tick (#2055). Three _workflow keys, declared in the config table and set in .claude/settings.local.json. The tick reuses warp-drive’s budget vocabulary and has no pricing table of its own. A spent budget still reconciles the plan and files the digest, and the tick exits 4.

Key	Default	What it bounds
`tick_max_issues`	`3`	The plan: `bob plan build --from farm --max-items N` seats at most N req/bug items in run order. Each item comes with the in-scope items it depends on, the rest are `[dropped]` with the cap named, and gating todos ride along.
`tick_wall_clock_minutes`	`300`	The wait. A run still going is stopped with `bob warp stop` — the graceful abort, never `--hard` or a tmux kill — so it finishes its own abort protocol. Reported as `budget: wall-clock`.
`tick_max_tokens`	`0` (off)	The spend. Passed to the run as the kickoff `--max-tokens` ceiling, which the session breaker enforces. After the run the tick reads its spend with `token-report.js --since <tick start> --json` and reports `budget: tokens` when it went over.

The digest carries budgets (the three limits in force), budget (which one was spent, or null) and tokens (total, sessions, usd, usd_qualified, limit).

Two failed ticks pause the timer (#2055). A bad night costs one night.
- A tick that ends in exit 1 or 4 adds a strike to $BOB_HOME/.bob-tick.strikes; an exit 0 clears the strikes, so only consecutive failures count.
- The second strike writes $BOB_HOME/.bob-tick.pause: paused_at, a reason naming both ticks with their exits and verdicts, and source: auto.
- It also sends exactly one bob notify --level block. The paused ticks after it send no further block message.
- A lock-held tick and a paused tick neither add a strike nor clear one.
- The digest carries strikes, auto_paused and pause.
The pause switch — bob tick --pause writes the same file by hand (source: manual). While it exists every tick exits 3 paused and runs nothing. bob tick --resume removes it and resets the strikes (#2054, #2055).
A plan that waits on another plan (#2298) starts nothing. The run step is skipped with the reason plan #N waits on flight plan #2092 (38/57 done), and the plan is still reconciled.
Dry-run first: the first week runs bob tick --dry-run and files only the digest, so you see what it would have done before it does it (#2054).

The eligibility rule above is the hard floor under all of this: an issue the rule excludes cannot be taken by any tick, paused or not.

How you see it

Each tick ends with one digest — on Telegram through the remote-decision bridge (bob notify, #2048) and on bob viz: taken, merged, blocked, bugs filed, release cut, loop notes added, rituals, missed backups (#3086), the wiki maintainer’s PR (#2389), the skill proposal (#2391), the briefing, tokens (#2054). The digest’s briefing is {ran, outcome, exit, file, note}; outcome is one of sent, quiet (nothing needed you and --on-quiet skip held), quiet-hours, no-bridge, transport, refused (an invalid brief declaration), error or, on a dry run, skipped. A tick at 02:30 inside 22:00-07:00 quiet hours records quiet-hours and makes nothing — the morning briefing comes from the bob-brief timer instead (Get your briefing). Every tick also publishes a fleet snapshot (audit.js snapshot), so audit.js view shows the farm’s state next to the laptops’.

After the snapshot the tick prepares evidence packets (#2233). It runs ritual.js prepare --due --all --within P2D --budget-seconds <left> --root <BOB_SOURCE>. Each farm-eligible ritual that is overdue or due within the next 48 hours gets its open run issue reused, or a run opened. The ritual’s collectors then run report-only (cost and churn, the status-report data, aged issues, usage gaps, doc and memory audits, dependency audit, readiness), and one packet comment is posted to the run. A later tick replaces that comment; it never adds another.

Budget: the pass is bounded by what is left of _workflow.tick_wall_clock_minutes after the run. Collectors spend no tokens, so the wall clock is the budget that applies (#2055). When the run used the whole wall clock, no packet is prepared and the digest says so.
Failures: a collector that cannot run is a named gap inside the packet, and a ritual that cannot be prepared is listed with its reason. Neither ever fails the tick.
Dry run: a dry run lists what it would prepare and writes nothing.

Then the digest’s rituals (#2232) is read with ritual.js due --all --json --root <BOB_SOURCE>. The read runs in a dry run too and is never a pipeline step: an overdue ritual, or a read that fails, never blocks the tick.

prepared lists each packet the tick posted (project, ritual, run issue, whether it opened the run, gap count).
prepare carries would_prepare (dry run), failed and, when preparation did not run, its reason.
overdue lists every ritual still overdue across BOB_SOURCE’s active manifests, most overdue first.
held lists every stable promotion a release-gating ritual holds — project, ritual, reason, open run, last run (#2236): what bob release promote --auto refused with exit 4 in step 7, which the tick counts as an outcome, never a failed step. The markdown Rituals row ends with promotion held N: …. A project with the gate off (_workflow.ritual_release_gate) is never listed.

A read that cannot run leaves overdue: null and names why in error. See Rituals.

Then the tick notices missed backups (#3086). scripts/backup/notice.js --apply --json reads the snapshots and sends one bob notify for each project whose declared D1 backup missed its scheduled run. Each miss is keyed by project, env and the run it missed in $BOB_HOME/.backup-notices.json, so the same miss is noticed once however many ticks see it; the next missed run is a new notice. A backup whose bucket could not be read is unknown: it is listed, never noticed. It is never a pipeline step. The digest’s backup_notices is the bob-backup-notices/1 document (stale, noticed, already, failed, unknown, and on a dry run would_notice, where nothing is sent), and the markdown has a Backups row. bob brief --send runs the same pass before a briefing, sharing the ledger. See Dev lifecycle.

Then the tick notices cost alarms (#4037). scripts/costs/notice.js --apply --json builds the Costs page’s own document and sends one bob notify when a route key is projected over its budget, one more when it runs out before the month ends, and one for each runaway run: a live run with no commit or phase move for the page’s idle window, with a link to its session page and the command that stops it. A key first seen running out sends only that notice. Each is recorded in $BOB_HOME/.cost-notices.json once bob notify delivered it or printed it for want of a bridge; a transport error is retried on the next tick. It is never a pipeline step. The digest’s cost_notices is the bob-cost-notices/1 document (alarms, noticed, already, failed, and on a dry run would_notice), and the markdown has a Cost alarms row. bob brief --send runs the same pass before a briefing, sharing the ledger.

Then the tick consolidates the project’s traces into the wiki (#2389). It runs bob wiki maintain --project <the tick's project> --source <BOB_SOURCE> --json once. The maintainer reads every record added since its watermark: loop notes, the decision log, harness defects, the ledger’s no-trigger and improvised records, lesson issues and session reports. It scrubs client records and opens one PR on BOB_SOURCE with the page edits. The project’s automation level decides how far it goes:

`_automation.active_level`	What the tick runs
3 or 4	a real run: one PR, or a no-op when nothing is new
1 or 2, or unset	`--dry-run`: it reads and scrubs, then writes nothing

A dry tick always runs --dry-run. This is never a pipeline step: a failed run consumes nothing, and the next tick retries the same records. The digest’s wiki_maintain is {verdict, level, read, refused, left_over, created, updated, pr, reason, error}, and the markdown has a Wiki row with the PR link. You merge that PR like any other; the tick never merges it. See Consolidate traces into the wiki.

When the tick’s project is bigbrain (its checkout shares BOB_SOURCE’s git directory), the tick then runs the skill proposer (#2391): bob wiki propose --auto --source <BOB_SOURCE> --json. It picks the item with the most new wiki evidence that has an eval suite, gates one change to it, and opens one PR labelled skill-proposal, as a draft when the gate rejects it. The level decides here too: Level 3 or 4 is a real run, Level 1 or 2 and a dry tick run --dry-run. In any other project it is skipped, and the digest says so. The digest’s skill_propose is {verdict, level, item, pages, gate, draft, pr, reason, error}, and the markdown has a Skill proposal row. It is never a pipeline step, and the tick never merges the PR.

The snapshot of a machine that declares a workload also carries bob_workload (#2056): the declaration (source, bob_home_channel, tick) and timer — installed, current (the units on disk match what the declaration renders today), active (systemctl --user is-active bob-tick.timer), linger and on_calendar. A timer that was never installed, has stopped, or has drifted from its declaration is therefore visible from any machine. An invalid declaration, or a declaration on a host that is not Linux, is error with timer: null — it never fails the snapshot. A machine with no declaration has no bob_workload key. You read one message per tick, plus one per pause.

Flight Plan — bob plan build --from farm and the engine flags
_workflow config table — farm_eligible
Capability #2033 (the farm side of #2023); requirements #2052 (this rule), #2053, #2054, #2055, #2056
Product vision — the farm self-maintenance roadmap (#2023) and this rule as a non-goal