diff options
| author | Adam Malczewski <[email protected]> | 2026-06-12 20:42:57 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-12 20:42:57 +0900 |
| commit | 980de470c29d9cae475ada77284025305eaf5474 (patch) | |
| tree | cf24ef3b2510509d03dd4a3beb4e1939a2fbcccd | |
| parent | a1639b72103e4f038950a9dfe51c86fdda9f2771 (diff) | |
| download | dispatch-980de470c29d9cae475ada77284025305eaf5474.tar.gz dispatch-980de470c29d9cae475ada77284025305eaf5474.zip | |
docs(harness): Task-tool summon mechanism rework (briefs, ORCHESTRATOR, HANDOFF) + .skills
| -rw-r--r-- | .dispatch/extension-agent.md | 12 | ||||
| -rw-r--r-- | .dispatch/package-agent.md | 28 | ||||
| -rw-r--r-- | .skills/ORCHESTRATOR.md | 441 | ||||
| -rw-r--r-- | HANDOFF.md | 9 | ||||
| -rw-r--r-- | ORCHESTRATOR.md | 180 |
5 files changed, 560 insertions, 110 deletions
diff --git a/.dispatch/extension-agent.md b/.dispatch/extension-agent.md index cfaeaa9..51a8543 100644 --- a/.dispatch/extension-agent.md +++ b/.dispatch/extension-agent.md @@ -1,12 +1,12 @@ -<!-- ORCHESTRATOR-ONLY meta: this supplement is concatenated AFTER package-agent.md (and the - inlined rules), BEFORE the TASK block, only for EXTENSION summons. It is never used alone. --> +<!-- ORCHESTRATOR-ONLY meta: the summon tells an EXTENSION agent to read this supplement right + after package-agent.md and before the scoped rules + the TASK. It is never used alone. --> # Extension Owner-Agent — Supplement -The package owner-agent brief above in this prompt governs you in full: an extension **is** a -package, so its single-writer directory ownership, visibility, verification, and report rules -all apply. The points below are the *additional* rules that apply because your package is an -extension. +The package owner-agent brief (`.dispatch/package-agent.md`, which you read first) governs you in +full: an extension **is** a package, so its single-writer directory ownership, visibility, +verification, and report rules all apply. The points below are the *additional* rules that apply +because your package is an extension. ## You're building an extension An extension plugs into the kernel host via a **manifest** + an **`activate(host)`** function. diff --git a/.dispatch/package-agent.md b/.dispatch/package-agent.md index 9af0485..fc70f28 100644 --- a/.dispatch/package-agent.md +++ b/.dispatch/package-agent.md @@ -1,13 +1,14 @@ -<!-- ORCHESTRATOR-ONLY meta (see ORCHESTRATOR.md §2/§3): every single-package summon is - assembled as package-agent.md [+ extension-agent.md] + the inlined .dispatch/rules/* + - the per-summon TASK block. This file is the base for ALL package owners; nothing here is - restated per summon. --> +<!-- ORCHESTRATOR-ONLY meta (see ORCHESTRATOR.md §2/§3): the summon points an Opus 4.8 Task + agent at this file, then [extension-agent.md], then the scoped .dispatch/rules/*, then the + per-unit prompts/<unit>.md TASK — the AGENT reads them itself (orchestrator never inlines + them). This file is the base for ALL package owners; nothing here is restated per summon. --> # Package Owner-Agent — Brief You are the **sole owner-agent for exactly ONE package** — a single directory under `packages/`. -Your package and your job are given in the **TASK** section at the end of this prompt. You build -it, test it, and write a report — nothing else. If no single package is named, stop and say so. +Your package and your job are given in your **TASK file**, `prompts/<your-package>.md` (the last +file you were told to read). You build it, test it, and write a report — nothing else. If no +single package is named, stop and say so. ## Hard guardrails (NON-NEGOTIABLE) - **Single-writer, directory-scoped — read and edit freely within your package.** Your unit is @@ -31,11 +32,10 @@ it, test it, and write a report — nothing else. If no single package is named, exports (and manifest, if any). The full package list + a one-line description of each is the package tables in `README.md`. Don't read unrelated packages' internals. -## Headless read boundary (you run non-interactively) -You run HEADLESS: a Read of any file OUTSIDE this repo triggers a permission prompt that -CANNOT be answered → the run HANGS until aborted. Read ONLY within this repo. If you believe -you need a file outside it, do NOT attempt the read — STOP and write the need in your report, -then end. +## Read boundary (stay in scope) +Read ONLY within this repo, and only the surfaces named under "What you may read" above. Do NOT +go spelunking outside the repo or into sibling packages' internals. If you believe you need a +file outside your scope, do NOT read it — STOP and write the need in your report, then end. ## Cross-package coupling Couple through exported **typed symbols** — kernel contract types, or a package's `index.ts` @@ -43,7 +43,7 @@ exports. A package that is a **library** is itself a sanctioned shared surface ( it). No string-keyed lookups into another feature's internals. ## Engineering standard -The authoritative rules (`.dispatch/rules/*`, inlined into this prompt) govern. In brief: +The authoritative rules (the `.dispatch/rules/*` files you were told to read) govern. In brief: - **Pure core / injected shell.** Decision logic is `input → output`: zero I/O, no ambient state, no singletons. Effects (fs, db, network, shell, clock, random) are **injected** at the edges. Put the pure part in its own module so it tests without mocks. @@ -55,7 +55,7 @@ The authoritative rules (`.dispatch/rules/*`, inlined into this prompt) govern. module wired between features. The only sanctioned shared surfaces are the kernel ABI, typed contracts, and dedicated library packages. - **Strict TS.** Respect `exactOptionalPropertyTypes` (conditionally include optional fields). -- **Biome is zero-tolerance** (`.dispatch/rules/biome-clean.md`, inlined). `bunx biome check` must +- **Biome is zero-tolerance** (`.dispatch/rules/biome-clean.md`). `bunx biome check` must end with ZERO warnings AND ZERO infos — not merely zero errors. Fix the code; never `// biome-ignore` or relax config. @@ -76,4 +76,4 @@ The orchestrator runs the authoritative full-graph `typecheck` / `test` / `check 4. **Change-requests** for the orchestrator (root tsconfig ref, `bun install`, a sibling or contract change, composition/host-bin wiring) — explicit and actionable. -Your specific **TASK** follows at the end of this prompt. +Your specific **TASK** is in `prompts/<your-package>.md` — read it last and execute it. diff --git a/.skills/ORCHESTRATOR.md b/.skills/ORCHESTRATOR.md new file mode 100644 index 0000000..620b983 --- /dev/null +++ b/.skills/ORCHESTRATOR.md @@ -0,0 +1,441 @@ +# ORCHESTRATOR.md — how to drive this project + +> **You are the orchestrator.** You do NOT write feature code yourself. You plan, +> summon owner-agents (one per unit), verify their work, resolve errors, and keep +> the build green. This file is your complete operating manual. Read it fully +> before acting. Also read: `AGENTS.md` (the subagent constitution — you enforce +> it), `GLOSSARY.md`, `.dispatch/rules/`, `tasks.md` (live progress), and +> `notes/restructure-plan.md` (the full design + rationale; §-refs below point +> into it). + +--- + +## 0. Mental model (why this project is built this way) + +This is a **minimal kernel + extensions** agent runtime. Every feature is an +extension. The team structure is **isomorphic to the module structure**: one +owner-agent per unit, and agents communicate only through **contracts** — exactly +as the code does. Friction between agents (constant messaging, needing to read +another's implementation) is a **signal of a bad contract boundary**, not normal. + +This is a synthesis of "The AI Harness" +(https://dev.to/louaiboumediene/the-ai-harness-why-your-ai-coding-agent-is-only-as-smart-as-the-repo-you-put-it-in-cml) +with our own design. The harness layers we use: +- **Constitution** (`AGENTS.md`) — loaded by every agent. Non-obvious, project- + specific rules only. +- **Safety reflexes** (`.dispatch/rules/*.md`) — tiny, crystallized scar tissue. +- **Glossary** (`GLOSSARY.md`) — one canonical name per concept. +- **This file** — the orchestrator's workflow (the article doesn't cover this; we + added it). +- **Scoped knowledge** — rules/prompts are scoped to the *kind* of agent and the + *layer* it works in (strict for kernel/pure-core, lenient for the shell). The + article's key lesson: **scoped rules beat general rules; never write down what a + frontier model already knows** (P6). + +The 8 principles (P1–P8) live in `notes/restructure-plan.md` §1. Internalize them; +they justify every rule below. + +--- + +## 1. The golden workflow (build/modify a feature) + +1. **Plan.** Decide the unit(s); split into dependency-topological **waves** of + disjoint units, and WIDEN each wave where you can (§2a). One agent owns one unit; + it may ONLY edit its assigned files. +2. **Overlap check FIRST (anti-synonym-drift, §5.6).** Before creating anything + new, check `GLOSSARY.md` + existing code. If the request *describes* an + existing concept under a new name, steer to the canonical term (e.g. + "web-notifier" → that's a `webhook`). New term? Propose the standard/training- + baked name and **ask the user** before adding it to the glossary. Never coin a + term silently. +3. **Boundary decision is the USER's (§5.2).** "New extension vs. extend an + existing one?" — surface it to the user; never decide granularity silently. +4. **Write the prompt** to `prompts/<unit>.md` (gitignored). See §3 for the + prompt recipe. +5. **Summon the wave** via `opencode run` (see §2); disjoint units run in PARALLEL + (§2a). RE-READ `.dispatch/rules/` + the §3 scoping map before each wave — assemble + from the files, not from memory. +6. **Verify** the reports + independently re-run checks (see §4). Trust nothing + until you've re-run `typecheck`/`test`/`check` yourself. +7. **Resolve** any contract gaps / errors (see §5). +8. **Commit** the milestone with a clear message + test count. Update `tasks.md`. + +--- + +## 2. Summoning agents via `opencode run` (the harness) + +OpenCode CLI is the summon mechanism (see `notes/opencode-agents.md`). + +**Working dir:** always the repo root, +`/home/tradam/projects/dispatch/arch-rewrite` (so the agents' `lsp` tool works — +TS language server is configured globally). + +**Model:** use `opencode-go/mimo-v2.5-pro` for BUILDING agents (capable coder). +`deepseek-v4-flash` is reserved as the *app's own runtime testbench*, not for +building. + +**Canonical invocation** — assemble the prompt by CONCATENATING the standardized briefs + the +scoped rules + the per-summon TASK. The invariant guardrails live ONCE in the briefs, so +`prompts/<unit>.md` is now JUST the TASK block (§3). Do NOT use `-f` (see gotcha); ALWAYS +redirect output to a file. +```bash +cd /home/tradam/projects/dispatch/arch-rewrite && \ +opencode run --dir /home/tradam/projects/dispatch/arch-rewrite \ + -m opencode-go/mimo-v2.5-pro \ + "$(cat .dispatch/package-agent.md) +$(cat .dispatch/extension-agent.md) +$(cat .dispatch/rules/one-owner.md .dispatch/rules/isolation-over-dry.md .dispatch/rules/biome-clean.md .dispatch/rules/pure-core.md .dispatch/rules/no-internal-mocks.md .dispatch/rules/typed-handles.md) + +## TASK +$(cat prompts/<unit>.md)" \ + > reports/<unit>.run.log 2>&1 +``` +**Assembly order is fixed: package brief → extension supplement → scoped rules → TASK** +(the supplement references "the package brief above"; the briefs reference "rules inlined into +this prompt"). Rules: +- **Non-extension package?** OMIT the `.dispatch/extension-agent.md` line. +- Inline ONLY the scoped rules matching the unit's layer (the §3 map) — not every rule on every agent. +- `AGENTS.md` is auto-loaded by opencode — never `cat` it. +- The briefs already instruct the agent on ownership, visibility, verify, and the report; the + TASK block must NOT repeat any of that. + +**MANDATORY — capture output to a file, never display it.** The agent's streamed +output is enormous and will overwhelm and CRASH this harness if it lands in your +terminal. ALWAYS redirect the summon's stdout+stderr to a log file (e.g. +`> reports/<unit>.run.log 2>&1`) and do NOT echo/`cat` that log back wholesale. +You don't need the raw stream: read the agent's `reports/<unit>.md` report (and, +if you must, `grep`/`tail` the log for a specific error). Treat dumping a full run +log into context as a hard failure. + +**Run discipline (from the tool harness):** +- **Do NOT background it. Use a large timeout** (e.g. 1800000 ms = 30 min) — these + are long tasks. Backgrounding loses the stream. +- One non-backgrounded `run_shell` per summon. For PARALLEL agents on disjoint + files, launch multiple summons (the harness allows concurrent tool calls) — but + ONLY when their file sets do not overlap (single-writer rule). Log parallel runs + in `tasks.md`. + +**GOTCHAS (learned the hard way):** +- **Headless cross-`--dir` read = HANG.** An agent's Read of any file OUTSIDE its + `--dir` triggers an interactive permission prompt that CANNOT be answered headlessly + → the run wedges until aborted. This bites CROSS-REPO: a `file:` dep symlink (e.g. + `dispatch-web/node_modules/@dispatch/ui-contract` → the sibling repo) resolves OUTSIDE + `--dir`, so an agent reading the dep's source hangs. Fixes: (a) keep everything the + agent must READ inside `--dir` — ship an **in-repo reference snapshot** of a cross-repo + contract and FORBID reading `node_modules/@dispatch/*`; OR (b) set `--dir` to a parent + containing all needed paths — but then the repo's `AGENTS.md` won't auto-load (you lose + the constitution). The briefs now tell agents: never read outside your scope — if you + think you need to, REPORT it and STOP, never attempt the read. +- `-f/--file` is an ARRAY flag and greedily eats your trailing message as another + filename → "File not found". **Inline with `"$(cat prompts/X.md)"` instead.** +- A quick smoke test works: `opencode run -m opencode-go/mimo-v2.5-pro "Reply with + exactly SMOKE_OK"` should print `SMOKE_OK`. +- `opencode models` lists models; `opencode agent list` lists agent profiles; + `opencode run --help` for flags. + +--- + +## 2a. Parallel execution — WAVES + +Throughput comes from running disjoint units at once. Organise it as waves: +- **A wave = units that (a) touch DISJOINT files and (b) have no compile-time dependency + on each other** (each imports only already-built packages + existing contracts). Launch a + wave by emitting one summon per unit as CONCURRENT tool calls (§2). Later waves depend on + earlier ones; the composition root (`packages/host-bin/`) is almost always the LAST wave. +- **Pre-author the seam to widen the wave.** Because the orchestrator OWNS contracts (§6), + author the shared contract / typed handle in `packages/kernel/src/contracts/*` FIRST, then + summon the producer AND the consumer in the SAME wave against that fixed type — neither needs + the other's implementation. Authoring the contract up front is what turns a sequential + producer→consumer chain into one parallel wave (and `lsp references` on the new symbol gives + the exact consumer set to summon). +- **Also widen by removing edges:** prefer a consumer-defined handle the producer implements, + or a generic utility over a feature-specific one, so a dependency disappears entirely. +- **One writer per file, always** — even across waves. If two units would edit the same file, + they are NOT separable; merge them into one unit or sequence them. +- **After a wave:** read every report, run the §4 checks ONCE for the whole wave, commit the + milestone (update `tasks.md`), then start the next wave. Don't open a new wave before the + prior one is green. + +--- + +## 3. The per-summon `prompts/<unit>.md` is JUST the TASK block + +The invariant guardrails — single-writer directory ownership, visibility, coupling, the +engineering standard, isolated verification, and the report format — live ONCE in the +standardized briefs the summon concatenates (§2): +- **`.dispatch/package-agent.md`** — the base for EVERY package owner. +- **`.dispatch/extension-agent.md`** — the extension-only supplement (added for extension summons). + +So `prompts/<unit>.md` no longer restates any of that. It contains ONLY the **TASK**: +1. **Your package:** `packages/<name>/` — name the WHAT, not the files (the owner owns the whole + directory and decides which files to touch). +2. **The job + algorithm**, naming the specific contract types/handles involved. +3. **The specific contract file(s)** to read (e.g. `packages/kernel/src/contracts/<x>.ts`) and + any sibling public surfaces it consumes. +4. **The required test cases** (named). + +Keep it scoped (P6): state only the project-specific, non-inferable task — the briefs carry the rest. + +**`.dispatch/rules/` scoping map** — include ONLY the rows matching the unit (per §0 +"scoped rules beat general rules"); do NOT dump every rule on every agent: +- **Every agent:** `one-owner.md`, `isolation-over-dry.md`, `biome-clean.md`. +- **Kernel unit:** `kernel-purity.md` + `pure-core.md` + `no-internal-mocks.md`. +- **Pure-core unit:** `pure-core.md` + `no-internal-mocks.md`. +- **Any extension coupling via hooks/services:** `typed-handles.md`. +- **Every extension (≈ all of them — they all log):** `extension-logging.md`. Use the + injected `host.logger`/`ctx.log`; keystone: each extension self-redacts its OWN secrets + in its OWN code — NO shared redaction helper (design rationale: + `notes/observability-design.md` §9). Include this on EVERY extension summon (an + extension that never logs is a coverage gap, not an exemption). +- **Frontend units** are summoned from the SEPARATE `../dispatch-web` repo using ITS + OWN harness (`package-agent.md` + `frontend-*.md` rules) + ITS OWN scoping map — NOT + these backend rules. See that repo's `ORCHESTRATOR.md`. + +**Tell each agent it has company (parallel waves).** Add to each wave TASK: sibling units are +being built in OTHER packages right now; `tsc -b`/vitest/biome are whole-PROJECT, so if a check +reports errors OUTSIDE your package, that's concurrent WIP — ignore it and ensure YOUR files are +clean. The orchestrator's post-wave run (§4) is the source of truth. + +**Make agents IMPLEMENT, not deliberate.** A summoned owner must edit files + run its checks + +write its report in the one run. If a summon returns only a plan, re-summon (§5a). + +--- + +## 4. Verification (the orchestrator's trust protocol) + +**Plan principle (§3.6 / §5 last row):** the orchestrator confirms work from +**contracts + test results + build/diagnostics output** — that is the *designed* +trust mechanism, and it works precisely because the boundaries are testable. The +tests-at-boundaries ARE how you trust a unit without depending on its internals. + +**Stay out of implementation files (§6 Visibility).** Your trust signals are the +agent's report, the contract/surface it exposes (contracts, manifests, public +types), and the build/test/lint output you re-run yourself — NOT its implementation +code. Do NOT open an extension's implementation files — not even to "skim", +double-check, or diagnose a bug. **There is NO "conflict exception."** When X and Y +don't work together, or a unit is broken, you diagnose from the `typecheck`/`test` +output + `lsp references` on the contract + the agent's report, then **summon the +owning agent** (or a temporary multi-knowledge agent, §5) to read its own code and +fix it. You diagnose from symptoms; the agent reads the code. + +After every agent, independently: +```bash +cd /home/tradam/projects/dispatch/arch-rewrite +bun run typecheck # tsc -b --pretty — must be clean (EXIT 0) +bun run test # vitest — note the pass count +bun run check # biome — must be clean +git status --short # confirm the agent stayed in its lane (no out-of-scope edits) +``` +- **Read ONLY the surfaces** (the contracts/hooks/public signatures the unit + exposes), not its implementation files — unless an implementation conflict or + trouble forces you in (§6). The surface plus green checks is enough to trust a + unit; subtle contract mistakes show up at the boundary, which is what the + contract + boundary tests are for. +- Confirm the agent touched ONLY its assigned files (one-owner rule). +- For pure units, confirm tests use NO internal `vi.mock("@dispatch/*")`. + +**Concurrency caveat (parallel waves):** `tsc -b`/vitest/biome are whole-project, so an agent's +OWN mid-wave check can transiently see a sibling's half-written file. Don't act on a report's +out-of-package errors; YOUR post-wave run is authoritative. Re-run a suite that depends on shared +external state before trusting it — and ALWAYS sweep leaked server/collector processes between +live runs (§8 bracket trick), since a leak silently poisons the next run's counts. + +--- + +## 5. Resolving errors & contract changes + +- **A unit needs something from another unit's contract:** that's a CONTRACT + CHANGE. The owner of the contract makes it. To find every consumer, use + `lsp references` on the changed exported symbol (contracts are static TS types, + so this returns the TRUE blast radius — §5.3). Then summon the affected owners + to update. The orchestrator dispatches this fan-out; agents don't reach across. +- **Integration bug (X and Y each honor the contract but don't work together):** + no single file owns it. Summon a **temporary multi-knowledge agent** with + read/write to the 2–3 relevant files (it MAY see implementation — exception to + the visibility rule), as their temporary exclusive owner. Dispatch proactively, + or when a file-owner requests it (§5.5). +- **CR (change-request) in a report:** if it's **build/config** (root + `tsconfig.json` ref, a `package.json` dep, `.gitignore`, `bun.lock`) the + orchestrator edits it directly, then re-verifies. If it's **implementation** (a + barrel `index.ts`, a sibling conforming to a contract change), the orchestrator + **summons the owning agent** — it does NOT edit implementation itself. +- **Live API errors:** an HTTP 429 `GoUsageLimitError` is an UPSTREAM rate limit, + not a bug. The `opencode-2` key has a monthly cap; `opencode-1` is the backup. + Swap `DISPATCH_API_KEY` in `.env` (both keys are there). + +--- + +## 5a. Agent-failure recovery patterns + +- **Plan-only / "shall I proceed?" agent.** A summon sometimes returns a PLAN and STOPS without + editing (no diff, no `reports/<unit>.md`). Detect via `git status` + the missing report. + Re-summon the SAME TASK prefixed: "IMPLEMENT THIS NOW — make all edits, run the checks, write + the report; do not stop to plan or ask." Don't hand-fix its work. +- **A behaviour change reds a SIBLING's tests (test fan-out).** When a unit's new behaviour + invalidates another unit's test ASSERTIONS, those tests belong to that OTHER owner — summon it + with a focused "fix these N failing tests to match the new behaviour" TASK (state the + behaviour). The orchestrator never edits feature tests itself. (Distinct from an INTEGRATION + bug where neither side is wrong — that's the temporary multi-knowledge agent in §5.) +- **Agent strayed out of its lane.** `git status --short` after every wave; if an agent touched a + file outside its package, keep it ONLY if it's legitimately the orchestrator's lane + (contracts / build / config / harness, §6) and note it — otherwise revert + re-summon with a + tighter scope. +- **Flaky green.** A wave that passes once but leaked a server/collector or relies on shared + external state can pass for the wrong reason; sweep (§8) and re-run before committing. + +--- + +## 6. Restrictions & invariants (NEVER violate) + +- **Single-writer:** never let two agents edit the same file concurrently. +- **Kernel purity:** no I/O / no concrete feature names in `packages/kernel` + (`.dispatch/rules/kernel-purity.md`). +- **Visibility rule (§5.1):** agents see only other units' CONTRACTS, never their + implementation. A contract documents **behavior & guarantees a consumer can + rely on, not just types** (P6 applied to contracts). An agent *needing* to read + another unit's code is a signal that contract is underspecified — fix the + contract, don't grant code access. (Exception: the temporary multi-knowledge + integration agent, §5 / ORCHESTRATOR §5, which MAY read implementation.) +- **The orchestrator NEVER reads or edits implementation.** You read ONLY contracts + (`packages/kernel/src/contracts/*`) + surfaces (manifests, public signatures) + + diagnostics (`typecheck`/`test` output, `lsp references` on contract symbols) + + agent reports. Do NOT open implementation `.ts` files (feature logic, tests, + composition roots) — not even during a bug. Clean context = level-headed + decisions; the subagents do the implementation. +- **What the orchestrator MAY edit directly:** (a) **contracts** + (`packages/kernel/src/contracts/*`); (b) **build wiring + config** (root/package + `tsconfig.json`, `package.json` deps, project refs, `.gitignore`, `bun.lock`); + (c) **harness/docs** (`ORCHESTRATOR.md`, `AGENTS.md`, `GLOSSARY.md`, + `.dispatch/rules/`, `notes/`, `tasks.md`, `prompts/`, `reports/`). Everything else + — all executable implementation `.ts`, including tests and composition roots like + `host-bin/src/main.ts` — changes ONLY by summoning the owning agent. +- **Roadblock → surface to the user.** If a needed change doesn't fit the above + (ambiguous ownership, a design question, a stuck agent), stop and ask rather than + reaching into implementation. +- **Subagents inherit this restriction.** Every prompt you write must instruct the + agent to read ONLY the surfaces (contracts/hooks) of OTHER units, with the sole + exception that it MAY read the implementation files of the task/extension it is + assigned to. It must not go spelunking through sibling units' implementations. +- **`onAny` is the ONLY allowed dynamic hook subscription** (observability/logging + firehose). All other cross-extension coupling is typed-symbol anchored (§5.4). +- **Contracts are static TYPES; loading is dynamic** (manifests via host). This + split is load-bearing — it's what makes `lsp references` fan-out work. +- **Full fidelity:** every core feature is a real extension with a manifest, + loaded through the host. Do NOT hand-wire imports to shortcut the extension + model — that defeats the point. +- **Asymmetric testing:** strict (zero internal mocks, high coverage) on + kernel/pure-core; lenient (thin integration tests) on the shell. +- **Destructive git ops:** be extremely careful. Back up irreplaceable files + (e.g. `notes/restructure-plan.md`) to `/tmp` before any `git reset --hard` / + `git clean`. `.env` is gitignored — preserve it. +- **Write things up before pivoting topics.** Keep `tasks.md` current in real + time. Don't leave decisions only in chat context (it can be lost). + +--- + +## 7. Repo geography + +``` +/home/tradam/projects/dispatch/arch-rewrite # THE worktree (branch arch/rewrite) + + AGENTS.md the subagent constitution (auto-loaded by opencode; you enforce it) + ORCHESTRATOR.md the orchestrator's operating manual (this file) + GLOSSARY.md canonical vocabulary + aliases-to-avoid (human-gated) + tasks.md live progress checklist / milestone log + README.md deployment, CLI usage, extension/package tables + + .dispatch/ + package-agent.md base owner-agent brief (every summon) + extension-agent.md extension-only supplement (appended for extension summons) + rules/ safety reflexes — tiny crystallized scar tissue + journal/ runtime observability journal (gitignored) + plans/ agent scratchpads (gitignored) + + notes/ + restructure-plan.md the full architecture design + rationale (P1–P8; §-refs) + observability-design.md logging/spans/collector/trace-store design (Phase A–B) + cli-design.md CLI design decisions + unit plan (built; §3 = settled decisions) + frontend-design.md future web frontend design (IDEATION; separate repo) + opencode-agents.md notes on summoning agents via the opencode CLI + + prompts/ (gitignored — orchestrator→agent TASK blocks) + reports/ (gitignored — agent→orchestrator reports) + .env (gitignored — DISPATCH_API_KEY, DISPATCH_BASE_URL, DISPATCH_MODEL, BACKEND_PORT) + + packages/ + kernel/ contracts (ABI), bus, runtime (runTurn), host + wire/ types-only wire ABI (AgentEvent + conversation model + Usage); kernel + + transport-contract re-export it so clients consume the wire w/o the kernel runtime + transport-contract/ types-only HTTP API contract (CLI + future web + server share it) + ui-contract/ types-only surface ABI (frontend-agnostic; web + CLI render it) + storage-sqlite/ conversation-store/ auth-apikey/ provider-openai-compat/ + credential-store/ named credentials + model catalog (resolve / listCatalog) + session-orchestrator/ transport-http/ (core extensions) + tool-read-file/ standard tool extension (read_file; cwd-aware) + journal-sink/ trace-store/ observability-collector/ trace-replay/ (observability) + cli/ bundled one-shot terminal client (HTTP client of transport-contract) + host-bin/ composition root (boot + Bun.serve + collector supervisor) +``` + +The genesis commit deleted all prior source; we rebuilt from scratch. The OLD +project lives at `/home/tradam/projects/dispatch/dispatch-source` (reference only +— do not edit). + +The **web frontend is a SEPARATE repo** at `/home/tradam/projects/dispatch/dispatch-web` +(own git, own harness — its own `AGENTS.md`/`ORCHESTRATOR.md`/`GLOSSARY.md`/`.dispatch/`). +It consumes `packages/ui-contract` + the wire types as a pinned `file:` dependency. +`lsp references` does NOT span the two repos, so cross-repo contract changes are +**couriered via the user** (see the FE `ORCHESTRATOR.md` §5). Design + plan: +`notes/frontend-design.md`. Do NOT edit the FE repo from here. + +--- + +## 8. Current status & how to run + +See `tasks.md` for the live checklist. As of MVP completion: +- Kernel + 6 core extensions + host-bin DONE. 178 tests pass; typecheck + biome + clean. +- **MVP verified live:** multi-turn curl against OpenCode Go flash works + (`conversationId` threads history). + +**Boot + smoke test:** +```bash +cd /home/tradam/projects/dispatch/arch-rewrite +KEY1=$(grep DISPATCH_API_KEY_OPENCODE1 .env | cut -d= -f2) +PORT=4567 DISPATCH_API_KEY="$KEY1" bun packages/host-bin/src/main.ts # boots server +# in another shell: +curl -s -X POST localhost:4567/chat -H 'content-type: application/json' \ + -d '{"conversationId":"c1","message":"Say hello in 3 words."}' +``` +Note the chat field is **`conversationId`** (threads multi-turn), not `tabId`. + +**Live validation & process cleanup — the `[x]` bracket trick (scar tissue).** When +you live-validate you background the app (`bun packages/host-bin/src/main.ts &`), and it +now spawns a child **observability collector** process. To list or kill those, ALWAYS +use the bracket trick in `ps`/`pgrep`/`pkill` patterns: +```bash +ps -eo pid,args | grep '[o]bservability-collector/src/main' # list (won't self-match) +pkill -9 -f '[h]ost-bin/src/main.ts' # kill the app +pkill -9 -f '[o]bservability-collector/src/main' # kill the collector +``` +**Why it matters:** a plain `pkill -f 'host-bin/src/main.ts'` matches its OWN command +line and kills the parent shell → the tool call prints NOTHING and times out, looking +exactly like a wedged session. `[h]ost-bin` matches the target "host-bin" while the +literal pattern `[h]ost-bin` does not match itself. ALWAYS clean up the backgrounded app ++ its spawned collector after each live run — leaked processes pollute the next run's +counts (this is precisely what made a correct supervisor look like it spawned 3 +collectors and left 2 behind). + +**Live boot-probe in ONE command WILL hit the tool timeout — that is NOT failure (scar tissue).** +A single bash command that boots the app (even detached via `setsid … & disown`), sleeps, runs a +probe, then kills it will still run to the tool's timeout: the tool waits on the spawned +server/collector session. The probe already ran — **read the probe's printed `RESULT: OK/FAIL` +line as the signal**, ignore the timeout, then run a SEPARATE `pkill` (bracket-trick) + `ps` +cleanup command (it returns immediately and confirms no leaks). Don't try to make the boot+probe +command "return cleanly" — it won't. (For a frontend-agnostic surface, the probe is a tiny +`bun` WebSocket client that asserts `catalog → subscribe → surface`.) + +**Next suggested work** (post-MVP, see `tasks.md` "Open items"): wire +auth→provider properly (auth-apikey is currently vestigial), then add the first +TOOL extension to exercise the dispatch loop (turns currently run with `tools: +[]`). @@ -93,9 +93,12 @@ update. Do this as ONE coordinated change; expect ~4–5 files. Update GLOSSARY --- ## Standing reminders (from ORCHESTRATOR.md — don't relearn the hard way) -- Summon with `opencode run -m opencode-go/qwen3.7-max`, **inline the prompt via - `"$(cat prompts/X.md)"`** (the `-f` flag is greedy and breaks). Don't background; - large timeout. +- Summon via the **Task tool** (`subagent_type: "Opus 4.8"`, ORCHESTRATOR §2). The Task + `prompt` is a SHORT pointer that tells the agent to READ the briefs + scoped rules + + `prompts/<unit>.md` ITSELF — **never `Read`/`cat` those files into your own context to + inline them** (that burns your tokens; the whole point of the file harness is to keep the + guardrails in the *subagent's* context, not yours). Parallel wave = multiple Task calls in + ONE message (disjoint file sets only). - **`deepseek-v4-flash` is the app's runtime testbench, NOT for building agents.** - Parallelize ONLY disjoint file sets (single-writer). Log parallel runs in tasks.md. - Verify independently (typecheck/test/check) + confirm single-lane edits. Trust diff --git a/ORCHESTRATOR.md b/ORCHESTRATOR.md index 620b983..15393c7 100644 --- a/ORCHESTRATOR.md +++ b/ORCHESTRATOR.md @@ -52,9 +52,10 @@ they justify every rule below. existing one?" — surface it to the user; never decide granularity silently. 4. **Write the prompt** to `prompts/<unit>.md` (gitignored). See §3 for the prompt recipe. -5. **Summon the wave** via `opencode run` (see §2); disjoint units run in PARALLEL - (§2a). RE-READ `.dispatch/rules/` + the §3 scoping map before each wave — assemble - from the files, not from memory. +5. **Summon the wave** via the Task tool (`subagent_type: "Opus 4.8"`, see §2); disjoint units + run in PARALLEL (§2a). RE-READ the §3 scoping map before each wave so you point each agent at + the right rule files. You MAY read the rules/briefs themselves (they are tiny) — what you must + NEVER do is INLINE their contents into a Task prompt (§2 TOKEN RULE). 6. **Verify** the reports + independently re-run checks (see §4). Trust nothing until you've re-run `typecheck`/`test`/`check` yourself. 7. **Resolve** any contract gaps / errors (see §5). @@ -62,76 +63,77 @@ they justify every rule below. --- -## 2. Summoning agents via `opencode run` (the harness) +## 2. Summoning agents via the Task tool (Opus 4.8) + +The **Task tool** is the summon mechanism; **`subagent_type: "Opus 4.8"`** is the BUILDING +agent (Claude Opus 4.8 — capable coder). `deepseek-v4-flash` remains the *app's own runtime +testbench*, never a builder. (The legacy `opencode run` CLI path is retired; +`notes/opencode-agents.md` is historical.) + +**Same session, full tools.** A Task subagent runs in THIS session's working directory (the repo +root, `/home/tradam/projects/dispatch/arch-rewrite`) with the normal coding toolset +(Read/Edit/Write/Bash/`lsp`) and the same permissions — so its `lsp`/typecheck work and there is +NO headless cross-dir permission hang. The visibility/ownership rules (§6) are NOT enforced by a +sandbox here — they hold only because the prompt states them (the briefs do). + +**THE TOKEN RULE — never INLINE the briefs/rules/TASK into the prompt.** This is the whole point +of the file-based harness: pasting `.dispatch/*` or `prompts/<unit>.md` contents into the Task +`prompt` burns YOUR context and duplicates what the agent can read itself — exactly what we avoid. +(Reading the small rule/brief files for your OWN understanding is allowed — the rule is about +prompt assembly, not about you being ignorant of the rules.) Instead the Task `prompt` +is a SHORT pointer that tells the subagent to READ those files ITSELF. The guardrail bytes then +land in the SUBAGENT's context, never the orchestrator's — the same property the old +`"$(cat …)"` shell-concat gave us (the concat just moves from the shell to the agent's own Read). + +**Canonical summon** — ONE Task call per unit: +- `description`: `"build <unit>"` +- `subagent_type`: `"Opus 4.8"` +- `prompt`: the SHORT pointer below — fill in `<unit>` + its scoped-rule files (§3 map). Do NOT + inline any file contents. -OpenCode CLI is the summon mechanism (see `notes/opencode-agents.md`). - -**Working dir:** always the repo root, -`/home/tradam/projects/dispatch/arch-rewrite` (so the agents' `lsp` tool works — -TS language server is configured globally). - -**Model:** use `opencode-go/mimo-v2.5-pro` for BUILDING agents (capable coder). -`deepseek-v4-flash` is reserved as the *app's own runtime testbench*, not for -building. - -**Canonical invocation** — assemble the prompt by CONCATENATING the standardized briefs + the -scoped rules + the per-summon TASK. The invariant guardrails live ONCE in the briefs, so -`prompts/<unit>.md` is now JUST the TASK block (§3). Do NOT use `-f` (see gotcha); ALWAYS -redirect output to a file. -```bash -cd /home/tradam/projects/dispatch/arch-rewrite && \ -opencode run --dir /home/tradam/projects/dispatch/arch-rewrite \ - -m opencode-go/mimo-v2.5-pro \ - "$(cat .dispatch/package-agent.md) -$(cat .dispatch/extension-agent.md) -$(cat .dispatch/rules/one-owner.md .dispatch/rules/isolation-over-dry.md .dispatch/rules/biome-clean.md .dispatch/rules/pure-core.md .dispatch/rules/no-internal-mocks.md .dispatch/rules/typed-handles.md) - -## TASK -$(cat prompts/<unit>.md)" \ - > reports/<unit>.run.log 2>&1 ``` -**Assembly order is fixed: package brief → extension supplement → scoped rules → TASK** -(the supplement references "the package brief above"; the briefs reference "rules inlined into -this prompt"). Rules: -- **Non-extension package?** OMIT the `.dispatch/extension-agent.md` line. -- Inline ONLY the scoped rules matching the unit's layer (the §3 map) — not every rule on every agent. -- `AGENTS.md` is auto-loaded by opencode — never `cat` it. -- The briefs already instruct the agent on ownership, visibility, verify, and the report; the - TASK block must NOT repeat any of that. - -**MANDATORY — capture output to a file, never display it.** The agent's streamed -output is enormous and will overwhelm and CRASH this harness if it lands in your -terminal. ALWAYS redirect the summon's stdout+stderr to a log file (e.g. -`> reports/<unit>.run.log 2>&1`) and do NOT echo/`cat` that log back wholesale. -You don't need the raw stream: read the agent's `reports/<unit>.md` report (and, -if you must, `grep`/`tail` the log for a specific error). Treat dumping a full run -log into context as a hard failure. - -**Run discipline (from the tool harness):** -- **Do NOT background it. Use a large timeout** (e.g. 1800000 ms = 30 min) — these - are long tasks. Backgrounding loses the stream. -- One non-backgrounded `run_shell` per summon. For PARALLEL agents on disjoint - files, launch multiple summons (the harness allows concurrent tool calls) — but - ONLY when their file sets do not overlap (single-writer rule). Log parallel runs - in `tasks.md`. - -**GOTCHAS (learned the hard way):** -- **Headless cross-`--dir` read = HANG.** An agent's Read of any file OUTSIDE its - `--dir` triggers an interactive permission prompt that CANNOT be answered headlessly - → the run wedges until aborted. This bites CROSS-REPO: a `file:` dep symlink (e.g. - `dispatch-web/node_modules/@dispatch/ui-contract` → the sibling repo) resolves OUTSIDE - `--dir`, so an agent reading the dep's source hangs. Fixes: (a) keep everything the - agent must READ inside `--dir` — ship an **in-repo reference snapshot** of a cross-repo - contract and FORBID reading `node_modules/@dispatch/*`; OR (b) set `--dir` to a parent - containing all needed paths — but then the repo's `AGENTS.md` won't auto-load (you lose - the constitution). The briefs now tell agents: never read outside your scope — if you - think you need to, REPORT it and STOP, never attempt the read. -- `-f/--file` is an ARRAY flag and greedily eats your trailing message as another - filename → "File not found". **Inline with `"$(cat prompts/X.md)"` instead.** -- A quick smoke test works: `opencode run -m opencode-go/mimo-v2.5-pro "Reply with - exactly SMOKE_OK"` should print `SMOKE_OK`. -- `opencode models` lists models; `opencode agent list` lists agent profiles; - `opencode run --help` for flags. +You are the single owner-agent for packages/<unit>/. Read these files IN FULL with your own tools +and follow them exactly, in this order (do NOT skip — they are your constitution, brief, rules, +and task; do NOT paste them back to me): + 1. AGENTS.md (project constitution; may already be in context) + 2. .dispatch/package-agent.md (base owner brief) + 3. .dispatch/extension-agent.md (ONLY if your unit is an extension — else skip) + 4. .dispatch/rules/one-owner.md .dispatch/rules/isolation-over-dry.md .dispatch/rules/biome-clean.md + (+ the unit's other scoped rules per the §3 map — list them here) + 5. prompts/<unit>.md (YOUR task) +Then IMPLEMENT it now: edit ONLY files under packages/<unit>/, run tsc -b / vitest / biome for +your package, and write your report to reports/<unit>.md. +Reply with ONLY a one-line status + the path reports/<unit>.md — no diffs, no logs. +``` + +The read list IS the old fixed assembly order (constitution → package brief → extension supplement +→ scoped rules → TASK); the ONLY change is the AGENT reads it instead of the shell concatenating +it. Name in line 4 ONLY the scoped-rule files matching the unit's layer (§3 map) — don't dump +every rule. `prompts/<unit>.md` stays JUST the TASK block (§3). + +**Output discipline.** The Task tool returns ONE short final message (not the agent's stream), so +the firehose can't flood you. Keep it that way: the agent replies tiny and writes the real report +to `reports/<unit>.md`, which you then `Read` from disk. NEVER ask an agent to paste diffs/logs +back; never `Read` a giant log into context — `Grep` it for a specific error if needed. + +**Parallel waves.** Launch a wave by emitting MULTIPLE Task calls IN ONE message (concurrent) — +one per unit — but ONLY when their file sets are disjoint (single-writer, §6/§2a). Sequence +dependent waves across separate messages (later waves compile against earlier ones). Log parallel +runs in `tasks.md`. No timeout/backgrounding knobs to manage — the Task tool handles long runs and +notifies on completion. + +**GOTCHAS:** +- **Don't burn your own tokens.** Re-read THE TOKEN RULE above: point the agent at the files; + never inline them. The biggest failure mode here is the orchestrator pasting briefs/rules/ + prompts into a fat Task prompt — that defeats the harness. (Reading them yourself is fine.) +- **No sandbox = state the rules.** Because the subagent shares your tools/cwd, nothing stops it + editing out of its lane except the prompt. Always include the ownership/visibility briefs (they + tell the agent: never edit outside `packages/<unit>/`; read only OTHER units' contracts; if you + think you must read another unit's impl, REPORT and STOP). +- **Make agents IMPLEMENT, not deliberate** (§3): the pointer says "IMPLEMENT it now … write the + report". A plan-only return → re-summon (§5a). +- **Smoke check:** `Task(subagent_type:"Opus 4.8", prompt:"Reply with exactly SMOKE_OK")` should + return `SMOKE_OK`. --- @@ -162,7 +164,7 @@ Throughput comes from running disjoint units at once. Organise it as waves: The invariant guardrails — single-writer directory ownership, visibility, coupling, the engineering standard, isolated verification, and the report format — live ONCE in the -standardized briefs the summon concatenates (§2): +standardized briefs the summon points the agent at (§2; the agent reads them itself): - **`.dispatch/package-agent.md`** — the base for EVERY package owner. - **`.dispatch/extension-agent.md`** — the extension-only supplement (added for extension summons). @@ -356,7 +358,7 @@ live runs (§8 bracket trick), since a leak silently poisons the next run's coun observability-design.md logging/spans/collector/trace-store design (Phase A–B) cli-design.md CLI design decisions + unit plan (built; §3 = settled decisions) frontend-design.md future web frontend design (IDEATION; separate repo) - opencode-agents.md notes on summoning agents via the opencode CLI + opencode-agents.md LEGACY — notes on the retired `opencode run` CLI summon path (§2 now uses the Task tool / Opus 4.8) prompts/ (gitignored — orchestrator→agent TASK blocks) reports/ (gitignored — agent→orchestrator reports) @@ -390,19 +392,28 @@ It consumes `packages/ui-contract` + the wire types as a pinned `file:` dependen --- -## 8. Current status & how to run +## 8. How to run (live validation) -See `tasks.md` for the live checklist. As of MVP completion: -- Kernel + 6 core extensions + host-bin DONE. 178 tests pass; typecheck + biome - clean. -- **MVP verified live:** multi-turn curl against OpenCode Go flash works - (`conversationId` threads history). +> This file is instructions and rules ONLY — it carries NO project state. Current +> status, test counts, and next work live EXCLUSIVELY in `tasks.md`. Never record +> status here; it drifts. -**Boot + smoke test:** +**Prefer the already-running dev stack.** If `bin/up` (`:24203`) or `../bin/up2` (`:25203`) +is up, probe THAT for read-only checks (GETs, validation 400s) instead of booting your own +instance — or ask the user. Boot a private instance only when the probe must WRITE. + +**Boot + smoke test (private instance):** `.env` is auto-loaded by Bun and already carries +`DISPATCH_API_KEY` — do NOT re-export it (an empty/botched export OVERRIDES `.env` and the +provider silently fails to register: "No providers registered"). `.env` also pins +`BACKEND_PORT`, which beats `PORT` — so set `BACKEND_PORT` explicitly, and ISOLATE the data +paths or you'll share SQLite files + spawn a duplicate collector against the dev stack: ```bash cd /home/tradam/projects/dispatch/arch-rewrite -KEY1=$(grep DISPATCH_API_KEY_OPENCODE1 .env | cut -d= -f2) -PORT=4567 DISPATCH_API_KEY="$KEY1" bun packages/host-bin/src/main.ts # boots server +BACKEND_PORT=4567 SURFACE_WS_PORT=4569 \ + DISPATCH_DB=/tmp/opencode/probe/dispatch.db \ + DISPATCH_TRACE_DB=/tmp/opencode/probe/traces.db \ + DISPATCH_JOURNAL=/tmp/opencode/probe/app.ndjson \ + bun packages/host-bin/src/main.ts # boots server (mkdir -p /tmp/opencode/probe first) # in another shell: curl -s -X POST localhost:4567/chat -H 'content-type: application/json' \ -d '{"conversationId":"c1","message":"Say hello in 3 words."}' @@ -434,8 +445,3 @@ line as the signal**, ignore the timeout, then run a SEPARATE `pkill` (bracket-t cleanup command (it returns immediately and confirms no leaks). Don't try to make the boot+probe command "return cleanly" — it won't. (For a frontend-agnostic surface, the probe is a tiny `bun` WebSocket client that asserts `catalog → subscribe → surface`.) - -**Next suggested work** (post-MVP, see `tasks.md` "Open items"): wire -auth→provider properly (auth-apikey is currently vestigial), then add the first -TOOL extension to exercise the dispatch loop (turns currently run with `tools: -[]`). |
