| Age | Commit message (Collapse) | Author |
|---|
|
log scope)
Make transport-http a full-fidelity extension that runs its own Bun.serve
inside activate(host) — symmetric with transport-ws. The Hono app is now built
with the extension-scoped host, so all HTTP edge logs are correctly attributed
extensionId=transport-http instead of the host-bin __host__ scope (verified
live in the journal).
- transport-http: createTransportHttpExtension() factory; activate builds the
app + Bun.serve, reads host.config httpPort (?? 24203); deactivate stops it.
- host-bin: drops the HTTP Bun.serve + createServer call; config.ts maps
BACKEND_PORT/PORT -> httpPort. host-bin now serves no transport (both
transports self-serve); boot log -> 'Dispatch booted'.
- +5 bun lifecycle tests wired into test:bun.
No contract change (composition wiring). Verified live: HTTP serves on :24203;
journal edge logs now scoped transport-http. typecheck clean, 498 vitest + 89
bun, biome clean.
|
|
gap #2)
Both HTTP + WS transport edges now emit structured logs via the injected
logger (D7-compliant: no per-AgentEvent/chat.delta frame logging). Verified
live — the journal contains the edge records.
- transport-ws: connection open/close (debug), chat.send accepted (info),
surface-op + malformed-chat.send (warn), abort-on-close (debug). +4 bun tests.
Correctly scoped extensionId=transport-ws (owns its Bun.serve).
- transport-http: /chat accepted (info) / 400 (warn) / turn-failure (error),
GET /conversations read (info), /models + store failure (error). +4 vitest.
Known follow-up: transport-http edge logs are attributed to '__host__' (not
'transport-http') because host-bin runs the HTTP server via createServer(getHostAPI())
rather than the extension owning its Bun.serve. Logs are captured + correlated;
only the per-extension filter is mis-scoped. Tracked in tasks.md.
typecheck clean, 498 vitest + 84 bun, biome clean.
|
|
The .dispatch/rules/extension-logging.md rule was '(pending)' in ORCHESTRATOR
§3 for the entire life of the observability substrate, so every extension
summon was built without logging/self-redaction guidance — leaving most
extensions silent (a coverage audit found conversation-store, transport-http,
credential-store, tool-read-file, storage-sqlite, auth-apikey, surface-* all
with zero logger refs).
- Author .dispatch/rules/extension-logging.md (tribal-knowledge only, P6/P7):
self-redact your own secrets in your own code (no shared helper; §6 tiers),
use injected host.logger/ctx.log, flat scalar attrs, no token-delta logging,
one-way logs, edge verbatim capture.
- Wire it into ORCHESTRATOR §3 as 'every extension' — include on EVERY
extension summon; remove the (pending) note.
- Record the coverage audit + remaining instrumentation debt (#1 reconcile.repair
span in conversation-store, #2 transport-edge logging) in tasks.md.
Future extensions now get logging by construction.
|
|
Close a gap found live: neither transport emitted turn-start/done/turn-sealed
(the wire defined them; nothing fired them). turn-sealed is the FE's
cache-commit signal (frontend-design §6.3); done ends the stream.
- kernel-runtime: runTurn emits turn-start first and done (with finishReason)
last, on every exit path (stop/tool-calls/max-steps/error/aborted).
- session-orchestrator: emits turn-sealed after conversationStore.append
succeeds (the kernel touches no DB, so the post-persist seal is the
orchestrator's). Not emitted if append throws.
No contract change (all three wire types already existed). Verified live: HTTP
/chat and WS chat both stream turn-start … done turn-sealed.
typecheck clean, 494 vitest + 80 bun, biome clean.
|
|
Add chat WS ops (chat.send / chat.delta / chat.error) + unified
WsClientMessage/WsServerMessage unions to @dispatch/transport-contract
(imports ui-contract; surface protocol unchanged — additive non-colliding
type variants, no channel wrapper). transport-ws drives
sessionOrchestrator.handleMessage, streaming each AgentEvent as chat.delta
over the same connection that carries surface ops; per-connection
AbortController cancels in-flight turns on socket close; error-isolated.
Verified live: one WS connection delivered the surface catalog AND a real
flash chat turn (chat.delta stream, reply 'Hello my friend').
Completes the FE Slice 2 backend prereqs. typecheck clean, 485 vitest + 80 bun,
biome clean.
Discovered (separate, pre-existing): runtime does not emit
turn-start/done/turn-sealed on either transport — needed for FE cache-commit;
tracked in tasks.md.
|
|
endpoint
Incremental rehydration endpoint for long-lived clients. Returns
ConversationHistoryResponse { chunks: StoredChunk[], latestSeq } — the RAW,
append-order, seq-filtered slice from conversation-store.loadSince, NOT
reconciled (reconcile conflicts with the per-chunk seq cursor, so it stays on
the turn path; the read path is a pure sync primitive).
- transport-contract: add ConversationHistoryResponse + StoredChunk re-export.
- transport-http: GET /conversations/:id route reaching the log directly via
conversationStoreHandle (dependsOn conversation-store); pure parseSinceSeq
(absent->0, invalid->400).
- build wiring: conversation-store dep + project ref.
FE Slice 2 backend prereq (read-side). typecheck clean, 481 vitest, biome clean.
|
|
Add StoredChunk { seq, role, chunk } to @dispatch/wire (re-exported via the
kernel contract shims). Keeps Chunk pure (provider-facing, no cursor); the
sync cursor lives only on the envelope.
conversation-store: rekey conv:<id>:msg:<seq> -> conv:<id>:chunk:<seq>;
append explodes messages into role-tagged seq'd chunks (1-based, gap-free,
monotonic) with internal boundary metadata so load() round-trips ChatMessage[]
losslessly and still reconciles; new loadSince(id, sinceSeq?) raw sync stream.
session-orchestrator test fake conforms to the widened interface.
FE Slice 2 backend prereq (per-chunk seq). typecheck clean, 469 vitest, biome clean.
|
|
split (B2)
FE slice 1 — backend-declared, frontend-agnostic surface system (verified live): new types-only @dispatch/ui-contract (SurfaceSpec / field kinds / region / ActionRef / catalog), surface-registry (typed service handle), transport-ws (Bun WS :24205, path-agnostic upgrade), surface-loaded-extensions (first real surface); kernel HostAPI.getExtensions; host-bin wiring; bin/up. Harness: retire AGENTS 'backend only', ORCHESTRATOR §3/§7/§8, frontend-design.md locked.
B2 — wire-types split (chat-slice prerequisite): new types-only @dispatch/wire single-sources the wire ABI (AgentEvent + 11 variants; conversation model Chunk/ChatMessage/Role/TurnId/StepId + 6 chunk variants; Usage) with zero @dispatch/* deps. @dispatch/kernel re-exports via shims so its public surface is byte-identical (zero consumer blast radius). transport-contract re-exports AgentEvent from @dispatch/wire and drops its @dispatch/kernel dependency, so HTTP clients (the web frontend) consume the wire without the kernel runtime.
tsc -b + biome clean; 460 vitest + 77 bun pass.
|
|
briefs + all notes/ with descriptions
|
|
- .dispatch/package-agent.md: base brief for every package owner (dir-scoped ownership,
visibility, engineering standard, isolated verify, report).
- .dispatch/extension-agent.md: thin supplement (manifest, activate/host, tighter quarantine);
references the package brief inline (injected), never instructs the agent to read a file.
- ORCHESTRATOR.md: §2 summon now concatenates briefs + scoped rules + TASK; §3 slimmed so each
prompts/<unit>.md is JUST the TASK block.
|
|
Superseded by the package/extension owner-agent briefs iterated with the user.
Reverts only the 3 markdown files (.dispatch/extension-agent.md, .dispatch/package-agent.md,
ORCHESTRATOR.md); no code was involved.
|
|
separate message
|
|
+ extension-agent.md
Rework ORCHESTRATOR §2 (summon) and §3 (TASK block): prompts are now assembled from
standardized briefs (.dispatch/package-agent.md + .dispatch/extension-agent.md for extensions)
+ cat'd scoped .dispatch/rules/* + a TASK block the orchestrator fills per summon. The old
per-unit prompts/<unit>.md workflow is retired. The agent never reads files — everything is
inlined by the orchestrator.
|
|
|
|
|
|
--text/--file/--cwd/--conversation)
HTTP client of transport-contract; pure-core arg/render/ndjson + injected fetch/fs shell.
Docs: GLOSSARY (credential/key/model name/model catalog), tasks.md milestone, ORCHESTRATOR geography.
|
|
per-turn cwd through orchestrator/transport/host-bin
|
|
transport-contract wire package
|
|
User-set ordering: (1) CLI MVP (line-oriented, NOT a TUI; may have basic selectors; same mirrored-backend methodology with a careful design pass first — seed at notes/cli-design.md), (2) web frontend (Svelte + DaisyUI, notes/frontend-design.md), (3) dedup/storage growth. CLI design seed frames the same open questions as the web FE (pure-core/shell split, unit boundaries, transport, testing) adapted to a terminal client.
|
|
methodology), then dedup/storage
User-set roadmap: (1) Frontend MVP — Svelte + DaisyUI, but ONLY after a careful design pass that maps the backend's methodology (minimal core + extensions, typed contracts, pure-core/inject-effects, one-owner, asymmetric testing) onto the frontend. Old Dispatch FE is reference-only; port 24204 reserved. Seed doc at notes/frontend-design.md (IDEATION mode — design WITH the user before any summon). (2) dedup/storage growth (D5 volume-control + prefix.fingerprint + §6 retention) — already designed, sequenced after the FE. Re-sequenced the deferred storage item. When FE build begins: retire AGENTS.md 'Backend only' line + author new frontend scoped rules + update ORCHESTRATOR §3/§7.
|
|
(prompt_tokens_details.cached_tokens) -> Usage.cacheReadTokens
The real flash fixture showed flash reports cache usage in the NESTED prompt_tokens_details.cached_tokens form (384 cached of 665 prompt); the parser only mapped the flat cache_read_tokens form, so cache tokens never surfaced. Now: cacheReadTokens = usage.cache_read_tokens ?? usage.prompt_tokens_details?.cached_tokens (flat wins; cacheWriteTokens flat-only, never fabricated; partial/null *_details safe). No kernel contract change (Usage already has the fields). +5 parser tests + a real-fixture regression (cacheReadTokens === 384).
These counts (+ a future prefix.fingerprint) are the cheap signals for body de-duplication. The broader trace-body storage-growth concern (verbatim body stored per request -> ~O(N^2) for long conversations) is logged DEFERRED in tasks.md; mitigation already designed (D5 volume control + §6 retention/rotation), not yet built. 339 tests, typecheck + biome 0/0.
|
|
sanitized flash capture (D5)
Installed a real DISPATCH_RECORD_FIXTURE capture (200 text/event-stream, deepseek-v4-flash, finish_reason stop, reply 'Hello there friend') as src/__fixtures__/flash-text-turn.json, replacing the hand-authored one. Auth header masked (Bearer sk-…redacted…UN0); fixture re-verified secret-free before commit. Text-turn replay assertions updated to real values (inputTokens 665 / outputTokens 90); structural assertions kept (multi-chunk replay, getCapturedRequest deep-equals the outgoing request, finish/stop event, concatenated deltas). The provider's request-building + SSE parsing now run against genuine flash bytes.
Real-data finding (logged in tasks.md): flash reports cache tokens via DeepSeek's NESTED prompt_tokens_details.cached_tokens (384 cached of 665 prompt); the parser only maps the FLAT cache_read/creation form, so cache tokens don't surface — an observability gap vs the §3.1 cache-debugging goal. Deferred pending decision. 334 tests, typecheck + biome 0/0.
|
|
header — case-insensitive mask (+3 regression tests)
A live capture exposed that provider record mode saved the request Authorization header with the API key in CLEARTEXT: onExchange redaction matched lowercase 'authorization', but streamChat sends 'Authorization' (capital A) and recordFetch captures headers verbatim, so the real header slipped through. The provider.request SPAN redaction was unaffected (it masks from config.apiKey directly — journal + trace-DB showed zero leaks); the leak was record-mode-only and caught PRE-COMMIT (fixture was /tmp-only, scrubbed). Fix: redact auth case-insensitively across all captured header keys (strip Bearer, maskSecret the token, re-prepend, preserve key casing).
New tests: reproduce the exact capital-Authorization leak (would have caught it), a lowercase case, and a guard that no authorization header of ANY casing survives carrying a raw sk- token. 334 tests (331 -> +3), typecheck + biome 0/0. This is the live-capture step (D5) earning its keep — real data exposed what the synthetic redaction test assumed away.
|
|
env-gated capture + hermetic fixture tests (331 tests)
provider-openai-compat now consumes @dispatch/trace-replay. (A) Opt-in record mode: when DISPATCH_RECORD_FIXTURE is set, the fetch edge wraps recordFetch and saves a fixture of the verbatim post-transform request + raw SSE response, self-redacting the auth header in the provider's OWN code (reuses its existing maskSecret graduated-tier mask — no shared helper, isolation over DRY). Zero overhead when unset; fail-safe. (B) Hermetic replay tests: stream.test.ts drives the provider off committed SSE fixtures via replayFetch (chunk-split to exercise SSE parsing across boundaries), asserting ProviderEvents + that the outgoing request still matches the recorded one (transform-drift regression). Injectable fetch via an internal StreamConfig.fetchFn — NO kernel contract change.
2 committed fixtures (text-turn + tool-call, currently hand-authored-faithful; a real flash text-turn swap follows). Verified: tsc -b clean, 331 vitest (327 -> +4: 2 replay + 2 redaction), biome 0/0. Provider 44 -> 48.
|
|
library (39 tests)
New standalone package @dispatch/trace-replay: replayFetch (pure — fixture -> fetch double + captured request, optional chunking to simulate streaming), recordFetch (tees a real fetch into a fixture WITHOUT consuming the caller's stream), and serialize/parse + save/load fixture I/O. Redaction-free by design: calling extensions self-redact in their OWN code before saving (isolation over DRY, D5/§9). Zero @dispatch/* deps, no bun:sqlite (runs under vitest). The shared unit realizing the §7/D5 replay affordance for hermetic provider tests; provider-openai-compat will consume it next.
Root tsconfig ref wired. Verified: tsc -b clean, 327 vitest (288 -> +39: replay 12 / record 8 / fixture 19), biome 0/0. Agent stayed in lane (packages/trace-replay only).
|
|
trick (pkill self-match scar)
A plain pkill -f 'host-bin/src/main.ts' matches its own command line and kills the parent shell (no output -> looks like a wedged/timed-out session). Use the [h]ost-bin bracket trick in ps/pgrep/pkill, and always clean up the backgrounded app + spawned collector after each live run (leaked processes inflated counts and made a correct supervisor look buggy).
|
|
drain-last / restart) — 288 tests
host-bin spawns the out-of-process collector before serving (real Bun.spawn adapted to a ChildHandle), restarts on unexpected exit (backoff + restart-guard cap), drains on SIGINT/SIGTERM (collector final-drain, SIGKILL fallback on timeout). createCollectorSupervisor takes an injected spawn so the lifecycle is unit-tested with a fake (no real subprocess). Collector failures never crash the app (D3 subordinate/fail-safe). New env DISPATCH_TRACE_DB (default ./.dispatch-data/traces.db).
Verified: tsc -b clean, 288 tests (279 + 9 supervisor), biome 0/0. Live (clean single run): 1 collector during, trace DB auto-populated (nested easy-view), 0 collectors after shutdown.
|
|
(+ buildSpanOpen parent propagation)
run-turn: step is now turnSpan.child; prompt/provider.request/tool-call are step's children (stepSpan.log passed into provider.stream). logger.ts: buildSpanOpen now propagates the child's computed parentSpanId onto the span-open record — a latent bug where span.child(...) never set parentSpanId on open (close was already correct).
Verified: tsc -b clean, 279 tests, biome 0/0. Live: span tree turn->step->{prompt,provider.request}; the trace CLI easy-view renders the nesting.
|
|
collector + trace CLI (345 tests)
trace-store (bun:sqlite): records+bodies schema (thin/fat split), idempotent insertRecords (FNV-1a id + INSERT OR IGNORE), getTurn/getBody, pure renderEasyView (D8 timeline skeleton), trace CLI. Its own DB, isolated from storage-sqlite.
observability-collector: out-of-process bin — tail journal -> splitLines/drainOnce -> trace-store.insertRecords; offset sidecar; at-least-once + idempotent; fail-safe; clean SIGINT/SIGTERM drain.
Build-config (orchestrator): root tsconfig refs; both excluded from vitest + added to test:bun (bun:sqlite); bun install.
Verified: tsc -b clean, 345 tests (273 vitest + 72 bun), biome 0 warnings/0 infos. Pipeline proven end-to-end: app -> journal -> collector -> SQLite -> 'trace <turnId>' easy-view. Known follow-up (next commit): kernel spans are flat (parent=ROOT) — run-turn nesting fix.
|
|
verbatim before/after -> LogRecord.body (273 tests)
contracts/logging.ts reduced to pure types; createLogger (+ helpers) moved to kernel/src/logging/ — @dispatch/kernel still exports it (host-bin/tool-read-file unaffected).
Span body channel (Option A): Logger.span / Span.child / Span.end accept an optional body string -> SpanOpenRecord.body / SpanCloseRecord.body. Large verbatim payloads now use body, not stringified attributes (store-fat-serve-thin; attributes stay thin/queryable for D9).
before: run-turn emits a 'prompt' span with the verbatim messages+tools in body (small scalars in attrs). after: provider.request span carries the verbatim request in body; attrs thin, auth self-redacted.
Verified: tsc -b clean, 273 tests, biome 0 warnings/0 infos. Live boot: prompt + provider.request bodies present and correlated (shared turnId); request.body no longer in attributes; auth-key leak count = 0.
|
|
+ self-redaction (267 tests)
Threads the step span's correlated logger into provider.stream (new optional ProviderStreamOptions.logger) so provider-openai-compat opens a child provider.request span at the fetch edge, capturing the verbatim post-transform request + response status/cache-tokens/raw-error. Auth header self-redacted in the provider's OWN code (graduated mask tiers; no shared helper). Capture is fail-safe (never throws into the turn). Adds the first hermetic provider HTTP test (stream.test.ts: fetch mocked, 15 cases). Large payloads use attributes for now; the LogRecord.body channel is a deferred ABI design (notes §10).
Verified: tsc -b clean, 267 tests (250->+17), biome 0 warnings/0 infos. Live boot: provider.request shares turnId with prompt:before (before<->after diffable); auth-key leak count = 0 (self-redaction proven on a real request).
|
|
contracts/build-config/docs, delegate all implementation
Remove the 'conflict exception' for reading implementation: the orchestrator diagnoses from typecheck/test output + lsp references on contracts + agent reports, then summons the owning agent to fix. Enumerates what the orchestrator MAY edit directly (contracts, build wiring/config, harness docs) vs. delegate (all executable .ts incl. tests + composition roots); roadblocks surface to the user.
|
|
sink (250 tests)
Structured, agent-first logging captured durably to an append-only journal file.
Kernel (contracts/logging.ts): leveled/attributed Logger + Span, auto-scoped per extension (host stamps manifest.id, unspoofable), incremental span records (open/close) for crash-reconstructable traces, injected LogSink (pure record-builder). ctx.log on ToolContract; runTurn opens turn/step/tool-call spans and captures the verbatim pre-mutation prompt (the 'before') on the step span.
journal-sink (new package, bootstrap dep — not an extension): LogSink appending NDJSON to a rotating journal; pure serialize + thin fs edge; fail-safe drop, never blocks a turn. host-bin injects it via HostDeps; session-orchestrator threads host.logger (childed per turn) into runTurn.
Redaction is per-extension self-redaction (no shared helper — isolation over DRY). The out-of-process collector + SQLite store + the verbatim 'after' provider.request capture are Phase B / next (notes/observability-design.md §10/§11).
Verified: tsc -b clean, 250 tests (218→+32), biome clean. Live boot: a turn's journal holds host logs + turn/step spans (open+close) + the prompt:before record with the verbatim messages array.
Harness: ORCHESTRATOR §3 rule-scoping map; .dispatch/rules/isolation-over-dry.md; notes/observability-design.md (design D1–D10 + Phase A/B plan).
|
|
consumers (218 tests)
Step 4 of the post-MVP backlog: resolve the last vocab drift. The canonical
term for a thread of turns is `conversationId` (GLOSSARY), but `AgentEvent`
variants and `RunTurnInput` still used the legacy `tabId` from the old frontend
"tab" concept, with session-orchestrator bridging `conversationId → tabId`.
Atomic, type-driven rename across the full 10-file consumer set:
- contracts/events.ts: all 11 AgentEvent variants tabId → conversationId
- contracts/runtime.ts: RunTurnInput.tabId → conversationId
- runtime/{events,run-turn,dispatch}.ts: factory params, ctx field, locals
- session-orchestrator: drop the redundant `tabId: conversationId` bridge line
- transport-http: emit wiring; external /chat field + X-Conversation-Id header
unchanged (already canonical) — only the emitted NDJSON event field flips
- tests (run-turn, app, logic): inputs + assertions now use conversationId
Pure rename, zero behavior change: typecheck clean, 218 tests pass (unchanged
count), biome clean, `grep tabId packages/` → zero matches. Verified live:
multi-turn curl emits conversationId-keyed NDJSON and threads history correctly.
GLOSSARY drift note removed. Closes the post-MVP backlog (Steps 1–4).
|
|
storage-sqlite manifest honesty
host CR-1: createHost.getHostAPI() returns the canonical post-activation HostAPI
(registration closed) via a single builder — host-bin deletes its
buildPostActivationHostAPI duplicate and calls host.getHostAPI().
storage-sqlite CR-2: remove false contributes.services:["storage"] (backend is a
kernel bootstrap dep injected as HostDeps.storageFactory, not a bus service);
document the intentional no-op activate.
typecheck clean, 218 tests pass, biome clean; live boot + curl verified.
|
|
First TOOL extension (standard tier, fs capability). Pure-core/shell split with
workdir containment (realpath symlink guard). host-bin registers it in
CORE_EXTENSIONS; flows into runTurn via session-orchestrator's resolveTools.
Verified: typecheck clean, 214 tests pass (was 185), biome clean. Live curl
against flash produced a real tool-call + tool-result round-trip with correct
final answer. Proves the kernel tool-dispatch loop end-to-end (plan §3.3).
|
|
file; restrict implementation-file reading to conflicts only (orchestrator + subagents)
|
|
- kernel HostAPI: add getAuthProviders()/getAuthProvider(id) read-views (mirrors getProviders)
- provider-openai-compat: activate() resolves creds via host.getAuthProvider("apikey").resolve(); dependsOn auth-apikey; model stays config-driven
- host-bin: mirror the new getters in post-activation HostAPI stub
- auth-apikey is no longer vestigial; auth seam exercised end-to-end
- 185 tests pass; typecheck + biome clean; verified live (curl returns real response)
|
|
ORCHESTRATOR↔plan §5/§3.6, add HANDOFF.md, host-bin reads BACKEND_PORT (24203)
|
|
opencode, prompt recipe, verification, error/contract resolution, invariants)
|
|
Go flash
|
|
Bun.serve; full-fidelity wiring (178 tests)
|
|
input tabId/turnId (CR-3); simplify orchestrator wiring (167 tests)
|
|
|
|
build graph (164 tests)
|
|
|
|
StorageNamespace + pure reconcile (16 tests)
|
|
|
|
activate, HostAPI (wraps bus); 50 tests
|
|
- storage-sqlite: bun:sqlite StorageNamespace backend + migrations (21 bun tests)
- auth-apikey: pure resolver from env → ApiKeyCredentials (4 tests)
- provider-openai-compat: OpenAI-compatible SSE stream → ProviderEvents
- orchestrator fixes: provider imports (@dispatch/kernel), missing dep,
exactOptionalPropertyTypes (omit-when-undefined), root tsconfig refs
- vitest excludes storage-sqlite (bun:sqlite); test:bun runs it under bun
|
