| Age | Commit message (Collapse) | Author |
|---|
|
User-set ordering: (1) CLI MVP (line-oriented, NOT a TUI; may have basic selectors; same mirrored-backend methodology with a careful design pass first — seed at notes/cli-design.md), (2) web frontend (Svelte + DaisyUI, notes/frontend-design.md), (3) dedup/storage growth. CLI design seed frames the same open questions as the web FE (pure-core/shell split, unit boundaries, transport, testing) adapted to a terminal client.
|
|
methodology), then dedup/storage
User-set roadmap: (1) Frontend MVP — Svelte + DaisyUI, but ONLY after a careful design pass that maps the backend's methodology (minimal core + extensions, typed contracts, pure-core/inject-effects, one-owner, asymmetric testing) onto the frontend. Old Dispatch FE is reference-only; port 24204 reserved. Seed doc at notes/frontend-design.md (IDEATION mode — design WITH the user before any summon). (2) dedup/storage growth (D5 volume-control + prefix.fingerprint + §6 retention) — already designed, sequenced after the FE. Re-sequenced the deferred storage item. When FE build begins: retire AGENTS.md 'Backend only' line + author new frontend scoped rules + update ORCHESTRATOR §3/§7.
|
|
(prompt_tokens_details.cached_tokens) -> Usage.cacheReadTokens
The real flash fixture showed flash reports cache usage in the NESTED prompt_tokens_details.cached_tokens form (384 cached of 665 prompt); the parser only mapped the flat cache_read_tokens form, so cache tokens never surfaced. Now: cacheReadTokens = usage.cache_read_tokens ?? usage.prompt_tokens_details?.cached_tokens (flat wins; cacheWriteTokens flat-only, never fabricated; partial/null *_details safe). No kernel contract change (Usage already has the fields). +5 parser tests + a real-fixture regression (cacheReadTokens === 384).
These counts (+ a future prefix.fingerprint) are the cheap signals for body de-duplication. The broader trace-body storage-growth concern (verbatim body stored per request -> ~O(N^2) for long conversations) is logged DEFERRED in tasks.md; mitigation already designed (D5 volume control + §6 retention/rotation), not yet built. 339 tests, typecheck + biome 0/0.
|
|
sanitized flash capture (D5)
Installed a real DISPATCH_RECORD_FIXTURE capture (200 text/event-stream, deepseek-v4-flash, finish_reason stop, reply 'Hello there friend') as src/__fixtures__/flash-text-turn.json, replacing the hand-authored one. Auth header masked (Bearer sk-…redacted…UN0); fixture re-verified secret-free before commit. Text-turn replay assertions updated to real values (inputTokens 665 / outputTokens 90); structural assertions kept (multi-chunk replay, getCapturedRequest deep-equals the outgoing request, finish/stop event, concatenated deltas). The provider's request-building + SSE parsing now run against genuine flash bytes.
Real-data finding (logged in tasks.md): flash reports cache tokens via DeepSeek's NESTED prompt_tokens_details.cached_tokens (384 cached of 665 prompt); the parser only maps the FLAT cache_read/creation form, so cache tokens don't surface — an observability gap vs the §3.1 cache-debugging goal. Deferred pending decision. 334 tests, typecheck + biome 0/0.
|
|
header — case-insensitive mask (+3 regression tests)
A live capture exposed that provider record mode saved the request Authorization header with the API key in CLEARTEXT: onExchange redaction matched lowercase 'authorization', but streamChat sends 'Authorization' (capital A) and recordFetch captures headers verbatim, so the real header slipped through. The provider.request SPAN redaction was unaffected (it masks from config.apiKey directly — journal + trace-DB showed zero leaks); the leak was record-mode-only and caught PRE-COMMIT (fixture was /tmp-only, scrubbed). Fix: redact auth case-insensitively across all captured header keys (strip Bearer, maskSecret the token, re-prepend, preserve key casing).
New tests: reproduce the exact capital-Authorization leak (would have caught it), a lowercase case, and a guard that no authorization header of ANY casing survives carrying a raw sk- token. 334 tests (331 -> +3), typecheck + biome 0/0. This is the live-capture step (D5) earning its keep — real data exposed what the synthetic redaction test assumed away.
|
|
env-gated capture + hermetic fixture tests (331 tests)
provider-openai-compat now consumes @dispatch/trace-replay. (A) Opt-in record mode: when DISPATCH_RECORD_FIXTURE is set, the fetch edge wraps recordFetch and saves a fixture of the verbatim post-transform request + raw SSE response, self-redacting the auth header in the provider's OWN code (reuses its existing maskSecret graduated-tier mask — no shared helper, isolation over DRY). Zero overhead when unset; fail-safe. (B) Hermetic replay tests: stream.test.ts drives the provider off committed SSE fixtures via replayFetch (chunk-split to exercise SSE parsing across boundaries), asserting ProviderEvents + that the outgoing request still matches the recorded one (transform-drift regression). Injectable fetch via an internal StreamConfig.fetchFn — NO kernel contract change.
2 committed fixtures (text-turn + tool-call, currently hand-authored-faithful; a real flash text-turn swap follows). Verified: tsc -b clean, 331 vitest (327 -> +4: 2 replay + 2 redaction), biome 0/0. Provider 44 -> 48.
|
|
library (39 tests)
New standalone package @dispatch/trace-replay: replayFetch (pure — fixture -> fetch double + captured request, optional chunking to simulate streaming), recordFetch (tees a real fetch into a fixture WITHOUT consuming the caller's stream), and serialize/parse + save/load fixture I/O. Redaction-free by design: calling extensions self-redact in their OWN code before saving (isolation over DRY, D5/§9). Zero @dispatch/* deps, no bun:sqlite (runs under vitest). The shared unit realizing the §7/D5 replay affordance for hermetic provider tests; provider-openai-compat will consume it next.
Root tsconfig ref wired. Verified: tsc -b clean, 327 vitest (288 -> +39: replay 12 / record 8 / fixture 19), biome 0/0. Agent stayed in lane (packages/trace-replay only).
|
|
trick (pkill self-match scar)
A plain pkill -f 'host-bin/src/main.ts' matches its own command line and kills the parent shell (no output -> looks like a wedged/timed-out session). Use the [h]ost-bin bracket trick in ps/pgrep/pkill, and always clean up the backgrounded app + spawned collector after each live run (leaked processes inflated counts and made a correct supervisor look buggy).
|
|
drain-last / restart) — 288 tests
host-bin spawns the out-of-process collector before serving (real Bun.spawn adapted to a ChildHandle), restarts on unexpected exit (backoff + restart-guard cap), drains on SIGINT/SIGTERM (collector final-drain, SIGKILL fallback on timeout). createCollectorSupervisor takes an injected spawn so the lifecycle is unit-tested with a fake (no real subprocess). Collector failures never crash the app (D3 subordinate/fail-safe). New env DISPATCH_TRACE_DB (default ./.dispatch-data/traces.db).
Verified: tsc -b clean, 288 tests (279 + 9 supervisor), biome 0/0. Live (clean single run): 1 collector during, trace DB auto-populated (nested easy-view), 0 collectors after shutdown.
|
|
(+ buildSpanOpen parent propagation)
run-turn: step is now turnSpan.child; prompt/provider.request/tool-call are step's children (stepSpan.log passed into provider.stream). logger.ts: buildSpanOpen now propagates the child's computed parentSpanId onto the span-open record — a latent bug where span.child(...) never set parentSpanId on open (close was already correct).
Verified: tsc -b clean, 279 tests, biome 0/0. Live: span tree turn->step->{prompt,provider.request}; the trace CLI easy-view renders the nesting.
|
|
collector + trace CLI (345 tests)
trace-store (bun:sqlite): records+bodies schema (thin/fat split), idempotent insertRecords (FNV-1a id + INSERT OR IGNORE), getTurn/getBody, pure renderEasyView (D8 timeline skeleton), trace CLI. Its own DB, isolated from storage-sqlite.
observability-collector: out-of-process bin — tail journal -> splitLines/drainOnce -> trace-store.insertRecords; offset sidecar; at-least-once + idempotent; fail-safe; clean SIGINT/SIGTERM drain.
Build-config (orchestrator): root tsconfig refs; both excluded from vitest + added to test:bun (bun:sqlite); bun install.
Verified: tsc -b clean, 345 tests (273 vitest + 72 bun), biome 0 warnings/0 infos. Pipeline proven end-to-end: app -> journal -> collector -> SQLite -> 'trace <turnId>' easy-view. Known follow-up (next commit): kernel spans are flat (parent=ROOT) — run-turn nesting fix.
|
|
verbatim before/after -> LogRecord.body (273 tests)
contracts/logging.ts reduced to pure types; createLogger (+ helpers) moved to kernel/src/logging/ — @dispatch/kernel still exports it (host-bin/tool-read-file unaffected).
Span body channel (Option A): Logger.span / Span.child / Span.end accept an optional body string -> SpanOpenRecord.body / SpanCloseRecord.body. Large verbatim payloads now use body, not stringified attributes (store-fat-serve-thin; attributes stay thin/queryable for D9).
before: run-turn emits a 'prompt' span with the verbatim messages+tools in body (small scalars in attrs). after: provider.request span carries the verbatim request in body; attrs thin, auth self-redacted.
Verified: tsc -b clean, 273 tests, biome 0 warnings/0 infos. Live boot: prompt + provider.request bodies present and correlated (shared turnId); request.body no longer in attributes; auth-key leak count = 0.
|
|
+ self-redaction (267 tests)
Threads the step span's correlated logger into provider.stream (new optional ProviderStreamOptions.logger) so provider-openai-compat opens a child provider.request span at the fetch edge, capturing the verbatim post-transform request + response status/cache-tokens/raw-error. Auth header self-redacted in the provider's OWN code (graduated mask tiers; no shared helper). Capture is fail-safe (never throws into the turn). Adds the first hermetic provider HTTP test (stream.test.ts: fetch mocked, 15 cases). Large payloads use attributes for now; the LogRecord.body channel is a deferred ABI design (notes §10).
Verified: tsc -b clean, 267 tests (250->+17), biome 0 warnings/0 infos. Live boot: provider.request shares turnId with prompt:before (before<->after diffable); auth-key leak count = 0 (self-redaction proven on a real request).
|
|
contracts/build-config/docs, delegate all implementation
Remove the 'conflict exception' for reading implementation: the orchestrator diagnoses from typecheck/test output + lsp references on contracts + agent reports, then summons the owning agent to fix. Enumerates what the orchestrator MAY edit directly (contracts, build wiring/config, harness docs) vs. delegate (all executable .ts incl. tests + composition roots); roadblocks surface to the user.
|
|
sink (250 tests)
Structured, agent-first logging captured durably to an append-only journal file.
Kernel (contracts/logging.ts): leveled/attributed Logger + Span, auto-scoped per extension (host stamps manifest.id, unspoofable), incremental span records (open/close) for crash-reconstructable traces, injected LogSink (pure record-builder). ctx.log on ToolContract; runTurn opens turn/step/tool-call spans and captures the verbatim pre-mutation prompt (the 'before') on the step span.
journal-sink (new package, bootstrap dep — not an extension): LogSink appending NDJSON to a rotating journal; pure serialize + thin fs edge; fail-safe drop, never blocks a turn. host-bin injects it via HostDeps; session-orchestrator threads host.logger (childed per turn) into runTurn.
Redaction is per-extension self-redaction (no shared helper — isolation over DRY). The out-of-process collector + SQLite store + the verbatim 'after' provider.request capture are Phase B / next (notes/observability-design.md §10/§11).
Verified: tsc -b clean, 250 tests (218→+32), biome clean. Live boot: a turn's journal holds host logs + turn/step spans (open+close) + the prompt:before record with the verbatim messages array.
Harness: ORCHESTRATOR §3 rule-scoping map; .dispatch/rules/isolation-over-dry.md; notes/observability-design.md (design D1–D10 + Phase A/B plan).
|
|
consumers (218 tests)
Step 4 of the post-MVP backlog: resolve the last vocab drift. The canonical
term for a thread of turns is `conversationId` (GLOSSARY), but `AgentEvent`
variants and `RunTurnInput` still used the legacy `tabId` from the old frontend
"tab" concept, with session-orchestrator bridging `conversationId → tabId`.
Atomic, type-driven rename across the full 10-file consumer set:
- contracts/events.ts: all 11 AgentEvent variants tabId → conversationId
- contracts/runtime.ts: RunTurnInput.tabId → conversationId
- runtime/{events,run-turn,dispatch}.ts: factory params, ctx field, locals
- session-orchestrator: drop the redundant `tabId: conversationId` bridge line
- transport-http: emit wiring; external /chat field + X-Conversation-Id header
unchanged (already canonical) — only the emitted NDJSON event field flips
- tests (run-turn, app, logic): inputs + assertions now use conversationId
Pure rename, zero behavior change: typecheck clean, 218 tests pass (unchanged
count), biome clean, `grep tabId packages/` → zero matches. Verified live:
multi-turn curl emits conversationId-keyed NDJSON and threads history correctly.
GLOSSARY drift note removed. Closes the post-MVP backlog (Steps 1–4).
|
|
storage-sqlite manifest honesty
host CR-1: createHost.getHostAPI() returns the canonical post-activation HostAPI
(registration closed) via a single builder — host-bin deletes its
buildPostActivationHostAPI duplicate and calls host.getHostAPI().
storage-sqlite CR-2: remove false contributes.services:["storage"] (backend is a
kernel bootstrap dep injected as HostDeps.storageFactory, not a bus service);
document the intentional no-op activate.
typecheck clean, 218 tests pass, biome clean; live boot + curl verified.
|
|
First TOOL extension (standard tier, fs capability). Pure-core/shell split with
workdir containment (realpath symlink guard). host-bin registers it in
CORE_EXTENSIONS; flows into runTurn via session-orchestrator's resolveTools.
Verified: typecheck clean, 214 tests pass (was 185), biome clean. Live curl
against flash produced a real tool-call + tool-result round-trip with correct
final answer. Proves the kernel tool-dispatch loop end-to-end (plan §3.3).
|
|
file; restrict implementation-file reading to conflicts only (orchestrator + subagents)
|
|
- kernel HostAPI: add getAuthProviders()/getAuthProvider(id) read-views (mirrors getProviders)
- provider-openai-compat: activate() resolves creds via host.getAuthProvider("apikey").resolve(); dependsOn auth-apikey; model stays config-driven
- host-bin: mirror the new getters in post-activation HostAPI stub
- auth-apikey is no longer vestigial; auth seam exercised end-to-end
- 185 tests pass; typecheck + biome clean; verified live (curl returns real response)
|
|
ORCHESTRATOR↔plan §5/§3.6, add HANDOFF.md, host-bin reads BACKEND_PORT (24203)
|
|
opencode, prompt recipe, verification, error/contract resolution, invariants)
|
|
Go flash
|
|
Bun.serve; full-fidelity wiring (178 tests)
|
|
input tabId/turnId (CR-3); simplify orchestrator wiring (167 tests)
|
|
|
|
build graph (164 tests)
|
|
|
|
StorageNamespace + pure reconcile (16 tests)
|
|
|
|
activate, HostAPI (wraps bus); 50 tests
|
|
- storage-sqlite: bun:sqlite StorageNamespace backend + migrations (21 bun tests)
- auth-apikey: pure resolver from env → ApiKeyCredentials (4 tests)
- provider-openai-compat: OpenAI-compatible SSE stream → ProviderEvents
- orchestrator fixes: provider imports (@dispatch/kernel), missing dep,
exactOptionalPropertyTypes (omit-when-undefined), root tsconfig refs
- vitest excludes storage-sqlite (bun:sqlite); test:bun runs it under bun
|
|
union (resolves runtime CR-1/2/3)
|
|
(eager/semaphore/dedup/concurrencySafe/abort), 16 tests
|
|
|
|
stateful shell
|
|
|
|
dispatch, hooks, extension/HostAPI, runtime, events)
|
|
stub)
|
|
|
|
- Add watchDirConfig() for per-directory config watching
- Register watchers for subdirectories with their own dispatch.toml
- Fix permission ordering: move "*" wildcard to front so findLast
reaches specific rules first (was silently breaking all specific
bash permission rules)
- Add comprehensive tests for watcher functionality
- Update mocks in test files
|
|
Gemini review caught a precedence-inversion bug in mergePermissions: when a
nested permission group exists in BOTH global and local configs, the previous
`{ ...existing, ...value }` spread updated an overridden pattern IN PLACE,
leaving its original (global) insertion slot. Since configToRuleset flattens
patterns in iteration order and evaluate() uses findLast (last match wins), a
more-general global pattern declared lower (e.g. "*") would sit AFTER the
local override and silently shadow it.
Example: global bash { "npm test"=allow, "*"=ask } + local bash
{ "npm test"=deny } resolved "npm test" to "ask" instead of "deny".
Fix: drop global patterns the local block also defines, keep remaining global
patterns in order, then append ALL local patterns last — reproducing a clean
"global rules then local rules" concatenation so local always wins. Adds a
regression test asserting order and evaluation outcome.
|
|
Load an optional global config at ~/.config/dispatch/dispatch.toml
(override via DISPATCH_GLOBAL_CONFIG) and deep-merge it underneath every
project/working-directory dispatch.toml, so machine-wide settings — most
notably globally available LSP servers — work in any repo without per-repo
config. Local always wins on conflicts.
- loader: add getGlobalConfigPath(), loadGlobalConfig(), mergeConfigs();
loadConfig(dir) now loads+merges global. [lsp] and [[keys]] merge by id;
[permissions] merge per-group with global patterns emitted first so local
rules win at evaluation time (findLast). A malformed global config is
downgraded to empty rather than breaking every repo.
- watcher: watch BOTH global and local dispatch.toml so hot-reload re-merges
on either change (dedupes when paths coincide).
- export new loader fns from config/index and core index.
- types/agent-manager: doc updates reflecting merged LSP resolution.
- dispatch.toml: document global-default merge behavior; activate biome and
typescript-language-server LSP entries.
- tests: merge precedence, lsp/keys merge-by-id, permissions merge,
filesystem integration, malformed-global resilience; isolate global path
in existing loader tests.
|
|
# Conflicts:
# packages/api/src/agent-manager.ts
# packages/api/tests/agent-manager.test.ts
# packages/frontend/src/lib/tabs.svelte.ts
|
|
Add a DEBUG=1 convenience flag to bin/up that turns on full LLM debug logging
(DISPATCH_DEBUG_LLM) plus the per-step usage/cache split (DISPATCH_DEBUG_USAGE)
and routes the JSON request/response files into the container's /app/logging,
which maps to the project-root logging/ dir on the host via the existing .:/app
bind mount. This makes it easy to verify cache hits (including the prompt-cache
warming replay) without docker exec.
- bin/up: DEBUG=1 sets DISPATCH_DEBUG_LLM/_USAGE=1 and DISPATCH_DEBUG_LLM_DIR=
/app/logging as DEFAULTS (explicit DISPATCH_DEBUG_* still win). Pre-creates
the host log dir; /app/* targets are already host-owned so no sudo chown,
while the /tmp fallback keeps its ownership fix.
- docker-compose.yml: forward DISPATCH_DEBUG_LLM_DIR into the api container.
- .gitignore: ignore logging/.
|
|
Root cause of the 'first warmup misses' + 'switch to chat misses' bugs:
Anthropic keys the MESSAGE-level prompt cache on `tool_choice` AND the
extended-thinking parameters (both rows in their cache-invalidation table mark
the messages cache as invalidated on change). The original warmCache() sent
toolChoice:'none' and NO thinking providerOptions, while real turns send
toolChoice:'auto' + thinking config for the effort. So warming and chat wrote
TWO different message-cache buckets:
- warmup #1 missed (no warm-only bucket existed yet), every later warmup hit
its own bucket;
- the next real chat message read the OTHER bucket → miss.
Fix: extract a shared buildStreamOptions() that produces the cache-affecting
params (toolChoice + thinking providerOptions + maxOutputTokens). Both run()
and warmCache() now call it with the SAME resolved reasoning effort, so the
warming replay refreshes the exact cache the next real message reads. The
trivial probe turn is still appended AFTER the last cache breakpoint, so it
never disturbs the cached prefix.
Threaded the per-tab reasoning effort (per-model -> per-tab selector -> default,
mirroring processMessage) from the frontend resolver through POST /chat/warm to
warmCacheForTab to warmCache.
Tests: updated the warmCache toolChoice test to assert it MATCHES a real turn,
added an invariant test driving run() and warmCache() and asserting identical
cache-affecting params, and assert effort forwarding in the frontend store.
check / test (780) / frontend build / typecheck all green.
|
|
Keep a tab's provider prompt-cache warm while idle by periodically replaying
the exact cached conversation prefix plus a single trivial throwaway turn,
resetting the provider's ~5-min cache TTL so the user's next real message hits
a warm cache.
Backend:
- Agent.warmCache(history): extracts buildLlmContext() shared with run(), then
re-sends the identical system+tools+history prefix (same Anthropic
cache_control breakpoints) plus a 'reply with just a .' probe turn via
toolChoice:none. Returns the request usage; mutates no history, emits/persists
nothing.
- AgentManager.warmCacheForTab(): resolves the same agent the next real turn
would use, replays the FULL genuine history, refuses while a turn is running.
- POST /chat/warm: returns ONLY the warming request's usage (never persisted,
never folded into the real usage aggregate).
Frontend:
- cache-warming.svelte.ts store: per-tab 4-min repeating idle timer with
countdown, warming-specific last-request cache %, and error capture. Arms on
turn end, pauses during a turn, disables+resets on a real user message.
- cache-warm-storage.ts: per-tab localStorage persistence of the toggle.
- Lifecycle hooks wired into tabs.svelte.ts (status/statuses/sendMessage/
hydrate/create/open/close).
- ModelSelector: bottom-of-panel checkbox + debug strip (last-% / countdown /
error), shown only when enabled. Warming cache data never touches the real
Cache Rate view.
Tests: core warmCache (5), api warm route (3) + warmCacheForTab (3), frontend
store (12) + storage (10). check / test (779) / frontend build / typecheck all
green.
|
|
# Conflicts:
# packages/frontend/src/lib/components/ChatInput.svelte
|
|
Summarize a conversation's older "head" into a structured anchored Markdown
summary while preserving the most recent turns verbatim, shrinking context size
while keeping the information needed to continue coherently. Triggered by a
"Compact conversation" button in Chat Settings (not an agent tool).
Approach informed by OpenCode's session/compaction.ts:
- Ported SUMMARY_TEMPLATE (Goal / Constraints / Progress / Key Decisions /
Next Steps / Critical Context / Relevant Files) and the anchored-summary
buildPrompt (re-summarizes a prior summary when present).
- Ported the TOOL_OUTPUT_MAX_CHARS (2000) cap on tool results in the summary
request.
- Simplified tail selection to a fixed recent-turn count (DEFAULT_TAIL_TURNS=2)
instead of OpenCode's token-budget splitTurn.
core:
- New src/compaction/ module (pure, DB-free): template, prompt builder,
head/tail selection, transcript renderer with tool-output capping, prior
summary extraction. Generic over ChatMessage so callers keep turnId/seq.
- db/chunks.ts: rekeyChunks(from,to) relocates a tab's full history to a
backup tab (reversible — nothing is deleted).
- AgentEvent: compaction-started / -complete / -error variants.
api:
- AgentManager.compactTab(tempTabId, sourceTabId): side-effect-free
resolveConnection() for the compactor model (configured compaction_model_*,
else the source tab's own key+model), one-shot tool-less summary generation
via a transient Agent, then relocate full history to a fresh backup tab and
re-seed the canonical source id with [summary turn + preserved tail]. Source
tab is locked (messages queue) during the run; queue drains afterward.
- Routes: POST /tabs/:id/compact, GET/PUT /tabs/settings/compaction-model.
frontend:
- "Compact conversation" button in ModelSelector (Chat Settings), between
Working Directory and the agent toggle; idle-gated.
- Compaction-model key+model selector in Settings, beside the title model.
- Transient placeholder tab shows a large, non-faded "Please wait, compacting
conversation…" screen; closing it cancels. Source input locked while running.
- Handle compaction-* events: reload compacted source, insert backup tab,
refocus source, discard placeholder.
tests: core compaction unit tests, rekeyChunks DB test, AgentManager.compactTab
orchestration tests, and compaction route tests. All green (713 tests), biome
clean, all typechecks pass, frontend builds.
|
|
|
