summaryrefslogtreecommitdiffhomepage
path: root/packages/core/src/db
AgeCommit message (Collapse)Author
2026-06-03feat(compaction): add UI-driven conversation compactionAdam Malczewski
Summarize a conversation's older "head" into a structured anchored Markdown summary while preserving the most recent turns verbatim, shrinking context size while keeping the information needed to continue coherently. Triggered by a "Compact conversation" button in Chat Settings (not an agent tool). Approach informed by OpenCode's session/compaction.ts: - Ported SUMMARY_TEMPLATE (Goal / Constraints / Progress / Key Decisions / Next Steps / Critical Context / Relevant Files) and the anchored-summary buildPrompt (re-summarizes a prior summary when present). - Ported the TOOL_OUTPUT_MAX_CHARS (2000) cap on tool results in the summary request. - Simplified tail selection to a fixed recent-turn count (DEFAULT_TAIL_TURNS=2) instead of OpenCode's token-budget splitTurn. core: - New src/compaction/ module (pure, DB-free): template, prompt builder, head/tail selection, transcript renderer with tool-output capping, prior summary extraction. Generic over ChatMessage so callers keep turnId/seq. - db/chunks.ts: rekeyChunks(from,to) relocates a tab's full history to a backup tab (reversible — nothing is deleted). - AgentEvent: compaction-started / -complete / -error variants. api: - AgentManager.compactTab(tempTabId, sourceTabId): side-effect-free resolveConnection() for the compactor model (configured compaction_model_*, else the source tab's own key+model), one-shot tool-less summary generation via a transient Agent, then relocate full history to a fresh backup tab and re-seed the canonical source id with [summary turn + preserved tail]. Source tab is locked (messages queue) during the run; queue drains afterward. - Routes: POST /tabs/:id/compact, GET/PUT /tabs/settings/compaction-model. frontend: - "Compact conversation" button in ModelSelector (Chat Settings), between Working Directory and the agent toggle; idle-gated. - Compaction-model key+model selector in Settings, beside the title model. - Transient placeholder tab shows a large, non-faded "Please wait, compacting conversation…" screen; closing it cancels. Source input locked while running. - Handle compaction-* events: reload compacted source, insert backup tab, refocus source, discard placeholder. tests: core compaction unit tests, rekeyChunks DB test, AgentManager.compactTab orchestration tests, and compaction route tests. All green (713 tests), biome clean, all typechecks pass, frontend builds.
2026-06-02feat(tabs): drag-reorder + double-click rename + per-tab chat draftAdam Malczewski
- TabBar: HTML5 drag-and-drop to reorder user tabs (subagent tabs untouched); double-click a tab title to rename (Enter/blur confirm, Escape cancel). - Store: add reorderTabs/renameTab/setDraft; per-tab in-memory `draft` and `manualTitle` fields. Manual rename suppresses first-message auto-title. - ChatInput: bind to the active tab's draft so switching tabs saves/restores unsent text instead of clobbering it. - Backend: updateTabPositions() + PATCH /tabs/reorder persist tab order to the existing `position` column; tabs without a stored position fall to the end then get explicit positions on first reorder. - Tests: store reorder/rename/auto-title-guard/draft coverage; core updateTabPositions coverage (FakeDatabase extended with transaction support).
2026-06-02fix: reconcile live cacheStats to DB truth on turn-sealedAdam Malczewski
Addresses the live-accumulator overshoot a Gemini review surfaced: the frontend adds every streamed usage event to cacheStats, but a rate-limited fallback attempt's usage is discarded server-side (never persisted). Live numbers overshot until a reload re-seeded from the DB aggregate. Fix: turn-sealed (emitted AFTER the atomic usage-row write) now carries the authoritative getUsageStatsForTab aggregate. The store REPLACES (not adds) cacheStats with it every turn — landing the just-sealed turn's usage AND self-healing any live drift, including the discarded-fallback overshoot. No extra round-trip (piggybacks turn-sealed); idempotent in the happy path. - core: add UsageStats type; getUsageStatsForTab returns it; turn-sealed gains optional usageStats field. - api: agent-manager reads getUsageStatsForTab post-flush and attaches it to the turn-sealed emit (try/catch: omit on DB error). - frontend: turn-sealed handler replaces cacheStats (undefined ⇒ untouched back-compat; null ⇒ clear). Tests: frontend reconcile/self-heal/back-compat/null-clear; api turn-sealed carries aggregate. 509 -> 514 passing; typecheck + biome green.
2026-06-02feat: persist per-tab token/cache usage across reloadAdam Malczewski
Persist usage as invisible type:"usage" chunk rows (side channel): - core: add "usage" ChunkType + UsageData; exclude usage rows from getChunksForTab/getTotalChunkCount; add getUsageStatsForTab aggregate (exported from barrel); defensive skip in groupRowsToMessages. - api: agent-manager accumulates per-attempt usageRows and flushes them in the same atomic appendChunks call as the turn's content (discarded on a superseded fallback attempt). GET /tabs enriches rows with usageStats. - frontend: hydrateFromBackend seeds cacheStats from usageStats (reload only; no re-seed on statuses reconnect, so no double-count with live events). Tests: core DB-backed usage persistence/aggregate; api usage-row-per-event + fallback discard; routes GET /tabs usageStats; frontend hydrate seed + no-double-count + live-accumulation-after-seed. 495 -> 509 passing.
2026-06-01feat(wake): probe 4 times per marked hour (:00 :15 :30 :45), coalesce ↵Adam Malczewski
same-tick fires Marking an hour on the Claude Wake Schedule panel now schedules FOUR probes within that hour instead of one. Rate-window edges are unforgiving — a single probe at :15 can miss the actual reset moment by up to 14 minutes; hitting :00 / :15 / :30 / :45 puts us within ~7 minutes of any reset that happens during that hour. When multiple slots come due in the same 30s scheduler tick (or recover together at boot), they coalesce into a SINGLE upstream wake call — no point hitting Anthropic 4× in the same window. DB schema - wake_schedule is now (hour, slot_minute, next_wake_at) PK (hour, slot_minute). Destructive migration: detect old single-row-per-hour schema by absence of the slot_minute column and DROP TABLE. No other table is touched. Per user direction: no back-compat for old rows. API - POST /models/wake-schedule/toggle add: { hour, timestamps: { '0': ms, '15': ms, '30': ms, '45': ms } } — all 4 slots required, all must be future Unix ms. Delete shape unchanged ({ hour }). - GET /models/wake-schedule shape: schedule: { '9': { '0': ts, '15': ts, '30': ts, '45': ts }, ... } probeSlotMinutes: [0, 15, 30, 45] resetOffsetHours, lastWake, pendingRetry (unchanged from prior commit) Frontend - Computes 4 timestamps client-side (next occurrence of HH:MM in local TZ) and sends them in one request. - markedHours summary now says 'Probes :00 :15 :30 :45 → reset by ~Xh later'. - Same in-flight tracking / current-hour ring / status row as before. Tests - wake-scheduler.test.ts unchanged (pure helpers still correct; added PROBE_SLOT_MINUTES + isProbeSlotMinute exports). - routes.test.ts rewritten for the new payload shape: 12 wake-schedule tests covering snapshot shape, add/remove (full 4-slot round-trip), validation (range, integer, past-slot, missing slot, non-object, missing timestamps), independent multi-hour scheduling, and re-toggle replacement. 417 tests total (was 414).
2026-06-01feat(tabs): tab-to-tab agent communication via short handlesAdam Malczewski
Add send_to_tab / read_tab tools so an agent can message or read another tab by a git-style short handle (shortest unique prefix of the tab UUID, min 4 chars), shown in the tab bar. - core/db/tabs: resolveTabPrefix + shortestUniquePrefix (open tabs only, LIKE-sanitized prefix matching) - new tools read-tab.ts / send-to-tab.ts (+ tests) decoupled from the DB TabRow via a minimal ResolvedTabRef projection - agent-manager: unified deliverMessage routing (busy -> queue, idle -> new turn) shared by POST /chat and send_to_tab; agent->agent auto-wake budget (MAX_AGENT_AUTO_WAKES) to bound ping-pong loops - summon/loader: send_to_tab + read_tab as grantable tools - frontend: shortHandleFor + handle badge in TabBar; perm toggles - notes: tab-comm / user-agents / todo-redesign plans - chore: biome format fixes (debug-logger, summon.test) Refs notes/plan-tab-comm.md
2026-05-30chore(notes): collect loose root docs into notes/; add reconcile edge-cases noteAdam Malczewski
Move all loose root-level .md files (plans, reports, gemini reviews, incident notes) into a single notes/ directory, and update the doc-reference breadcrumbs in code comments/test labels to the notes/ path. Add notes/queue-interrupt-reconcile-edge-cases.md: documents why the queue/interrupt/turn-sealed reconcile path keeps surfacing edge cases (a catalog of the four review-pass bugs, the no-loss/no-duplicate invariants, the recommended membership-based reconcile refactor, and interleaving-test guidance).
2026-05-30feat(chunks): chunk-native frontend store with turn-sealed reconcile + ↵Adam Malczewski
per-chunk eviction Replace the stored ChatMessage[] with a chunk-native model: tab.chunks (sealed ChunkRow[]) + tab.live (transient in-flight turn buffer) + derived tab.renderGroups. This enables per-chunk eviction (trimming WITHIN a large turn) and raw-chunk pagination (loadOlderChunks), removing the whole-message eviction limitation. Backend: - Emit turn-start/turn-sealed around each turn; expose currentTurnId in the status snapshot. turn-sealed fires after the durable write (status:idle fires before it). - New GET /tabs/:id/chunks raw paginated endpoint (limit/before). - Wrap appendChunks in a single SQLite transaction. Frontend: - turn-sealed drives a turn-aware reconcile that folds the sealed turn into chunks while preserving a concurrent newer in-flight turn and pending queued messages; deferred while the user is scrolled up. - Stable turn-scoped render keys (${turnId}:${role}:${n}) avoid remount/flash. Reconcile correctness (three review passes): - preserve a concurrent newer turn when an earlier deferred reconcile flushes; - keep optimistic queued user messages (no loss); - turn-start backfill skips pending queued rows and tags only the turn initiator; - bind consumed interrupt messages to the in-flight turn so they collapse on seal (no lingering/duplicated bubble). Tests: chat-store reconcile/eviction/pagination suite; api chunks endpoint + events.
2026-05-30refactor(chunks): append-only chunk log with per-step cache-stable wireAdam Malczewski
Replace the message-as-container model with a flat, append-only chunk log. - chunks table (id, tab_id, seq, turn_id, step, role, type, data_json): one row per chunk; tool_call (assistant) and tool_result (tool) are SEPARATE rows linked by callId. Message/turn are derived groupings, not stored. - chunks/transform.ts: DB-free explode (Chunk[] -> rows) / group (rows -> messages), shared by backend and the browser frontend. - Cache fix: toModelMessages segments each turn at tool-batch boundaries into stable [assistant, tool] pairs per step, so earlier steps serialize byte-identically across requests (kills the prompt-cache churn). - agent-manager persists a turn's chunks on seal (once), discarding a failed fallback attempt's partial chunks; rebuilds agent history from the log. - GET /messages windows the log by chunk seq then groups; loadMoreMessages merges a turn split across the window boundary by turnId. - One-shot migration drops the legacy messages table and clears tabs; settings/credentials/keys/usage preserved. Full suite green (317 tests); biome, tsc, and svelte-check clean.
2026-05-29feat: disappearing chat history — chunk-limited frontend window with ↵Adam Malczewski
backend pagination Frontend keeps only a bounded window of chunks in memory (configurable via settings slider, default 100). Older messages are evicted when at the bottom and re-fetched from the backend on scroll-up. - Backend: paginated GET /tabs/:id/messages with ?limit=N&before=seq - Store: evictMessages trims oldest messages until total chunks ≤ limit - Store: loadMoreMessages fetches next page and prepends with dedup - ChatPanel: smart scroll hooks trigger eviction on return-to-bottom - ChatPanel: onNearTop loads older history with scroll-position maintenance - Settings: chunk limit slider in Memory section - Fix: oldestLoadedSeq recalculated after eviction (pagination cursor stays valid) - Fix: seq preserved on ChatMessage for cursor tracking - Fix: scrolledUpTabs cleaned up on tab switch (no memory leak) - Fix: evictMessages reads appSettings.chunkLimit directly (live updates)
2026-05-28fix(core): strip stale [USER INTERRUPT] from LLM history; inject into last ↵Adam Malczewski
tool of batch The interrupt block embedded in a tool-result was persistent in the assistant message history, so the imperative 'You MUST address these before continuing' got re-evaluated as fresh on every subsequent LLM step. Result: the model repeatedly thought about and re-acknowledged the same interrupt 5-10+ times per chat (verified in production DB traces — e.g. tab 4c5727aa had 11 thinking chunks quoting a single interrupt verbatim). agent.ts (toModelMessages): strip [USER INTERRUPT] from every tool- result except the one in the freshest tool-batch (last chunk of the last assistant message, which itself must be the last message). The strip is a serialization-time transform only — this.messages, the DB row, and the UI display all keep the full text. The LLM sees the imperative exactly once: the step immediately after injection. agent.ts (tool execution loop): batch queued messages across the group's tool calls and inject them only into the LAST executable tool's result. Previously the first tool to dequeue won; now the interrupt lands in a single deterministic spot regardless of timing. Tool-level handlers (run-shell/youtube/retrieve) are untouched — they still embed their own interrupt text when they background work. Also fix pre-existing tabs.test.ts: it referenced a getDescendantIds function that didn't exist (added: BFS, leaf-first, cycle-safe, skips archived) and imported bun:sqlite directly which vite couldn't resolve (rewrote with a minimal FakeDatabase + vi.mock pattern matching the rest of the suite).
2026-05-27refactor: ChatMessage.chunks[] union — interleaved thinking, tool ↵Adam Malczewski
batching, error/system chunks
2026-05-22feat: agent builder, CWD support, auto-save, UI polish, unavailable tool ↵Adam Malczewski
handling - Agent Builder: full CRUD with card grid, drag-and-drop model reorder, edit/delete - Auto-save on edit with 600ms debounce, AbortController for concurrency, fieldset disabled until name entered - Agent definitions stored as TOML with cwd field, loaded from global/project dirs - Working directory: per-tab CWD override in Chat Settings, agent default CWD, auto-create on first message - CWD validation: check-dir endpoint with ~ expansion, real-time validity indicator - Subagent CWD validated against parent's effective CWD using path.relative - Unavailable tool calls: caught gracefully, shown as tool call with error badge, model retries - UI: tab bar border radius, sidebar border removed, chat input ghost style, scroll-to-bottom rectangle - Skills dir collapse uses CSS rotation, Model Choice renamed to Chat Settings, System Prompt view removed - Reusable SkillsBrowser/ToolPermissions with external mode for Agent Builder - ModelSelector: Agent/Manual toggle, agent list, Agent Settings link - Page router, skills recursive scanning, bin/up gopass removed, docker volume mounts
2026-05-22feat: two-row tab bar with temp/persistent subagent tabs, tab persistenceAdam Malczewski
- Split tab bar into user tabs (top) and subagent tabs (bottom row) - Bottom row only visible when active user tab has child agents - Parent user tab stays highlighted when viewing a subagent tab - Temp tabs auto-disappear when agent completes, click to promote to persistent - 'Open Tab' button on summon tool calls to view/reopen subagent history - Reopening archived tabs fetches full chat history from backend - Add parent_tab_id column to DB with safe ALTER TABLE migration - Persist keyId, modelId, parentTabId on child tab creation - Flush partial assistant messages on abort/error (finally block) - Defensive JSON.parse in message restoration - Allow all Vite hosts for local development - Fix nested button in ToolCallDisplay (button inside button)
2026-05-21feat: tab system with per-tab agents, DB persistence, and DaisyUI tabs-lift UIAdam Malczewski
- Add tabs, messages, and settings tables to SQLite database - Backend: refactor AgentManager to manage per-tab Agent instances via Map<tabId, TabAgent> - Backend: WebSocket events tagged with tabId for multiplexing - Backend: tab CRUD routes (create, list, update, archive, messages) - Backend: persist user and assistant messages to DB during chat - Frontend: new tabStore replaces single chatStore with multi-tab reactive state - Frontend: TabBar component using DaisyUI tabs-lift style with status dots - Frontend: Settings sidebar panel for title generation model selection - Frontend: wire ChatPanel, ChatInput, Header to use tabStore - Fix HMR listener accumulation via wsClient.clearCallbacks() - Delete old single-chat chatStore (chat.svelte.ts)
2026-05-21feat: SQLite database for all credentials, keys, wake schedule, and usage cacheAdam Malczewski
- Add SQLite database at ~/.local/share/dispatch/dispatch.db with tables: credentials, api_keys, wake_schedule, usage_cache - Store Claude OAuth credentials in DB with import button in Model Status UI - Store OpenCode/Copilot API keys in DB with paste-to-import modal - Store OpenCode cookie and workspace IDs in DB - Migrate wake schedule from .wake-schedule.json to DB - Migrate usage cache from in-memory Map + localStorage to DB - Remove all env var and file fallbacks — DB is the single source of truth - Add seed scripts: bin/import-credentials.ts, bin/seed-opencode-keys.ts - Docker: container runs as host UID/GID with matching home directory - Clean up dispatch.toml: remove env fields, update comments - Progress bar time markers for usage cycle tracking