diff options
| author | Adam Malczewski <[email protected]> | 2026-05-30 20:06:31 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-05-30 20:06:31 +0900 |
| commit | 0f39b6f78957aacf206012ad2193d9b0c1940c1f (patch) | |
| tree | ff5f2da8b4f3cdf56cf50d44b8fec75a489ad6fe /packages/core/src/db/index.ts | |
| parent | 8c58a973b0d021689cebad5c0cc6d56956bbc2f6 (diff) | |
| download | dispatch-0f39b6f78957aacf206012ad2193d9b0c1940c1f.tar.gz dispatch-0f39b6f78957aacf206012ad2193d9b0c1940c1f.zip | |
refactor(chunks): append-only chunk log with per-step cache-stable wire
Replace the message-as-container model with a flat, append-only chunk log.
- chunks table (id, tab_id, seq, turn_id, step, role, type, data_json): one
row per chunk; tool_call (assistant) and tool_result (tool) are SEPARATE
rows linked by callId. Message/turn are derived groupings, not stored.
- chunks/transform.ts: DB-free explode (Chunk[] -> rows) / group (rows ->
messages), shared by backend and the browser frontend.
- Cache fix: toModelMessages segments each turn at tool-batch boundaries into
stable [assistant, tool] pairs per step, so earlier steps serialize
byte-identically across requests (kills the prompt-cache churn).
- agent-manager persists a turn's chunks on seal (once), discarding a failed
fallback attempt's partial chunks; rebuilds agent history from the log.
- GET /messages windows the log by chunk seq then groups; loadMoreMessages
merges a turn split across the window boundary by turnId.
- One-shot migration drops the legacy messages table and clears tabs;
settings/credentials/keys/usage preserved.
Full suite green (317 tests); biome, tsc, and svelte-check clean.
Diffstat (limited to 'packages/core/src/db/index.ts')
| -rw-r--r-- | packages/core/src/db/index.ts | 46 |
1 files changed, 38 insertions, 8 deletions
diff --git a/packages/core/src/db/index.ts b/packages/core/src/db/index.ts index e63b266..18dd1b5 100644 --- a/packages/core/src/db/index.ts +++ b/packages/core/src/db/index.ts @@ -93,16 +93,46 @@ export function getDatabase(): Database { // Column already exists — ignore } - _db.run(`CREATE TABLE IF NOT EXISTS messages ( - id TEXT PRIMARY KEY, - tab_id TEXT NOT NULL REFERENCES tabs(id), - seq INTEGER NOT NULL, - role TEXT NOT NULL, - content_json TEXT NOT NULL, - created_at INTEGER NOT NULL + // ─── Append-only chunk log (replaces the old `messages` blob table) ── + // + // A conversation is stored as a flat, append-only stream of chunk rows + // keyed by a per-tab monotonic `seq`. "Message" and "turn" are DERIVED + // groupings (see db/chunks.ts), never stored containers. This is what + // powers per-chunk frontend pagination AND the stable per-step wire + // format that fixes Anthropic prompt-cache churn (see plan-chunk-log.md). + // + // role : 'user' | 'assistant' | 'tool' | 'system' + // type : 'text' | 'thinking' | 'tool_call' | 'tool_result' | 'error' | 'system' + // step : LLM round-trip index within a turn (user/system rows = 0) + // data_json: the type-specific payload (see ChunkData in types) + _db.run(`CREATE TABLE IF NOT EXISTS chunks ( + id TEXT PRIMARY KEY, + tab_id TEXT NOT NULL, + seq INTEGER NOT NULL, + turn_id TEXT NOT NULL, + step INTEGER NOT NULL DEFAULT 0, + role TEXT NOT NULL, + type TEXT NOT NULL, + data_json TEXT NOT NULL, + created_at INTEGER NOT NULL )`); - _db.run(`CREATE INDEX IF NOT EXISTS idx_messages_tab ON messages(tab_id, seq)`); + _db.run(`CREATE INDEX IF NOT EXISTS idx_chunks_tab_seq ON chunks(tab_id, seq)`); + + // One-shot migration off the legacy `messages` blob model. Beta software, + // no backward compatibility: the old chat history is destroyed (tabs + + // messages), while settings / credentials / api_keys / usage_cache / + // wake_schedule are preserved. Detect the old schema by the presence of + // the `messages` table; once dropped, this branch never runs again. + const hasLegacyMessages = _db + .query("SELECT name FROM sqlite_master WHERE type='table' AND name='messages'") + .get() as { name: string } | null; + if (hasLegacyMessages) { + _db.run("DROP TABLE IF EXISTS messages"); + // Clear conversation containers too (fresh slate for the new model). + _db.run("DELETE FROM tabs"); + _db.run("DELETE FROM chunks"); + } _db.run(`CREATE TABLE IF NOT EXISTS settings ( key TEXT PRIMARY KEY, |
