diff options
| author | Adam Malczewski <[email protected]> | 2026-06-22 00:36:52 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-22 00:36:52 +0900 |
| commit | c1bfd62b0c734484efcff09d6cd521acdbab2640 (patch) | |
| tree | f9194ea3214d5f15be6fed659a230997a1309ab9 | |
| parent | 7ff9f94c41a9870e124a50133cd74b42295ab9ac (diff) | |
| download | dispatch-c1bfd62b0c734484efcff09d6cd521acdbab2640.tar.gz dispatch-c1bfd62b0c734484efcff09d6cd521acdbab2640.zip | |
feat: conversation compacting (manual + automatic)
Implement roadmap item 10: conversation compaction to reclaim context
window without losing the thread.
Wire (0.11.0):
- Add CompactionResult type
- Add ConversationCompactedMessage WS event
Transport-contract (0.15.0):
- Add CompactResponse, CompactThresholdResponse, SetCompactThresholdRequest
- Add ConversationCompactedMessage to WsServerMessage union
- Re-export CompactionResult
Conversation-store:
- replaceHistory: delete all chunks, reset seq, append new messages
- getCompactThreshold / setCompactThreshold (per-conversation setting)
- compactThresholdKey added to keys.ts
Session-orchestrator:
- CompactionService interface + compactionHandle
- conversationCompacted hook descriptor
- createCompactionService: load history, split old/recent, call provider
to summarize, replaceHistory with [system: summary] + recent N
- Auto-trigger: resolveCompaction lazy dep, fires after turn settles
(checks threshold, non-blocking)
- Hook declared in manifest contributes.hooks + services
Transport-http:
- POST /conversations/:id/compact (manual trigger)
- GET /conversations/:id/compact-threshold (read setting)
- PUT /conversations/:id/compact-threshold (set setting)
Transport-ws:
- Subscribe to conversationCompacted hook
- Broadcast conversation.compacted WS message
CLI:
- dispatch compact <conversationId> command
FE handoff: frontend-compaction-handoff.md
23 files changed, 672 insertions, 5 deletions
diff --git a/frontend-compaction-handoff.md b/frontend-compaction-handoff.md new file mode 100644 index 0000000..0514af9 --- /dev/null +++ b/frontend-compaction-handoff.md @@ -0,0 +1,115 @@ +# FE handoff — conversation compacting + +Courier this to `../dispatch-web`. All changes are ADDITIVE. + +## What shipped (backend) + +Conversation compaction: summarize old history and replace it with a summary + +retain the most recent N messages. Two modes: +- **Manual**: `POST /conversations/:id/compact` — triggers immediately. +- **Automatic**: after each turn settles, the backend checks if the last turn's + input tokens exceeded the per-conversation `compactThreshold`. If so, + compaction runs automatically (fire-and-forget, non-blocking). + +## Bump pinned deps +- `@dispatch/wire` → `0.11.0` +- `@dispatch/transport-contract` → `0.15.0` + +## New types + +```ts +// @dispatch/wire +export interface CompactionResult { + readonly summary: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} + +// @dispatch/transport-contract — WS message (server → client) +export interface ConversationCompactedMessage { + readonly type: "conversation.compacted"; + readonly conversationId: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} +// Added to WsServerMessage union. + +// @dispatch/transport-contract — HTTP response types +export interface CompactResponse { + readonly conversationId: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} + +export interface CompactThresholdResponse { + readonly conversationId: string; + readonly threshold: number; // 0 = manual only +} + +export interface SetCompactThresholdRequest { + readonly threshold: number; +} +``` + +## `POST /conversations/:id/compact` — manual compaction + +Triggers compaction on demand. Optional JSON body: +```json +{ "keepLastN": 10, "modelName": "umans/umans-glm-5.2" } +``` +- `keepLastN` (default 10): how many recent messages to retain. +- `modelName`: override the model used for summarization (defaults to the + conversation's provider). + +200 response: `CompactResponse` +409 response: `{ error: string }` — conversation is generating, too short, etc. +503: compaction service not available. + +## `GET /conversations/:id/compact-threshold` — read threshold + +200: `CompactThresholdResponse { conversationId, threshold }` +`threshold: 0` means auto-compact is disabled (manual only). + +## `PUT /conversations/:id/compact-threshold` — set threshold + +Body: `SetCompactThresholdRequest { threshold: number }` +`threshold: 0` disables auto-compact. Any positive number enables it. + +200: `CompactThresholdResponse` + +## `conversation.compacted` WS message + +Broadcast to all connected WS clients when compaction completes. The FE should +reload the conversation history via `GET /conversations/:id` to reflect the +compacted state (the old messages are replaced by a system summary + recent N). + +## How compaction works (backend) + +1. Load full conversation history. +2. Split: old messages (to summarize) + recent N (to keep, default 10). +3. Call the model with a summarization system prompt + old messages. +4. Replace the entire history with: `[system: "Summary of previous conversation: ..."] + recent N messages`. +5. Emit `conversationCompacted` hook → WS broadcast. +6. The summary preserves key decisions, file paths, and unresolved questions. + +## What the FE needs to do + +1. **Compact button** in the conversation toolbar → `POST /conversations/:id/compact`. + Show a loading indicator while waiting for the response. + +2. **Settings UI** for compact threshold: `PUT /conversations/:id/compact-threshold` + with `{ threshold: number }`. A number input (0 = manual only). Read the + current value via `GET /conversations/:id/compact-threshold`. + +3. **Handle `conversation.compacted` WS messages**: reload the conversation + history via `GET /conversations/:id`. The first message will now be a system + summary instead of the original conversation. + +4. **Visual indicator**: show a badge or divider indicating the conversation + has been compacted (e.g. "N messages summarized" from the WS payload or the + HTTP response). + +## CLI + +`dispatch compact <conversationId>` — triggers manual compaction. Resolves +short IDs like other commands. diff --git a/packages/cli/src/args.ts b/packages/cli/src/args.ts index 30fa309..ecf6e2e 100644 --- a/packages/cli/src/args.ts +++ b/packages/cli/src/args.ts @@ -34,6 +34,7 @@ export type ParsedCommand = readonly status?: string; readonly all: boolean; } + | { readonly kind: "compact"; readonly server: string; readonly conversationId: string } | { readonly kind: "open"; readonly server: string; readonly conversationId: string } | { readonly kind: "read"; readonly server: string; readonly conversationId: string } | { @@ -108,6 +109,28 @@ export function parseArgs(argv: readonly string[], opts: ParseOpts): ParsedComma }; } + if (first === "compact") { + let server = opts.defaultServer; + let conversationId: string | undefined; + for (let i = 1; i < argv.length; i++) { + const arg = argv[i] as string; + if (arg === "--server") { + if (i + 1 >= argv.length) return { kind: "error", message: "--server requires a value" }; + server = argv[++i] as string; + } else if (arg.startsWith("--")) { + return { kind: "error", message: `Unknown flag: ${arg}` }; + } else if (conversationId !== undefined) { + return { kind: "error", message: `Unexpected argument for 'compact': ${arg}` }; + } else { + conversationId = arg; + } + } + if (conversationId === undefined) { + return { kind: "error", message: "'compact' requires a conversation id" }; + } + return { kind: "compact", server, conversationId }; + } + if (first === "read") { let server = opts.defaultServer; let conversationId: string | undefined; diff --git a/packages/cli/src/http.ts b/packages/cli/src/http.ts index 876f570..585c678 100644 --- a/packages/cli/src/http.ts +++ b/packages/cli/src/http.ts @@ -10,6 +10,7 @@ import type { AgentEvent, ChatRequest, + CompactResponse, ConversationListResponse, LastMessageResponse, ModelsResponse, @@ -183,6 +184,26 @@ export async function openConversation( return (await res.json()) as OpenConversationResponse; } +interface CompactConversationOpts { + readonly server: string; + readonly conversationId: string; +} + +export async function compactConversation( + deps: FetchDeps, + opts: CompactConversationOpts, +): Promise<CompactResponse> { + const url = `${opts.server}/conversations/${encodeURIComponent(opts.conversationId)}/compact`; + const res = await deps.fetchImpl(url, { method: "POST" }); + + if (!res.ok) { + const body = await res.text(); + throw new Error(`POST /conversations/:id/compact failed with status ${res.status}: ${body}`); + } + + return (await res.json()) as CompactResponse; +} + /** * The outcome of short-ID resolution: either the full conversation id to use, * or a human-readable error describing why resolution failed. diff --git a/packages/cli/src/main.ts b/packages/cli/src/main.ts index e709f21..4e07da9 100644 --- a/packages/cli/src/main.ts +++ b/packages/cli/src/main.ts @@ -9,6 +9,7 @@ import { readFile } from "node:fs/promises"; import { parseArgs } from "./args.js"; import { formatCatalog } from "./catalog.js"; import { + compactConversation, enqueueMessage, fetchConversations, fetchLastMessage, @@ -23,6 +24,7 @@ import { extractLastText, formatConversationList, renderEvent } from "./render.j const USAGE = `Usage: dispatch models [--server <url>] dispatch list [<prefix>] [--status <active|idle|closed>] [--all] [--server <url>] + dispatch compact <conversationId> [--server <url>] dispatch read <conversationId> [--server <url>] dispatch open <conversationId> [--server <url>] dispatch send <conversationId> --text "..." [--queue] [--open] [--cwd <dir>] [--effort <level>] [--server <url>] @@ -79,6 +81,24 @@ async function main(): Promise<void> { if (last.content.length > 0) process.stdout.write(`${last.content}\n`); break; } + case "compact": { + const resolved = await resolveConversationId( + { fetchImpl: globalThis.fetch }, + { server: parsed.server, shortId: parsed.conversationId }, + ); + if (typeof resolved !== "string") { + process.stderr.write(`${resolved.error}\n`); + process.exit(1); + } + const result = await compactConversation( + { fetchImpl: globalThis.fetch }, + { server: parsed.server, conversationId: resolved }, + ); + process.stdout.write( + `Compacted ${resolved}: ${result.messagesSummarized} messages summarized, ${result.messagesKept} kept.\n`, + ); + break; + } case "open": { const resolved = await resolveConversationId( { fetchImpl: globalThis.fetch }, diff --git a/packages/conversation-store/src/keys.ts b/packages/conversation-store/src/keys.ts index 9b7c2cc..a609f82 100644 --- a/packages/conversation-store/src/keys.ts +++ b/packages/conversation-store/src/keys.ts @@ -54,6 +54,10 @@ export function reasoningEffortKey(conversationId: string): string { return `conv:${conversationId}:reasoning-effort`; } +export function compactThresholdKey(conversationId: string): string { + return `conv:${conversationId}:compact-threshold`; +} + export function metaKey(conversationId: string): string { return `conv:${conversationId}:meta`; } diff --git a/packages/conversation-store/src/store.ts b/packages/conversation-store/src/store.ts index 8d78df8..2f9f475 100644 --- a/packages/conversation-store/src/store.ts +++ b/packages/conversation-store/src/store.ts @@ -15,6 +15,7 @@ import { CONVERSATION_INDEX_KEY, chunkKey, chunkPrefix, + compactThresholdKey, cwdKey, metaKey, metricsKey, @@ -86,6 +87,20 @@ export interface ConversationStore { conversationId: string, status: ConversationStatus, ) => Promise<void>; + /** + * Replace the entire conversation history with the given messages. Deletes + * all existing chunks, resets the seq counter, and appends the new messages. + * Used by compaction to replace old history with a summary + recent messages. + * Metadata (createdAt, title, status) is preserved. + */ + readonly replaceHistory: ( + conversationId: string, + messages: readonly ChatMessage[], + ) => Promise<void>; + /** Get the compact threshold (token count, 0 = manual only), or null if unset. */ + readonly getCompactThreshold: (conversationId: string) => Promise<number | null>; + /** Set the compact threshold (token count, 0 = manual only). */ + readonly setCompactThreshold: (conversationId: string, threshold: number) => Promise<void>; } export const conversationStoreHandle = defineService<ConversationStore>("conversation-store/store"); @@ -529,5 +544,32 @@ export function createConversationStore( }; await storage.set(metaKey(conversationId), JSON.stringify(row)); }, + + async replaceHistory(conversationId, messages) { + // Delete all existing chunks. + const keys = await storage.keys(chunkPrefix(conversationId)); + for (const k of keys) { + await storage.delete(k); + } + // Reset the seq counter so the new messages start from seq 1. + await storage.set(seqKey(conversationId), "0"); + // Append the new messages (re-uses the append logic for seq + // numbering + metadata upsert). + await this.append(conversationId, messages); + }, + + async getCompactThreshold(conversationId) { + const raw = await storage.get(compactThresholdKey(conversationId)); + if (raw === null) return null; + const n = Number.parseInt(raw, 10); + return Number.isNaN(n) ? null : n; + }, + + async setCompactThreshold(conversationId, threshold) { + await storage.set(compactThresholdKey(conversationId), String(threshold)); + if (logger !== undefined) { + logger.debug("compact-threshold set", { conversationId, threshold }); + } + }, }; } diff --git a/packages/kernel/src/contracts/conversation.ts b/packages/kernel/src/contracts/conversation.ts index 9b7d2f6..009d295 100644 --- a/packages/kernel/src/contracts/conversation.ts +++ b/packages/kernel/src/contracts/conversation.ts @@ -8,6 +8,7 @@ export type { ChatMessage, Chunk, + CompactionResult, ConversationMeta, ConversationStatus, ErrorChunk, diff --git a/packages/kernel/src/contracts/index.ts b/packages/kernel/src/contracts/index.ts index 3aa9740..65bf910 100644 --- a/packages/kernel/src/contracts/index.ts +++ b/packages/kernel/src/contracts/index.ts @@ -15,6 +15,7 @@ export type { export type { ChatMessage, Chunk, + CompactionResult, ConversationMeta, ConversationStatus, ErrorChunk, diff --git a/packages/session-orchestrator/src/extension.ts b/packages/session-orchestrator/src/extension.ts index cbb3def..ae90902 100644 --- a/packages/session-orchestrator/src/extension.ts +++ b/packages/session-orchestrator/src/extension.ts @@ -5,6 +5,8 @@ import { runTurn } from "@dispatch/kernel"; import { messageQueueHandle } from "@dispatch/message-queue"; import { cacheWarmHandle, + compactionHandle, + createCompactionService, createSessionOrchestrator, createWarmService, sessionOrchestratorHandle, @@ -21,13 +23,18 @@ export const manifest: Manifest = { dependsOn: ["conversation-store", "credential-store"], activation: "eager", contributes: { - services: ["session-orchestrator/orchestrator", "session-orchestrator/warm"], + services: [ + "session-orchestrator/orchestrator", + "session-orchestrator/warm", + "session-orchestrator/compaction", + ], hooks: [ "session-orchestrator/turn-started", "session-orchestrator/turn-settled", "session-orchestrator/warm-completed", "session-orchestrator/conversation-closed", "session-orchestrator/conversation-status-changed", + "session-orchestrator/conversation-compacted", ], }, }; @@ -60,6 +67,16 @@ export function activate(host: HostAPI): void { const loaded = host.getExtensions().some((m) => m.id === "message-queue"); return loaded ? host.getService(messageQueueHandle) : undefined; }, + resolveCompaction: () => { + // Lazily resolve the compaction service (registered below after + // the orchestrator). By the time this is called at runtime + // (after a turn settles), the service is registered. + try { + return host.getService(compactionHandle); + } catch { + return undefined; + } + }, }); host.provideService(sessionOrchestratorHandle, orchestrator); @@ -86,6 +103,29 @@ export function activate(host: HostAPI): void { ); host.provideService(cacheWarmHandle, warmService); + + const compactionService = createCompactionService( + { + conversationStore, + resolveProvider: () => selectFirstProvider(host.getProviders()), + resolveTools: () => [...host.getTools().values()], + resolveModel: (modelName: string) => { + const store = host.getService(credentialStoreHandle); + const r = store.resolve(modelName); + if (r === undefined) return undefined; + const provider = host.getProviders().get(r.providerId); + return provider ? { provider, model: r.model } : undefined; + }, + applyToolsFilter: (assembly) => host.applyFilters(toolsFilter, assembly), + runTurn, + logger: host.logger, + now: () => Date.now(), + emit: (hook, payload) => host.emit(hook, payload), + }, + activeConversations, + ); + + host.provideService(compactionHandle, compactionService); } export const extension: Extension = { diff --git a/packages/session-orchestrator/src/index.ts b/packages/session-orchestrator/src/index.ts index d2aacd9..fa8d9e9 100644 --- a/packages/session-orchestrator/src/index.ts +++ b/packages/session-orchestrator/src/index.ts @@ -1,12 +1,17 @@ export { extension, manifest } from "./extension.js"; export { + type CompactionService, type ConversationClosedPayload, + type ConversationCompactedPayload, type ConversationOpenedPayload, type ConversationStatusChangedPayload, cacheWarmHandle, + compactionHandle, conversationClosed, + conversationCompacted, conversationOpened, conversationStatusChanged, + createCompactionService, createSessionOrchestrator, createWarmService, type EnqueueInput, diff --git a/packages/session-orchestrator/src/orchestrator.test.ts b/packages/session-orchestrator/src/orchestrator.test.ts index 53c1ce7..e657fe2 100644 --- a/packages/session-orchestrator/src/orchestrator.test.ts +++ b/packages/session-orchestrator/src/orchestrator.test.ts @@ -90,6 +90,11 @@ function createInMemoryStore(): ConversationStore & { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; } @@ -540,6 +545,11 @@ describe("turn-sealed event", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const { orchestrator } = createSessionOrchestrator({ @@ -606,6 +616,11 @@ describe("turn-sealed event", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const { orchestrator } = createSessionOrchestrator({ @@ -961,6 +976,11 @@ describe("turn metrics persistence", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const { orchestrator } = createSessionOrchestrator({ diff --git a/packages/session-orchestrator/src/orchestrator.ts b/packages/session-orchestrator/src/orchestrator.ts index 4ce83ca..e2d126c 100644 --- a/packages/session-orchestrator/src/orchestrator.ts +++ b/packages/session-orchestrator/src/orchestrator.ts @@ -2,6 +2,7 @@ import type { ConversationStore } from "@dispatch/conversation-store"; import type { AgentEvent, ChatMessage, + CompactionResult, ConversationStatus, EventHookDescriptor, Logger, @@ -128,6 +129,21 @@ export const conversationStatusChanged: EventHookDescriptor<ConversationStatusCh "session-orchestrator/conversation-status-changed", ); +/** Payload for the conversationCompacted bus event. */ +export interface ConversationCompactedPayload { + readonly conversationId: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} + +/** + * Fired when a conversation's history has been compacted (old messages + * summarized, recent messages retained). Transport-ws subscribes and + * broadcasts a `conversation.compacted` WS message so the FE reloads history. + */ +export const conversationCompacted: EventHookDescriptor<ConversationCompactedPayload> = + defineEventHook<ConversationCompactedPayload>("session-orchestrator/conversation-compacted"); + /** Payload for the warmCompleted bus event. */ export interface WarmCompletedPayload { readonly conversationId: string; @@ -158,6 +174,26 @@ export const cacheWarmHandle: ServiceHandle<WarmService> = defineService<WarmSer "session-orchestrator/warm", ); +// --- Compaction service --- + +export interface CompactionService { + /** + * Compact a conversation: summarize old messages and replace history with + * the summary + the most recent `keepLastN` messages. Returns the result + * or an error object. No-ops if the conversation is too short (≤ keepLastN + * messages). When `auto` is true, checks the compact-threshold setting and + * only compacts if the last turn's input tokens exceeded it. + */ + readonly compact: ( + conversationId: string, + opts?: { readonly keepLastN?: number; readonly modelName?: string; readonly auto?: boolean }, + ) => Promise<CompactionResult | { readonly error: string }>; +} + +export const compactionHandle: ServiceHandle<CompactionService> = defineService<CompactionService>( + "session-orchestrator/compaction", +); + export interface SessionOrchestrator { startTurn(input: StartTurnInput): StartTurnResult; /** @@ -214,6 +250,12 @@ export interface SessionOrchestratorDeps { * stays reproducible from its inputs and tests use a fake queue. */ readonly resolveQueue?: () => MessageQueueService | undefined; + /** + * Lazily resolves the compaction service, or `undefined` when not loaded. + * Used for automatic compaction after a turn settles (if the compact + * threshold is exceeded). Lazy so activation order doesn't matter. + */ + readonly resolveCompaction?: () => CompactionService | undefined; /** Apply the per-turn tools filter chain. Injected for testability. */ readonly applyToolsFilter: (assembly: ToolAssembly) => Promise<ToolAssembly>; /** Base logger (auto-scoped to this extension); childed per turn for span capture. */ @@ -444,6 +486,13 @@ export function createSessionOrchestrator( status: "idle", }); void deps.conversationStore.setConversationStatus(conversationId, "idle"); + // Fire-and-forget auto-compaction: check threshold and + // compact if exceeded. Non-blocking — the next turn + // starts fresh either way. + const compaction = deps.resolveCompaction?.(); + if (compaction !== undefined) { + void compaction.compact(conversationId, { auto: true }).catch(() => {}); + } } }); } @@ -642,3 +691,139 @@ export function createWarmService( }, }; } + +const DEFAULT_KEEP_LAST_N = 10; + +const COMPACTION_SYSTEM_PROMPT = + "You are a conversation summarizer. Summarize the following conversation concord concisely but comprehensively. " + + "Focus on key decisions, context, file paths, and any unresolved questions. " + + "The summary must preserve enough detail for the conversation to continue with full context."; + +function formatMessagesForSummary(messages: readonly ChatMessage[]): string { + return messages + .map((msg) => { + const text = msg.chunks + .map((c) => { + if (c.type === "text") return c.text; + if (c.type === "tool-call") return `[tool: ${c.toolName}]`; + if (c.type === "tool-result") return `[tool result: ${c.content.slice(0, 200)}]`; + return ""; + }) + .join(""); + return `${msg.role}: ${text}`; + }) + .join("\n\n"); +} + +export function createCompactionService( + deps: SessionOrchestratorDeps & { + readonly emit: <TPayload>(hook: EventHookDescriptor<TPayload>, payload: TPayload) => void; + }, + activeConversations: ReadonlySet<string>, +): CompactionService { + return { + async compact(conversationId, opts) { + if (activeConversations.has(conversationId)) { + return { error: "conversation is generating" }; + } + + const history = await deps.conversationStore.load(conversationId); + const keepLastN = opts?.keepLastN ?? DEFAULT_KEEP_LAST_N; + + if (history.length <= keepLastN) { + return { error: "conversation too short to compact" }; + } + + // Auto mode: check threshold + if (opts?.auto === true) { + const threshold = await deps.conversationStore.getCompactThreshold(conversationId); + if (threshold === null || threshold <= 0) return { error: "auto-compact disabled" }; + const metrics = await deps.conversationStore.loadMetrics(conversationId); + const lastTurn = metrics[metrics.length - 1]; + if (lastTurn === undefined) return { error: "no metrics" }; + const lastInputTokens = lastTurn.usage.inputTokens + (lastTurn.usage.cacheReadTokens ?? 0); + if (lastInputTokens < threshold) return { error: "threshold not exceeded" }; + } + + // Split: old messages to summarize + recent messages to keep. + const toSummarize = history.slice(0, history.length - keepLastN); + const toKeep = history.slice(history.length - keepLastN); + + // Resolve provider + let provider: ProviderContract; + let modelOverride: string | undefined; + if (opts?.modelName !== undefined && deps.resolveModel !== undefined) { + const resolved = deps.resolveModel(opts.modelName); + if (resolved === undefined) return { error: `unknown model: ${opts.modelName}` }; + provider = resolved.provider; + modelOverride = resolved.model; + } else { + provider = deps.resolveProvider(); + } + + // Build the summarization request: system prompt + conversation text + instruction + const conversationText = formatMessagesForSummary(toSummarize); + const summaryRequest: ChatMessage = { + role: "user", + chunks: [ + { + type: "text", + text: `Please summarize the following conversation:\n\n${conversationText}`, + }, + ], + }; + + const providerOpts: ProviderStreamOptions = { + maxTokens: 2000, + ...(modelOverride !== undefined ? { model: modelOverride } : {}), + ...(deps.logger !== undefined + ? { logger: deps.logger.child({ conversationId, attrs: { compaction: true } }) } + : {}), + }; + + // Call the provider and accumulate the summary + let summary = ""; + for await (const event of provider.stream([summaryRequest], [], { + ...providerOpts, + systemPrompt: COMPACTION_SYSTEM_PROMPT, + })) { + if ((event as ProviderEvent).type === "text-delta") { + summary += (event as { delta: string }).delta; + } else if ((event as ProviderEvent).type === "error") { + return { error: (event as { message: string }).message }; + } + } + + if (summary.trim().length === 0) { + return { error: "model produced empty summary" }; + } + + // Replace history: [system: summary] + recent messages + const summaryMessage: ChatMessage = { + role: "system", + chunks: [ + { + type: "text", + text: `The following is a summary of the previous conversation:\n\n${summary}`, + }, + ], + }; + + await deps.conversationStore.replaceHistory(conversationId, [summaryMessage, ...toKeep]); + + const result: CompactionResult = { + summary, + messagesSummarized: toSummarize.length, + messagesKept: toKeep.length, + }; + + deps.emit(conversationCompacted, { + conversationId, + messagesSummarized: toSummarize.length, + messagesKept: toKeep.length, + }); + + return result; + }, + }; +} diff --git a/packages/session-orchestrator/src/queue.test.ts b/packages/session-orchestrator/src/queue.test.ts index 745ac27..ae24ddf 100644 --- a/packages/session-orchestrator/src/queue.test.ts +++ b/packages/session-orchestrator/src/queue.test.ts @@ -86,6 +86,11 @@ function createInMemoryStore(): ConversationStore & { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; } diff --git a/packages/transport-contract/package.json b/packages/transport-contract/package.json index 682069b..b79b3b3 100644 --- a/packages/transport-contract/package.json +++ b/packages/transport-contract/package.json @@ -1,6 +1,6 @@ { "name": "@dispatch/transport-contract", - "version": "0.14.0", + "version": "0.15.0", "type": "module", "private": true, "main": "dist/index.js", diff --git a/packages/transport-contract/src/index.ts b/packages/transport-contract/src/index.ts index 237f4ab..86eb807 100644 --- a/packages/transport-contract/src/index.ts +++ b/packages/transport-contract/src/index.ts @@ -32,6 +32,7 @@ import type { export type { AgentEvent, + CompactionResult, ConversationMeta, ConversationStatus, QueuedMessage, @@ -486,7 +487,8 @@ export type WsServerMessage = | ChatDeltaMessage | ChatErrorMessage | ConversationOpenMessage - | ConversationStatusChangedMessage; + | ConversationStatusChangedMessage + | ConversationCompactedMessage; // ─── Conversation list + metadata ──────────────────────────────────────────── @@ -512,6 +514,18 @@ export interface ConversationStatusChangedMessage { } /** + * Broadcast to all connected WS clients when a conversation's history has been + * compacted (summarized). The frontend should reload the conversation history + * via `GET /conversations/:id` to reflect the compacted state. + */ +export interface ConversationCompactedMessage { + readonly type: "conversation.compacted"; + readonly conversationId: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} + +/** * Response for `GET /conversations` — the list of all known conversations, * sorted by `lastActivityAt` descending (most recent first). Each entry carries * enough metadata for a conversation picker UI (id, title, timestamps). @@ -555,3 +569,29 @@ export interface TitleResponse { readonly conversationId: string; readonly title: string; } + +/** + * Response for `POST /conversations/:id/compact` — confirms the conversation + * history was compacted (old messages summarized, recent messages retained). + */ +export interface CompactResponse { + readonly conversationId: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} + +/** + * Response for `GET /conversations/:id/compact-threshold` — the token count + * at which automatic compaction triggers (0 = manual only). + */ +export interface CompactThresholdResponse { + readonly conversationId: string; + readonly threshold: number; +} + +/** + * Request body for `PUT /conversations/:id/compact-threshold`. + */ +export interface SetCompactThresholdRequest { + readonly threshold: number; +} diff --git a/packages/transport-http/src/app.test.ts b/packages/transport-http/src/app.test.ts index 789efce..51f791f 100644 --- a/packages/transport-http/src/app.test.ts +++ b/packages/transport-http/src/app.test.ts @@ -138,6 +138,11 @@ function createFakeConversationStore( return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; } @@ -859,6 +864,11 @@ describe("GET /conversations/:id", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const app = createApp({ conversationStore: store, @@ -927,6 +937,11 @@ describe("GET /conversations/:id", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const app = createApp({ conversationStore: store, @@ -1064,6 +1079,11 @@ describe("GET /conversations/:id/metrics", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const app = createApp({ conversationStore: brokenStore, @@ -2034,6 +2054,11 @@ describe("PUT /conversations/:id/reasoning-effort", () => { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; const app = createApp({ conversationStore: store, diff --git a/packages/transport-http/src/app.ts b/packages/transport-http/src/app.ts index e9f56c9..fd78f3e 100644 --- a/packages/transport-http/src/app.ts +++ b/packages/transport-http/src/app.ts @@ -1,6 +1,8 @@ import type { AgentEvent, HostAPI, Logger } from "@dispatch/kernel"; import type { CloseConversationResponse, + CompactResponse, + CompactThresholdResponse, ConversationHistoryResponse, ConversationListResponse, ConversationMetricsResponse, @@ -12,6 +14,7 @@ import type { OpenConversationResponse, QueueResponse, ReasoningEffortResponse, + SetCompactThresholdRequest, ThroughputResponse, TitleResponse, WarmResponse, @@ -36,6 +39,7 @@ import { serializeEventLine, } from "./logic.js"; import { + type CompactionService, type ConversationStore, type CredentialStore, conversationOpened, @@ -52,6 +56,7 @@ export interface CreateServerOptions { readonly orchestrator: SessionOrchestrator; readonly credentialStore: CredentialStore; readonly warmService?: WarmService; + readonly compactionService?: CompactionService; readonly lspService?: LspService; /** Optional — defaults to a no-op store (recording disabled, empty reports). */ readonly throughputStore?: ThroughputStore; @@ -683,6 +688,79 @@ export function createApp(opts: CreateServerOptions): Hono { } }); + // ─── Compaction ────────────────────────────────────────────────────────── + + app.post("/conversations/:id/compact", async (c) => { + if (opts.compactionService === undefined) { + return c.json({ error: "Compaction service not available" }, 503); + } + const conversationId = c.req.param("id"); + let body: unknown = {}; + try { + body = await c.req.json(); + } catch { + // No body is fine — use defaults. + } + const obj = body as Record<string, unknown>; + const keepLastN = + typeof obj.keepLastN === "number" && Number.isFinite(obj.keepLastN) && obj.keepLastN > 0 + ? Math.floor(obj.keepLastN) + : undefined; + const modelName = typeof obj.modelName === "string" ? obj.modelName : undefined; + + log.info("conversations: compact request", { conversationId }); + + const result = await opts.compactionService.compact(conversationId, { + ...(keepLastN !== undefined ? { keepLastN } : {}), + ...(modelName !== undefined ? { modelName } : {}), + }); + + if ("error" in result) { + log.warn("conversations: compact returned error", { + conversationId, + error: result.error, + }); + return c.json({ error: result.error }, 409); + } + + const response: CompactResponse = { + conversationId, + messagesSummarized: result.messagesSummarized, + messagesKept: result.messagesKept, + }; + return c.json(response, 200); + }); + + app.get("/conversations/:id/compact-threshold", async (c) => { + const conversationId = c.req.param("id"); + const threshold = (await opts.conversationStore.getCompactThreshold(conversationId)) ?? 0; + const response: CompactThresholdResponse = { conversationId, threshold }; + return c.json(response, 200); + }); + + app.put("/conversations/:id/compact-threshold", async (c) => { + const conversationId = c.req.param("id"); + let body: unknown; + try { + body = await c.req.json(); + } catch { + return c.json({ error: "Invalid JSON body" }, 400); + } + const parsed = body as SetCompactThresholdRequest; + if ( + typeof parsed.threshold !== "number" || + !Number.isFinite(parsed.threshold) || + parsed.threshold < 0 + ) { + return c.json({ error: "threshold must be a non-negative number" }, 400); + } + const threshold = Math.floor(parsed.threshold); + await opts.conversationStore.setCompactThreshold(conversationId, threshold); + log.info("conversations: compact-threshold set", { conversationId, threshold }); + const response: CompactThresholdResponse = { conversationId, threshold }; + return c.json(response, 200); + }); + // ─── Static frontend serving (catch-all, API routes take precedence) ────── if (opts.webDir !== undefined) { const webDir = opts.webDir; diff --git a/packages/transport-http/src/extension.ts b/packages/transport-http/src/extension.ts index 3555f8e..351dd4a 100644 --- a/packages/transport-http/src/extension.ts +++ b/packages/transport-http/src/extension.ts @@ -2,6 +2,7 @@ import type { Extension, HostAPI, Manifest } from "@dispatch/kernel"; import { createApp } from "./app.js"; import { cacheWarmHandle, + compactionHandle, conversationStoreHandle, credentialStoreHandle, lspServiceHandle, @@ -30,6 +31,8 @@ export const manifest: Manifest = { "/conversations", "/conversations/:id", "/conversations/:id/close", + "/conversations/:id/compact", + "/conversations/:id/compact-threshold", "/conversations/:id/cwd", "/conversations/:id/last", "/conversations/:id/lsp", @@ -61,6 +64,7 @@ export function createTransportHttpExtension(): Extension & { const credentialStore = host.getService(credentialStoreHandle); const throughputStore = host.getService(throughputStoreHandle); const warmService = host.getService(cacheWarmHandle); + const compactionService = host.getService(compactionHandle); const lspService = host.getService(lspServiceHandle); const logger = host.logger; @@ -70,6 +74,7 @@ export function createTransportHttpExtension(): Extension & { credentialStore, throughputStore, warmService, + compactionService, lspService, logger, emit: host.emit.bind(host), diff --git a/packages/transport-http/src/seam.ts b/packages/transport-http/src/seam.ts index 1c89a34..3c507dc 100644 --- a/packages/transport-http/src/seam.ts +++ b/packages/transport-http/src/seam.ts @@ -4,9 +4,14 @@ export type { CredentialStore } from "@dispatch/credential-store"; export { credentialStoreHandle } from "@dispatch/credential-store"; export type { LspServerStatus, LspService } from "@dispatch/lsp"; export { lspServiceHandle } from "@dispatch/lsp"; -export type { SessionOrchestrator, WarmService } from "@dispatch/session-orchestrator"; +export type { + CompactionService, + SessionOrchestrator, + WarmService, +} from "@dispatch/session-orchestrator"; export { cacheWarmHandle, + compactionHandle, conversationOpened, sessionOrchestratorHandle, } from "@dispatch/session-orchestrator"; diff --git a/packages/transport-http/src/server.bun.test.ts b/packages/transport-http/src/server.bun.test.ts index a15a2c7..e770867 100644 --- a/packages/transport-http/src/server.bun.test.ts +++ b/packages/transport-http/src/server.bun.test.ts @@ -65,6 +65,11 @@ function fakeConversationStore(): ConversationStore { return null; }, async setConversationStatus() {}, + async replaceHistory() {}, + async getCompactThreshold() { + return null; + }, + async setCompactThreshold() {}, }; } diff --git a/packages/transport-ws/src/extension.ts b/packages/transport-ws/src/extension.ts index f59bb1f..c3b9771 100644 --- a/packages/transport-ws/src/extension.ts +++ b/packages/transport-ws/src/extension.ts @@ -9,6 +9,7 @@ import type { Extension, HostAPI } from "@dispatch/kernel"; import type { SessionOrchestrator } from "@dispatch/session-orchestrator"; import { + conversationCompacted, conversationOpened, conversationStatusChanged, sessionOrchestratorHandle, @@ -147,6 +148,19 @@ export function createTransportWsExtension(): Extension { }), ); + // Broadcast `conversation.compacted` to all connected clients so + // the FE reloads the conversation history after compaction. + disposers.push( + host.on(conversationCompacted, ({ conversationId, messagesSummarized, messagesKept }) => { + broadcast({ + type: "conversation.compacted", + conversationId, + messagesSummarized, + messagesKept, + }); + }), + ); + server = Bun.serve<ConnectionState>({ port, fetch(req, srv) { diff --git a/packages/wire/package.json b/packages/wire/package.json index d884c1e..e375c8f 100644 --- a/packages/wire/package.json +++ b/packages/wire/package.json @@ -1,6 +1,6 @@ { "name": "@dispatch/wire", - "version": "0.10.0", + "version": "0.11.0", "type": "module", "private": true, "main": "dist/index.js", diff --git a/packages/wire/src/index.ts b/packages/wire/src/index.ts index 53f1f02..69f35f0 100644 --- a/packages/wire/src/index.ts +++ b/packages/wire/src/index.ts @@ -522,3 +522,16 @@ export interface ConversationMeta { readonly title: string; readonly status: ConversationStatus; } + +// ─── Compaction ────────────────────────────────────────────────────────────── + +/** + * Result of a compaction operation. `summary` is the text the model produced; + * `messagesKept` is how many recent messages were retained after the summary; + * `messagesSummarized` is how many old messages were replaced by the summary. + */ +export interface CompactionResult { + readonly summary: string; + readonly messagesSummarized: number; + readonly messagesKept: number; +} |
