summaryrefslogtreecommitdiffhomepage
path: root/packages/kernel/src/contracts
diff options
context:
space:
mode:
authorAdam Malczewski <[email protected]>2026-06-28 22:16:57 +0900
committerAdam Malczewski <[email protected]>2026-06-28 22:16:57 +0900
commita5855cad41b2e4b0a196e52cd4c1ca5d4c0ca25f (patch)
treedeb7f267299b2a992b390f69b4ade7267da06b14 /packages/kernel/src/contracts
parent7f1381c4452846e5a2689d868ab0ee2bc90042c9 (diff)
downloaddispatch-a5855cad41b2e4b0a196e52cd4c1ca5d4c0ca25f.tar.gz
dispatch-a5855cad41b2e4b0a196e52cd4c1ca5d4c0ca25f.zip
feat: in-flight compaction — compact mid-turn at every step boundary
At the end of every step that produces tool calls (the tool-result boundary), the orchestrator now checks whether the conversation's context size has reached the compaction threshold (compact-percent of the model's context window). If it has, it compacts the history mid-turn — summarizing old messages and replacing the running history with [summary, ...recent] — then continues the prompt, so a long-running turn (e.g. left overnight) does not run out of context. This mirrors Opencode's approach (check at step boundaries, compact, continue) and is DISTINCT from the pre-existing post-seal auto-compact, which runs only AFTER a turn ends and refuses while a conversation is active. The new path runs DURING the turn (saving the running turn) using the live step usage rather than persisted metrics (written only at turn end). Architecture (kernel purity preserved): - Kernel contract: add an optional, feature-agnostic `onStepBoundary` hook to RunTurnInput. The runtime calls it at each tool-result boundary with the step's usage + the current messages, and adopts whatever message list it returns (undefined/empty = unchanged). Mirrors `drainSteering`. - Kernel runtime: ~21 lines — call the hook after onStepComplete + drainSteering; replace working messages in place when a list is returned. No I/O, no feature names. - Orchestrator: extract a shared `performCompaction` helper (summarize + fork + replaceHistory + emit) used by both `compact()` and the new in-flight path. Wire `onStepBoundary` in `runTurnDetached` to check the threshold via stepUsage vs contextWindow*percent and, on exceed, compact and return [summary, ...kernel's recent N] (preserves mid-turn steering + the vision-transformed provider view; the store gets the canonical recent messages). `compact()` is refactored to use the shared core; its active-conversation guard and metrics-based auto-threshold are unchanged. Tests (+13): 8 kernel onStepBoundary tests (replacement adopted by next step; undefined/empty = unchanged; not called on text-only turns; called once per tool-call step after drainSteering; receives stepUsage; omitted = no-op; replacement persists across steps); 5 orchestrator in-flight tests (triggers above threshold + turn continues; no-op below threshold; no-op when percent=0; no-op on text-only turns; the manual compact() service still refuses while active but in-flight runs anyway). Also fixed the in-memory test store's getCompactPercent/setCompactPercent (were no-ops). Verification: typecheck clean; 2012 tests pass (was 1999; +13); biome 0 errors (12 pre-existing warnings in untouched files).
Diffstat (limited to 'packages/kernel/src/contracts')
-rw-r--r--packages/kernel/src/contracts/runtime.ts27
1 files changed, 27 insertions, 0 deletions
diff --git a/packages/kernel/src/contracts/runtime.ts b/packages/kernel/src/contracts/runtime.ts
index 71d2211..2bab47a 100644
--- a/packages/kernel/src/contracts/runtime.ts
+++ b/packages/kernel/src/contracts/runtime.ts
@@ -142,6 +142,33 @@ export interface RunTurnInput {
readonly onStepComplete?: (messages: readonly ChatMessage[]) => Promise<void> | void;
/**
+ * Optional. Called by the runtime at each tool-result boundary — after a
+ * step that produced tool calls has been finalized (`onStepComplete` has run
+ * and any `drainSteering` messages have been appended), before the next step
+ * begins — giving the caller a chance to REPLACE the running message
+ * history. The caller returns a new message list which the runtime adopts
+ * as its working history for all subsequent steps, or `undefined`/an empty
+ * array to keep the history unchanged.
+ *
+ * The runtime receives the step's token `usage` (so the caller can check a
+ * threshold against the model's context window) and the current `messages`
+ * (the full prompt the next step would otherwise see). Generic and
+ * feature-agnostic: the runtime calls it and adopts whatever message list it
+ * returns — it names no feature, owns no threshold policy, and performs no
+ * I/O, mirroring `drainSteering`. The shell uses this to compact the history
+ * in-flight when the context window nears capacity, so a long-running turn
+ * (e.g. left overnight) does not run out of context mid-turn: it summarizes
+ * the old history and continues with the summary + recent messages. Only
+ * invoked when a step PRODUCED tool calls (there is a "next step" to compact
+ * before); a step that ends without tool calls ends the turn, so there is no
+ * boundary to compact at. Injected (not ambient) so the kernel stays pure.
+ */
+ readonly onStepBoundary?: (ctx: {
+ readonly stepUsage: Usage;
+ readonly messages: readonly ChatMessage[];
+ }) => Promise<readonly ChatMessage[] | undefined> | (readonly ChatMessage[] | undefined);
+
+ /**
* Optional injected retry strategy for retryable provider errors (e.g. HTTP
* 429 / 5xx "overloaded"). When omitted, a retryable error ends the step
* exactly as before (backward-compatible). When provided, the runtime wraps