diff options
| author | Adam Malczewski <[email protected]> | 2026-06-11 12:45:21 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-11 12:45:21 +0900 |
| commit | 27fd0be36b2f6395249de5aacc86e41fe4e0207f (patch) | |
| tree | 67ae766c1985344878d6a2e71da18834fa73e47d /packages/transport-contract/src | |
| parent | c2b4c05d91fa88b8d02c055a0e15c22abd8e21f3 (diff) | |
| download | dispatch-27fd0be36b2f6395249de5aacc86e41fe4e0207f.tar.gz dispatch-27fd0be36b2f6395249de5aacc86e41fe4e0207f.zip | |
feat(cache-warming): manual POST /chat/warm trigger endpoint
A frontend 'warm now' button (and fast tests) can trigger a warm on demand
instead of waiting for the automatic timer.
- transport-contract: WarmRequest / WarmResponse wire types
- transport-http: POST /chat/warm → cacheWarmHandle.warm(); 200 with cachePct,
409 when the conversation is generating, 400 on missing conversationId
Live-verified vs claude haiku: seed turn cacheWrite=6799 → POST /chat/warm
returns cacheReadTokens=6799 cachePct=100 (100% hit). 760 vitest + 109 bun green.
Diffstat (limited to 'packages/transport-contract/src')
| -rw-r--r-- | packages/transport-contract/src/index.ts | 44 |
1 files changed, 44 insertions, 0 deletions
diff --git a/packages/transport-contract/src/index.ts b/packages/transport-contract/src/index.ts index 5b59a2d..fbb61fc 100644 --- a/packages/transport-contract/src/index.ts +++ b/packages/transport-contract/src/index.ts @@ -154,6 +154,50 @@ export interface ThroughputResponse { readonly models: readonly ThroughputModelStat[]; } +/** + * Request body for `POST /chat/warm` — manually trigger a prompt-cache WARMING + * request for a conversation (e.g. a frontend "warm now" button, or fast tests + * that don't want to wait for the automatic warming timer). + * + * The warm replays the conversation's existing prefix to the provider to refresh + * its prompt cache; it is NEVER persisted and NEVER streamed (no `AgentEvent`s). + * Pass the same `model`/`cwd` the conversation chats with so the warm request's + * prefix is byte-identical to a real turn (which is what makes the cache hit). + */ +export interface WarmRequest { + /** The conversation whose prompt cache to warm. */ + readonly conversationId: string; + + /** + * The model name in `<credentialName>/<model>` form the conversation uses, so + * the warm resolves the same provider + prefix. Omit to use the server default. + */ + readonly model?: string; + + /** Working directory matching the conversation's turns (for cwd-aware tool assembly). */ + readonly cwd?: string; +} + +/** + * Response body for `POST /chat/warm` (HTTP 200). The warm request's usage — + * never folded into the conversation's real usage. A client surfaces `cachePct` + * as the "last warming" cache-hit indicator. + * + * When warming cannot run because the conversation is currently generating, the + * server responds `409` with `{ error }` instead of this body. + */ +export interface WarmResponse { + readonly inputTokens: number; + readonly outputTokens: number; + readonly cacheReadTokens: number; + readonly cacheWriteTokens: number; + /** + * Cache-hit percent: `round(clamp(cacheReadTokens / inputTokens, 0, 1) * 100)` + * (0 when `inputTokens <= 0`). + */ + readonly cachePct: number; +} + // ─── WebSocket chat ops ─────────────────────────────────────────────────────── // The persistent WS connection multiplexes chat ops (below) with surface ops // (`@dispatch/ui-contract`). The unified unions at the bottom compose both. Chat |
