diff options
| author | Adam Malczewski <[email protected]> | 2026-06-11 16:06:48 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-11 16:06:48 +0900 |
| commit | e45cab2a2d9d7bf5e48ace7111fd84b1b9bf2df3 (patch) | |
| tree | e9cd8665a3eea609ef1e027906be4abdfe67d876 /backend-handoff-cache-warming-timer.md | |
| parent | b3f7ba523f644224364d155b575fa3f9f13c5eb9 (diff) | |
| download | dispatch-web-e45cab2a2d9d7bf5e48ace7111fd84b1b9bf2df3.tar.gz dispatch-web-e45cab2a2d9d7bf5e48ace7111fd84b1b9bf2df3.zip | |
feat(cache-warming,surfaces,metrics,markdown): conversation-scoped surfaces, cache warming + retention, markdown
Consumes the backend cache-warming + cache-rate handoffs end-to-end and adds supporting infra:
- protocol/transport: conversation-scoped surfaces (conversationId on subscribe/invoke/surface + staleness routing); store auto-subscribes the catalog with the focused conversation and re-scopes on switch.
- surface-host: generic Number field renderer + custom rendererId dispatch (graceful skip on unknown).
- cache-warming feature: enabled toggle, min+sec interval, AUTHORITATIVE countdown from the surface's cache-warming-timer nextWarmAt, manual Warm now (POST /chat/warm), lastWarmAt-keyed history, cache-retention stat, expectedCacheRate headline.
- metrics: cross-turn expected-cache (retention) derivation + bubble badge; cache-rate fix needs no code change (inputTokens now total).
- markdown feature: marked + marked-highlight + highlight.js + dompurify, rendered in ChatView.
- fixes (gemini review): {#key activeConversationId} remount of CacheWarmingView to stop history/feedback leaking across tabs; guard NaN interval inputs from committing 0.
- docs/contracts: regenerated transport/ui-contract mirrors; backend-handoff updated (CR-3 resolved).
Verified: svelte-check 0 errors, biome clean, 494 tests pass, vite build OK.
Diffstat (limited to 'backend-handoff-cache-warming-timer.md')
| -rw-r--r-- | backend-handoff-cache-warming-timer.md | 80 |
1 files changed, 80 insertions, 0 deletions
diff --git a/backend-handoff-cache-warming-timer.md b/backend-handoff-cache-warming-timer.md new file mode 100644 index 0000000..2b1e8c7 --- /dev/null +++ b/backend-handoff-cache-warming-timer.md @@ -0,0 +1,80 @@ +# FE → backend handoff — cache-warming: next-warm timestamp + manual-warm timer reset + +> **Courier doc** (dispatch-web → arch-rewrite, carried by the user). `lsp` does not span the repos. +> **From:** dispatch-web orchestrator · **To:** arch-rewrite orchestrator · **Courier:** the user. +> Focused ask split out of `backend-handoff.md` CR-3. Two requests, both ADDITIVE / backward-compatible. + +## Why +The FE shipped a Cache Warming view: enabled toggle, minutes+seconds interval, a **live countdown to +the next warm**, a manual **Warm now** button, and a **history** of each warm's hit %. Two of those — +the countdown and a reliably-fresh "last cache %" — can't be done accurately from what the wire +exposes today. The FE currently fakes them best-effort (countdown anchored to the last warm the FE +*observed* + interval; manual warm % taken from the HTTP response because the surface doesn't refresh). +We'd like to make them authoritative. + +## What I found in the backend (so these asks are precise, not guesses) +Read of `packages/cache-warming/src/warmer.ts` + `packages/transport-http/src/app.ts`: + +1. **`POST /chat/warm` bypasses the warmer.** The handler (`transport-http/src/app.ts:240–289`) calls + `warmService.warm(conversationId, …)` **directly**. It never goes through `CacheWarmer`, so a manual + warm does NOT: re-arm the automatic timer, update the warmer's `lastPct`, or call `onSurfaceChange()`. + ⇒ the cache-warming **surface does not refresh** after a manual warm (no `update` pushed), and the + automatic timer keeps counting from the *previous* warm — a manual warm doesn't reset the cycle. +2. **No next-warm time is tracked.** `armTimer` (`warmer.ts:99`) arms a relative + `timers.setTimer(fn, state.intervalMs)` but stores no absolute fire time, and `ConversationState` + (`warmer.ts:61`) is `{ enabled, intervalMs, active, lastPct, token }`. There is nothing the surface + could carry to tell a client *when* the next warm fires. + +## Ask 1 — serve a machine-readable next-warm (and last-warm) timestamp on the surface +Record the absolute fire time when the timer is armed and expose it (plus the last warm's time) on the +**conversation-scoped `cache-warming` surface**, pushed on every change via the existing +`onSurfaceChange()` (warm completes, toggle, interval change, turn start/settle). + +- `nextWarmAt`: epoch-ms the next automatic warm is scheduled to fire, or `null` when not scheduled + (disabled, or `active` i.e. a turn is generating so the timer is cancelled). +- `lastWarmAt`: epoch-ms of the most recent completed warm, or `null` if none yet. + +**Suggested shape — no `@dispatch/ui-contract` bump needed:** add ONE `custom` field to the spec +(the escape hatch already exists; non-supporting clients gracefully skip it): +```ts +{ + kind: "custom", + rendererId: "cache-warming-timer", + payload: { + nextWarmAt: number | null, // epoch-ms, or null when not scheduled + lastWarmAt: number | null, // epoch-ms, or null when never warmed + }, +} +``` +(If you'd rather not add a field, a `stat` carrying an ISO/epoch string works too, but a machine- +readable `custom` payload is cleanest for the countdown — the FE needs the number, not a display +string.) The FE will read this in its cache-warming feature and render an exact countdown; it already +parses the surface fields itself, so wiring a `cache-warming-timer` renderer is a small FE change. + +## Ask 2 — reset the automatic timer on a manual warm (and refresh the surface) +Confirm or change: a manual `POST /chat/warm` should be treated as "a warm just happened" for that +conversation, i.e. route it through (or notify) the `CacheWarmer` so it: +1. **re-arms the automatic timer** from *now* (the countdown restarts at the full interval), and +2. **updates `lastPct`** from the manual warm's result and **calls `onSurfaceChange()`** so subscribers + get an `update` (this also fixes the surface "last cache %" not refreshing after a manual warm). + +Per the source, none of this happens today (Ask-1 finding #1). The minimal change is a +`CacheWarmer.warmNow(conversationId)` that does what the automatic `fireWarm` already does — warm → +set `lastPct` → `onSurfaceChange()` → `armTimer()` — and have the `/chat/warm` handler call THAT +instead of `warmService.warm` directly. (If you intend manual warms to be a *separate*, non-cycle- +resetting probe, tell us and we'll keep the FE's manual entry purely local — but the user's intent is +that a manual warm resets the cycle.) + +## What the FE does once this lands +- Drop the FE's best-effort countdown anchor and use `nextWarmAt` directly → exact, drift-free + countdown that also reflects generation pauses (null while `active`). +- Render history from the authoritative surface signal (using `lastWarmAt` changes), removing the + FE-side de-dup/identical-pct workaround noted in `reports/cache-warming-feature.md`. + +## References +- Backend: `packages/cache-warming/src/warmer.ts`, `packages/transport-http/src/app.ts:240`, + `packages/cache-warming/src/pure.ts` (surface spec builder), the cache-warming surface + (`id:"cache-warming"`, region `"side"`, conversation-scoped). +- FE: `src/features/cache-warming/` (view-model + view), `backend-handoff.md` CR-3 (superseded by this + doc), the original `frontend-cache-warming-handoff.md`. +- No contract version bump required if Ask 1 uses the `custom` escape hatch; Ask 2 is server-internal. |
