diff options
| author | Adam Malczewski <[email protected]> | 2026-06-11 14:11:13 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-11 14:11:13 +0900 |
| commit | 7ffb6b28f5b6bdbfc53ebed94fc68af557612189 (patch) | |
| tree | e66d9ea9d326ef771cc473d81ca5716ff78b08a8 /packages/cache-warming/src/warmer.test.ts | |
| parent | 763e5fb1c7fbfb4c7bbd43ffb935e42e5f5b5a42 (diff) | |
| download | dispatch-7ffb6b28f5b6bdbfc53ebed94fc68af557612189.tar.gz dispatch-7ffb6b28f5b6bdbfc53ebed94fc68af557612189.zip | |
fix(cache-warming): accurate cache rate + expectedCacheRate (retention) metric
The Claude cache % read 100% whenever anything was cached, because the metric's
denominator (inputTokens) excluded cached tokens on Anthropic. Fixed upstream in
../claude/provider-anthropic (inputTokens = total prompt); this commit adds the
companion retention metric and exposes it:
- transport-contract: WarmResponse += expectedCacheRate
- transport-http: POST /chat/warm returns expectedCacheRate = cacheRead/(cacheRead+cacheWrite)
- cache-warming: computeExpectedCacheRate + a per-conversation 'cache retention' surface stat
- handoff: documents the fix + cache-rate vs expected-cache (cross-turn) for the FE
Live-verified vs claude haiku: real turn cache rate 61% (was inflated 100%);
warm within TTL expectedCacheRate=100%, after expiry=0%.
Diffstat (limited to 'packages/cache-warming/src/warmer.test.ts')
| -rw-r--r-- | packages/cache-warming/src/warmer.test.ts | 47 |
1 files changed, 47 insertions, 0 deletions
diff --git a/packages/cache-warming/src/warmer.test.ts b/packages/cache-warming/src/warmer.test.ts index 9865877..86908a2 100644 --- a/packages/cache-warming/src/warmer.test.ts +++ b/packages/cache-warming/src/warmer.test.ts @@ -182,6 +182,30 @@ describe("CacheWarmer", () => { expect(state.lastPct).toBe(80); }); + it("a completed warm stores both lastPct (rate) and lastExpectedPct (retention)", async () => { + const timers = fakeTimers(); + const warmer = createCacheWarmer({ + warm: async () => ({ + inputTokens: 1000, + outputTokens: 10, + cacheReadTokens: 700, + cacheWriteTokens: 300, + }), + storage: memStorage(), + logger: makeLogger(), + timers, + onSurfaceChange: () => {}, + }); + + warmer.onTurnSettled("conv-1", {}); + timers.flush(); + + await new Promise((r) => setTimeout(r, 10)); + const state = warmer.getState("conv-1"); + expect(state.lastPct).toBe(70); + expect(state.lastExpectedPct).toBe(70); + }); + it("re-arms timer after warm completes", async () => { const timers = fakeTimers(); let warmCount = 0; @@ -316,4 +340,27 @@ describe("CacheWarmer", () => { await warmer.setIntervalMs("conv-1", 30_000); expect(changeCount).toBe(2); }); + + it("the per-conversation spec includes a cache-retention stat", async () => { + const timers = fakeTimers(); + const warmer = createCacheWarmer({ + warm: async () => ({ + inputTokens: 1000, + outputTokens: 10, + cacheReadTokens: 900, + cacheWriteTokens: 100, + }), + storage: memStorage(), + logger: makeLogger(), + timers, + onSurfaceChange: () => {}, + }); + + warmer.onTurnSettled("conv-1", {}); + timers.flush(); + await new Promise((r) => setTimeout(r, 10)); + + const state = warmer.getState("conv-1"); + expect(state.lastExpectedPct).toBe(90); + }); }); |
