diff options
| author | Adam Malczewski <[email protected]> | 2026-06-03 14:49:04 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-03 14:49:04 +0900 |
| commit | 656aad2752991ff32e98fed270fa330355650c17 (patch) | |
| tree | ef6ca33b8bf988f6790c19eb021432c34e7ae798 /packaging | |
| parent | e87e6b39285c8001045d1ebdac873b182c0f7868 (diff) | |
| download | dispatch-656aad2752991ff32e98fed270fa330355650c17.tar.gz dispatch-656aad2752991ff32e98fed270fa330355650c17.zip | |
fix: warm the SAME Anthropic message-cache bucket as real turns
Root cause of the 'first warmup misses' + 'switch to chat misses' bugs:
Anthropic keys the MESSAGE-level prompt cache on `tool_choice` AND the
extended-thinking parameters (both rows in their cache-invalidation table mark
the messages cache as invalidated on change). The original warmCache() sent
toolChoice:'none' and NO thinking providerOptions, while real turns send
toolChoice:'auto' + thinking config for the effort. So warming and chat wrote
TWO different message-cache buckets:
- warmup #1 missed (no warm-only bucket existed yet), every later warmup hit
its own bucket;
- the next real chat message read the OTHER bucket → miss.
Fix: extract a shared buildStreamOptions() that produces the cache-affecting
params (toolChoice + thinking providerOptions + maxOutputTokens). Both run()
and warmCache() now call it with the SAME resolved reasoning effort, so the
warming replay refreshes the exact cache the next real message reads. The
trivial probe turn is still appended AFTER the last cache breakpoint, so it
never disturbs the cached prefix.
Threaded the per-tab reasoning effort (per-model -> per-tab selector -> default,
mirroring processMessage) from the frontend resolver through POST /chat/warm to
warmCacheForTab to warmCache.
Tests: updated the warmCache toolChoice test to assert it MATCHES a real turn,
added an invariant test driving run() and warmCache() and asserting identical
cache-affecting params, and assert effort forwarding in the frontend store.
check / test (780) / frontend build / typecheck all green.
Diffstat (limited to 'packaging')
0 files changed, 0 insertions, 0 deletions
