summaryrefslogtreecommitdiffhomepage
path: root/tasks.md
diff options
context:
space:
mode:
authorAdam Malczewski <[email protected]>2026-06-05 17:53:00 +0900
committerAdam Malczewski <[email protected]>2026-06-05 17:53:00 +0900
commit8c417472e7801369c3dfd004c9c85d7d69372f7c (patch)
tree3da8eac532855a14013b502afaf30d669010c6be /tasks.md
parent74986a54093f6492cc4420f5917e5215f42a8f89 (diff)
downloaddispatch-8c417472e7801369c3dfd004c9c85d7d69372f7c.tar.gz
dispatch-8c417472e7801369c3dfd004c9c85d7d69372f7c.zip
feat(observability): map DeepSeek nested cache tokens (prompt_tokens_details.cached_tokens) -> Usage.cacheReadTokens
The real flash fixture showed flash reports cache usage in the NESTED prompt_tokens_details.cached_tokens form (384 cached of 665 prompt); the parser only mapped the flat cache_read_tokens form, so cache tokens never surfaced. Now: cacheReadTokens = usage.cache_read_tokens ?? usage.prompt_tokens_details?.cached_tokens (flat wins; cacheWriteTokens flat-only, never fabricated; partial/null *_details safe). No kernel contract change (Usage already has the fields). +5 parser tests + a real-fixture regression (cacheReadTokens === 384). These counts (+ a future prefix.fingerprint) are the cheap signals for body de-duplication. The broader trace-body storage-growth concern (verbatim body stored per request -> ~O(N^2) for long conversations) is logged DEFERRED in tasks.md; mitigation already designed (D5 volume control + §6 retention/rotation), not yet built. 339 tests, typecheck + biome 0/0.
Diffstat (limited to 'tasks.md')
-rw-r--r--tasks.md17
1 files changed, 12 insertions, 5 deletions
diff --git a/tasks.md b/tasks.md
index a44a781..e643d02 100644
--- a/tasks.md
+++ b/tasks.md
@@ -266,11 +266,18 @@ independent of the SQLite trace-store; the lib is redaction-free (caller self-re
(`src/__fixtures__/flash-text-turn.json`); reply "Hello there friend"; text-turn replay
assertions updated to real values (inputTokens 665 / outputTokens 90); secret-free
re-verified pre-commit. **334 tests**, typecheck + biome 0/0.
-- [ ] **FINDING → decide** (real-data, D5): flash returns cache tokens in DeepSeek's NESTED
- `prompt_tokens_details.cached_tokens` (665 prompt / 384 cached); the openai-compat SSE
- parser only maps the FLAT `cache_read/creation` form, so cache tokens never surface — an
- observability gap for the §3.1 cache-debugging goal. Surfaced to user: fix the parser
- (provider unit) or defer?
+- [x] **Cache-token mapping fixed** (real-data, D5): parser now maps nested
+ `prompt_tokens_details.cached_tokens` → `Usage.cacheReadTokens` (flat `cache_read_tokens`
+ still wins via `??`; `cacheWriteTokens` flat-only, never fabricated; partial/null details
+ safe; no contract change). +5 parser tests + real-fixture regression (`cacheReadTokens===384`).
+ 339 tests. reports/provider-cache-tokens.md.
+- [ ] **DEFERRED — trace body de-dup / storage growth** (user-flagged): the `provider.request`
+ span stores the FULL post-transform request body on EVERY request → ~O(N²) body text for
+ long conversations (history is resent each turn; cache hits are the signature of it).
+ Mitigation already DESIGNED, not built: D5 "Volume control" (persist body only when
+ `prefix.fingerprint` changed) + §6 retention/rotation/compression; thin/fat split already
+ built. `cacheReadTokens` (just added) + the future `prefix.fingerprint` are the cheap dedup
+ signals. Revisit when cache-warming / longer conversations land.
Summons: prompts/phase-a-{kernel-logging,journal-sink}.md;
reports/phase-a-{kernel-logging,journal-sink}.md.