summaryrefslogtreecommitdiffhomepage
path: root/tasks.md
diff options
context:
space:
mode:
authorAdam Malczewski <[email protected]>2026-06-10 10:43:40 +0900
committerAdam Malczewski <[email protected]>2026-06-10 10:43:40 +0900
commit3a0cdd2c8453f059a746465e3aa6d9b5caa3b399 (patch)
treeb76133d2a593ccd510673a745e5912599f850696 /tasks.md
parent80e14ab59732aabbf06035d13138500f133e921d (diff)
downloaddispatch-3a0cdd2c8453f059a746465e3aa6d9b5caa3b399.tar.gz
dispatch-3a0cdd2c8453f059a746465e3aa6d9b5caa3b399.zip
trace-store: content-addressed body dedup + retention/prune
Wave 1 of the dedup/storage-growth milestone (notes §12). - bodies table is now content-addressed (SHA-256 hash key); identical verbatim bodies (cache-warming resends, any repeat) collapse to one stored row, referenced by hash from records. Transparent to insert/read callers. - at-rest gzip compression for bodies >1 KiB (node:zlib), decompressed on read. - prune(policy): age-based delete + drop-oldest byte-cap eviction + orphan-body GC. Exports RetentionPolicy/PruneSummary/DEFAULT_RETENTION (7d / 256 MiB). typecheck EXIT 0; biome clean; vitest 576; bun 89->100, 0 fail.
Diffstat (limited to 'tasks.md')
-rw-r--r--tasks.md28
1 files changed, 23 insertions, 5 deletions
diff --git a/tasks.md b/tasks.md
index 4d7da4a..d86086a 100644
--- a/tasks.md
+++ b/tasks.md
@@ -5,7 +5,7 @@
> Keep this lean and current; do not let it re-accrete a step-by-step changelog.
## Status (current)
-`tsc -b` EXIT 0 · biome clean · **546 vitest + 89 bun = 635 tests**.
+`tsc -b` EXIT 0 · biome clean · **576 vitest + 89 bun = 665 tests**.
Built and verified live (full-fidelity: every feature is a manifest-loaded
extension through the host):
@@ -68,11 +68,29 @@ server/collector procs poison the next run's counts.
- [x] **FE courier handoff** written: `frontend-metrics-pass2-handoff.md` (in
this repo; user couriers to `../dispatch-web`; ORCHESTRATOR §7).
+## dedup / storage growth (current milestone — building)
+Design DECIDED + recorded: `notes/observability-design.md` §12. User-gated calls:
+extend existing pipeline (no new ext); scope = **de-dup + retention/rotation**
+(D9 roll-ups deferred); dedup = **content-addressed bodies** (body-hash, NOT
+fingerprint-gated). Glossary terms approved (add on land): deduplication /
+content-addressed body, prefix fingerprint, warm vs real, retention / rotation.
+- [x] **Wave 1 — `trace-store`** (done): content-addressed `bodies` table
+ (SHA-256), at-rest gzip (>1 KiB), `prune(policy)` (age + drop-oldest size cap +
+ orphan GC) / `RetentionPolicy` / `PruneSummary` / `DEFAULT_RETENTION` (7d/256MiB);
+ read paths transparent. bun 89→100.
+- [ ] **Wave 2 — `observability-collector`:** call `store.prune()` on a cadence;
+ body inserts flow through the content-addressed path.
+- [ ] On land: add the 4 glossary terms; (optional) host-bin env-override for
+ retention policy.
+
## Open items
-- **dedup / storage growth (deferred):** trace-body de-dup (D5 volume control +
- `prefix.fingerprint`) + rotation/compression/retention
- (`notes/observability-design.md` §6, D9). `cacheReadTokens` is the cheap dedup
- signal; thin/fat split already built.
+- **`prefix.fingerprint` / `warm|real` cache-bust attributes (deferred):** decoupled
+ from dedup by the content-addressed decision; also gated on cache-warming being
+ built (not yet) so `warm|real` can't be honestly stamped. Later cache-bust-debug
+ milestone (`notes/observability-design.md` §3.1, §12).
+- **D9 analytics roll-ups (deferred):** rollup table shape + `GROUP BY` indexes +
+ retention asymmetry + periodic rollup job (`notes/observability-design.md` §2 D9,
+ §12). The scheduler mechanism (`host.scheduler.register`) already exists.
- **D8 `prompt.assembly` segments:** deferred-by-design (await the context-filter
chain).