diff options
| author | Adam Malczewski <[email protected]> | 2026-06-10 08:29:59 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-10 08:29:59 +0900 |
| commit | 6db12ff70acb3333d05a5020ab66da4172a5225a (patch) | |
| tree | de5cc6314a3a6dd966d7c4fdb9b20adb04ae8307 /packages/wire | |
| parent | 4248cd1d546a4c1fb4e68940c11b5e309c2c2736 (diff) | |
| download | dispatch-6db12ff70acb3333d05a5020ab66da4172a5225a.tar.gz dispatch-6db12ff70acb3333d05a5020ab66da4172a5225a.zip | |
feat(metrics): durable per-turn/step token+timing metrics (observability spans + persisted replay)
Two-part token-data improvement:
#2 Observability spans (kernel run-turn): turn & step span-close now stamp
ALL four Usage fields — added usage.cacheReadTokens/cacheWriteTokens (were
silently dropped) and normalized usage_* -> usage.* to match the
provider.request span (consistent D9 GROUP BY). No contract change.
#3 Persisted replay metrics (conversation-store + read endpoint): new
StepMetrics/TurnMetrics wire types; conversation-store persists per-turn
metrics in a separate key space (appendMetrics/loadMetrics, turn-append
order); session-orchestrator accumulates per-step+turn metrics from the
event stream (pure metrics.ts) and persists after seal; transport-http
serves GET /conversations/:id/metrics -> ConversationMetricsResponse.
Contracts: @dispatch/wire + @dispatch/transport-contract bumped 0.3.0->0.4.0
(additive). GLOSSARY: turn metrics / step metrics.
typecheck EXIT 0, biome clean, 546 vitest + 89 bun = 635 tests.
Diffstat (limited to 'packages/wire')
| -rw-r--r-- | packages/wire/package.json | 2 | ||||
| -rw-r--r-- | packages/wire/src/index.ts | 41 |
2 files changed, 42 insertions, 1 deletions
diff --git a/packages/wire/package.json b/packages/wire/package.json index 762c06e..790c7e1 100644 --- a/packages/wire/package.json +++ b/packages/wire/package.json @@ -1,6 +1,6 @@ { "name": "@dispatch/wire", - "version": "0.3.0", + "version": "0.4.0", "type": "module", "private": true, "main": "dist/index.js", diff --git a/packages/wire/src/index.ts b/packages/wire/src/index.ts index a4790de..aa6f9d0 100644 --- a/packages/wire/src/index.ts +++ b/packages/wire/src/index.ts @@ -153,6 +153,47 @@ export interface Usage { readonly cacheWriteTokens?: number; } +// ─── Persisted metrics ─────────────────────────────────────────────────────── + +/** + * Durable per-step metrics for a completed step — the persisted, replayable + * counterpart of the live `usage` + `step-complete` events. Combines the step's + * token usage with its generation timing so a client reopening a past + * conversation renders the same per-step token/latency breakdown it would have + * seen live. Built from the turn's events, stored by `conversation-store`, and + * served by `GET /conversations/:id/metrics`. + */ +export interface StepMetrics { + readonly stepId: StepId; + /** The step's token usage (all four counters; cache fields optional per `Usage`). */ + readonly usage: Usage; + /** Time to first token (stream start → first text/reasoning delta). Optional — see `TurnStepCompleteEvent.ttftMs`. */ + readonly ttftMs?: number; + /** Decode time (first token → stream end). Optional — see `TurnStepCompleteEvent.decodeMs`. */ + readonly decodeMs?: number; + /** Total generation time for the step (stream start → stream end). Optional: present only when a clock was available. */ + readonly genTotalMs?: number; +} + +/** + * Durable per-turn metrics for a completed (sealed) turn — the persisted, + * replayable counterpart of the live `done` event's aggregate `usage` + + * `durationMs`, plus the per-step breakdown. `usage` is the aggregate across all + * steps; `steps` carries each step's `StepMetrics` in step order. Persisted per + * turn by `conversation-store` (returned in turn-append order) and served by + * `GET /conversations/:id/metrics`. (`turnId` is the plain wire string carried + * on every `AgentEvent`, the join key to the live stream.) + */ +export interface TurnMetrics { + readonly turnId: string; + /** Aggregate token usage across all steps in the turn. */ + readonly usage: Usage; + /** Total wall-clock duration of the turn (turn start → turn end). Optional: present only when a clock was available. */ + readonly durationMs?: number; + /** Per-step metrics in step order. */ + readonly steps: readonly StepMetrics[]; +} + // ─── Outward events ───────────────────────────────────────────────────────── /** |
