diff options
| author | Adam Malczewski <[email protected]> | 2026-06-10 08:29:59 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-10 08:29:59 +0900 |
| commit | 6db12ff70acb3333d05a5020ab66da4172a5225a (patch) | |
| tree | de5cc6314a3a6dd966d7c4fdb9b20adb04ae8307 /GLOSSARY.md | |
| parent | 4248cd1d546a4c1fb4e68940c11b5e309c2c2736 (diff) | |
| download | dispatch-6db12ff70acb3333d05a5020ab66da4172a5225a.tar.gz dispatch-6db12ff70acb3333d05a5020ab66da4172a5225a.zip | |
feat(metrics): durable per-turn/step token+timing metrics (observability spans + persisted replay)
Two-part token-data improvement:
#2 Observability spans (kernel run-turn): turn & step span-close now stamp
ALL four Usage fields — added usage.cacheReadTokens/cacheWriteTokens (were
silently dropped) and normalized usage_* -> usage.* to match the
provider.request span (consistent D9 GROUP BY). No contract change.
#3 Persisted replay metrics (conversation-store + read endpoint): new
StepMetrics/TurnMetrics wire types; conversation-store persists per-turn
metrics in a separate key space (appendMetrics/loadMetrics, turn-append
order); session-orchestrator accumulates per-step+turn metrics from the
event stream (pure metrics.ts) and persists after seal; transport-http
serves GET /conversations/:id/metrics -> ConversationMetricsResponse.
Contracts: @dispatch/wire + @dispatch/transport-contract bumped 0.3.0->0.4.0
(additive). GLOSSARY: turn metrics / step metrics.
typecheck EXIT 0, biome clean, 546 vitest + 89 bun = 635 tests.
Diffstat (limited to 'GLOSSARY.md')
| -rw-r--r-- | GLOSSARY.md | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/GLOSSARY.md b/GLOSSARY.md index 65aa6f2..845b45f 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -20,6 +20,8 @@ | **stepId** | The identifier of a step, stamped on each `tool-call`/`tool-result` event and tool chunk it produces, so a client groups a parallel/batched tool-call set by equality. Branded `StepId`; the runtime derives it deterministically as `<turnId>#<stepIndex>` (0-based). Generation provenance carried ON the tool chunk (unlike `seq`, which is a store-assigned sync cursor on the `StoredChunk` envelope). Treat as opaque. | batchId, step index (as the wire key) | | **TTFT** (time to first token) | Per-step latency from the generation stream starting to the first content token (text **or** reasoning) arriving. Inherently per LLM round-trip — each step re-prefills, so a turn has one TTFT per step; step 0's is the turn's user-visible first-token latency, and the sum across steps is the total prefill overhead. Captured as observability span timing only (not on the wire). | time-to-first-byte (when meaning tokens) | | **decode time** | The generation time of a step *after* the first token — first token → stream end, i.e. the step's generation total minus its TTFT. The model's token-production time with first-token latency removed. Observability span timing. | — | +| **turn metrics** | The durable, replayable per-turn metrics record persisted for a sealed turn: aggregate `Usage` (tokens) + turn `durationMs` + its per-step `StepMetrics`. The persisted counterpart of the live `done` event's metrics; persisted per turn by `conversation-store` (returned in turn-append order), served by `GET /conversations/:id/metrics`. Distinct from observability span timing (trace-store) and from transient live wire events. | usage record, turn stats | +| **step metrics** | The durable per-step metrics within a `TurnMetrics`: the step's `Usage` (tokens) + `ttftMs`/`decodeMs`/`genTotalMs` timing. The persisted counterpart of the live `usage` + `step-complete` events, keyed by `stepId`. | step stats | | **tool call** | A model's request to run a tool within a step. | function call (when meaning a tool call) | | **chunk** | One ordered piece of a message (text, thinking, tool-call/result, etc.), append-only in the log. | block, segment | | **seq** | The monotonic, gap-free, per-conversation sequence number stamped on each chunk as it is appended to the log. The sync cursor: a client requests `?sinceSeq=N` to fetch only newer chunks. Storage/sync metadata, never message content. | cursor (when meaning the number), offset, index | |
