From f8bf715abc8a89ec0c6370b40403c509b1ce2870 Mon Sep 17 00:00:00 2001 From: Adam Malczewski Date: Wed, 10 Jun 2026 10:06:27 +0900 Subject: feat(metrics): per-turn + per-step token/timing metrics bubbles Consume wire@0.4.0 / transport-contract@0.4.0 metrics: usage.stepId, step-complete (ttft/decode/genTotal), done.durationMs/usage, and the durable GET /conversations/:id/metrics endpoint. - core/metrics: pure live-fold + durable-merge reducer; decode-rate TPS; head-aligned, stable placement; progressive per-step rows (each shown as its step ends) with the turn-total row gated on the done event. - features/chat: store folds metric events + hydrates durable TurnMetrics; ChatView renders inline step bubbles + a turn-total bubble. - app: MetricsSync HTTP effect (tolerates 404) injected into chat stores. - scripts/live-probe: drives the metrics path; live-verified 17/17 vs bin/up. - docs: regenerate .dispatch wire/transport mirrors to 0.4.0; glossary terms (turn/step metrics, TTFT, decode time, TPS, metrics bubble); trim handoff. --- GLOSSARY.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'GLOSSARY.md') diff --git a/GLOSSARY.md b/GLOSSARY.md index 2d25fd3..538ba7e 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -15,6 +15,10 @@ | **AgentEvent** | An outward event the runtime emits during a turn (text-delta, tool-call, usage, done, turn-sealed, …). | — | | **model name** | The selectable id in `/` form. | model id, model reference | | **model catalog** | The list of available model names. | model list | +| **turn metrics** | The durable, replayable per-turn metrics record for a sealed turn: aggregate `Usage` (tokens) + turn `durationMs` + its per-step `StepMetrics` (`TurnMetrics`). Persisted backend-side keyed by `turnId`, served by `GET /conversations/:id/metrics`. The persisted counterpart of the live `done` event's metrics; the FE folds the SAME shape from the live `usage`/`step-complete`/`done` events for the in-flight turn. | usage record, turn stats | +| **step metrics** | The durable per-step metrics within a `TurnMetrics`: the step's `Usage` (tokens) + `ttftMs`/`decodeMs`/`genTotalMs` timing, keyed by `stepId` (`StepMetrics`). The persisted counterpart of the live `usage` + `step-complete` events. | step stats | +| **TTFT** (time to first token) | Per-step latency: generation stream start → first content token (text or reasoning). One per step (each step re-prefills). On the wire as `step-complete.ttftMs` / `StepMetrics.ttftMs` (optional). | time-to-first-byte | +| **decode time** | Per-step generation time after the first token (first token → stream end = `genTotalMs − ttftMs`). On the wire as `step-complete.decodeMs` / `StepMetrics.decodeMs` (optional). | — | ## Frontend-specific | Term | Meaning | Aliases to avoid | @@ -29,3 +33,5 @@ | **feature module** | A self-contained FE feature (chat, history explorer, …); feature-as-a-library, composed at the root. | — | | **composition root** | The single place (`src/app/`) that imports + wires feature modules + the surface host. | — | | **surface interpreter** | The generic renderer: field kind → component. Knows kinds, never surface ids. | — | +| **metrics bubble** | The FE chat element that renders a turn's **turn metrics** (one per-turn total) and **step metrics** (one per step) as muted system-style bubbles at a turn's tail. UI presentation of `TurnMetrics`/`StepMetrics`; never a surface. | telemetry bubble, usage bubble, stats bubble | +| **TPS** (tokens per second) | A FE-DERIVED decode rate: `outputTokens / (decodeMs / 1000)` (per step; per turn over Σ `decodeMs`), falling back to `genTotalMs` when `decodeMs` is absent. The backend-recommended basis (excludes first-token latency). Not carried on the wire; omitted when timing is absent. | throughput | -- cgit v1.2.3