diff options
| author | Adam Malczewski <[email protected]> | 2026-06-07 17:07:47 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-07 17:07:47 +0900 |
| commit | 3ecc9778fe278d6665b1e9a918f44c16f6992b87 (patch) | |
| tree | 57e14412d782469d2a744ce9da9eded8bb2a5bd6 /GLOSSARY.md | |
| parent | d2ac57045e9884e5f948b95014e853111cd6bc3d (diff) | |
| download | dispatch-3ecc9778fe278d6665b1e9a918f44c16f6992b87.tar.gz dispatch-3ecc9778fe278d6665b1e9a918f44c16f6992b87.zip | |
feat(kernel-runtime): per-step TTFT + decode timing spans (observability)
Split each step's generation into a ttft span (stream start -> first text|reasoning
token) and a decode span (first token -> stream end), children of the step span.
decode = generation total - TTFT; both retrievable from the trace-store. First token
counts reasoning deltas; a step with no content token ends ttft with firstToken:false
(no misleading decode). Span-based (no clock injection), no wire/contract change.
+3 runtime tests. GLOSSARY: TTFT + decode time.
typecheck clean; 512 vitest; biome 0/0.
Diffstat (limited to 'GLOSSARY.md')
| -rw-r--r-- | GLOSSARY.md | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/GLOSSARY.md b/GLOSSARY.md index 0ec7ea2..65aa6f2 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -18,6 +18,8 @@ | **turn** | One user message → assistant response cycle (may span multiple steps). | — | | **step** | One LLM round-trip within a turn (may emit multiple tool calls). | iteration | | **stepId** | The identifier of a step, stamped on each `tool-call`/`tool-result` event and tool chunk it produces, so a client groups a parallel/batched tool-call set by equality. Branded `StepId`; the runtime derives it deterministically as `<turnId>#<stepIndex>` (0-based). Generation provenance carried ON the tool chunk (unlike `seq`, which is a store-assigned sync cursor on the `StoredChunk` envelope). Treat as opaque. | batchId, step index (as the wire key) | +| **TTFT** (time to first token) | Per-step latency from the generation stream starting to the first content token (text **or** reasoning) arriving. Inherently per LLM round-trip — each step re-prefills, so a turn has one TTFT per step; step 0's is the turn's user-visible first-token latency, and the sum across steps is the total prefill overhead. Captured as observability span timing only (not on the wire). | time-to-first-byte (when meaning tokens) | +| **decode time** | The generation time of a step *after* the first token — first token → stream end, i.e. the step's generation total minus its TTFT. The model's token-production time with first-token latency removed. Observability span timing. | — | | **tool call** | A model's request to run a tool within a step. | function call (when meaning a tool call) | | **chunk** | One ordered piece of a message (text, thinking, tool-call/result, etc.), append-only in the log. | block, segment | | **seq** | The monotonic, gap-free, per-conversation sequence number stamped on each chunk as it is appended to the log. The sync cursor: a client requests `?sinceSeq=N` to fetch only newer chunks. Storage/sync metadata, never message content. | cursor (when meaning the number), offset, index | |
