summaryrefslogtreecommitdiffhomepage
path: root/GLOSSARY.md
diff options
context:
space:
mode:
authorAdam Malczewski <[email protected]>2026-06-07 17:07:47 +0900
committerAdam Malczewski <[email protected]>2026-06-07 17:07:47 +0900
commit3ecc9778fe278d6665b1e9a918f44c16f6992b87 (patch)
tree57e14412d782469d2a744ce9da9eded8bb2a5bd6 /GLOSSARY.md
parentd2ac57045e9884e5f948b95014e853111cd6bc3d (diff)
downloaddispatch-3ecc9778fe278d6665b1e9a918f44c16f6992b87.tar.gz
dispatch-3ecc9778fe278d6665b1e9a918f44c16f6992b87.zip
feat(kernel-runtime): per-step TTFT + decode timing spans (observability)
Split each step's generation into a ttft span (stream start -> first text|reasoning token) and a decode span (first token -> stream end), children of the step span. decode = generation total - TTFT; both retrievable from the trace-store. First token counts reasoning deltas; a step with no content token ends ttft with firstToken:false (no misleading decode). Span-based (no clock injection), no wire/contract change. +3 runtime tests. GLOSSARY: TTFT + decode time. typecheck clean; 512 vitest; biome 0/0.
Diffstat (limited to 'GLOSSARY.md')
-rw-r--r--GLOSSARY.md2
1 files changed, 2 insertions, 0 deletions
diff --git a/GLOSSARY.md b/GLOSSARY.md
index 0ec7ea2..65aa6f2 100644
--- a/GLOSSARY.md
+++ b/GLOSSARY.md
@@ -18,6 +18,8 @@
| **turn** | One user message → assistant response cycle (may span multiple steps). | — |
| **step** | One LLM round-trip within a turn (may emit multiple tool calls). | iteration |
| **stepId** | The identifier of a step, stamped on each `tool-call`/`tool-result` event and tool chunk it produces, so a client groups a parallel/batched tool-call set by equality. Branded `StepId`; the runtime derives it deterministically as `<turnId>#<stepIndex>` (0-based). Generation provenance carried ON the tool chunk (unlike `seq`, which is a store-assigned sync cursor on the `StoredChunk` envelope). Treat as opaque. | batchId, step index (as the wire key) |
+| **TTFT** (time to first token) | Per-step latency from the generation stream starting to the first content token (text **or** reasoning) arriving. Inherently per LLM round-trip — each step re-prefills, so a turn has one TTFT per step; step 0's is the turn's user-visible first-token latency, and the sum across steps is the total prefill overhead. Captured as observability span timing only (not on the wire). | time-to-first-byte (when meaning tokens) |
+| **decode time** | The generation time of a step *after* the first token — first token → stream end, i.e. the step's generation total minus its TTFT. The model's token-production time with first-token latency removed. Observability span timing. | — |
| **tool call** | A model's request to run a tool within a step. | function call (when meaning a tool call) |
| **chunk** | One ordered piece of a message (text, thinking, tool-call/result, etc.), append-only in the log. | block, segment |
| **seq** | The monotonic, gap-free, per-conversation sequence number stamped on each chunk as it is appended to the log. The sync cursor: a client requests `?sinceSeq=N` to fetch only newer chunks. Storage/sync metadata, never message content. | cursor (when meaning the number), offset, index |