diff options
| author | Adam Malczewski <[email protected]> | 2026-05-28 06:54:48 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-05-28 06:54:48 +0900 |
| commit | 8b17d929e70a43749fd962554214bf8ba3e9380f (patch) | |
| tree | bdff1f409a8fe78850044c23b38436d84cbbcca9 /packages/core/src | |
| parent | 25b6aac6d4df02e29a2ad4333272bb0998ecd410 (diff) | |
| download | dispatch-8b17d929e70a43749fd962554214bf8ba3e9380f.tar.gz dispatch-8b17d929e70a43749fd962554214bf8ba3e9380f.zip | |
refactor(core): upgrade ai-sdk v4 → v6 + Anthropic/openai-compatible reasoning round-trip + max-thinking budget audit
Migrates the LLM stack from [email protected] + @ai-sdk/[email protected] +
@ai-sdk/[email protected] to [email protected] + @ai-sdk/[email protected]
+ @ai-sdk/[email protected]. Full design in plan-v6-upgrade.md;
two rounds of Gemini code review captured in report.md.
Motivation: the recurring 'reasoning-signature without reasoning' error
on Claude Opus 4.7 was a v4 SDK artefact — @ai-sdk/[email protected] emitted
Anthropic signature_delta as a separate stream chunk that orphaned when
the model produced a signed-but-empty thinking block, and our chunk
store had no signature field so the round-trip back to Anthropic was
rejected on the next turn. In v6, signatures arrive inside
providerMetadata on the reasoning-end event, and the orphan-signature
class of bug is gone at the SDK level.
Core changes:
• ThinkingChunk gains optional metadata?: Record<string, unknown>
(the v6 providerMetadata blob). A non-undefined metadata 'seals'
the chunk: subsequent reasoning-delta opens a new chunk rather
than extending the sealed one.
• AgentEvent gains { type: 'reasoning-end'; metadata? } (replaces
the v4 reasoning-signature variant).
• toModelMessages (replaces toCoreMessages):
- returns ModelMessage[] (was CoreMessage[])
- thinking → { type: 'reasoning', text, providerOptions: metadata }
- tool-batch entries → { type: 'tool-call', input } (was 'args')
- tool results → { output: { type: 'text', value } } ToolResultOutput
• Claude OAuth uses createAnthropic({ authToken }) natively — no more
custom-fetch x-api-key → Bearer swap.
• rewriteBodyForOpus47 deleted — Opus 4.7 adaptive thinking is native
via providerOptions.anthropic.thinking = { type: 'adaptive' }.
• V1 middleware → V3 (specificationVersion: 'v3').
• v4-era normalizeMessages openai-compatible middleware deleted; the
v6 openai-compatible provider extracts reasoning_content natively
from { type: 'reasoning' } content parts.
• applyAnthropicStructuralNormalisations (mirrors opencode
provider/transform.ts:53-148): drops empty text/reasoning parts,
scrubs non-[a-zA-Z0-9_-] toolCallIds, splits [tool-call, non-tool]
assistant turns (Anthropic rejects tool_use followed by text).
• applyOpenAICompatibleReasoningNormalisation (mirrors opencode
transform.ts:217-249): lifts reasoning text into
providerOptions.openaiCompatible.reasoning_content (always, even
empty). Solves DeepSeek 'The reasoning_content in the thinking
mode must be passed back' — the v6 SDK skips emitting
reasoning_content when text is empty (dist/index.mjs:245), but
DeepSeek requires the field present once thinking was used.
• Tools: tool({ inputSchema: jsonSchema(zodToJsonSchema(...)) })
(was parameters: ZodSchema). AI SDK tools have no execute
callback — the agent runs tools manually for permission prompts
and shell-output streaming. New dep: zod-to-json-schema@^3.25.2.
• fullStream event loop rewritten for v6 event shape: text-delta
(text not textDelta), reasoning-start/delta/end, tool-input-*,
tool-call (input not args), tool-result, tool-error (new), abort
(new), start-step/finish-step, finish.
Max-thinking audit (matches opencode transform.ts:642-671 budgets):
• Claude enabled-thinking max budget 16000 → 31999 (Anthropic ceiling)
• Claude enabled-thinking high budget 10000 → 16000
• maxOutputTokens 'budget + 8000' → fixed 32000 (matches opencode's
OUTPUT_TOKEN_MAX; model self-allocates thinking vs response within)
• Opus 4.7 adaptive thinking gains display: 'summarized' and sibling
effort field (without these, thinking content is hidden by Anthropic
and the model barely thinks).
Frontend mirrors:
• types.ts — ThinkingChunk.metadata?, AgentEvent reasoning-end
• tabs.svelte.ts — routes reasoning-end through applyChunkEvent
• ChatMessage.svelte — hides empty thinking chunks; hides the entire
assistant bubble when no chunk has renderable content
Gemini-review-driven fixes:
• tool-error and abort stream events now surface as error chunks
(were silently ignored)
• toolCallId scrubbing pass (opencode transform.ts:96-122 parity)
• Empty-reasoning-cull explicit test coverage for both Anthropic
structural normalisation and DeepSeek path
Test counts (223 tests across 3 packages, all green):
• tests/chunks/append.test.ts: 44 (was 38) — reasoning-end sealing,
orphan walk-back, multi-block interleaving
• tests/agent/agent.test.ts: 24 (was 5) — exhaustive v6 event
mappings, structural normalisations, signature/reasoning_content
round-trip, tool-error/abort branches, DeepSeek scenario, empty
reasoning edge case
• tests/llm/provider.test.ts: 9 (was 22) — dropped 13 obsolete v4
middleware tests; new minimal tests confirm no middleware wrapping
on default openai-compat path and that createAnthropic gets
authToken vs apiKey correctly for OAuth vs api-key flows
• tests/tools/registry.test.ts: 10 (was 4) — v6 tool() contract
(inputSchema, no execute, JSON Schema for nested zod)
• packages/api/tests/agent-manager.test.ts: 12 (was 7) — mock Agent
emits v6 reasoning events; reasoning-end broadcast + ordering
• packages/frontend/tests/chat-store.test.ts: 35 (was 32) —
reasoning-end flow through Svelte $state store
typecheck clean (tsc --noEmit on core + api, svelte-check on frontend),
biome clean across 124 files.
Diffstat (limited to 'packages/core/src')
| -rw-r--r-- | packages/core/src/agent/agent.ts | 370 | ||||
| -rw-r--r-- | packages/core/src/chunks/append.ts | 66 | ||||
| -rw-r--r-- | packages/core/src/llm/provider.ts | 151 | ||||
| -rw-r--r-- | packages/core/src/tools/registry.ts | 48 | ||||
| -rw-r--r-- | packages/core/src/types/index.ts | 24 |
5 files changed, 470 insertions, 189 deletions
diff --git a/packages/core/src/agent/agent.ts b/packages/core/src/agent/agent.ts index 4e58378..7def41c 100644 --- a/packages/core/src/agent/agent.ts +++ b/packages/core/src/agent/agent.ts @@ -1,5 +1,6 @@ import { dirname } from "node:path"; -import type { CoreMessage, CoreSystemMessage } from "ai"; +import type { ProviderOptions } from "@ai-sdk/provider-utils"; +import type { ModelMessage, SystemModelMessage } from "ai"; import { streamText } from "ai"; import { appendEventToChunks } from "../chunks/append.js"; import { buildBillingHeaderValue, SYSTEM_IDENTITY } from "../credentials/claude.js"; @@ -20,7 +21,7 @@ import type { } from "../types/index.js"; /** - * Rebuild AI SDK `CoreMessage[]` from our internal `ChatMessage[]`. + * Rebuild AI SDK `ModelMessage[]` from our internal `ChatMessage[]`. * * Strip rules (see plan-chunk-refactor.md): * - `role: "system"` messages are skipped wholesale (they're display-only @@ -28,8 +29,11 @@ import type { * - `error` chunks are skipped — the turn ended; the LLM doesn't need them. * - `system` chunks are skipped — display-only notices. * - `text` chunks → `{ type: "text", text }` parts. - * - `thinking` chunks → `{ type: "reasoning", text }` parts (handles - * Claude's `interleaved-thinking-2025-05-14` round-trip). + * - `thinking` chunks → `{ type: "reasoning", text, providerOptions? }` parts. + * The `providerOptions` carries the Anthropic signature blob (if present) so + * Anthropic can validate extended-thinking round-trips. Non-Anthropic + * reasoning has no metadata and is sent without providerOptions (accepted + * by Anthropic — it just won't verify a missing signature). * - `tool-batch` chunks → one `{ type: "tool-call" }` part per entry * inside the current assistant message, followed by a separate * `{ role: "tool", content: [{ type: "tool-result", ... }] }` message @@ -42,8 +46,8 @@ import type { * LLM, so this case only arises mid-step (where the message hasn't been * round-tripped to the LLM yet) and is benign. */ -function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreMessage[] { - const result: CoreMessage[] = []; +function toModelMessages(messages: ChatMessage[], useToolPrefix?: boolean): ModelMessage[] { + const result: ModelMessage[] = []; for (const msg of messages) { if (msg.role === "system") continue; @@ -62,8 +66,8 @@ function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreM // role === "assistant" const parts: Array< | { type: "text"; text: string } - | { type: "reasoning"; text: string } - | { type: "tool-call"; toolCallId: string; toolName: string; args: Record<string, unknown> } + | { type: "reasoning"; text: string; providerOptions?: ProviderOptions } + | { type: "tool-call"; toolCallId: string; toolName: string; input: unknown } > = []; const trailingToolResults: Array<{ toolCallId: string; @@ -77,16 +81,26 @@ function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreM parts.push({ type: "text", text: chunk.text }); break; case "thinking": - parts.push({ type: "reasoning", text: chunk.text }); + // v6: carry providerOptions (Anthropic signature blob) if present. + // Non-Anthropic reasoning has no metadata → send without providerOptions + // (Anthropic accepts it; the round-trip just won't carry a signature). + parts.push({ + type: "reasoning", + text: chunk.text, + ...(chunk.metadata !== undefined + ? { providerOptions: chunk.metadata as ProviderOptions } + : {}), + }); break; case "tool-batch": for (const entry of chunk.calls) { const toolName = useToolPrefix ? prefixToolName(entry.name) : entry.name; + // v6: `input` replaces v4's `args` parts.push({ type: "tool-call", toolCallId: entry.id, toolName, - args: entry.arguments, + input: entry.arguments, }); if (entry.result !== undefined) { trailingToolResults.push({ @@ -114,6 +128,7 @@ function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreM } for (const tr of trailingToolResults) { + // v6: `output` (ToolResultOutput) replaces v4's `result` (raw string) result.push({ role: "tool", content: [ @@ -121,7 +136,7 @@ function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreM type: "tool-result", toolCallId: tr.toolCallId, toolName: tr.toolName, - result: tr.result, + output: { type: "text" as const, value: tr.result }, }, ], }); @@ -131,6 +146,163 @@ function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreM } /** + * Apply OpenAI-compatible reasoning normalisation. + * + * Cribbed from opencode `provider/transform.ts:217-249`. Solves DeepSeek's + * "The reasoning_content in the thinking mode must be passed back to the + * API" error. + * + * The v6 `@ai-sdk/[email protected]` provider extracts `reasoning_content` + * from assistant `{ type: "reasoning", text }` parts natively + * (see `node_modules/@ai-sdk/openai-compatible/dist/index.mjs:215-216` and :245). + * But that native path SKIPS emission when `reasoning.length === 0` — + * "reasoning_content" is omitted from the wire. DeepSeek (and similar + * "thinking mode" providers) require the field to be present in every + * follow-up turn once it was emitted at least once, even if the value is + * the empty string. + * + * Strategy: for every assistant message that has any `reasoning` parts, + * concatenate their text into `providerOptions.openaiCompatible.reasoning_content` + * (preserving empty strings) AND strip those parts from content. The + * resulting payload has a single source of truth for `reasoning_content` + * coming via the message-level `providerOptions` spread at line 247 of the + * SDK provider, which fires regardless of empty-vs-non-empty text. + * + * Only applied for the default (non-Anthropic) openai-compatible path. + */ +function applyOpenAICompatibleReasoningNormalisation(msgs: ModelMessage[]): ModelMessage[] { + return msgs.map((msg) => { + if (msg.role !== "assistant" || !Array.isArray(msg.content)) return msg; + + // Find reasoning parts. If there are none, this message never had + // a thinking turn — DeepSeek doesn't require `reasoning_content` + // in that case, so we pass through unchanged. + const reasoningParts = msg.content.filter( + (p): p is Extract<typeof p, { type: "reasoning" }> => p.type === "reasoning", + ); + if (reasoningParts.length === 0) return msg; + + const reasoningText = reasoningParts.map((p) => p.text).join(""); + const filteredContent = msg.content.filter((p) => p.type !== "reasoning"); + + const existingOpts = msg.providerOptions ?? {}; + const existingCompat = (existingOpts.openaiCompatible ?? {}) as Record<string, unknown>; + + return { + ...msg, + content: filteredContent, + providerOptions: { + ...existingOpts, + openaiCompatible: { + ...existingCompat, + // Always set, even when empty. This is the key fix — + // the SDK's content-side extraction skips empty + // reasoning, but DeepSeek requires the field present. + reasoning_content: reasoningText, + }, + }, + } as ModelMessage; + }); +} + +/** + * Apply Anthropic structural normalisations to a `ModelMessage[]`. + * + * Cribbed from opencode `provider/transform.ts:53-148`. Two passes: + * + * 1. **Empty-text/reasoning filter**: Drop `text` / `reasoning` parts + * whose `text === ""`. Drop messages whose content array becomes + * empty. Anthropic rejects assistant turns with empty content. + * + * 2. **`toolCallId` scrubbing**: Anthropic only accepts tool call IDs + * that match `[a-zA-Z0-9_-]`. Our IDs are crypto.randomUUID() values + * which are already safe, but tool-call IDs assigned by upstream + * sources (provider-executed tools, subagent retrieval, etc.) may + * not be. Defensively scrub. Mirrors opencode `provider/transform.ts:96-122`. + * + * 3. **`[tool-call, text]` split**: Anthropic rejects assistant turns + * where `tool_use` blocks are followed by non-tool-call content + * ("`tool_use` ids were found without `tool_result` blocks immediately + * after"). If an assistant message has mixed ordering, split it into + * `[non-tool parts] + [tool-call parts]`. + * + * Only applied for Anthropic-backed providers (Claude OAuth or + * opencode-anthropic). Skip for openai-compatible / OpenCode Zen. + */ +const SCRUB_TOOL_CALL_ID = (id: string): string => id.replace(/[^a-zA-Z0-9_-]/g, "_"); + +function applyAnthropicStructuralNormalisations(msgs: ModelMessage[]): ModelMessage[] { + // Pass 1: Filter empty text/reasoning parts; drop messages that become empty. + msgs = msgs + .map((msg) => { + if (typeof msg.content === "string") { + if (msg.content === "") return undefined; + return msg; + } + if (!Array.isArray(msg.content)) return msg; + const filtered = msg.content.filter((part) => { + if (part.type === "text" || part.type === "reasoning") { + return (part as { text: string }).text !== ""; + } + return true; + }); + if (filtered.length === 0) return undefined; + return { ...msg, content: filtered }; + }) + .filter((msg): msg is ModelMessage => msg !== undefined && msg.content !== ""); + + // Pass 2: Scrub toolCallId chars on both assistant tool-call parts and + // tool-role tool-result parts. Anthropic rejects anything outside + // [a-zA-Z0-9_-]. Our internal IDs are crypto.randomUUID() and safe, but + // upstream provider-executed tools or external sources may not be. + msgs = msgs.map((msg) => { + if (msg.role === "assistant" && Array.isArray(msg.content)) { + return { + ...msg, + content: msg.content.map((part) => { + if (part.type === "tool-call" || part.type === "tool-result") { + return { ...part, toolCallId: SCRUB_TOOL_CALL_ID(part.toolCallId) }; + } + return part; + }), + }; + } + if (msg.role === "tool" && Array.isArray(msg.content)) { + return { + ...msg, + content: msg.content.map((part) => { + if (part.type === "tool-result") { + return { ...part, toolCallId: SCRUB_TOOL_CALL_ID(part.toolCallId) }; + } + return part; + }), + }; + } + return msg; + }); + + // Pass 3: Split assistant messages where tool-calls are followed by non-tool-call parts. + // [text, tool-call, text] → [text, text] + [tool-call] (text-only first, tools-only second) + msgs = msgs.flatMap((msg) => { + if (msg.role !== "assistant" || !Array.isArray(msg.content)) return [msg]; + const parts = msg.content; + const firstToolCallIdx = parts.findIndex((part) => part.type === "tool-call"); + if (firstToolCallIdx === -1) return [msg]; // no tool calls → pass through + // If everything from the first tool-call onward is also a tool-call, it's already valid + if (!parts.slice(firstToolCallIdx).some((part) => part.type !== "tool-call")) return [msg]; + // Has non-tool-call content AFTER a tool-call → split + const nonToolParts = parts.filter((part) => part.type !== "tool-call"); + const toolParts = parts.filter((part) => part.type === "tool-call"); + return [ + { ...msg, content: nonToolParts }, + { ...msg, content: toolParts }, + ]; + }); + + return msgs; +} + +/** * Apply Anthropic prompt-caching breakpoints to a message list. * * Anthropic caches the entire request prefix up to (and including) any block @@ -151,8 +323,8 @@ function toCoreMessages(messages: ChatMessage[], useToolPrefix?: boolean): CoreM * served via `@ai-sdk/openai` (GPT) and `@ai-sdk/google` (Gemini) likewise * use server-side automatic caching. */ -function applyAnthropicCaching(msgs: CoreMessage[]): void { - const targets = new Set<CoreMessage>(); +function applyAnthropicCaching(msgs: ModelMessage[]): void { + const targets = new Set<ModelMessage>(); const systemMsgs = msgs.filter((m) => m.role === "system").slice(0, 2); for (const m of systemMsgs) targets.add(m); @@ -447,14 +619,25 @@ export class Agent { // `system` shortcut parameter takes a plain string with nowhere to // attach `providerOptions.anthropic.cacheControl`. Moving it inline // also lets us apply rolling cache breakpoints to the last messages. - const systemMessage: CoreSystemMessage = { role: "system", content: systemPrompt }; - const coreMessages: CoreMessage[] = [ + const systemMessage: SystemModelMessage = { + role: "system", + content: systemPrompt, + }; + + let coreMessages: ModelMessage[] = [ systemMessage, - ...toCoreMessages(stepMessages, isClaudeOAuth), + ...toModelMessages(stepMessages, isClaudeOAuth), ]; + // Apply provider-specific structural normalisations before + // sending. Anthropic and openai-compatible paths each have + // their own message-shape requirements; the two passes are + // mutually exclusive. if (usesAnthropicSDK) { + coreMessages = applyAnthropicStructuralNormalisations(coreMessages); applyAnthropicCaching(coreMessages); + } else { + coreMessages = applyOpenAICompatibleReasoningNormalisation(coreMessages); } const streamOptions: Parameters<typeof streamText>[0] = { @@ -464,34 +647,68 @@ export class Agent { }; if (isClaudeOAuth && effort !== "none") { - // Opus 4.7 rejects `thinking: { type: "enabled" }` ("reasoning- - // signature without reasoning") and only supports adaptive thinking. - // `@ai-sdk/anthropic` v1.x can't emit `type: "adaptive"`, so we - // leave `providerOptions.anthropic.thinking` unset and let the - // custom fetch in `createClaudeOAuthProvider` inject the adaptive - // shape into the request body. We still set `maxTokens` here so - // the SDK serializes it — adaptive thinking spends from this - // budget rather than a separate one. + // v6 native support for Opus 4.7 adaptive thinking via + // providerOptions. No more rewriteBodyForOpus47 body- + // injection hack needed. + // + // Budgets mirror opencode transform.ts: + // - max budget = 31999 (Anthropic's documented ceiling; + // `Math.min(31_999, model.limit.output - 1)` in + // opencode line 668) + // - high budget = 16000 (opencode line 662: + // `Math.min(16_000, Math.floor(model.limit.output/2-1))`) + // - medium/low: no opencode equivalent — sensible + // intermediate defaults. + // + // `maxOutputTokens` is the TOTAL output cap (thinking + + // response combined). Anthropic's standard output max is + // 32K for Claude 4.x; matching opencode's OUTPUT_TOKEN_MAX + // constant. The `budget_tokens` field is a thinking + // ceiling, not a requirement, so the model self-regulates + // thinking vs response within this total budget. + // + // Opus 4.7 (adaptive) specifics (mirrors opencode + // transform.ts:642-655): + // - `thinking.type = "adaptive"` — the model decides how + // much to think within `maxOutputTokens`. + // - `thinking.display = "summarized"` — adaptive thinking + // for Opus 4.7 defaults to `display: "omitted"`, which + // means NO thinking content is streamed to us. Setting + // `summarized` is what makes the thinking visible in + // the UI. + // - `effort` (sibling of `thinking`) — adaptive needs an + // explicit effort level. Without it, Anthropic uses a + // conservative default and the model barely thinks. const isOpus47 = this.config.model === "claude-opus-4-7"; const budgetTokens = effort === "max" - ? 16000 + ? 31999 : effort === "high" - ? 10000 + ? 16000 : effort === "medium" ? 5000 : effort === "low" ? 2000 : 0; - if (isOpus47) { - streamOptions.maxTokens = budgetTokens + 8000; - } else { - streamOptions.providerOptions = { - anthropic: { thinking: { type: "enabled" as const, budgetTokens } }, - }; - streamOptions.maxTokens = budgetTokens + 8000; - } + // Cap at Anthropic's standard 32K output (matches opencode's + // OUTPUT_TOKEN_MAX); the model splits this between thinking + // (up to budgetTokens) and response. + streamOptions.maxOutputTokens = 32000; + streamOptions.providerOptions = { + anthropic: isOpus47 + ? { + thinking: { type: "adaptive" as const, display: "summarized" as const }, + effort, + } + : { thinking: { type: "enabled" as const, budgetTokens } }, + }; } else if (!usesAnthropicSDK && effort !== "none") { + // OpenAI-compatible models (OpenCode Zen: DeepSeek, GLM, + // Kimi, etc.). The `reasoningEffort` field is forwarded + // verbatim to providers that support it. `"max"` is the + // strongest level we expose; the provider may map it + // internally (e.g. DeepSeek interprets it as deep + // reasoning). streamOptions.providerOptions = { openaiCompatible: { reasoningEffort: effort } }; } @@ -513,31 +730,100 @@ export class Agent { try { for await (const event of result.fullStream) { if (event.type === "text-delta") { + // v6: text-delta carries `text` (not `textDelta`) const internalEvent: AgentEvent = { type: "text-delta", - delta: event.textDelta, + delta: event.text, }; appendEventToChunks(chunks, internalEvent); yield internalEvent; - } else if (event.type === "reasoning") { + } else if (event.type === "reasoning-delta") { + // v6 new event: reasoning-delta carries `text` (not `textDelta`) const internalEvent: AgentEvent = { type: "reasoning-delta", - delta: event.textDelta, + delta: event.text, }; appendEventToChunks(chunks, internalEvent); yield internalEvent; + } else if (event.type === "reasoning-end") { + // Only emit when providerMetadata is present — non-Anthropic + // models send reasoning-end without any metadata to round-trip. + // Anthropic's signature lives inside providerMetadata as + // { anthropic: { signature: "..." } }. + if (event.providerMetadata !== undefined) { + const internalEvent: AgentEvent = { + type: "reasoning-end", + metadata: event.providerMetadata as Record<string, unknown>, + }; + appendEventToChunks(chunks, internalEvent); + yield internalEvent; + } } else if (event.type === "tool-call") { + // v6: tool call input is in `input` (not `args`) const rawName = event.toolName; const toolName = isClaudeOAuth ? unprefixToolName(rawName) : rawName; const toolCall: ToolCall = { id: event.toolCallId, name: toolName, - arguments: event.args as Record<string, unknown>, + arguments: event.input as Record<string, unknown>, }; stepToolCalls.push(toolCall); const internalEvent: AgentEvent = { type: "tool-call", toolCall }; appendEventToChunks(chunks, internalEvent); yield internalEvent; + } else if (event.type === "tool-error") { + // Anthropic-side / provider-executed tool failures + // (e.g. server tools or repair-tool-call paths) surface + // as a fresh stream event rather than as a thrown + // `tool-result` from our manual executor. Forward both + // a synthetic tool-result (so the tool-batch entry the + // model expects has an `isError: true` result) and + // abort the step as an error chunk — without this the + // model would silently wait for a result that never + // arrives. + const evt = event as unknown as { + toolCallId: string; + toolName: string; + error: unknown; + }; + const toolName = isClaudeOAuth ? unprefixToolName(evt.toolName) : evt.toolName; + const errMessage = evt.error instanceof Error ? evt.error.message : String(evt.error); + + const trEvent: AgentEvent = { + type: "tool-result", + toolResult: { + toolCallId: evt.toolCallId, + toolName, + result: `Error: ${errMessage}`, + isError: true, + }, + }; + appendEventToChunks(chunks, trEvent); + yield trEvent; + + const errorMsg = formatError(evt.error, this.config); + const errChunkEvent: AgentEvent = { type: "error", error: errorMsg }; + appendEventToChunks(chunks, errChunkEvent); + yield errChunkEvent; + this.status = "error"; + yield { type: "status", status: "error" }; + return; + } else if (event.type === "abort") { + // Stream aborted upstream. Surface as an error so the + // frontend tab transitions out of `running`. We don't + // currently call abortController.abort() ourselves, so + // this would only fire from external signal propagation. + const reason = + typeof (event as { reason?: unknown }).reason === "string" + ? (event as { reason: string }).reason + : "stream aborted"; + const errorMsg = formatError(new Error(reason), this.config); + const internalEvent: AgentEvent = { type: "error", error: errorMsg }; + appendEventToChunks(chunks, internalEvent); + yield internalEvent; + this.status = "error"; + yield { type: "status", status: "error" }; + return; } else if (event.type === "error") { const errRecord = event.error as unknown as Record<string, unknown>; const statusCode = @@ -554,9 +840,17 @@ export class Agent { yield { type: "status", status: "error" }; return; } + // Ignored events (intentional): + // start, text-start, text-end, reasoning-start, + // tool-input-start, tool-input-delta, tool-input-end, + // tool-result (only fires if tool has execute; ours don't), + // start-step, finish-step, finish, raw, + // source, file, tool-output-denied, tool-approval-* } } catch (streamErr) { const errMsg = streamErr instanceof Error ? streamErr.message : String(streamErr); + // v6 SDK error message: "Model tried to call unavailable tool '...'" + // (v4 was: "tried to call unavailable tool '...'") const unavailMatch = errMsg.match(/tried to call unavailable tool '([^']+)'/i); if (!unavailMatch) throw streamErr; diff --git a/packages/core/src/chunks/append.ts b/packages/core/src/chunks/append.ts index 9c2a367..fe384bd 100644 --- a/packages/core/src/chunks/append.ts +++ b/packages/core/src/chunks/append.ts @@ -8,18 +8,31 @@ * * Open/close rules — see plan-chunk-refactor.md for the full table. * - * | Chunk | Opens on | Coalesces | - * |---------------|-------------------------------------------------------|------------------------------------------------------------| - * | `text` | first `text-delta` after a non-text chunk | consecutive `text-delta` events append to `.text` | - * | `thinking` | first `reasoning-delta` after a non-thinking chunk | consecutive `reasoning-delta` events append to `.text` | - * | `tool-batch` | first `tool-call` after a non-tool-batch chunk | consecutive `tool-call` events push a new entry to `.calls`| - * | `error` | every `error` event | NEVER (always single-event) | - * | `system` | every `notice`/`model-changed`/`config-reload`/... | NEVER (two consecutive system events → two chunks) | + * | Chunk | Opens on | Coalesces | + * |---------------|-----------------------------------------------------------------|------------------------------------------------------------| + * | `text` | first `text-delta` after a non-text chunk | consecutive `text-delta` events append to `.text` | + * | `thinking` | first `reasoning-delta` after a non-thinking chunk | consecutive `reasoning-delta` events append to `.text` | + * | | OR after the last thinking chunk was sealed by `reasoning-end` | (only into the most recent UNSEALED thinking chunk) | + * | `tool-batch` | first `tool-call` after a non-tool-batch chunk | consecutive `tool-call` events push a new entry to `.calls`| + * | `error` | every `error` event | NEVER (always single-event) | + * | `system` | every `notice`/`model-changed`/`config-reload`/... | NEVER (two consecutive system events → two chunks) | * * Side-effect events (no new chunk): - * - `tool-result` → finds the call by `id` across all `tool-batch` chunks (most-recent first) - * and updates its `result` / `isError`. - * - `shell-output` → appends to the most recent entry of the most recent `tool-batch` chunk. + * - `tool-result` → finds the call by `id` across all `tool-batch` + * chunks (most-recent first) and updates its + * `result` / `isError`. + * - `shell-output` → appends to the most recent entry of the most + * recent `tool-batch` chunk. + * - `reasoning-end` → attaches `metadata` (the AI SDK v6 + * `providerMetadata` blob) to the most recent + * UNSEALED `thinking` chunk. The metadata is also + * the "sealed" marker — subsequent + * `reasoning-delta`s will open a new chunk rather + * than extending this one. Anthropic's signature + * lives inside this blob; round-tripping it on the + * next turn is mandatory for Anthropic to accept + * the conversation. Orphan `reasoning-end` events + * (no unsealed thinking chunk) are dropped. * * Ignored events: * - `status`, `done`, `task-list-update`, `tab-created`, `message-queued`, @@ -56,9 +69,14 @@ export function appendEventToChunks(chunks: Chunk[], event: AgentEvent): void { } case "reasoning-delta": { - // Open or extend the current thinking chunk. + // Open a new thinking chunk if the last chunk is not a thinking + // chunk OR if it's already sealed by metadata. Anthropic emits + // each thinking content block with its own metadata; a fresh + // reasoning-delta after a sealed thinking chunk is the start of + // a new block, not a continuation — extending the sealed chunk + // would corrupt the metadata/text mapping. const last = chunks[chunks.length - 1]; - if (last && last.type === "thinking") { + if (last && last.type === "thinking" && last.metadata === undefined) { last.text += event.delta; } else { chunks.push({ type: "thinking", text: event.delta }); @@ -66,6 +84,30 @@ export function appendEventToChunks(chunks: Chunk[], event: AgentEvent): void { return; } + case "reasoning-end": { + // Attach `providerMetadata` to the most recent unsealed + // thinking chunk. Anthropic's signature lives inside this + // blob; without it on the next request, Anthropic rejects the + // thinking block. The walk-back is a defensive backstop — + // Anthropic's SSE delivers a content block's deltas strictly + // in order and `appendEventToChunks` runs synchronously per + // event, so the most recent thinking chunk is normally the + // last chunk in the array. + if (event.metadata === undefined) return; + for (let i = chunks.length - 1; i >= 0; i--) { + const c = chunks[i]; + if (!c || c.type !== "thinking") continue; + if (c.metadata !== undefined) { + // Already sealed; the orphan metadata has no home. + return; + } + c.metadata = event.metadata; + return; + } + // No thinking chunk found at all — drop silently. + return; + } + case "tool-call": { // Open or extend the current tool-batch chunk. const last = chunks[chunks.length - 1]; diff --git a/packages/core/src/llm/provider.ts b/packages/core/src/llm/provider.ts index a7d800c..81f7f51 100644 --- a/packages/core/src/llm/provider.ts +++ b/packages/core/src/llm/provider.ts @@ -1,42 +1,6 @@ import { createAnthropic } from "@ai-sdk/anthropic"; import { createOpenAICompatible } from "@ai-sdk/openai-compatible"; -import type { LanguageModelV1, LanguageModelV1Middleware, LanguageModelV1Prompt } from "ai"; -import { wrapLanguageModel } from "ai"; - -function normalizeMessages(msgs: unknown[]): unknown[] { - return msgs.map((msg: unknown) => { - const message = msg as Record<string, unknown>; - if (message.role !== "assistant" || !Array.isArray(message.content)) return message; - - const content = message.content as Array<Record<string, unknown>>; - const reasoningParts = content.filter( - (p) => p.type === "reasoning" || p.type === "redacted-reasoning", - ); - const reasoningText = reasoningParts.map((p) => p.text ?? "").join(""); - - const filteredContent = content.filter( - (p) => p.type !== "reasoning" && p.type !== "redacted-reasoning", - ); - - const existingMetadata = (message.providerMetadata ?? {}) as Record<string, unknown>; - const existingOpenAICompat = (existingMetadata.openaiCompatible ?? {}) as Record< - string, - unknown - >; - - return { - ...message, - content: filteredContent, - providerMetadata: { - ...existingMetadata, - openaiCompatible: { - ...existingOpenAICompat, - reasoning_content: reasoningText, - }, - }, - }; - }); -} +import type { LanguageModelV3 } from "@ai-sdk/provider"; export interface ProviderConfig { apiKey: string; @@ -63,9 +27,9 @@ function unprefixToolName(name: string): string { // Explicit factory return type so the inferred type doesn't leak references // into transitive `@ai-sdk/provider` paths (which would trip TS2742). -// `@ai-sdk/anthropic` v1.x already returns `LanguageModelV1`-spec models; -// `@ai-sdk/openai-compatible` v0.2.x and `wrapLanguageModel` likewise. -export type ModelFactory = (modelId: string) => LanguageModelV1; +// `@ai-sdk/anthropic` v3.x and `@ai-sdk/openai-compatible` v2.x both return +// `LanguageModelV3`-spec models; `wrapLanguageModel` likewise. +export type ModelFactory = (modelId: string) => LanguageModelV3; export function createProvider(config: ProviderConfig): ModelFactory { if (config.provider === "anthropic") { @@ -76,115 +40,52 @@ export function createProvider(config: ProviderConfig): ModelFactory { return createApiKeyAnthropicProvider(config); } - // Default: OpenAI-compatible provider + // Default: OpenAI-compatible provider (OpenCode Zen — DeepSeek, GLM, + // Kimi, MiniMax, etc.). + // + // `@ai-sdk/[email protected]` handles reasoning round-tripping + // natively: it reads `{ type: "reasoning", text }` parts from each + // assistant message's content and emits them as `reasoning_content` + // on the wire (see node_modules/@ai-sdk/openai-compatible/dist/index.mjs + // lines 215-216 and 245). Our `toModelMessages` in agent.ts already + // emits reasoning parts from `ThinkingChunk`s, so no middleware is + // needed. + // + // (The v4-era `normalizeMessages` middleware that lived here was + // actively breaking DeepSeek: it stripped reasoning parts from + // content AND wrote them under `providerMetadata` — wrong key in v3 + // prompts, which use `providerOptions`. The result was that + // reasoning_content never reached the wire and DeepSeek rejected the + // follow-up turn with "must be passed back".) const provider = createOpenAICompatible({ name: "opencode-zen", apiKey: config.apiKey, baseURL: config.baseURL, }); - return (modelId: string) => { - const middleware: LanguageModelV1Middleware = { - middlewareVersion: "v1" as const, - transformParams: async ({ type, params }) => { - if (type === "stream" && params.prompt) { - return { - ...params, - prompt: normalizeMessages(params.prompt as unknown[]) as LanguageModelV1Prompt, - }; - } - return params; - }, - }; - - return wrapLanguageModel({ - model: provider(modelId), - middleware: [middleware], - }); - }; + return (modelId: string) => provider(modelId); } /** * Claude OAuth provider. Used by Dispatch's `anthropic` provider keys - * (claude-pro, claude-max). Swaps `x-api-key` for `Authorization: Bearer` - * to satisfy Anthropic's OAuth flow, and mimics Claude Code CLI request - * headers so the request bills against the user's Claude subscription. - * - * The custom fetch also rewrites the outgoing JSON body for Claude Opus 4.7: - * that model rejects `thinking: { type: "enabled", budget_tokens }` (the only - * shape `@ai-sdk/anthropic` v1.x can emit) with "reasoning-signature without - * reasoning", and instead requires `thinking: { type: "adaptive" }`. `ai` v4 - * is pinned to V1-spec providers, so we can't upgrade to v3 of the Anthropic - * SDK without breaking everything. Doing the rewrite here keeps the rest of - * the agent path SDK-agnostic and limits the special case to one model. + * (claude-pro, claude-max). Uses `authToken` to send `Authorization: Bearer` + * (natively supported by `@ai-sdk/anthropic` v3.x), and mimics Claude Code CLI + * request headers so the request bills against the user's Claude subscription. */ function createClaudeOAuthProvider(config: ProviderConfig): ModelFactory { - const accessToken = config.claudeCredentials?.accessToken ?? config.apiKey; - - const customFetch = Object.assign( - async (url: RequestInfo | URL, init?: RequestInit): Promise<Response> => { - const headers = new Headers(init?.headers); - headers.delete("x-api-key"); - headers.set("authorization", `Bearer ${accessToken}`); - - const body = rewriteBodyForOpus47(init?.body); - return globalThis.fetch(url, { ...init, headers, body }); - }, - { preconnect: globalThis.fetch.preconnect?.bind(globalThis.fetch) }, - ); - const anthropic = createAnthropic({ - apiKey: "sk-ant-oauth-placeholder", baseURL: config.baseURL || "https://api.anthropic.com/v1", + authToken: config.claudeCredentials?.accessToken ?? config.apiKey, headers: { "anthropic-dangerous-direct-browser-access": "true", "x-app": "cli", "user-agent": "claude-cli/2.1.112 (external, sdk-cli)", }, - fetch: customFetch as typeof globalThis.fetch, }); - return (modelId: string) => anthropic(modelId); } /** - * If the request body is a JSON `/messages` payload targeting Claude Opus 4.7 - * and the caller signaled they want thinking (by setting `max_tokens` above - * Anthropic's default 4096), insert `thinking: { type: "adaptive" }`. - * - * Skipping the rewrite when `max_tokens` is small (or absent) keeps `effort: - * "none"` requests as plain non-thinking calls — agent.ts only sets a high - * `max_tokens` when thinking is wanted, so this acts as a clean signal. - * - * Returns the body unchanged for any other model, any non-string body, or any - * payload that fails to parse, leaving non-Anthropic providers, non-Opus-4.7 - * Claude models, and streaming/binary uploads unaffected. - */ -function rewriteBodyForOpus47(body: BodyInit | null | undefined): BodyInit | null | undefined { - if (typeof body !== "string") return body; - let parsed: Record<string, unknown>; - try { - parsed = JSON.parse(body) as Record<string, unknown>; - } catch { - return body; - } - if (parsed.model !== "claude-opus-4-7") return body; - const maxTokens = typeof parsed.max_tokens === "number" ? parsed.max_tokens : 0; - if (maxTokens <= 4096) return body; - parsed.thinking = { type: "adaptive" }; - // Anthropic rejects requests that combine extended thinking (enabled or - // adaptive) with any temperature other than 1. `ai` v4 defaults - // `temperature: 0`, and the v1 Anthropic SDK normally strips it when its - // own `isThinking` flag is set — but we're injecting `thinking` here, - // behind the SDK's back, so we have to strip it ourselves. Same for - // `top_p` and `top_k`, which are likewise rejected with thinking. - delete parsed.temperature; - delete parsed.top_p; - delete parsed.top_k; - return JSON.stringify(parsed); -} - -/** * Plain-API-key Anthropic-format provider. Used to hit gateways that speak * Anthropic's `/messages` protocol with a standard `x-api-key` header — most * importantly OpenCode Go's MiniMax and Qwen routes. Unlike the Claude OAuth diff --git a/packages/core/src/tools/registry.ts b/packages/core/src/tools/registry.ts index 0c7b110..a09535e 100644 --- a/packages/core/src/tools/registry.ts +++ b/packages/core/src/tools/registry.ts @@ -1,7 +1,25 @@ -import { tool } from "ai"; -import { z } from "zod"; +import type { Tool } from "ai"; +import { jsonSchema, tool } from "ai"; +import { zodToJsonSchema } from "zod-to-json-schema"; import type { ToolDefinition } from "../types/index.js"; +/** + * Convert an internal `ToolDefinition` (Zod-parameterised) to an AI SDK v6 + * `Tool` object. + * + * Critically, NO `execute` function is attached. The agent's manual tool + * loop (see agent.ts) handles execution itself — for permission prompts, + * shell-output streaming, and queued-message injection. Without `execute`, + * the SDK never auto-runs tools; it only surfaces `tool-call` events from + * `fullStream` that agent.ts collects and dispatches. + */ +function toAISDKTool(def: ToolDefinition): Tool { + return tool({ + description: def.description, + inputSchema: jsonSchema(zodToJsonSchema(def.parameters)), + }); +} + export function createToolRegistry(tools: ToolDefinition[]) { const toolMap = new Map<string, ToolDefinition>(tools.map((t) => [t.name, t])); @@ -14,19 +32,21 @@ export function createToolRegistry(tools: ToolDefinition[]) { return toolMap.get(name); }, - getAISDKTools() { - const result: Record<string, ReturnType<typeof tool>> = {}; + /** + * Returns AI SDK v6 `Tool` objects keyed by tool name, for passing + * directly to `streamText({ tools })`. + * + * Each tool has: + * - `description` — forwarded verbatim from the internal definition. + * - `inputSchema` — Zod schema converted to JSONSchema7 via + * `zod-to-json-schema`, then wrapped with the v6 + * `jsonSchema()` helper. + * - NO `execute` — intentional; see `toAISDKTool` above. + */ + getAISDKTools(): Record<string, Tool> { + const result: Record<string, Tool> = {}; for (const [name, def] of toolMap) { - const schema = def.parameters; - // Do NOT pass execute here — agent.ts handles tool execution - // manually via executeToolWithStreaming. Passing execute would - // cause the AI SDK to auto-execute tools AND agent.ts to execute - // them again, resulting in double execution. - const t = tool({ - description: def.description, - parameters: schema instanceof z.ZodObject ? schema : z.object({}), - }); - result[name] = t as unknown as ReturnType<typeof tool>; + result[name] = toAISDKTool(def); } return result; }, diff --git a/packages/core/src/types/index.ts b/packages/core/src/types/index.ts index 8985bd9..e59eb68 100644 --- a/packages/core/src/types/index.ts +++ b/packages/core/src/types/index.ts @@ -26,6 +26,23 @@ export interface TextChunk { export interface ThinkingChunk { type: "thinking"; text: string; + /** + * Full Anthropic `providerMetadata` blob captured from the v6 + * `reasoning-end` stream event (typically `{ anthropic: { signature + * } }` plus any other provider-side metadata). Round-tripped verbatim + * as `providerOptions` on the `ReasoningPart` of the next request so + * Anthropic can validate the thinking block's signature. + * + * Also acts as a "sealed" marker for `appendEventToChunks`: once + * `metadata` is set, the next `reasoning-delta` opens a new thinking + * chunk rather than extending this one (each Anthropic content block + * gets its own metadata, so two consecutive thinking blocks must not + * be coalesced). + * + * Optional: non-Anthropic models produce no metadata, and pre-v6 + * persisted chunks have neither field. + */ + metadata?: Record<string, unknown>; } export interface ToolBatchChunk { @@ -82,6 +99,13 @@ export type AgentEvent = | { type: "status"; status: AgentStatus } | { type: "text-delta"; delta: string } | { type: "reasoning-delta"; delta: string } + /** + * Emitted on the v6 `reasoning-end` stream event when it carries + * `providerMetadata`. `appendEventToChunks` attaches the metadata to + * the most recent unsealed `thinking` chunk; `toModelMessages` reads + * it back as `providerOptions` on the next request's `ReasoningPart`. + */ + | { type: "reasoning-end"; metadata?: Record<string, unknown> } | { type: "tool-call"; toolCall: ToolCall } | { type: "tool-result"; toolResult: ToolResult } | { type: "shell-output"; data: string; stream: "stdout" | "stderr" } |
