diff options
| author | Adam Malczewski <[email protected]> | 2026-05-28 06:54:48 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-05-28 06:54:48 +0900 |
| commit | 8b17d929e70a43749fd962554214bf8ba3e9380f (patch) | |
| tree | bdff1f409a8fe78850044c23b38436d84cbbcca9 /packages/core/tests | |
| parent | 25b6aac6d4df02e29a2ad4333272bb0998ecd410 (diff) | |
| download | dispatch-8b17d929e70a43749fd962554214bf8ba3e9380f.tar.gz dispatch-8b17d929e70a43749fd962554214bf8ba3e9380f.zip | |
refactor(core): upgrade ai-sdk v4 → v6 + Anthropic/openai-compatible reasoning round-trip + max-thinking budget audit
Migrates the LLM stack from [email protected] + @ai-sdk/[email protected] +
@ai-sdk/[email protected] to [email protected] + @ai-sdk/[email protected]
+ @ai-sdk/[email protected]. Full design in plan-v6-upgrade.md;
two rounds of Gemini code review captured in report.md.
Motivation: the recurring 'reasoning-signature without reasoning' error
on Claude Opus 4.7 was a v4 SDK artefact — @ai-sdk/[email protected] emitted
Anthropic signature_delta as a separate stream chunk that orphaned when
the model produced a signed-but-empty thinking block, and our chunk
store had no signature field so the round-trip back to Anthropic was
rejected on the next turn. In v6, signatures arrive inside
providerMetadata on the reasoning-end event, and the orphan-signature
class of bug is gone at the SDK level.
Core changes:
• ThinkingChunk gains optional metadata?: Record<string, unknown>
(the v6 providerMetadata blob). A non-undefined metadata 'seals'
the chunk: subsequent reasoning-delta opens a new chunk rather
than extending the sealed one.
• AgentEvent gains { type: 'reasoning-end'; metadata? } (replaces
the v4 reasoning-signature variant).
• toModelMessages (replaces toCoreMessages):
- returns ModelMessage[] (was CoreMessage[])
- thinking → { type: 'reasoning', text, providerOptions: metadata }
- tool-batch entries → { type: 'tool-call', input } (was 'args')
- tool results → { output: { type: 'text', value } } ToolResultOutput
• Claude OAuth uses createAnthropic({ authToken }) natively — no more
custom-fetch x-api-key → Bearer swap.
• rewriteBodyForOpus47 deleted — Opus 4.7 adaptive thinking is native
via providerOptions.anthropic.thinking = { type: 'adaptive' }.
• V1 middleware → V3 (specificationVersion: 'v3').
• v4-era normalizeMessages openai-compatible middleware deleted; the
v6 openai-compatible provider extracts reasoning_content natively
from { type: 'reasoning' } content parts.
• applyAnthropicStructuralNormalisations (mirrors opencode
provider/transform.ts:53-148): drops empty text/reasoning parts,
scrubs non-[a-zA-Z0-9_-] toolCallIds, splits [tool-call, non-tool]
assistant turns (Anthropic rejects tool_use followed by text).
• applyOpenAICompatibleReasoningNormalisation (mirrors opencode
transform.ts:217-249): lifts reasoning text into
providerOptions.openaiCompatible.reasoning_content (always, even
empty). Solves DeepSeek 'The reasoning_content in the thinking
mode must be passed back' — the v6 SDK skips emitting
reasoning_content when text is empty (dist/index.mjs:245), but
DeepSeek requires the field present once thinking was used.
• Tools: tool({ inputSchema: jsonSchema(zodToJsonSchema(...)) })
(was parameters: ZodSchema). AI SDK tools have no execute
callback — the agent runs tools manually for permission prompts
and shell-output streaming. New dep: zod-to-json-schema@^3.25.2.
• fullStream event loop rewritten for v6 event shape: text-delta
(text not textDelta), reasoning-start/delta/end, tool-input-*,
tool-call (input not args), tool-result, tool-error (new), abort
(new), start-step/finish-step, finish.
Max-thinking audit (matches opencode transform.ts:642-671 budgets):
• Claude enabled-thinking max budget 16000 → 31999 (Anthropic ceiling)
• Claude enabled-thinking high budget 10000 → 16000
• maxOutputTokens 'budget + 8000' → fixed 32000 (matches opencode's
OUTPUT_TOKEN_MAX; model self-allocates thinking vs response within)
• Opus 4.7 adaptive thinking gains display: 'summarized' and sibling
effort field (without these, thinking content is hidden by Anthropic
and the model barely thinks).
Frontend mirrors:
• types.ts — ThinkingChunk.metadata?, AgentEvent reasoning-end
• tabs.svelte.ts — routes reasoning-end through applyChunkEvent
• ChatMessage.svelte — hides empty thinking chunks; hides the entire
assistant bubble when no chunk has renderable content
Gemini-review-driven fixes:
• tool-error and abort stream events now surface as error chunks
(were silently ignored)
• toolCallId scrubbing pass (opencode transform.ts:96-122 parity)
• Empty-reasoning-cull explicit test coverage for both Anthropic
structural normalisation and DeepSeek path
Test counts (223 tests across 3 packages, all green):
• tests/chunks/append.test.ts: 44 (was 38) — reasoning-end sealing,
orphan walk-back, multi-block interleaving
• tests/agent/agent.test.ts: 24 (was 5) — exhaustive v6 event
mappings, structural normalisations, signature/reasoning_content
round-trip, tool-error/abort branches, DeepSeek scenario, empty
reasoning edge case
• tests/llm/provider.test.ts: 9 (was 22) — dropped 13 obsolete v4
middleware tests; new minimal tests confirm no middleware wrapping
on default openai-compat path and that createAnthropic gets
authToken vs apiKey correctly for OAuth vs api-key flows
• tests/tools/registry.test.ts: 10 (was 4) — v6 tool() contract
(inputSchema, no execute, JSON Schema for nested zod)
• packages/api/tests/agent-manager.test.ts: 12 (was 7) — mock Agent
emits v6 reasoning events; reasoning-end broadcast + ordering
• packages/frontend/tests/chat-store.test.ts: 35 (was 32) —
reasoning-end flow through Svelte $state store
typecheck clean (tsc --noEmit on core + api, svelte-check on frontend),
biome clean across 124 files.
Diffstat (limited to 'packages/core/tests')
| -rw-r--r-- | packages/core/tests/agent/agent.test.ts | 841 | ||||
| -rw-r--r-- | packages/core/tests/chunks/append.test.ts | 97 | ||||
| -rw-r--r-- | packages/core/tests/llm/provider.test.ts | 392 | ||||
| -rw-r--r-- | packages/core/tests/tools/registry.test.ts | 109 |
4 files changed, 1115 insertions, 324 deletions
diff --git a/packages/core/tests/agent/agent.test.ts b/packages/core/tests/agent/agent.test.ts index 3d11fc5..82ca830 100644 --- a/packages/core/tests/agent/agent.test.ts +++ b/packages/core/tests/agent/agent.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it, vi } from "vitest"; import { z } from "zod"; -import type { AgentConfig } from "../../src/types/index.js"; +import type { AgentConfig, AgentEvent } from "../../src/types/index.js"; // Mock bun:sqlite to avoid Bun-only import in vitest/Node vi.mock("../../src/db/index.js", () => ({ @@ -31,6 +31,7 @@ vi.mock("@ai-sdk/openai-compatible", () => ({ })); const { Agent } = await import("../../src/agent/agent.js"); +const { streamText } = await import("ai"); function makeConfig(overrides: Partial<AgentConfig> = {}): AgentConfig { return { @@ -58,6 +59,21 @@ function makeMockStreamResult(events: Array<{ type: string; [key: string]: unkno } as ReturnType<typeof import("ai").streamText>; } +// v6 finish event — only finishReason, rawFinishReason, totalUsage (no usage/providerMetadata/response) +const finishStop = { + type: "finish", + finishReason: "stop", + rawFinishReason: "stop", + totalUsage: { inputTokens: 10, outputTokens: 5 }, +}; + +const finishToolCalls = { + type: "finish", + finishReason: "tool-calls", + rawFinishReason: "tool_use", + totalUsage: { inputTokens: 10, outputTokens: 5 }, +}; + describe("Agent", () => { it("starts in idle status", () => { const agent = new Agent(makeConfig()); @@ -70,17 +86,11 @@ describe("Agent", () => { }); it("yields running then idle status events around a simple message", async () => { - const { streamText } = await import("ai"); vi.mocked(streamText).mockReturnValue( makeMockStreamResult([ - { type: "text-delta", textDelta: "Hello!" }, - { - type: "finish", - finishReason: "stop", - usage: {}, - providerMetadata: undefined, - response: {}, - }, + // v6: text-delta uses `text` (not `textDelta`) + { type: "text-delta", id: "t0", text: "Hello!" }, + finishStop, ]), ); @@ -99,18 +109,12 @@ describe("Agent", () => { }); it("yields text-delta events", async () => { - const { streamText } = await import("ai"); vi.mocked(streamText).mockReturnValue( makeMockStreamResult([ - { type: "text-delta", textDelta: "Hello" }, - { type: "text-delta", textDelta: " world" }, - { - type: "finish", - finishReason: "stop", - usage: {}, - providerMetadata: undefined, - response: {}, - }, + // v6: text-delta uses `text` (not `textDelta`) + { type: "text-delta", id: "t0", text: "Hello" }, + { type: "text-delta", id: "t0", text: " world" }, + finishStop, ]), ); @@ -127,18 +131,8 @@ describe("Agent", () => { }); it("adds user message and assistant message to history", async () => { - const { streamText } = await import("ai"); vi.mocked(streamText).mockReturnValue( - makeMockStreamResult([ - { type: "text-delta", textDelta: "Response" }, - { - type: "finish", - finishReason: "stop", - usage: {}, - providerMetadata: undefined, - response: {}, - }, - ]), + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "Response" }, finishStop]), ); const agent = new Agent(makeConfig()); @@ -158,18 +152,8 @@ describe("Agent", () => { }); it("yields done event with final message", async () => { - const { streamText } = await import("ai"); vi.mocked(streamText).mockReturnValue( - makeMockStreamResult([ - { type: "text-delta", textDelta: "Done!" }, - { - type: "finish", - finishReason: "stop", - usage: {}, - providerMetadata: undefined, - response: {}, - }, - ]), + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "Done!" }, finishStop]), ); const agent = new Agent(makeConfig()); @@ -187,8 +171,6 @@ describe("Agent", () => { }); it("yields tool-call and tool-result events", async () => { - const { streamText } = await import("ai"); - // First call: LLM emits a tool-call // Second call (after tool execution): LLM emits text response with no tool calls vi.mocked(streamText) @@ -198,27 +180,16 @@ describe("Agent", () => { type: "tool-call", toolCallId: "tc1", toolName: "read_file", - args: { path: "hello.txt" }, - }, - { - type: "finish", - finishReason: "tool-calls", - usage: {}, - providerMetadata: undefined, - response: {}, + // v6: `input` replaces `args` + input: { path: "hello.txt" }, }, + finishToolCalls, ]), ) .mockReturnValueOnce( makeMockStreamResult([ - { type: "text-delta", textDelta: "Here is the file." }, - { - type: "finish", - finishReason: "stop", - usage: {}, - providerMetadata: undefined, - response: {}, - }, + { type: "text-delta", id: "t0", text: "Here is the file." }, + finishStop, ]), ); @@ -247,4 +218,754 @@ describe("Agent", () => { toolResult: { toolCallId: "tc1", result: "file contents" }, }); }); + + it("yields reasoning-delta events", async () => { + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([ + // v6: reasoning-delta uses `text` (not `textDelta`) + { type: "reasoning-delta", id: "r0", text: "thinking about this..." }, + { type: "reasoning-delta", id: "r0", text: " more thoughts" }, + { type: "text-delta", id: "t0", text: "Answer" }, + finishStop, + ]), + ); + + const agent = new Agent(makeConfig()); + const events = []; + for await (const event of agent.run("think")) { + events.push(event); + } + + const reasoningDeltas = events.filter((e) => e.type === "reasoning-delta"); + expect(reasoningDeltas).toHaveLength(2); + expect(reasoningDeltas[0]).toMatchObject({ delta: "thinking about this..." }); + expect(reasoningDeltas[1]).toMatchObject({ delta: " more thoughts" }); + }); + + it("yields reasoning-end event when providerMetadata is present", async () => { + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([ + { type: "reasoning-delta", id: "r0", text: "some reasoning" }, + { + type: "reasoning-end", + id: "r0", + providerMetadata: { anthropic: { signature: "sig-1" } }, + }, + { type: "text-delta", id: "t0", text: "Answer" }, + finishStop, + ]), + ); + + const agent = new Agent(makeConfig()); + const events = []; + for await (const event of agent.run("think")) { + events.push(event); + } + + const reasoningEndEvent = events.find((e) => e.type === "reasoning-end"); + expect(reasoningEndEvent).toBeDefined(); + expect(reasoningEndEvent).toMatchObject({ + type: "reasoning-end", + metadata: { anthropic: { signature: "sig-1" } }, + }); + }); + + it("does NOT yield reasoning-end event when providerMetadata is absent", async () => { + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([ + { type: "reasoning-delta", id: "r0", text: "some reasoning" }, + { + type: "reasoning-end", + id: "r0", + // No providerMetadata — non-Anthropic model + }, + { type: "text-delta", id: "t0", text: "Answer" }, + finishStop, + ]), + ); + + const agent = new Agent(makeConfig()); + const events = []; + for await (const event of agent.run("think")) { + events.push(event); + } + + const reasoningEndEvent = events.find((e) => e.type === "reasoning-end"); + expect(reasoningEndEvent).toBeUndefined(); + }); + + // ─── New v6 round-trip tests ────────────────────────────────────────────── + + it("signed thinking round-trip: ThinkingChunk.metadata → ReasoningPart.providerOptions", async () => { + // Pre-seed the agent with a prior assistant message containing a ThinkingChunk + // with metadata (the Anthropic signature blob). + // Anthropic-path provider — for openai-compatible the metadata + // would be lifted into providerOptions.openaiCompatible instead; + // that path is covered by the DeepSeek tests further down. + const agent = new Agent(makeConfig({ provider: "opencode-anthropic" })); + agent.messages.push({ + role: "user", + chunks: [{ type: "text", text: "prior user message" }], + }); + agent.messages.push({ + role: "assistant", + chunks: [ + { + type: "thinking", + text: "I thought about it", + metadata: { anthropic: { signature: "S" } }, + }, + { type: "text", text: "prior response" }, + ], + }); + + // Next turn: just return a simple text response + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "New answer" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + // consume + } + + // Inspect the messages passed to streamText in this (last) call + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + expect(callArgs).toBeDefined(); + const messages = callArgs?.messages as Array<{ + role: string; + content: unknown; + }>; + + // Find the assistant message in the rebuilt ModelMessage[] + const assistantMsg = messages.find((m) => m.role === "assistant"); + expect(assistantMsg).toBeDefined(); + const content = assistantMsg?.content as Array<Record<string, unknown>>; + const reasoningPart = content.find((p) => p.type === "reasoning"); + expect(reasoningPart).toBeDefined(); + expect(reasoningPart).toMatchObject({ + type: "reasoning", + text: "I thought about it", + providerOptions: { anthropic: { signature: "S" } }, + }); + }); + + it("tool-call input round-trip: ToolBatchEntry.arguments → ToolCallPart.input (not args)", async () => { + // Pre-seed the agent with a prior assistant message containing a tool-batch chunk + const agent = new Agent(makeConfig()); + agent.messages.push({ + role: "user", + chunks: [{ type: "text", text: "run a tool" }], + }); + agent.messages.push({ + role: "assistant", + chunks: [ + { + type: "tool-batch", + calls: [ + { + id: "call-1", + name: "read_file", + arguments: { path: "/foo/bar.txt" }, + result: "file contents", + }, + ], + }, + ], + }); + + // Next turn: just return a simple text response + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "Done" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + // consume + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + + // The assistant message should contain a tool-call part with `input` (not `args`) + const assistantMsg = messages.find((m) => m.role === "assistant"); + expect(assistantMsg).toBeDefined(); + const content = assistantMsg?.content as Array<Record<string, unknown>>; + const toolCallPart = content.find((p) => p.type === "tool-call"); + expect(toolCallPart).toBeDefined(); + expect(toolCallPart).toMatchObject({ + type: "tool-call", + toolCallId: "call-1", + toolName: "read_file", + input: { path: "/foo/bar.txt" }, // v6: input not args + }); + // Explicitly assert `args` is NOT present + expect(toolCallPart).not.toHaveProperty("args"); + }); + + it("tool-result output round-trip: result string → { type: 'text', value } ToolResultOutput", async () => { + // Pre-seed the agent with a prior assistant message containing a tool-batch chunk + const agent = new Agent(makeConfig()); + agent.messages.push({ + role: "user", + chunks: [{ type: "text", text: "run a tool" }], + }); + agent.messages.push({ + role: "assistant", + chunks: [ + { + type: "tool-batch", + calls: [ + { + id: "call-2", + name: "read_file", + arguments: { path: "/foo/baz.txt" }, + result: "the file content here", + }, + ], + }, + ], + }); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "Done" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + // consume + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + + // The tool message should contain a tool-result part with `output` (ToolResultOutput) + const toolMsg = messages.find((m) => m.role === "tool"); + expect(toolMsg).toBeDefined(); + const toolContent = toolMsg?.content as Array<Record<string, unknown>>; + expect(toolContent[0]).toMatchObject({ + type: "tool-result", + toolCallId: "call-2", + toolName: "read_file", + output: { type: "text", value: "the file content here" }, + }); + // Explicitly assert `result` (v4 raw string) is NOT present + expect(toolContent[0]).not.toHaveProperty("result"); + }); + + it("Anthropic [tool-call, text] split: mixed-order assistant message gets split into [text]+[tool-call]", async () => { + // Pre-seed an assistant message with chunks in [tool-batch, text] order — + // which produces [tool-call, text] in the ModelMessage content, a shape + // Anthropic rejects. Only applies for anthropic / opencode-anthropic provider. + const agent = new Agent(makeConfig({ provider: "opencode-anthropic" })); + agent.messages.push({ + role: "user", + chunks: [{ type: "text", text: "run a tool and explain" }], + }); + agent.messages.push({ + role: "assistant", + chunks: [ + // Note: tool-batch appears BEFORE text in chunks — this is the + // problematic ordering that Anthropic rejects + { + type: "tool-batch", + calls: [ + { + id: "call-3", + name: "read_file", + arguments: { path: "/tmp/x.txt" }, + result: "x contents", + }, + ], + }, + { type: "text", text: "Here is my explanation." }, + ], + }); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + // consume + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + + // After Anthropic structural normalisation, we should have TWO assistant messages: + // 1st: text-only content + // 2nd: tool-call-only content + const assistantMsgs = messages.filter((m) => m.role === "assistant"); + expect(assistantMsgs.length).toBeGreaterThanOrEqual(2); + + // Find the text-only assistant message and the tool-call-only assistant message + const textOnlyMsg = assistantMsgs.find((m) => { + const c = m.content as Array<Record<string, unknown>>; + return Array.isArray(c) && c.every((p) => p.type !== "tool-call"); + }); + const toolOnlyMsg = assistantMsgs.find((m) => { + const c = m.content as Array<Record<string, unknown>>; + return Array.isArray(c) && c.every((p) => p.type === "tool-call"); + }); + + // Narrow the optionals — toBeDefined() already verified non-null, + // but TypeScript needs the explicit assertion via local consts so + // we can pass them to indexOf without `!`. + if (!textOnlyMsg || !toolOnlyMsg) throw new Error("type guard"); + + // Text message comes first (before tool-call message) — Anthropic requires this ordering + expect(messages.indexOf(textOnlyMsg)).toBeLessThan(messages.indexOf(toolOnlyMsg)); + }); + + it("Anthropic [tool-call, text] split: openai-compatible provider preserves original order (no split)", async () => { + // For non-Anthropic providers, the [tool-call, text] split should NOT be applied. + // (No provider set → defaults to openai-compatible) + const agent = new Agent(makeConfig()); + agent.messages.push({ + role: "user", + chunks: [{ type: "text", text: "run a tool and explain" }], + }); + agent.messages.push({ + role: "assistant", + chunks: [ + { + type: "tool-batch", + calls: [ + { + id: "call-4", + name: "read_file", + arguments: { path: "/tmp/y.txt" }, + result: "y contents", + }, + ], + }, + { type: "text", text: "Here is my explanation." }, + ], + }); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + // consume + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + + // For openai-compatible provider, only ONE assistant message with mixed content + const assistantMsgs = messages.filter((m) => m.role === "assistant"); + expect(assistantMsgs).toHaveLength(1); + const content = assistantMsgs[0]?.content as Array<Record<string, unknown>>; + // Both tool-call and text parts should be in the same message + expect(content.some((p) => p.type === "tool-call")).toBe(true); + expect(content.some((p) => p.type === "text")).toBe(true); + }); + + it("empty-text-part filter (Anthropic): empty text chunk is not sent", async () => { + // Pre-seed an assistant message where a text chunk has empty text. + const agent = new Agent(makeConfig({ provider: "opencode-anthropic" })); + agent.messages.push({ + role: "user", + chunks: [{ type: "text", text: "hello" }], + }); + agent.messages.push({ + role: "assistant", + chunks: [ + { type: "text", text: "" }, // empty text — should be filtered out + { type: "text", text: "non-empty response" }, + ], + }); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + // consume + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + + const assistantMsg = messages.find((m) => m.role === "assistant"); + expect(assistantMsg).toBeDefined(); + const content = assistantMsg?.content as Array<Record<string, unknown>>; + + // Empty text part should have been filtered out + const emptyTextParts = content.filter((p) => p.type === "text" && p.text === ""); + expect(emptyTextParts).toHaveLength(0); + + // The non-empty text part should still be there + const nonEmptyTextParts = content.filter((p) => p.type === "text" && p.text !== ""); + expect(nonEmptyTextParts).toHaveLength(1); + expect(nonEmptyTextParts[0]).toMatchObject({ text: "non-empty response" }); + }); + + it("empty-reasoning-part filter (Anthropic): empty reasoning chunk is not sent", async () => { + // Anthropic's adaptive thinking mode occasionally produces a signed- + // but-empty thinking block. We persist it (for signature round-trip + // fidelity) but strip the empty `reasoning` part before sending it + // back, or Anthropic rejects with "thinking block must have content". + const agent = new Agent(makeConfig({ provider: "opencode-anthropic" })); + agent.messages.push({ role: "user", chunks: [{ type: "text", text: "hi" }] }); + agent.messages.push({ + role: "assistant", + chunks: [ + // Signed-but-empty thinking block + { type: "thinking", text: "", metadata: { anthropic: { signature: "sig-empty" } } }, + { type: "text", text: "answer" }, + ], + }); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + /* consume */ + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + const assistantMsg = messages.find((m) => m.role === "assistant"); + const content = assistantMsg?.content as Array<Record<string, unknown>>; + + // Empty reasoning part must have been filtered out by the + // Anthropic structural normalisation pass. + const emptyReasoning = content.filter((p) => p.type === "reasoning" && p.text === ""); + expect(emptyReasoning).toHaveLength(0); + + // The text part should still be there + expect(content.some((p) => p.type === "text" && p.text === "answer")).toBe(true); + }); + + it("toolCallId scrubbing (Anthropic): non-[a-zA-Z0-9_-] chars in tool IDs are sanitised", async () => { + // Anthropic rejects toolCallId outside [a-zA-Z0-9_-]. Our internal + // crypto.randomUUID IDs are safe, but defensively scrub for any + // upstream-assigned IDs (subagent retrieval, provider-executed + // tools, MCP, etc.). Mirrors opencode transform.ts:96-122. + const agent = new Agent(makeConfig({ provider: "opencode-anthropic" })); + agent.messages.push({ role: "user", chunks: [{ type: "text", text: "do the thing" }] }); + agent.messages.push({ + role: "assistant", + chunks: [ + { + type: "tool-batch", + calls: [ + { + id: "call.with/dots:and:slashes", // invalid chars + name: "fake_tool", + arguments: { x: 1 }, + result: "ok", + isError: false, + }, + ], + }, + ], + }); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + /* consume */ + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + + // Assistant tool-call part must have scrubbed ID + const assistantMsg = messages.find((m) => m.role === "assistant"); + const assistantContent = assistantMsg?.content as Array<Record<string, unknown>>; + const toolCallPart = assistantContent.find((p) => p.type === "tool-call"); + expect(toolCallPart).toBeDefined(); + expect(toolCallPart?.toolCallId).toBe("call_with_dots_and_slashes"); + + // Matching tool-result message must use the SAME scrubbed ID + // so Anthropic can pair them. + const toolMsg = messages.find((m) => m.role === "tool"); + const toolContent = toolMsg?.content as Array<Record<string, unknown>>; + expect(toolContent?.[0]?.toolCallId).toBe("call_with_dots_and_slashes"); + }); + + it("reasoning metadata captured from stream is round-tripped on the next turn", async () => { + // End-to-end integrity for the providerMetadata round-trip — the + // bug that prompted the entire migration. Stream a turn that + // emits reasoning-delta + reasoning-end with metadata, then run + // ANOTHER turn and verify the metadata reaches the model via + // ReasoningPart.providerOptions. + const agent = new Agent(makeConfig({ provider: "opencode-anthropic" })); + const sig = { anthropic: { signature: "round-trip-sig-1" } }; + + // Turn 1: model emits reasoning + signed reasoning-end + vi.mocked(streamText).mockReturnValueOnce( + makeMockStreamResult([ + { type: "reasoning-delta", id: "r0", text: "let me think" }, + { type: "reasoning-end", id: "r0", providerMetadata: sig }, + { type: "text-delta", id: "t0", text: "answer" }, + finishStop, + ]), + ); + + for await (const _ of agent.run("first question")) { + /* consume */ + } + + // After turn 1, the persisted chunks should include a ThinkingChunk + // with the captured metadata. The agent's messages array IS the + // canonical persisted shape (the DB just JSON-stringifies it). + const turn1Assistant = agent.messages.find( + (m, i) => m.role === "assistant" && i === agent.messages.length - 1, + ); + expect(turn1Assistant).toBeDefined(); + const thinkingChunk = turn1Assistant?.chunks.find((c) => c.type === "thinking"); + expect(thinkingChunk).toBeDefined(); + expect(thinkingChunk).toMatchObject({ text: "let me think", metadata: sig }); + + // Turn 2: drive another turn, capture what streamText receives. + vi.mocked(streamText).mockReturnValueOnce( + makeMockStreamResult([ + { type: "text-delta", id: "t1", text: "follow-up answer" }, + finishStop, + ]), + ); + for await (const _ of agent.run("second question")) { + /* consume */ + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ role: string; content: unknown }>; + const turn2Assistant = messages + .filter((m) => m.role === "assistant") + .find((m) => Array.isArray(m.content)); + const turn2Content = turn2Assistant?.content as Array<Record<string, unknown>>; + const reasoningPart = turn2Content.find((p) => p.type === "reasoning"); + expect(reasoningPart).toBeDefined(); + expect(reasoningPart).toMatchObject({ + type: "reasoning", + text: "let me think", + providerOptions: sig, + }); + }); + + it("tool-error stream event yields a synthetic tool-result + error chunk and aborts the turn", async () => { + // Provider-executed tools (Anthropic server tools) bypass our + // manual executor and surface as a `tool-error` stream event. + // We must: + // 1. Synthesize a tool-result with isError=true so the chunks + // reflect that the tool ran and failed. + // 2. Emit an error chunk so the UI shows the failure. + // 3. Transition the agent to "error" status (no further steps). + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([ + { + type: "tool-error", + toolCallId: "tc_server", + toolName: "server_tool", + error: new Error("upstream tool failure"), + }, + finishStop, + ]), + ); + + const agent = new Agent(makeConfig()); + const events: AgentEvent[] = []; + for await (const event of agent.run("trigger")) { + events.push(event); + } + + // Synthetic tool-result with the upstream error + const trEvent = events.find((e) => e.type === "tool-result"); + expect(trEvent).toBeDefined(); + expect(trEvent).toMatchObject({ + type: "tool-result", + toolResult: { toolCallId: "tc_server", isError: true }, + }); + + // Error chunk + const errEvent = events.find((e) => e.type === "error"); + expect(errEvent).toBeDefined(); + const errMsg = errEvent && "error" in errEvent ? errEvent.error : ""; + expect(typeof errMsg).toBe("string"); + expect((errMsg as string).includes("upstream tool failure")).toBe(true); + + // Status transitions to error + const errStatusEvent = events.filter((e) => e.type === "status").at(-1); + expect(errStatusEvent).toMatchObject({ type: "status", status: "error" }); + }); + + it("abort stream event surfaces as an error event and stops the turn", async () => { + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([ + { type: "text-delta", id: "t0", text: "starting..." }, + { type: "abort", reason: "user cancelled" }, + ]), + ); + + const agent = new Agent(makeConfig()); + const events: AgentEvent[] = []; + for await (const event of agent.run("hi")) { + events.push(event); + } + + const errEvent = events.find((e) => e.type === "error"); + expect(errEvent).toBeDefined(); + const errMsg = errEvent && "error" in errEvent ? errEvent.error : ""; + expect( + (errMsg as string).toLowerCase().includes("aborted") || + (errMsg as string).includes("user cancelled"), + ).toBe(true); + + const lastStatus = events.filter((e) => e.type === "status").at(-1); + expect(lastStatus).toMatchObject({ type: "status", status: "error" }); + }); + + it("openai-compatible reasoning round-trip: ThinkingChunk -> providerOptions.openaiCompatible.reasoning_content (DeepSeek scenario)", async () => { + // Reproducer for the "reasoning_content must be passed back" error + // from DeepSeek via OpenCode Go. + // + // applyOpenAICompatibleReasoningNormalisation strips the + // `{ type: "reasoning", text }` parts and lifts the concatenated + // text into `providerOptions.openaiCompatible.reasoning_content`. + // The v6 SDK provider serializes the message-level + // `providerOptions.openaiCompatible.*` into the wire `assistant` + // message via its `metadata` spread (line 247 of the SDK dist). + // This route emits `reasoning_content` regardless of empty/non- + // empty text — which is what DeepSeek requires. + const agent = new Agent( + makeConfig({ + model: "deepseek-v4-pro", + // no provider field → default openai-compatible path + }), + ); + agent.messages.push( + { role: "user", chunks: [{ type: "text", text: "ping" }] }, + { + role: "assistant", + chunks: [ + { type: "thinking", text: "let me reason about this" }, + { type: "text", text: "ok done" }, + ], + }, + ); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + /* consume */ + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ + role: string; + content: unknown; + providerOptions?: { openaiCompatible?: { reasoning_content?: string } }; + }>; + const assistantMsg = messages.find((m) => m.role === "assistant"); + if (!assistantMsg || !Array.isArray(assistantMsg.content)) { + throw new Error("expected structured assistant content"); + } + const content = assistantMsg.content as Array<Record<string, unknown>>; + + // Reasoning parts have been stripped from content (lifted into + // providerOptions instead). + expect(content.find((p) => p.type === "reasoning")).toBeUndefined(); + + // reasoning_content is set on providerOptions.openaiCompatible. + // This is what reaches DeepSeek and prevents the rejection. + expect(assistantMsg.providerOptions?.openaiCompatible?.reasoning_content).toBe( + "let me reason about this", + ); + + // The text part still survives in content. + const textPart = content.find((p) => p.type === "text"); + expect(textPart).toMatchObject({ type: "text", text: "ok done" }); + + // And critically, the message must NOT carry a providerMetadata + // key (the v4-era misnamed key). v3 prompts use `providerOptions`. + expect((assistantMsg as Record<string, unknown>).providerMetadata).toBeUndefined(); + }); + + it("openai-compatible empty-reasoning edge case: forces reasoning_content='' so DeepSeek does not reject", async () => { + // DeepSeek will reject the follow-up turn with "must be passed + // back" if a prior assistant turn emitted reasoning AND the + // follow-up doesn't include `reasoning_content` (even empty). + // The v6 SDK's content-side path skips emission when reasoning + // is empty (see `dist/index.mjs:245`); our normalisation routes + // it via providerOptions instead, which fires unconditionally. + const agent = new Agent(makeConfig({ model: "deepseek-v4-pro" })); + agent.messages.push( + { role: "user", chunks: [{ type: "text", text: "ping" }] }, + { + role: "assistant", + chunks: [ + // Empty thinking — captured but produced no actual text. + { type: "thinking", text: "" }, + { type: "text", text: "answer" }, + ], + }, + ); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("follow-up")) { + /* consume */ + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ + role: string; + content: unknown; + providerOptions?: { openaiCompatible?: { reasoning_content?: string } }; + }>; + const assistantMsg = messages.find((m) => m.role === "assistant"); + if (!assistantMsg) throw new Error("expected assistant message"); + + // The empty-string reasoning_content is explicitly set on + // providerOptions. (`""` is intentional and required — + // `assistantMsg.providerOptions?.openaiCompatible?.reasoning_content` + // must not be `undefined`.) + const rc = assistantMsg.providerOptions?.openaiCompatible?.reasoning_content; + expect(rc).toBeDefined(); + expect(rc).toBe(""); + }); + + it("openai-compatible normalisation does NOT run for messages without any reasoning parts", async () => { + // DeepSeek only requires `reasoning_content` AFTER a thinking + // turn. For purely-text assistant messages, we should leave + // providerOptions alone. + const agent = new Agent(makeConfig({ model: "deepseek-v4-pro" })); + agent.messages.push( + { role: "user", chunks: [{ type: "text", text: "hi" }] }, + { + role: "assistant", + chunks: [{ type: "text", text: "hello back" }], + }, + ); + + vi.mocked(streamText).mockReturnValue( + makeMockStreamResult([{ type: "text-delta", id: "t0", text: "ok" }, finishStop]), + ); + + for await (const _ of agent.run("again")) { + /* consume */ + } + + const callArgs = vi.mocked(streamText).mock.calls.at(-1)?.[0]; + const messages = callArgs?.messages as Array<{ + role: string; + providerOptions?: { openaiCompatible?: { reasoning_content?: string } }; + }>; + const assistantMsg = messages.find((m) => m.role === "assistant"); + + // No reasoning chunks → no providerOptions injection. (May still + // be undefined entirely if nothing else set it.) + const rc = assistantMsg?.providerOptions?.openaiCompatible?.reasoning_content; + expect(rc).toBeUndefined(); + }); }); diff --git a/packages/core/tests/chunks/append.test.ts b/packages/core/tests/chunks/append.test.ts index c8917c9..dc277d2 100644 --- a/packages/core/tests/chunks/append.test.ts +++ b/packages/core/tests/chunks/append.test.ts @@ -6,6 +6,10 @@ import type { AgentEvent, ChatMessage, Chunk } from "../../src/types/index.js"; const td = (delta: string): AgentEvent => ({ type: "text-delta", delta }); const rd = (delta: string): AgentEvent => ({ type: "reasoning-delta", delta }); +const re = (metadata?: Record<string, unknown>): AgentEvent => ({ + type: "reasoning-end", + ...(metadata !== undefined ? { metadata } : {}), +}); const tc = (id: string, name = "fake_tool", args: Record<string, unknown> = {}): AgentEvent => ({ type: "tool-call", toolCall: { id, name, arguments: args }, @@ -295,6 +299,99 @@ describe("appendEventToChunks — transition matrix", () => { ]); }); + // ─── reasoning-end (v6 SDK metadata round-trip) ────────────────── + + it("reasoning-delta then reasoning-end seals the thinking chunk with metadata", () => { + const meta = { anthropic: { signature: "sig-1" } }; + const chunks = run([rd("plan"), re(meta)]); + expect(chunks).toEqual([{ type: "thinking", text: "plan", metadata: meta }]); + }); + + it("two reasoning-deltas then reasoning-end coalesces text and seals once", () => { + const meta = { anthropic: { signature: "abc" } }; + const chunks = run([rd("a"), rd("b"), re(meta)]); + expect(chunks).toEqual([{ type: "thinking", text: "ab", metadata: meta }]); + }); + + it("reasoning-delta → reasoning-end → reasoning-delta opens a NEW chunk", () => { + // Each Anthropic thinking content block gets its own metadata. + // Extending a sealed chunk would corrupt the text/metadata mapping. + const meta1 = { anthropic: { signature: "sig-1" } }; + const chunks = run([rd("first"), re(meta1), rd("second")]); + expect(chunks).toEqual([ + { type: "thinking", text: "first", metadata: meta1 }, + { type: "thinking", text: "second" }, + ]); + }); + + it("rd → re → rd → re produces two independently sealed chunks", () => { + const m1 = { anthropic: { signature: "s1" } }; + const m2 = { anthropic: { signature: "s2" } }; + const chunks = run([rd("first"), re(m1), rd("second"), re(m2)]); + expect(chunks).toEqual([ + { type: "thinking", text: "first", metadata: m1 }, + { type: "thinking", text: "second", metadata: m2 }, + ]); + }); + + it("orphan reasoning-end (no prior thinking chunk) is a no-op", () => { + const chunks = run([re({ anthropic: { signature: "orphan" } })]); + expect(chunks).toEqual([]); + }); + + it("reasoning-end after an already-sealed thinking chunk does NOT overwrite", () => { + const m1 = { anthropic: { signature: "first" } }; + const m2 = { anthropic: { signature: "second" } }; + const chunks = run([rd("a"), re(m1), re(m2)]); + expect(chunks).toEqual([{ type: "thinking", text: "a", metadata: m1 }]); + }); + + it("reasoning-end without metadata is a silent no-op (does not seal)", () => { + // v6 may emit reasoning-end with no providerMetadata for + // non-Anthropic providers. Don't seal those chunks — a subsequent + // reasoning-delta should continue extending. + const chunks = run([rd("hello"), re(), rd(" world")]); + expect(chunks).toEqual([{ type: "thinking", text: "hello world" }]); + }); + + it("re walks back across an intervening text chunk to seal the right thinking", () => { + // Defensive: even if a non-thinking chunk lands between the + // reasoning text and its end-event, the metadata still attaches + // to the unsealed thinking chunk. + const meta = { anthropic: { signature: "late" } }; + const chunks = run([rd("plan"), td("midstream"), re(meta)]); + expect(chunks).toEqual([ + { type: "thinking", text: "plan", metadata: meta }, + { type: "text", text: "midstream" }, + ]); + }); + + it("interleaved rd / re / tool-call / rd / re produces correct chunk sequence", () => { + // Anthropic's interleaved-thinking emits a thinking block, then + // a tool call, then another thinking block. Each thinking block + // gets its own metadata. + const m1 = { anthropic: { signature: "before-tool" } }; + const m2 = { anthropic: { signature: "after-tool" } }; + const chunks = run([ + rd("plan tool"), + re(m1), + tc("t1", "read_file"), + rd("plan response"), + re(m2), + ]); + expect(chunks.map((c) => c.type)).toEqual(["thinking", "tool-batch", "thinking"]); + expect(chunks[0]).toEqual({ + type: "thinking", + text: "plan tool", + metadata: m1, + }); + expect(chunks[2]).toEqual({ + type: "thinking", + text: "plan response", + metadata: m2, + }); + }); + it("non-content events (status / done / task-list-update / message-queued etc.) are no-ops", () => { const chunks = run([ td("hello"), diff --git a/packages/core/tests/llm/provider.test.ts b/packages/core/tests/llm/provider.test.ts index fb5dca5..6171e6b 100644 --- a/packages/core/tests/llm/provider.test.ts +++ b/packages/core/tests/llm/provider.test.ts @@ -1,298 +1,178 @@ import { describe, expect, it, vi } from "vitest"; -// We test normalizeMessages through the middleware by mocking the provider -// layers and capturing what transformParams does to the prompt. - -// Mock wrapLanguageModel to capture the middleware -vi.mock("ai", async () => { - const actual = await import("ai"); - return { - ...actual, - wrapLanguageModel: vi.fn(({ model, middleware }) => { - // Return a wrapper that exposes the middleware for testing - const wrapped = actual.wrapLanguageModel({ model, middleware }); - (wrapped as unknown as Record<string, unknown>)._middleware = middleware; - return wrapped; - }), - streamText: vi.fn(), - }; -}); +// Mock @ai-sdk/anthropic to capture what options createAnthropic is called with +const mockAnthropicInstance = vi.fn((modelId: string) => ({ + specificationVersion: "v3" as const, + provider: "anthropic.messages", + modelId, + supportedUrls: {}, + doGenerate: vi.fn(), + doStream: vi.fn(), +})); +const mockCreateAnthropic = vi.fn(() => mockAnthropicInstance); +vi.mock("@ai-sdk/anthropic", () => ({ + createAnthropic: mockCreateAnthropic, +})); -// Mock provider factory +// Mock @ai-sdk/openai-compatible to capture what options createOpenAICompatible +// is called with, and what model id the returned factory is called with. +const mockOpenAICompatibleFactory = vi.fn((modelId: string) => ({ + specificationVersion: "v3" as const, + provider: "openai-compatible", + modelId, + supportedUrls: {}, + doGenerate: vi.fn(), + doStream: vi.fn(), +})); +const mockCreateOpenAICompatible = vi.fn(() => mockOpenAICompatibleFactory); vi.mock("@ai-sdk/openai-compatible", () => ({ - createOpenAICompatible: vi.fn(() => (modelId: string) => ({ - id: `mock-${modelId}`, - doGenerate: vi.fn(), - doStream: vi.fn(), - })), + createOpenAICompatible: mockCreateOpenAICompatible, })); const { createProvider } = await import("../../src/llm/provider.js"); -// A helper that runs the middleware's transformParams on a prompt -// and returns the resulting normalized prompt. -async function runTransform(prompt: unknown[]): Promise<unknown[]> { - const wrappedModel = createProvider({ - apiKey: "test-key", - baseURL: "https://example.com/v1", - })("test-model"); - - const middleware = ( - wrappedModel as unknown as { - _middleware: Array<{ - transformParams: (args: { - type: string; - params: Record<string, unknown>; - }) => Promise<unknown>; - }>; - } - )._middleware; - - const result = await middleware[0]?.transformParams({ - type: "stream", - params: { prompt }, - }); - - return (result as Record<string, unknown>).prompt as unknown[]; -} +describe("createProvider (default OpenAI-compatible path)", () => { + it("does not wrap the model in a middleware layer — v6 SDK handles reasoning round-trip natively", () => { + mockCreateOpenAICompatible.mockClear(); + mockOpenAICompatibleFactory.mockClear(); -describe("createProvider middleware", () => { - it("passes through non-stream calls unchanged", async () => { - const wrappedModel = createProvider({ + const model = createProvider({ apiKey: "test-key", baseURL: "https://example.com/v1", - })("test-model"); - - const middleware = ( - wrappedModel as unknown as { - _middleware: Array<{ - transformParams: (args: { - type: string; - params: Record<string, unknown>; - }) => Promise<unknown>; - }>; - } - )._middleware; - - const params = { prompt: [], temperature: 0.5 }; - const result = (await middleware[0]?.transformParams({ - type: "generate", - params, - })) as Record<string, unknown>; - - expect(result).toEqual(params); + })("deepseek-v4-pro"); + + // The factory should have been invoked with the model id directly, + // without going through `wrapLanguageModel`. If a middleware were + // still in place, the returned object would carry an `_middleware` + // property (set by our test mock pattern). The bare provider model + // has no such property — verifying the v4-era normalizeMessages + // middleware is gone. + expect(mockOpenAICompatibleFactory).toHaveBeenCalledWith("deepseek-v4-pro"); + expect((model as { _middleware?: unknown })._middleware).toBeUndefined(); }); - it("strips reasoning parts and sets reasoning_content on providerMetadata", async () => { - const prompt = [ - { - role: "assistant", - content: [ - { type: "reasoning", text: "I should use the list_files tool." }, - { type: "text", text: "Let me check the directory." }, - { - type: "tool-call", - toolCallId: "call_1", - toolName: "list_files", - args: { path: "." }, - }, - ], - }, - ]; - - const normalized = await runTransform(prompt); - const msg = normalized[0] as Record<string, unknown>; - - // Reasoning parts removed from content - const content = msg.content as Array<Record<string, unknown>>; - expect(content).toHaveLength(2); - expect(content.find((p) => p.type === "reasoning")).toBeUndefined(); - expect(content.find((p) => p.type === "text")).toBeDefined(); - expect(content.find((p) => p.type === "tool-call")).toBeDefined(); - - // reasoning_content set on providerMetadata - const pm = msg.providerMetadata as Record<string, unknown>; - const compat = pm.openaiCompatible as Record<string, unknown>; - expect(compat.reasoning_content).toBe("I should use the list_files tool."); - }); - - it("sets empty reasoning_content when no reasoning parts exist", async () => { - const prompt = [ - { - role: "assistant", - content: [{ type: "text", text: "Hello!" }], - }, - ]; - - const normalized = await runTransform(prompt); - const msg = normalized[0] as Record<string, unknown>; + it("passes name, apiKey, baseURL to createOpenAICompatible", () => { + mockCreateOpenAICompatible.mockClear(); - // Content unchanged - const content = msg.content as Array<Record<string, unknown>>; - expect(content).toHaveLength(1); - expect(content[0]?.type).toBe("text"); + createProvider({ + apiKey: "zen-key", + baseURL: "https://opencode.ai/zen/v1", + })("deepseek-v4-pro"); - // reasoning_content always set, even empty - const pm = msg.providerMetadata as Record<string, unknown>; - const compat = pm.openaiCompatible as Record<string, unknown>; - expect(compat.reasoning_content).toBe(""); + expect(mockCreateOpenAICompatible).toHaveBeenCalledWith({ + name: "opencode-zen", + apiKey: "zen-key", + baseURL: "https://opencode.ai/zen/v1", + }); }); +}); - it("does not modify user messages", async () => { - const prompt = [ - { - role: "user", - content: [{ type: "text", text: "What dir am I in?" }], - }, - ]; - - const normalized = await runTransform(prompt); - expect(normalized).toEqual(prompt); +describe("createClaudeOAuthProvider", () => { + it("passes authToken (not apiKey) to createAnthropic for OAuth flow", () => { + mockCreateAnthropic.mockClear(); + + createProvider({ + provider: "anthropic", + apiKey: "fallback-api-key", + baseURL: "", + claudeCredentials: { accessToken: "oauth-access-token" }, + })("claude-opus-4-5"); + + expect(mockCreateAnthropic).toHaveBeenCalledOnce(); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record<string, unknown>; + expect(callArgs.authToken).toBe("oauth-access-token"); + expect(callArgs.apiKey).toBeUndefined(); }); - it("does not modify system messages", async () => { - const prompt = [{ role: "system", content: "You are a helpful assistant." }]; + it("falls back to apiKey as authToken when claudeCredentials are absent", () => { + mockCreateAnthropic.mockClear(); - const normalized = await runTransform(prompt); - expect(normalized).toEqual(prompt); - }); + createProvider({ + provider: "anthropic", + apiKey: "sk-ant-api-key", + baseURL: "", + })("claude-opus-4-5"); - it("handles assistant with plain string content (not array)", async () => { - const prompt = [ - { - role: "assistant", - content: "Hello world", - }, - ]; - - const normalized = await runTransform(prompt); - expect(normalized).toEqual(prompt); + expect(mockCreateAnthropic).toHaveBeenCalledOnce(); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record<string, unknown>; + expect(callArgs.authToken).toBe("sk-ant-api-key"); + expect(callArgs.apiKey).toBeUndefined(); }); - it("handles redacted-reasoning type parts", async () => { - const prompt = [ - { - role: "assistant", - content: [ - { type: "redacted-reasoning", text: "[redacted chain of thought]" }, - { type: "text", text: "Here is the result." }, - ], - }, - ]; - - const normalized = await runTransform(prompt); - const msg = normalized[0] as Record<string, unknown>; + it("includes required Claude CLI headers", () => { + mockCreateAnthropic.mockClear(); - const content = msg.content as Array<Record<string, unknown>>; - expect(content.find((p) => p.type === "redacted-reasoning")).toBeUndefined(); - expect(content.find((p) => p.type === "text")).toBeDefined(); + createProvider({ + provider: "anthropic", + apiKey: "test-key", + baseURL: "", + claudeCredentials: { accessToken: "tok" }, + })("claude-opus-4-5"); - const pm = msg.providerMetadata as Record<string, unknown>; - const compat = pm.openaiCompatible as Record<string, unknown>; - expect(compat.reasoning_content).toBe("[redacted chain of thought]"); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record< + string, + Record<string, string> + >; + expect(callArgs.headers?.["anthropic-dangerous-direct-browser-access"]).toBe("true"); + expect(callArgs.headers?.["x-app"]).toBe("cli"); + expect(callArgs.headers?.["user-agent"]).toMatch(/claude-cli/); }); - it("preserves existing providerMetadata fields", async () => { - const prompt = [ - { - role: "assistant", - content: [ - { type: "reasoning", text: "thinking..." }, - { type: "text", text: "done" }, - ], - providerMetadata: { - openaiCompatible: { custom_field: "keep-me" }, - }, - }, - ]; + it("uses default Anthropic baseURL when none provided", () => { + mockCreateAnthropic.mockClear(); - const normalized = await runTransform(prompt); - const msg = normalized[0] as Record<string, unknown>; - const pm = msg.providerMetadata as Record<string, unknown>; - const compat = pm.openaiCompatible as Record<string, unknown>; + createProvider({ + provider: "anthropic", + apiKey: "test-key", + baseURL: "", + claudeCredentials: { accessToken: "tok" }, + })("claude-opus-4-5"); - expect(compat.custom_field).toBe("keep-me"); - expect(compat.reasoning_content).toBe("thinking..."); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record<string, string>; + expect(callArgs.baseURL).toBe("https://api.anthropic.com/v1"); }); - it("handles multi-message prompts with mixed roles", async () => { - const prompt = [ - { role: "system", content: "Be helpful." }, - { role: "user", content: [{ type: "text", text: "hi" }] }, - { - role: "assistant", - content: [ - { type: "reasoning", text: "I'll say hello." }, - { type: "text", text: "Hi there!" }, - ], - }, - ]; + it("uses configured baseURL when provided", () => { + mockCreateAnthropic.mockClear(); - const normalized = await runTransform(prompt); - - // System and user unchanged - expect(normalized[0]).toEqual(prompt[0]); - expect(normalized[1]).toEqual(prompt[1]); + createProvider({ + provider: "anthropic", + apiKey: "test-key", + baseURL: "https://custom.proxy.example.com/v1", + claudeCredentials: { accessToken: "tok" }, + })("claude-opus-4-5"); - // Assistant transformed - const msg = normalized[2] as Record<string, unknown>; - const pm = msg.providerMetadata as Record<string, unknown>; - const compat = pm.openaiCompatible as Record<string, unknown>; - expect(compat.reasoning_content).toBe("I'll say hello."); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record<string, string>; + expect(callArgs.baseURL).toBe("https://custom.proxy.example.com/v1"); }); +}); - it("concatenates multiple reasoning parts", async () => { - const prompt = [ - { - role: "assistant", - content: [ - { type: "reasoning", text: "Step 1: " }, - { type: "reasoning", text: "Step 2: " }, - { type: "reasoning", text: "Step 3." }, - { type: "text", text: "Final answer." }, - ], - }, - ]; +describe("createApiKeyAnthropicProvider", () => { + it("passes apiKey (not authToken) to createAnthropic", () => { + mockCreateAnthropic.mockClear(); - const normalized = await runTransform(prompt); - const msg = normalized[0] as Record<string, unknown>; - const pm = msg.providerMetadata as Record<string, unknown>; - const compat = pm.openaiCompatible as Record<string, unknown>; - expect(compat.reasoning_content).toBe("Step 1: Step 2: Step 3."); + createProvider({ + provider: "opencode-anthropic", + apiKey: "zen-api-key", + baseURL: "", + })("minimax-model"); + + expect(mockCreateAnthropic).toHaveBeenCalledOnce(); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record<string, unknown>; + expect(callArgs.apiKey).toBe("zen-api-key"); + expect(callArgs.authToken).toBeUndefined(); }); - it("applies to every assistant message in a multi-step history", async () => { - const prompt = [ - { - role: "assistant", - content: [ - { type: "reasoning", text: "First thought." }, - { type: "tool-call", toolCallId: "c1", toolName: "list_files", args: {} }, - ], - }, - { - role: "assistant", - content: [ - { type: "reasoning", text: "Second thought." }, - { type: "text", text: "All done." }, - ], - }, - ]; + it("uses default OpenCode Zen baseURL when none provided", () => { + mockCreateAnthropic.mockClear(); - const normalized = await runTransform(prompt); + createProvider({ + provider: "opencode-anthropic", + apiKey: "zen-api-key", + baseURL: "", + })("minimax-model"); - const msg1 = normalized[0] as Record<string, unknown>; - const compat1 = (msg1.providerMetadata as Record<string, unknown>).openaiCompatible as Record< - string, - unknown - >; - expect(compat1.reasoning_content).toBe("First thought."); - - const msg2 = normalized[1] as Record<string, unknown>; - const compat2 = (msg2.providerMetadata as Record<string, unknown>).openaiCompatible as Record< - string, - unknown - >; - expect(compat2.reasoning_content).toBe("Second thought."); + const callArgs = mockCreateAnthropic.mock.calls[0]?.[0] as Record<string, string>; + expect(callArgs.baseURL).toBe("https://opencode.ai/zen/go/v1"); }); }); diff --git a/packages/core/tests/tools/registry.test.ts b/packages/core/tests/tools/registry.test.ts index b6f1fca..cad75d2 100644 --- a/packages/core/tests/tools/registry.test.ts +++ b/packages/core/tests/tools/registry.test.ts @@ -17,6 +17,21 @@ const anotherTool: ToolDefinition = { execute: async (_args) => "another result", }; +/** A non-trivial tool that exercises nested objects, required fields, and enums. */ +const complexTool: ToolDefinition = { + name: "complex_tool", + description: "A tool with nested parameters", + parameters: z.object({ + command: z.string().describe("Shell command to run"), + options: z.object({ + timeout: z.number().optional().describe("Timeout in milliseconds"), + shell: z.enum(["bash", "sh", "zsh"]).describe("Shell to use"), + }), + flags: z.array(z.string()).optional().describe("Additional flags"), + }), + execute: async (_args) => "complex result", +}; + describe("createToolRegistry", () => { it("returns all tools via getTools()", () => { const registry = createToolRegistry([mockTool, anotherTool]); @@ -38,13 +53,91 @@ describe("createToolRegistry", () => { expect(registry.getTool("nonexistent")).toBeUndefined(); }); - it("getAISDKTools returns correct format", () => { - const registry = createToolRegistry([mockTool, anotherTool]); - const aiTools = registry.getAISDKTools(); - expect(aiTools).toHaveProperty("mock_tool"); - expect(aiTools).toHaveProperty("another_tool"); - // Each should have description and parameters (AI SDK tool format) - expect(aiTools.mock_tool).toHaveProperty("description"); - expect(aiTools.mock_tool).toHaveProperty("parameters"); + describe("getAISDKTools", () => { + it("returns correct keys for all tools", () => { + const registry = createToolRegistry([mockTool, anotherTool]); + const aiTools = registry.getAISDKTools(); + expect(aiTools).toHaveProperty("mock_tool"); + expect(aiTools).toHaveProperty("another_tool"); + }); + + it("AI SDK tools have description from ToolDefinition", () => { + const registry = createToolRegistry([mockTool]); + const aiTools = registry.getAISDKTools(); + expect(aiTools.mock_tool.description).toBe("A mock tool for testing"); + }); + + it("AI SDK tools surface schema via inputSchema, not parameters", () => { + const registry = createToolRegistry([mockTool]); + const aiTools = registry.getAISDKTools(); + // v6 uses inputSchema; v4 used parameters — this verifies the migration + expect(aiTools.mock_tool).toHaveProperty("inputSchema"); + expect(aiTools.mock_tool).not.toHaveProperty("parameters"); + }); + + it("AI SDK tools have no execute callback so the SDK does not auto-run", () => { + const registry = createToolRegistry([mockTool, anotherTool, complexTool]); + const aiTools = registry.getAISDKTools(); + for (const [name, sdkTool] of Object.entries(aiTools)) { + expect( + (sdkTool as Record<string, unknown>).execute, + `Tool "${name}" should not have an execute callback`, + ).toBeUndefined(); + } + }); + + it("inputSchema produces valid JSONSchema7 for a simple tool", () => { + const registry = createToolRegistry([mockTool]); + const aiTools = registry.getAISDKTools(); + const schema = aiTools.mock_tool.inputSchema; + // jsonSchema() wraps the raw JSONSchema7; it should expose the schema + // as a `jsonSchema` property on the Schema object + expect(schema).toBeDefined(); + // The wrapped schema object should carry the JSON Schema definition + const schemaObj = schema as { jsonSchema: Record<string, unknown> }; + expect(schemaObj.jsonSchema).toBeDefined(); + expect(schemaObj.jsonSchema.type).toBe("object"); + const props = schemaObj.jsonSchema.properties as Record<string, unknown>; + expect(props).toHaveProperty("input"); + }); + + it("inputSchema produces correct JSONSchema7 for a non-trivial nested tool", () => { + const registry = createToolRegistry([complexTool]); + const aiTools = registry.getAISDKTools(); + const schema = aiTools.complex_tool.inputSchema; + expect(schema).toBeDefined(); + const schemaObj = schema as { jsonSchema: Record<string, unknown> }; + expect(schemaObj.jsonSchema.type).toBe("object"); + + const props = schemaObj.jsonSchema.properties as Record<string, Record<string, unknown>>; + + // Top-level required field "command" + expect(props).toHaveProperty("command"); + expect(props.command.type).toBe("string"); + + // Nested object "options" + expect(props).toHaveProperty("options"); + expect(props.options.type).toBe("object"); + const optProps = props.options.properties as Record<string, Record<string, unknown>>; + expect(optProps).toHaveProperty("shell"); + expect(optProps.shell.enum).toEqual(["bash", "sh", "zsh"]); + + // Optional array "flags" present as a property + expect(props).toHaveProperty("flags"); + expect(props.flags.type).toBe("array"); + + // Required fields should include "command" and "options" + const required = schemaObj.jsonSchema.required as string[]; + expect(required).toContain("command"); + expect(required).toContain("options"); + }); + + it("getTool still returns the original ToolDefinition with execute", () => { + const registry = createToolRegistry([mockTool]); + const def = registry.getTool("mock_tool"); + expect(def).toBeDefined(); + expect(typeof def?.execute).toBe("function"); + expect(def?.name).toBe("mock_tool"); + }); }); }); |
