diff options
| author | Adam Malczewski <[email protected]> | 2026-06-27 04:18:59 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-27 04:18:59 +0900 |
| commit | 3566a20ebbded754070fce66af48690d1a904879 (patch) | |
| tree | ad4941abdbe685e2d4ae85d9ad3d0b6e9cf82c2d /src/core/wire | |
| parent | 2fa03f8d7410c2b8d6be8e10ad088863e83d7177 (diff) | |
| download | dispatch-web-3566a20ebbded754070fce66af48690d1a904879.tar.gz dispatch-web-3566a20ebbded754070fce66af48690d1a904879.zip | |
feat(vision): image paste + transcript image rendering + vision badge
Vision & vision-handoff frontend (consumes the backend's additive
[email protected] / [email protected] image types — no version bump).
Contracts mirrored:
- .dispatch/wire.reference.md: ImageChunk added to the Chunk union +
ImageChunk/ImageInput interfaces.
- .dispatch/transport-contract.reference.md: ChatRequest.images,
ModelMetadata.vision, + ImageChunk/ImageInput re-exports.
Core (core/chunks):
- conformance: assertChunkExhaustive handles the new 'image' variant
(the guard caught it — its purpose).
- appendUserMessage(state, text, images?) echoes a [text, image, ...]
user run; the user-message event dedup scans the trailing user run
(not just the last chunk) so an image-bearing echo doesn't duplicate
the text; applyHistory's during-gen dedup matches a multi-chunk echo
by content equality (chunkContentEquals + trailingRun helpers).
UI:
- ChatView renders user 'image' chunks as lazy <img> bubbles; a
non-vision model's persisted [image, analysis-text] both render.
read_image tool renders generically (no special-casing).
- Composer: clipboard paste / file picker / drag-drop of images ->
base64 data URLs, thumbnail previews with remove, forwarded on
chat.send (omitted when none). Image-only sends allowed; steering
(chat.queue) never forwards images.
- ModelSelector: vision badge (isVisionModel) marks vision-capable
models; indicator shows native-vision vs vision-handoff hint.
Store wiring: ChatStore.send + AppStore.send + App.svelte handleSend
thread images through; chat.send still omits cwd (only images added).
Verification: svelte-check 0/0; vitest 901/901 (run twice, +34 new);
biome clean; vite build OK. See backend-handoff.md §2j.
Not merged or pushed.
Diffstat (limited to 'src/core/wire')
| -rw-r--r-- | src/core/wire/conformance.test.ts | 40 | ||||
| -rw-r--r-- | src/core/wire/conformance.ts | 2 |
2 files changed, 41 insertions, 1 deletions
diff --git a/src/core/wire/conformance.test.ts b/src/core/wire/conformance.test.ts index 880af07..0d955d5 100644 --- a/src/core/wire/conformance.test.ts +++ b/src/core/wire/conformance.test.ts @@ -128,9 +128,32 @@ describe("classifies every Chunk type", () => { }, { type: "error" as const, message: "e" }, { type: "system" as const, text: "s" }, + { type: "image" as const, url: "data:image/png;base64,AAAA", mimeType: "image/png" }, ]; const labels = chunks.map(assertChunkExhaustive); - expect(labels).toEqual(["text", "thinking", "tool-call", "tool-result", "error", "system"]); + expect(labels).toEqual([ + "text", + "thinking", + "tool-call", + "tool-result", + "error", + "system", + "image", + ]); + }); + + it("covers all 7 Chunk variants", () => { + // Keeps the exhaustive guard honest: a new Chunk.type variant must be added + // both here and to `assertChunkExhaustive` or the `satisfies never` errors. + expect([ + "text", + "thinking", + "tool-call", + "tool-result", + "error", + "system", + "image", + ] as const).toHaveLength(7); }); }); @@ -222,6 +245,21 @@ describe("ChatSendMessage shape is constructible", () => { expect(msg.model).toBe("default/gpt-4"); expect(msg.cwd).toBe("/tmp"); }); + + it("constructs a ChatSendMessage with pasted images", () => { + const msg: ChatSendMessage = { + type: "chat.send", + conversationId: "c1", + message: "what's in this image?", + images: [ + { url: "data:image/png;base64,AAAA", mimeType: "image/png" }, + { url: "https://example.com/cat.jpg" }, + ], + }; + expect(msg.images).toHaveLength(2); + expect(msg.images?.[0]?.mimeType).toBe("image/png"); + expect(msg.images?.[1]?.mimeType).toBeUndefined(); + }); }); describe("ConversationHistoryResponse shape is constructible", () => { diff --git a/src/core/wire/conformance.ts b/src/core/wire/conformance.ts index 412d80a..bfa67dc 100644 --- a/src/core/wire/conformance.ts +++ b/src/core/wire/conformance.ts @@ -60,6 +60,8 @@ export function assertChunkExhaustive(chunk: Chunk): string { return "error"; case "system": return "system"; + case "image": + return "image"; default: return chunk satisfies never; } |
