summaryrefslogtreecommitdiffhomepage
path: root/.dispatch
diff options
context:
space:
mode:
authorAdam Malczewski <[email protected]>2026-06-27 19:15:04 +0900
committerAdam Malczewski <[email protected]>2026-06-27 19:15:04 +0900
commitfa0bd9c0e433b1abddc814b48a358c94954c7d36 (patch)
treededf7adf827de6908b5c7cbb4b4153aba38c1882 /.dispatch
parent3566a20ebbded754070fce66af48690d1a904879 (diff)
downloaddispatch-web-fa0bd9c0e433b1abddc814b48a358c94954c7d36.tar.gz
dispatch-web-fa0bd9c0e433b1abddc814b48a358c94954c7d36.zip
feat(vision): consult_vision tool + vision settings API
Backend vision update (additive to [email protected] / [email protected], no version bump). Contracts mirrored (.dispatch/transport-contract.reference.md): - VisionSettingsResponse + SetVisionSettingsRequest (GET/PUT /settings/vision). - Delta note: read_image -> consult_vision; numbered placeholders; compaction. Tool rendering (ChatView): - read_image test -> consult_vision (rendering is generic by toolName). - +2 tests: numbered-placeholder text chunk + [Compacted image] text chunk (both regular text chunks, render as-is — no special handling). New vision feature library (src/features/vision/): - logic/view-model.ts (32 tests): VisionSettings/VisionSettingsPatch types (consumer-defines-port), LoadVisionSettings/SaveVisionSettings ports + results, normalizeVisionSettings (network-seam coercion), parseImageLimit/ imageLimitChanged, compactionModelOptions (vision-capable models via chat's public isVisionModel + Auto sentinel), round-trip helpers, imageLimitLabel. - ui/VisionSettingsView.svelte (9 tests): imageLimit input + Save, compactionModel dropdown (Auto + vision-capable models), load-on-mount, save-on-change, error/saved feedback. - index.ts. Cross-unit seam: isVisionModel added to features/chat public index.ts (additive); imported through the public surface, not internals. Store wiring (src/app/store.svelte.ts): - visionSettings state + refreshVisionSettings (GET /settings/vision, normalized at the seam) + setVisionSettings (PUT, partial, returns merged) + VisionSettingsResult; seeded on boot; exposed on AppStore. +4 store tests. Mounted in App.svelte: new "Vision" sidebar view kind + VisionSettingsView in viewContent (not conversation-scoped); load/save adapters; visionManifest. Verification: svelte-check 0/0; vitest 948/948 (run twice, +47 since the prior vision commit); biome clean; vite build OK. See backend-handoff.md §2j. Not merged or pushed.
Diffstat (limited to '.dispatch')
-rw-r--r--.dispatch/transport-contract.reference.md37
1 files changed, 34 insertions, 3 deletions
diff --git a/.dispatch/transport-contract.reference.md b/.dispatch/transport-contract.reference.md
index 152a1b0..047f050 100644
--- a/.dispatch/transport-contract.reference.md
+++ b/.dispatch/transport-contract.reference.md
@@ -13,9 +13,19 @@
> (each entry: `{ url, mimeType? }` — a base64 data URL or `http(s)://` URL; validated non-array/no-url/
> empty-url → 400, empty array treated as absent). `ModelMetadata` gains `vision?: boolean` (true when the
> model natively accepts images; absent → the server's vision handoff transcribes images to text before the
-> model sees them). `ImageChunk`/`ImageInput` are `@dispatch/wire` types (re-exported here). A non-vision
-> model's persisted user message keeps the original `image` chunk AND adds a `text` transcription chunk
-> (`[Image analysis (via <model>)]: …`) in the SAME message — render both. See `backend-handoff.md` §2j.
+> model sees them). `ImageChunk`/`ImageInput` are `@dispatch/wire` types (re-exported here).
+>
+> **2026-06-26 update (consult_vision + vision settings — ADDITIVE, NO version bump):** the `read_image`
+> tool is REPLACED by `consult_vision` (`{ question: string, imageIds?: number[], path?: string }`) — it
+> opens a NEW conversation tab with a vision-capable model, attaches the image + question, and returns the
+> vision model's answer (rendered like any tool call/result). Non-vision models now get NUMBERED
+> PLACEHOLDERS (`[Image N attached — call consult_vision with imageIds=[N] and a specific question to
+> analyze it]`) instead of auto-transcriptions — these are regular `text` chunks (render as-is). Image
+> compaction transcribes the oldest images past `imageLimit` to `[Compacted image]: <description>` text
+> chunks (also regular `text` — render as-is; the persisted `image` chunk stays for rendering). NEW global
+> vision settings API: `GET /settings/vision` → `VisionSettingsResponse` (`{ imageLimit, compactionModel }`),
+> `PUT /settings/vision` ← `SetVisionSettingsRequest` (partial: `imageLimit?` non-negative int, 0 = disable
+> compaction; `compactionModel?` `<key>/<model>` or null = auto). See `backend-handoff.md` §2j.
>
> **2026-06-25 delta (SSH handoff #2 — ADDITIVE to `[email protected]`, NO version bump):** adds the
> computer HTTP API types: `ComputerListResponse` (`GET /computers`), `ComputerResponse` (`GET /computers/:alias`),
@@ -459,6 +469,27 @@ export interface SystemPromptVariablesResponse {
readonly variables: readonly SystemPromptVariable[];
}
+// ─── Vision settings (global) ───────────────────────────────────────────────
+
+/**
+ * Response of `GET /settings/vision` — the global vision configuration shared
+ * across all conversations and vision models.
+ */
+export interface VisionSettingsResponse {
+ /** Max native images per turn (default 10); 0 disables image compaction. */
+ readonly imageLimit: number;
+ /** Which model transcribes old images (null = auto-select a vision model). */
+ readonly compactionModel: string | null;
+}
+
+/** Body of `PUT /settings/vision` — a partial update. */
+export interface SetVisionSettingsRequest {
+ /** Non-negative integer (0 = disable compaction). */
+ readonly imageLimit?: number;
+ /** A model name (`<key>/<model>`) or null (auto). */
+ readonly compactionModel?: string | null;
+}
+
// ─── Message queue (steering) ─────────────────────────────────────────────────
/**