diff options
| author | Adam Malczewski <[email protected]> | 2026-06-27 19:15:04 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-27 19:15:04 +0900 |
| commit | fa0bd9c0e433b1abddc814b48a358c94954c7d36 (patch) | |
| tree | dedf7adf827de6908b5c7cbb4b4153aba38c1882 /.dispatch | |
| parent | 3566a20ebbded754070fce66af48690d1a904879 (diff) | |
| download | dispatch-web-fa0bd9c0e433b1abddc814b48a358c94954c7d36.tar.gz dispatch-web-fa0bd9c0e433b1abddc814b48a358c94954c7d36.zip | |
feat(vision): consult_vision tool + vision settings API
Backend vision update (additive to [email protected] / [email protected],
no version bump).
Contracts mirrored (.dispatch/transport-contract.reference.md):
- VisionSettingsResponse + SetVisionSettingsRequest (GET/PUT /settings/vision).
- Delta note: read_image -> consult_vision; numbered placeholders; compaction.
Tool rendering (ChatView):
- read_image test -> consult_vision (rendering is generic by toolName).
- +2 tests: numbered-placeholder text chunk + [Compacted image] text chunk
(both regular text chunks, render as-is — no special handling).
New vision feature library (src/features/vision/):
- logic/view-model.ts (32 tests): VisionSettings/VisionSettingsPatch types
(consumer-defines-port), LoadVisionSettings/SaveVisionSettings ports +
results, normalizeVisionSettings (network-seam coercion), parseImageLimit/
imageLimitChanged, compactionModelOptions (vision-capable models via chat's
public isVisionModel + Auto sentinel), round-trip helpers, imageLimitLabel.
- ui/VisionSettingsView.svelte (9 tests): imageLimit input + Save,
compactionModel dropdown (Auto + vision-capable models), load-on-mount,
save-on-change, error/saved feedback.
- index.ts.
Cross-unit seam: isVisionModel added to features/chat public index.ts
(additive); imported through the public surface, not internals.
Store wiring (src/app/store.svelte.ts):
- visionSettings state + refreshVisionSettings (GET /settings/vision,
normalized at the seam) + setVisionSettings (PUT, partial, returns merged) +
VisionSettingsResult; seeded on boot; exposed on AppStore. +4 store tests.
Mounted in App.svelte: new "Vision" sidebar view kind + VisionSettingsView
in viewContent (not conversation-scoped); load/save adapters; visionManifest.
Verification: svelte-check 0/0; vitest 948/948 (run twice, +47 since the
prior vision commit); biome clean; vite build OK. See backend-handoff.md §2j.
Not merged or pushed.
Diffstat (limited to '.dispatch')
| -rw-r--r-- | .dispatch/transport-contract.reference.md | 37 |
1 files changed, 34 insertions, 3 deletions
diff --git a/.dispatch/transport-contract.reference.md b/.dispatch/transport-contract.reference.md index 152a1b0..047f050 100644 --- a/.dispatch/transport-contract.reference.md +++ b/.dispatch/transport-contract.reference.md @@ -13,9 +13,19 @@ > (each entry: `{ url, mimeType? }` — a base64 data URL or `http(s)://` URL; validated non-array/no-url/ > empty-url → 400, empty array treated as absent). `ModelMetadata` gains `vision?: boolean` (true when the > model natively accepts images; absent → the server's vision handoff transcribes images to text before the -> model sees them). `ImageChunk`/`ImageInput` are `@dispatch/wire` types (re-exported here). A non-vision -> model's persisted user message keeps the original `image` chunk AND adds a `text` transcription chunk -> (`[Image analysis (via <model>)]: …`) in the SAME message — render both. See `backend-handoff.md` §2j. +> model sees them). `ImageChunk`/`ImageInput` are `@dispatch/wire` types (re-exported here). +> +> **2026-06-26 update (consult_vision + vision settings — ADDITIVE, NO version bump):** the `read_image` +> tool is REPLACED by `consult_vision` (`{ question: string, imageIds?: number[], path?: string }`) — it +> opens a NEW conversation tab with a vision-capable model, attaches the image + question, and returns the +> vision model's answer (rendered like any tool call/result). Non-vision models now get NUMBERED +> PLACEHOLDERS (`[Image N attached — call consult_vision with imageIds=[N] and a specific question to +> analyze it]`) instead of auto-transcriptions — these are regular `text` chunks (render as-is). Image +> compaction transcribes the oldest images past `imageLimit` to `[Compacted image]: <description>` text +> chunks (also regular `text` — render as-is; the persisted `image` chunk stays for rendering). NEW global +> vision settings API: `GET /settings/vision` → `VisionSettingsResponse` (`{ imageLimit, compactionModel }`), +> `PUT /settings/vision` ← `SetVisionSettingsRequest` (partial: `imageLimit?` non-negative int, 0 = disable +> compaction; `compactionModel?` `<key>/<model>` or null = auto). See `backend-handoff.md` §2j. > > **2026-06-25 delta (SSH handoff #2 — ADDITIVE to `[email protected]`, NO version bump):** adds the > computer HTTP API types: `ComputerListResponse` (`GET /computers`), `ComputerResponse` (`GET /computers/:alias`), @@ -459,6 +469,27 @@ export interface SystemPromptVariablesResponse { readonly variables: readonly SystemPromptVariable[]; } +// ─── Vision settings (global) ─────────────────────────────────────────────── + +/** + * Response of `GET /settings/vision` — the global vision configuration shared + * across all conversations and vision models. + */ +export interface VisionSettingsResponse { + /** Max native images per turn (default 10); 0 disables image compaction. */ + readonly imageLimit: number; + /** Which model transcribes old images (null = auto-select a vision model). */ + readonly compactionModel: string | null; +} + +/** Body of `PUT /settings/vision` — a partial update. */ +export interface SetVisionSettingsRequest { + /** Non-negative integer (0 = disable compaction). */ + readonly imageLimit?: number; + /** A model name (`<key>/<model>`) or null (auto). */ + readonly compactionModel?: string | null; +} + // ─── Message queue (steering) ───────────────────────────────────────────────── /** |
