diff options
| author | Adam Malczewski <[email protected]> | 2026-06-22 14:58:40 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-22 14:58:40 +0900 |
| commit | e6e2d31b987043d0325a93d23295af0408b723ec (patch) | |
| tree | d3208766c1e8123b8ca81c7a7c5a91c2c269e536 /packages/credential-store/src | |
| parent | 519b79999d49156bbfaecc91a2d882fba2475fef (diff) | |
| download | dispatch-e6e2d31b987043d0325a93d23295af0408b723ec.tar.gz dispatch-e6e2d31b987043d0325a93d23295af0408b723ec.zip | |
feat: context window from model endpoints + percentage-based auto-compact
ModelInfo (kernel contract):
- Add contextWindow?: number field
OpenAI-stream listModels:
- Parse contextWindow from common field names (context_length,
context_window, max_context_length, max_tokens)
Transport-contract:
- ModelsResponse: add optional modelInfo map (model name → { contextWindow? })
- Add ModelMetadata type
- Rename CompactThresholdResponse → CompactPercentResponse
- Rename SetCompactThresholdRequest → SetCompactPercentRequest
Credential store:
- Add getModelInfo(modelName) method — resolves full ModelInfo
(including contextWindow) for a <credential>/<model> string
Transport-http:
- GET /models now includes modelInfo with contextWindow per model
- Rename compact-threshold endpoints → compact-percent
Session-orchestrator:
- Auto-compact now uses contextSize (not overcounted usage.inputTokens)
compared against contextWindow * (percent / 100)
- Default percent: 85 (was flat 350000)
- resolveModelInfo dep added to look up contextWindow
- Passes modelName from the settled turn to the compaction service
Conversation store:
- Rename getCompactThreshold/setCompactThreshold → getCompactPercent/setCompactPercent
- compactThresholdKey → compact-percent key
Diffstat (limited to 'packages/credential-store/src')
| -rw-r--r-- | packages/credential-store/src/registry.ts | 26 |
1 files changed, 25 insertions, 1 deletions
diff --git a/packages/credential-store/src/registry.ts b/packages/credential-store/src/registry.ts index c1ba771..89d8084 100644 --- a/packages/credential-store/src/registry.ts +++ b/packages/credential-store/src/registry.ts @@ -1,4 +1,4 @@ -import type { ProviderContract } from "@dispatch/kernel"; +import type { ModelInfo, ProviderContract } from "@dispatch/kernel"; export interface Credential { readonly name: string; @@ -24,6 +24,13 @@ export interface CredentialStore { * missing or has no listModels. */ listCatalog(): Promise<readonly string[]>; + + /** + * Returns the full `ModelInfo` for a `<credentialName>/<model>` string, or + * undefined if unknown. Caches the result of `listModels` per credential. + * Used to look up `contextWindow` for auto-compaction. + */ + getModelInfo(modelName: string): Promise<ModelInfo | undefined>; } export interface CredentialStoreDeps { @@ -76,5 +83,22 @@ export function createCredentialStore(deps: CredentialStoreDeps): CredentialStor return results; }, + + async getModelInfo(modelName: string): Promise<ModelInfo | undefined> { + const slashIndex = modelName.indexOf("/"); + if (slashIndex === -1) return undefined; + const credentialName = modelName.slice(0, slashIndex); + const modelId = modelName.slice(slashIndex + 1); + if (!modelId) return undefined; + + const providerId = credentialMap.get(credentialName); + if (providerId === undefined) return undefined; + + const provider = deps.getProvider(providerId); + if (provider?.listModels === undefined) return undefined; + + const models = await provider.listModels(); + return models.find((m) => m.id === modelId); + }, }; } |
