feat: add persistent model-aware reasoning effort settings
This commit is contained in:
+16
-3
@@ -46,6 +46,10 @@ Chat upload limits:
|
||||
- Gemini model lists are loaded from Google's native Models API and filtered to Gemini `generateContent` model ids.
|
||||
- The backend loads provider model lists at startup and refreshes them about once every 24 hours. If a later provider refresh fails, the response keeps the last loaded model list for that provider and sets `error` to the latest failure message.
|
||||
|
||||
Each provider also includes `reasoningCapabilities`, a map keyed by exact model ID. For example, OpenAI includes `"gpt-5.2": { "efforts": ["none", "low", "medium", "high", "xhigh"], "defaultEffort": "none" }`. This metadata comes from a conservative backend allowlist verified against [OpenAI model documentation](https://developers.openai.com/api/docs/models/gpt-5.2) and the [reasoning guidance](https://developers.openai.com/api/docs/guides/deployment-checklist#set-up-reasoningeffort), independently of model-list refreshes. It describes capabilities, not account access. `defaultEffort: null` means the provider default has not been established; it does not mean `none`. Defaults vary by model (for example GPT-5.2 uses `none`, [GPT-5.5 uses `medium`](https://developers.openai.com/api/docs/models/gpt-5.5)).
|
||||
|
||||
Clients must use the returned effort levels for the selected provider/model. Missing entries, missing metadata from an older server, unknown aliases/snapshots, and other providers fall back to **Model default**, with no effort sent to the provider. Do not infer support from a `gpt-` prefix or from an OpenAI-compatible endpoint. The full web Chat Settings slider includes Model default plus the supported levels; choices are saved per chat and remembered per provider/model in browser storage for new chats.
|
||||
|
||||
## Chat Tools
|
||||
|
||||
### `GET /v1/chat-tools`
|
||||
@@ -103,6 +107,7 @@ Behavior notes:
|
||||
"initiatedModel": "gpt-4.1-mini",
|
||||
"lastUsedProvider": "openai",
|
||||
"lastUsedModel": "gpt-4.1-mini",
|
||||
"reasoningEffort": null,
|
||||
"additionalSystemPrompt": null,
|
||||
"enabledTools": ["web_search", "fetch_url"]
|
||||
},
|
||||
@@ -149,6 +154,7 @@ History search (`q`) notes:
|
||||
"title": "optional title",
|
||||
"provider": "optional openai|anthropic|xai|gemini",
|
||||
"model": "optional model id",
|
||||
"reasoningEffort": null,
|
||||
"additionalSystemPrompt": "optional stored system prompt",
|
||||
"enabledTools": ["web_search", "fetch_url"],
|
||||
"messages": [
|
||||
@@ -165,6 +171,7 @@ History search (`q`) notes:
|
||||
|
||||
Behavior notes:
|
||||
- `provider` and `model` must be supplied together when present.
|
||||
- `reasoningEffort` is optional: `null` (the default) or `none|minimal|low|medium|high|xhigh|max`. The server stores only an effort supported by that exact provider/model; unsupported combinations become `null`. A string outside this enum is rejected. `null` means omit the provider parameter, whereas `"none"` explicitly disables reasoning on models that support it.
|
||||
- Newly created non-fork chats have `parentChatId: null` and `titleGenerationPending: false`.
|
||||
- When `provider`/`model` are supplied, the new chat initializes `initiatedProvider`/`initiatedModel` and `lastUsedProvider`/`lastUsedModel`.
|
||||
- `additionalSystemPrompt` is trimmed and stored on the chat; blank values are stored as `null`.
|
||||
@@ -181,18 +188,19 @@ Behavior notes:
|
||||
Behavior notes:
|
||||
- With no `messageId`, the server copies the complete source transcript. With an assistant response `messageId`, it copies the transcript through that response, inclusive.
|
||||
- The source chat is unchanged. Copied messages receive new ids while retaining their role, content, name, creation time, order, and metadata except for the source request's transport-only `clientRequestId`. Attachments, tool-call metadata, and per-response `metadata.generation` are retained. For older replies with an exact call-log match, generation metadata is recovered before copying. LLM call logs, star/project memberships, and other child threads are not copied.
|
||||
- The fork inherits the source chat's user, initiated/last-used provider and model, additional system prompt, and enabled tool settings.
|
||||
- The fork inherits the source chat's user, initiated/last-used provider and model, supported reasoning effort, additional system prompt, and enabled tool settings.
|
||||
- A fork of a root or another fork always sets `parentChatId` to the single root chat id. Root chats keep `parentChatId: null`.
|
||||
- A whole-chat fork starts with `Fork of <source title>` (or `Fork of Untitled chat`). A message fork starts with `Fork of '<message snippet>'`.
|
||||
- The placeholder title is returned with `titleGenerationPending: true`. After the first new prompt, call `POST /v1/chats/title/suggest`; the placeholder is eligible for replacement exactly once.
|
||||
|
||||
### `PATCH /v1/chats/:chatId`
|
||||
- Body: any subset of `{ "title": string, "additionalSystemPrompt": string|null, "enabledTools": string[] }`
|
||||
- Body: any subset of `{ "title": string, "provider": Provider, "model": string, "reasoningEffort": ReasoningEffort|null, "additionalSystemPrompt": string|null, "enabledTools": string[] }`
|
||||
- Response: `{ "chat": ChatSummary }`
|
||||
- Blank titles are rejected. The server trims surrounding whitespace before storing the title.
|
||||
- Setting a title clears `titleGenerationPending`, preventing an in-flight or later automatic suggestion from replacing the manual title.
|
||||
- `additionalSystemPrompt: null` clears the stored prompt. Blank string values are also stored as `null`.
|
||||
- `enabledTools: []` disables Sybil-managed tools for this chat. Omitted settings are left unchanged.
|
||||
- `provider` and `model` must be supplied together. They update `lastUsedProvider`/`lastUsedModel` immediately, atomically with `reasoningEffort`, without changing the original `initiatedProvider`/`initiatedModel`. Changing the model/provider without an effort clears the old effort. Keeping the same model without an effort preserves it. An effort-only patch uses the saved model. `reasoningEffort: null` resets to Model default; unsupported combinations normalize to `null` as on creation.
|
||||
- Updating chat fields changes the returned chat's `updatedAt`.
|
||||
- Not found: `404 { "message": "chat not found" }`
|
||||
|
||||
@@ -317,7 +325,8 @@ Notes:
|
||||
"additionalSystemPrompt": "optional one-off system prompt",
|
||||
"enabledTools": ["web_search", "fetch_url"],
|
||||
"temperature": 0.2,
|
||||
"maxTokens": 256
|
||||
"maxTokens": 256,
|
||||
"reasoningEffort": null
|
||||
}
|
||||
```
|
||||
- Response:
|
||||
@@ -345,6 +354,8 @@ Behavior notes:
|
||||
- Saved ordinary assistant responses also store the completion's provider/model in `metadata.generation`, exposed by `GET /v1/chats/:chatId`; changing the chat selection does not relabel previous responses.
|
||||
- `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint.
|
||||
- `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
|
||||
- Optional `reasoningEffort` uses the enum/null contract above. Omission inherits the saved effort only when the request provider/model exactly matches the saved pair; otherwise it uses Model default. Explicit `null` resets it. The effective validated value is persisted alongside the last-used model on ordinary persisted calls, including failed calls. Inline `@sybil` calls use the separate Agent configuration.
|
||||
- The backend checks support again before sending OpenAI requests: Responses uses `reasoning: { effort }`; Chat Completions uses `reasoning_effort`. Defaults and unsupported combinations omit both fields. With an explicit effort other than `none`, OpenAI `temperature` is omitted to avoid incompatible sampling parameters. For allowlisted reasoning models, Chat Completions maps `maxTokens` to `max_completion_tokens`; Responses uses `max_output_tokens`.
|
||||
- `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap.
|
||||
- Server persists final assistant output and call metadata (`LlmCall`) in DB.
|
||||
- Server updates chat-level model metadata on each call: `lastUsedProvider`/`lastUsedModel`; first successful/failed call also initializes `initiatedProvider`/`initiatedModel` if unset.
|
||||
@@ -497,6 +508,7 @@ Saved selections for unsupported provider IDs are returned as `null` for both th
|
||||
"initiatedModel": "string|null",
|
||||
"lastUsedProvider": "openai|anthropic|xai|gemini|null",
|
||||
"lastUsedModel": "string|null",
|
||||
"reasoningEffort": "none|minimal|low|medium|high|xhigh|max|null",
|
||||
"additionalSystemPrompt": null,
|
||||
"enabledTools": ["web_search", "fetch_url"]
|
||||
}
|
||||
@@ -551,6 +563,7 @@ Saved selections for unsupported provider IDs are returned as `null` for both th
|
||||
"initiatedModel": "string|null",
|
||||
"lastUsedProvider": "openai|anthropic|xai|gemini|null",
|
||||
"lastUsedModel": "string|null",
|
||||
"reasoningEffort": "none|minimal|low|medium|high|xhigh|max|null",
|
||||
"additionalSystemPrompt": null,
|
||||
"enabledTools": ["web_search", "fetch_url"],
|
||||
"messages": [Message]
|
||||
|
||||
@@ -58,7 +58,8 @@ Web presentation note: `/lite` uses these same chat streams and existing search
|
||||
"additionalSystemPrompt": "optional one-off system prompt",
|
||||
"enabledTools": ["web_search", "fetch_url"],
|
||||
"temperature": 0.2,
|
||||
"maxTokens": 256
|
||||
"maxTokens": 256,
|
||||
"reasoningEffort": null
|
||||
}
|
||||
```
|
||||
|
||||
@@ -73,6 +74,8 @@ Notes:
|
||||
- For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates.
|
||||
- `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint.
|
||||
- `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
|
||||
- Optional `reasoningEffort` accepts `null` or `none|minimal|low|medium|high|xhigh|max`. Load the selected provider/model's supported levels from `GET /v1/models` → `reasoningCapabilities` ([REST contract](./rest.md#models)). Omission inherits the chat's saved effort only for the exact same provider/model; `null` explicitly selects Model default. Unknown models and unsupported combinations normalize to `null`; invalid enum strings are rejected. The backend omits the effort parameter for Model default and unsupported combinations, letting the provider choose its model-specific default. `"none"` is an explicit setting and differs from `null`.
|
||||
- Ordinary persisted streams save the effective effort with the last-used provider/model. Non-persistent streams and Quick Questions do not save it. Inline `@sybil` turns use the separate Agent settings. OpenAI Responses requests use `reasoning: { effort }`, and Chat Completions requests use `reasoning_effort`, including subsequent tool rounds. Explicit reasoning efforts other than `none` omit `temperature`; allowlisted reasoning models use Chat Completions `max_completion_tokens` for `maxTokens`. SSE event shapes are unchanged.
|
||||
- `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap.
|
||||
- Attachments are optional and are persisted under `message.metadata.attachments` on stored user messages when `persist` is `true`.
|
||||
- For a persisted ordinary chat, a literal case-insensitive `@sybil` token in the latest user message invokes the existing persistent Sybil Agent instead of the selected provider. Email addresses, code spans/fences, indented code, and Markdown quote lines do not trigger it. The visible user message and inline Agent reply remain in the ordinary chat and are marked with backend-owned `metadata.agentAside`; the backend excludes both from later selected-provider context without deleting the visible rows. This routing does not apply to the special Agent tab or Quick Questions.
|
||||
@@ -112,7 +115,8 @@ Request body:
|
||||
"enabledTools": ["web_search", "fetch_url"],
|
||||
"userLocation": "optional city, region, country",
|
||||
"temperature": 0.2,
|
||||
"maxTokens": 256
|
||||
"maxTokens": 256,
|
||||
"reasoningEffort": null
|
||||
}
|
||||
```
|
||||
|
||||
@@ -122,7 +126,7 @@ Behavior notes:
|
||||
- The server prepends a Quick Question system prompt that asks for a succinct, direct, self-contained answer without follow-up questions. Clients do not send or maintain this prompt.
|
||||
- Quick Questions are always non-persistent. The endpoint does not create a chat or store messages, tool-call logs, assistant output, or `LlmCall` metadata.
|
||||
- The response uses the same `meta`, `tool_call`, `delta`, `done`, and `error` SSE events as chat completion streams. The `meta` event has `chatId: null` and `callId: null`.
|
||||
- `enabledTools`, `temperature`, and `maxTokens` are optional and behave as they do for chat completion streams. When `enabledTools` is omitted, all available Sybil-managed tools are enabled by default.
|
||||
- `enabledTools`, `temperature`, `maxTokens`, and `reasoningEffort` are optional and behave as they do for chat completion streams. When `enabledTools` is omitted, all available Sybil-managed tools are enabled by default. Omitted/null `reasoningEffort` uses the model default; Quick Questions never inherit ordinary chat settings. Existing Quick Question clients omit it.
|
||||
- User location is inferred from the same request headers as chat completion streams when `userLocation` is omitted.
|
||||
|
||||
## Event Stream Contract
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE "Chat" ADD COLUMN "reasoningEffort" TEXT;
|
||||
@@ -52,6 +52,7 @@ model Chat {
|
||||
initiatedModel String?
|
||||
lastUsedProvider String?
|
||||
lastUsedModel String?
|
||||
reasoningEffort String?
|
||||
|
||||
additionalSystemPrompt String?
|
||||
enabledTools Json?
|
||||
|
||||
@@ -12,7 +12,8 @@ import { searchBrave } from "../search/brave.js";
|
||||
import { exaClient } from "../search/exa.js";
|
||||
import { searchSearxng } from "../search/searxng.js";
|
||||
import { extractPdfText, hasPdfMagic, MAX_PDF_PAGES_PER_FETCH } from "./pdf-text.js";
|
||||
import type { ChatMessage } from "./types.js";
|
||||
import type { ChatMessage, Provider } from "./types.js";
|
||||
import type { ReasoningEffort } from "./reasoning-effort.js";
|
||||
|
||||
export const MAX_TOOL_ROUNDS = env.CHAT_MAX_TOOL_ROUNDS;
|
||||
const DEFAULT_WEB_RESULTS = 5;
|
||||
@@ -270,12 +271,14 @@ export type ToolAwareStreamingEvent =
|
||||
|
||||
export type ToolAwareCompletionParams = {
|
||||
client: any;
|
||||
provider?: Provider;
|
||||
model: string;
|
||||
messages: ChatMessage[];
|
||||
enabledTools?: string[];
|
||||
userLocation?: string;
|
||||
temperature?: number;
|
||||
maxTokens?: number;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
onToolEvent?: (event: ToolExecutionEvent) => void | Promise<void>;
|
||||
logContext?: {
|
||||
provider: string;
|
||||
|
||||
@@ -4,11 +4,13 @@ import {
|
||||
listModelCatalogProviders,
|
||||
} from "./provider-adapters.js";
|
||||
import type { Provider } from "./types.js";
|
||||
import { getReasoningCatalog, type ReasoningCapabilities } from "./reasoning-effort.js";
|
||||
|
||||
export type ProviderModelSnapshot = {
|
||||
models: string[];
|
||||
loadedAt: string | null;
|
||||
error: string | null;
|
||||
reasoningCapabilities?: Record<string, ReasoningCapabilities>;
|
||||
};
|
||||
|
||||
export type ModelCatalogSnapshot = Partial<Record<Provider, ProviderModelSnapshot>>;
|
||||
@@ -92,6 +94,7 @@ export function getModelCatalogSnapshot(): ModelCatalogSnapshot {
|
||||
models: [...entry.models],
|
||||
loadedAt: entry.loadedAt,
|
||||
error: entry.error,
|
||||
reasoningCapabilities: getReasoningCatalog(provider),
|
||||
};
|
||||
}
|
||||
return snapshot;
|
||||
|
||||
@@ -3,8 +3,10 @@ import { prisma } from "../db.js";
|
||||
import { buildToolLogMessageData } from "./chat-tools.js";
|
||||
import { getProviderChatAdapter } from "./provider-adapters.js";
|
||||
import type { MultiplexRequest, MultiplexResponse } from "./types.js";
|
||||
import { normalizeReasoningEffort } from "./reasoning-effort.js";
|
||||
|
||||
export async function runMultiplex(req: MultiplexRequest): Promise<MultiplexResponse> {
|
||||
req = { ...req, reasoningEffort: normalizeReasoningEffort(req.provider, req.model, req.reasoningEffort) };
|
||||
const t0 = performance.now();
|
||||
const chatId = req.chatId ?? (await prisma.chat.create({ data: {}, select: { id: true } })).id;
|
||||
|
||||
@@ -25,6 +27,7 @@ export async function runMultiplex(req: MultiplexRequest): Promise<MultiplexResp
|
||||
data: {
|
||||
lastUsedProvider: req.provider,
|
||||
lastUsedModel: req.model,
|
||||
reasoningEffort: req.reasoningEffort,
|
||||
},
|
||||
}),
|
||||
prisma.chat.updateMany({
|
||||
@@ -49,6 +52,7 @@ export async function runMultiplex(req: MultiplexRequest): Promise<MultiplexResp
|
||||
userLocation: req.userLocation,
|
||||
temperature: req.temperature,
|
||||
maxTokens: req.maxTokens,
|
||||
reasoningEffort: req.reasoningEffort,
|
||||
logContext: {
|
||||
provider: req.provider,
|
||||
model: req.model,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { getReasoningCapabilities, reasoningRequestOptions } from "../reasoning-effort.js";
|
||||
import {
|
||||
appendDanglingToolIntentCorrection,
|
||||
buildChatToolSystemPrompt,
|
||||
@@ -114,7 +115,8 @@ export async function completeWithChatCompletionsApi(params: ToolAwareCompletion
|
||||
model: params.model,
|
||||
messages: normalizePlainMessages(params.messages, params.userLocation),
|
||||
temperature: params.temperature,
|
||||
max_tokens: params.maxTokens,
|
||||
...(getReasoningCapabilities(params.provider, params.model) ? { max_completion_tokens: params.maxTokens } : { max_tokens: params.maxTokens }),
|
||||
...reasoningRequestOptions(params, "chat-completions"),
|
||||
} as any);
|
||||
|
||||
const usageAcc: Required<NonNullable<ToolAwareCompletionResult["usage"]>> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
|
||||
@@ -142,7 +144,8 @@ export async function completeWithChatCompletionsApi(params: ToolAwareCompletion
|
||||
model: params.model,
|
||||
messages: conversation,
|
||||
temperature: params.temperature,
|
||||
max_tokens: params.maxTokens,
|
||||
...(getReasoningCapabilities(params.provider, params.model) ? { max_completion_tokens: params.maxTokens } : { max_tokens: params.maxTokens }),
|
||||
...reasoningRequestOptions(params, "chat-completions"),
|
||||
tools: enabledTools,
|
||||
tool_choice: "auto",
|
||||
} as any);
|
||||
@@ -227,7 +230,8 @@ export async function* streamWithChatCompletionsApi(params: ToolAwareCompletionP
|
||||
model: params.model,
|
||||
messages: normalizePlainMessages(params.messages, params.userLocation),
|
||||
temperature: params.temperature,
|
||||
max_tokens: params.maxTokens,
|
||||
...(getReasoningCapabilities(params.provider, params.model) ? { max_completion_tokens: params.maxTokens } : { max_tokens: params.maxTokens }),
|
||||
...reasoningRequestOptions(params, "chat-completions"),
|
||||
stream: true,
|
||||
} as any);
|
||||
|
||||
@@ -267,7 +271,8 @@ export async function* streamWithChatCompletionsApi(params: ToolAwareCompletionP
|
||||
model: params.model,
|
||||
messages: conversation,
|
||||
temperature: params.temperature,
|
||||
max_tokens: params.maxTokens,
|
||||
...(getReasoningCapabilities(params.provider, params.model) ? { max_completion_tokens: params.maxTokens } : { max_tokens: params.maxTokens }),
|
||||
...reasoningRequestOptions(params, "chat-completions"),
|
||||
tools: enabledTools,
|
||||
tool_choice: "auto",
|
||||
stream: true,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { reasoningRequestOptions } from "../reasoning-effort.js";
|
||||
import {
|
||||
appendDanglingToolIntentCorrection,
|
||||
buildChatToolSystemPrompt,
|
||||
@@ -155,6 +156,7 @@ export async function completeWithResponsesApi(params: ToolAwareCompletionParams
|
||||
input,
|
||||
temperature: params.temperature,
|
||||
max_output_tokens: params.maxTokens,
|
||||
...reasoningRequestOptions(params, "responses"),
|
||||
tools: toResponsesTools(enabledTools),
|
||||
tool_choice: "auto",
|
||||
parallel_tool_calls: true,
|
||||
@@ -225,6 +227,7 @@ export async function* streamWithResponsesApi(params: ToolAwareCompletionParams)
|
||||
input,
|
||||
temperature: params.temperature,
|
||||
max_output_tokens: params.maxTokens,
|
||||
...reasoningRequestOptions(params, "responses"),
|
||||
tools: toResponsesTools(enabledTools),
|
||||
tool_choice: "auto",
|
||||
parallel_tool_calls: true,
|
||||
|
||||
@@ -10,6 +10,7 @@ import { completeWithMessagesApi, streamWithMessagesApi } from "./protocols/mess
|
||||
import { completeWithResponsesApi, streamWithResponsesApi } from "./protocols/responses-api.js";
|
||||
import { anthropicClient, geminiClient, openaiClient, xaiClient } from "./providers.js";
|
||||
import type { ChatMessage, Provider } from "./types.js";
|
||||
import { normalizeReasoningEffort, type ReasoningEffort } from "./reasoning-effort.js";
|
||||
|
||||
type ProviderAdapterParams = {
|
||||
model: string;
|
||||
@@ -18,6 +19,7 @@ type ProviderAdapterParams = {
|
||||
userLocation?: string;
|
||||
temperature?: number;
|
||||
maxTokens?: number;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
logContext?: ToolAwareCompletionParams["logContext"];
|
||||
};
|
||||
|
||||
@@ -126,15 +128,19 @@ async function fetchJson(url: URL): Promise<any> {
|
||||
return body;
|
||||
}
|
||||
|
||||
function withClient(params: ProviderAdapterParams, client: any, enabledTools?: string[]): ToolAwareCompletionParams {
|
||||
function withClient(params: ProviderAdapterParams, provider: Provider, client: any, enabledTools?: string[]): ToolAwareCompletionParams {
|
||||
const reasoningEffort = normalizeReasoningEffort(provider, params.model, params.reasoningEffort);
|
||||
return {
|
||||
client,
|
||||
provider,
|
||||
model: params.model,
|
||||
messages: params.messages,
|
||||
enabledTools,
|
||||
userLocation: params.userLocation,
|
||||
temperature: params.temperature,
|
||||
// OpenAI reasoning efforts other than none do not accept temperature.
|
||||
temperature: reasoningEffort && reasoningEffort !== "none" ? undefined : params.temperature,
|
||||
maxTokens: params.maxTokens,
|
||||
reasoningEffort,
|
||||
logContext: params.logContext,
|
||||
};
|
||||
}
|
||||
@@ -154,11 +160,11 @@ function createProviderChatAdapter(provider: Provider, spec: ProviderBackendSpec
|
||||
provider,
|
||||
complete(params) {
|
||||
const selected = selectChatProtocol(spec, params);
|
||||
return selected.protocol.complete(withClient(params, spec.createClient(), selected.enabledTools));
|
||||
return selected.protocol.complete(withClient(params, provider, spec.createClient(), selected.enabledTools));
|
||||
},
|
||||
stream(params) {
|
||||
const selected = selectChatProtocol(spec, params);
|
||||
return selected.protocol.stream(withClient(params, spec.createClient(), selected.enabledTools));
|
||||
return selected.protocol.stream(withClient(params, provider, spec.createClient(), selected.enabledTools));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -13,6 +13,7 @@ export type QuickQuestionRequest = {
|
||||
userLocation?: string;
|
||||
temperature?: number;
|
||||
maxTokens?: number;
|
||||
reasoningEffort?: MultiplexRequest["reasoningEffort"];
|
||||
};
|
||||
|
||||
export function buildQuickQuestionMultiplexRequest({
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
export const REASONING_EFFORTS = ["none", "minimal", "low", "medium", "high", "xhigh", "max"] as const;
|
||||
export type ReasoningEffort = (typeof REASONING_EFFORTS)[number];
|
||||
export type ReasoningCapabilities = {
|
||||
efforts: ReasoningEffort[];
|
||||
defaultEffort: ReasoningEffort | null;
|
||||
};
|
||||
|
||||
// Verified against official model pages on 2026-11-15:
|
||||
// https://developers.openai.com/api/docs/models/{model}
|
||||
// https://developers.openai.com/api/docs/guides/deployment-checklist#set-up-reasoningeffort
|
||||
// Use exact documented aliases/snapshots. Do not infer support from a prefix:
|
||||
// chat-latest, pro/codex variants, fine-tunes, and future snapshots can differ.
|
||||
const modelGroups: { models: string[]; capabilities: ReasoningCapabilities }[] = [
|
||||
{
|
||||
models: ["gpt-5", "gpt-5-2025-08-07"],
|
||||
capabilities: { efforts: ["minimal", "low", "medium", "high"], defaultEffort: "medium" },
|
||||
},
|
||||
{
|
||||
models: ["gpt-5.1", "gpt-5.1-2025-11-13"],
|
||||
capabilities: { efforts: ["none", "low", "medium", "high"], defaultEffort: "none" },
|
||||
},
|
||||
{
|
||||
models: ["gpt-5.2", "gpt-5.2-2025-12-11", "gpt-5.4", "gpt-5.4-2026-03-05",
|
||||
"gpt-5.4-mini", "gpt-5.4-mini-2026-03-17", "gpt-5.4-nano", "gpt-5.4-nano-2026-03-17"],
|
||||
capabilities: { efforts: ["none", "low", "medium", "high", "xhigh"], defaultEffort: "none" },
|
||||
},
|
||||
{
|
||||
models: ["gpt-5.5", "gpt-5.5-2026-04-23"],
|
||||
capabilities: { efforts: ["none", "low", "medium", "high", "xhigh"], defaultEffort: "medium" },
|
||||
},
|
||||
{
|
||||
models: ["gpt-5.6", "gpt-5.6-sol", "gpt-6-sol", "gpt-6-luna"],
|
||||
capabilities: { efforts: ["none", "low", "medium", "high", "xhigh", "max"], defaultEffort: "medium" },
|
||||
},
|
||||
{
|
||||
models: ["gpt-6.1-sol"],
|
||||
capabilities: { efforts: ["low", "medium", "high", "xhigh", "max"], defaultEffort: "medium" },
|
||||
},
|
||||
{
|
||||
models: ["gpt-6-astra"],
|
||||
capabilities: { efforts: ["low", "medium", "high", "xhigh", "max"], defaultEffort: null },
|
||||
},
|
||||
];
|
||||
|
||||
const openAIModels = new Map(modelGroups.flatMap(({ models, capabilities }) => models.map((model) => [model, capabilities] as const)));
|
||||
|
||||
export function getReasoningCapabilities(provider: string | null | undefined, model: string | null | undefined) {
|
||||
return provider === "openai" && model ? openAIModels.get(model) ?? null : null;
|
||||
}
|
||||
|
||||
export function getReasoningCatalog(provider: string): Record<string, ReasoningCapabilities> {
|
||||
if (provider !== "openai") return {};
|
||||
return Object.fromEntries([...openAIModels].map(([model, value]) => [model, { ...value, efforts: [...value.efforts] }]));
|
||||
}
|
||||
|
||||
export function normalizeReasoningEffort(provider: string | null | undefined, model: string | null | undefined, effort: unknown): ReasoningEffort | null {
|
||||
const capabilities = getReasoningCapabilities(provider, model);
|
||||
return capabilities?.efforts.includes(effort as ReasoningEffort) ? effort as ReasoningEffort : null;
|
||||
}
|
||||
|
||||
export function reasoningRequestOptions(params: { provider?: string; model: string; reasoningEffort?: ReasoningEffort | null }, protocol: "responses" | "chat-completions") {
|
||||
const effort = normalizeReasoningEffort(params.provider, params.model, params.reasoningEffort);
|
||||
if (!effort) return {};
|
||||
return protocol === "responses" ? { reasoning: { effort } } : { reasoning_effort: effort };
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
} from "./chat-tools.js";
|
||||
import { getProviderChatAdapter } from "./provider-adapters.js";
|
||||
import type { MultiplexRequest, Provider } from "./types.js";
|
||||
import { normalizeReasoningEffort } from "./reasoning-effort.js";
|
||||
|
||||
type StreamUsage = {
|
||||
inputTokens?: number;
|
||||
@@ -26,6 +27,7 @@ function getChatIdOrCreate(chatId?: string) {
|
||||
}
|
||||
|
||||
export async function* runMultiplexStream(req: MultiplexRequest): AsyncGenerator<StreamEvent> {
|
||||
req = { ...req, reasoningEffort: normalizeReasoningEffort(req.provider, req.model, req.reasoningEffort) };
|
||||
const t0 = performance.now();
|
||||
const shouldPersist = req.persist !== false;
|
||||
const chatId = shouldPersist ? await getChatIdOrCreate(req.chatId) : null;
|
||||
@@ -50,6 +52,7 @@ export async function* runMultiplexStream(req: MultiplexRequest): AsyncGenerator
|
||||
data: {
|
||||
lastUsedProvider: req.provider,
|
||||
lastUsedModel: req.model,
|
||||
reasoningEffort: req.reasoningEffort,
|
||||
},
|
||||
}),
|
||||
prisma.chat.updateMany({
|
||||
@@ -77,6 +80,7 @@ export async function* runMultiplexStream(req: MultiplexRequest): AsyncGenerator
|
||||
userLocation: req.userLocation,
|
||||
temperature: req.temperature,
|
||||
maxTokens: req.maxTokens,
|
||||
reasoningEffort: req.reasoningEffort,
|
||||
logContext: {
|
||||
provider: req.provider,
|
||||
model: req.model,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { ReasoningEffort } from "./reasoning-effort.js";
|
||||
|
||||
export const PROVIDERS = ["openai", "anthropic", "xai", "gemini"] as const;
|
||||
|
||||
export type Provider = (typeof PROVIDERS)[number];
|
||||
@@ -42,6 +44,7 @@ export type MultiplexRequest = {
|
||||
userLocation?: string;
|
||||
temperature?: number;
|
||||
maxTokens?: number;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
};
|
||||
|
||||
export type MultiplexResponse = {
|
||||
|
||||
+41
-11
@@ -16,6 +16,7 @@ import { runMultiplexStream, type StreamEvent } from "./llm/streaming.js";
|
||||
import { getAvailableChatTools, normalizeEnabledChatTools } from "./llm/chat-tools.js";
|
||||
import { getModelCatalogSnapshot } from "./llm/model-catalog.js";
|
||||
import { buildQuickQuestionMultiplexRequest } from "./llm/quick-question.js";
|
||||
import { normalizeReasoningEffort, REASONING_EFFORTS, type ReasoningEffort } from "./llm/reasoning-effort.js";
|
||||
import { openaiClient } from "./llm/providers.js";
|
||||
import { serializeProviderFields } from "./llm/provider-ids.js";
|
||||
import { exaClient } from "./search/exa.js";
|
||||
@@ -24,6 +25,7 @@ import { searchWorkspace } from "./workspace-search.js";
|
||||
import { PROVIDERS, type ChatAttachment } from "./llm/types.js";
|
||||
|
||||
const ProviderSchema = z.enum(PROVIDERS);
|
||||
const ReasoningEffortSchema = z.enum(REASONING_EFFORTS).nullable();
|
||||
const MAX_ADDITIONAL_SYSTEM_PROMPT_CHARS = 12_000;
|
||||
const MAX_FORK_MESSAGE_SNIPPET_CHARS = 48;
|
||||
const EnabledToolsSchema = z.array(z.string().trim().min(1).max(80)).max(20).transform((value) => normalizeEnabledChatTools(value));
|
||||
@@ -209,6 +211,7 @@ const CompletionStreamBody = z
|
||||
userLocation: z.string().trim().min(1).max(200).optional(),
|
||||
temperature: z.number().min(0).max(2).optional(),
|
||||
maxTokens: z.number().int().positive().optional(),
|
||||
reasoningEffort: ReasoningEffortSchema.optional(),
|
||||
})
|
||||
.superRefine((value, ctx) => {
|
||||
if (value.persist === false && value.chatId) {
|
||||
@@ -235,6 +238,7 @@ const QuickQuestionStreamBody = z.object({
|
||||
userLocation: z.string().trim().min(1).max(200).optional(),
|
||||
temperature: z.number().min(0).max(2).optional(),
|
||||
maxTokens: z.number().int().positive().optional(),
|
||||
reasoningEffort: ReasoningEffortSchema.optional(),
|
||||
});
|
||||
|
||||
function mergeAttachmentsIntoMetadata(metadata: unknown, attachments?: ChatAttachment[]) {
|
||||
@@ -269,16 +273,16 @@ function prependAdditionalSystemPrompt<T extends { messages: IncomingChatMessage
|
||||
};
|
||||
}
|
||||
|
||||
async function applyStoredChatSettings<T extends { chatId?: string; messages: IncomingChatMessage[]; additionalSystemPrompt?: string; enabledTools?: string[] }>(
|
||||
async function applyStoredChatSettings<T extends { chatId?: string; provider: string; model: string; messages: IncomingChatMessage[]; additionalSystemPrompt?: string; enabledTools?: string[]; reasoningEffort?: ReasoningEffort | null }>(
|
||||
body: T
|
||||
) {
|
||||
if (!body.chatId || (body.additionalSystemPrompt !== undefined && body.enabledTools !== undefined)) {
|
||||
if (!body.chatId || (body.additionalSystemPrompt !== undefined && body.enabledTools !== undefined && body.reasoningEffort !== undefined)) {
|
||||
return prependAdditionalSystemPrompt(body);
|
||||
}
|
||||
|
||||
const chat = await prisma.chat.findUnique({
|
||||
where: { id: body.chatId },
|
||||
select: { additionalSystemPrompt: true, enabledTools: true },
|
||||
select: { additionalSystemPrompt: true, enabledTools: true, reasoningEffort: true, lastUsedProvider: true, lastUsedModel: true },
|
||||
});
|
||||
if (!chat) return prependAdditionalSystemPrompt(body);
|
||||
|
||||
@@ -286,6 +290,10 @@ async function applyStoredChatSettings<T extends { chatId?: string; messages: In
|
||||
...body,
|
||||
additionalSystemPrompt: body.additionalSystemPrompt ?? chat.additionalSystemPrompt ?? undefined,
|
||||
enabledTools: body.enabledTools ?? normalizeEnabledChatTools(chat.enabledTools),
|
||||
// Omission inherits only for the saved model; null explicitly resets it.
|
||||
reasoningEffort: normalizeReasoningEffort(body.provider, body.model, body.reasoningEffort !== undefined
|
||||
? body.reasoningEffort
|
||||
: chat.lastUsedProvider === body.provider && chat.lastUsedModel === body.model ? chat.reasoningEffort : null),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -556,6 +564,7 @@ const chatSummarySelect = {
|
||||
initiatedModel: true,
|
||||
lastUsedProvider: true,
|
||||
lastUsedModel: true,
|
||||
reasoningEffort: true,
|
||||
additionalSystemPrompt: true,
|
||||
enabledTools: true,
|
||||
projectItems: starredProjectItemsSelect,
|
||||
@@ -1082,7 +1091,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
} : undefined,
|
||||
orderBy: [{ updatedAt: "desc" }, { id: "asc" }],
|
||||
take: limit + 1,
|
||||
select: { id: true, title: true, parentChatId: true, createdAt: true, updatedAt: true, initiatedProvider: true, initiatedModel: true, lastUsedProvider: true, lastUsedModel: true },
|
||||
select: { id: true, title: true, parentChatId: true, createdAt: true, updatedAt: true, initiatedProvider: true, initiatedModel: true, lastUsedProvider: true, lastUsedModel: true, reasoningEffort: true },
|
||||
});
|
||||
const hasMore = chats.length > limit;
|
||||
const page = hasMore ? chats.slice(0, limit) : chats;
|
||||
@@ -1103,6 +1112,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
title: z.string().optional(),
|
||||
provider: ProviderSchema.optional(),
|
||||
model: z.string().trim().min(1).optional(),
|
||||
reasoningEffort: ReasoningEffortSchema.optional(),
|
||||
additionalSystemPrompt: z.string().max(MAX_ADDITIONAL_SYSTEM_PROMPT_CHARS).optional(),
|
||||
enabledTools: EnabledToolsSchema.optional(),
|
||||
messages: z.array(CompletionMessageSchema).optional(),
|
||||
@@ -1133,6 +1143,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
initiatedModel: body.model,
|
||||
lastUsedProvider: body.provider,
|
||||
lastUsedModel: body.model,
|
||||
reasoningEffort: normalizeReasoningEffort(body.provider, body.model, body.reasoningEffort),
|
||||
additionalSystemPrompt: normalizeAdditionalSystemPrompt(body.additionalSystemPrompt),
|
||||
enabledTools: body.enabledTools as any,
|
||||
messages: body.messages?.length
|
||||
@@ -1170,6 +1181,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
initiatedModel: true,
|
||||
lastUsedProvider: true,
|
||||
lastUsedModel: true,
|
||||
reasoningEffort: true,
|
||||
additionalSystemPrompt: true,
|
||||
enabledTools: true,
|
||||
userId: true,
|
||||
@@ -1199,6 +1211,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
initiatedModel: source.initiatedModel,
|
||||
lastUsedProvider: source.lastUsedProvider,
|
||||
lastUsedModel: source.lastUsedModel,
|
||||
reasoningEffort: normalizeReasoningEffort(source.lastUsedProvider, source.lastUsedModel, source.reasoningEffort),
|
||||
additionalSystemPrompt: source.additionalSystemPrompt,
|
||||
enabledTools: (source.enabledTools ?? undefined) as any,
|
||||
userId: source.userId,
|
||||
@@ -1231,11 +1244,16 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
const Params = z.object({ chatId: z.string() });
|
||||
const Body = z.object({
|
||||
title: z.string().trim().min(1).optional(),
|
||||
provider: ProviderSchema.optional(),
|
||||
model: z.string().trim().min(1).optional(),
|
||||
reasoningEffort: ReasoningEffortSchema.optional(),
|
||||
additionalSystemPrompt: z.string().max(MAX_ADDITIONAL_SYSTEM_PROMPT_CHARS).nullable().optional(),
|
||||
enabledTools: EnabledToolsSchema.optional(),
|
||||
});
|
||||
}).refine((value) => Boolean(value.provider) === Boolean(value.model), "provider and model must be supplied together");
|
||||
const { chatId } = Params.parse(req.params);
|
||||
const body = Body.parse(req.body ?? {});
|
||||
const parsed = Body.safeParse(req.body ?? {});
|
||||
if (!parsed.success) return app.httpErrors.badRequest(parsed.error.message);
|
||||
const body = parsed.data;
|
||||
|
||||
const data: Record<string, unknown> = {};
|
||||
if (body.title !== undefined) {
|
||||
@@ -1244,13 +1262,24 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
}
|
||||
if (body.additionalSystemPrompt !== undefined) data.additionalSystemPrompt = normalizeAdditionalSystemPrompt(body.additionalSystemPrompt);
|
||||
if (body.enabledTools !== undefined) data.enabledTools = body.enabledTools;
|
||||
|
||||
const updated = await prisma.chat.updateMany({
|
||||
where: { id: chatId },
|
||||
data: data as any,
|
||||
const updated = await prisma.$transaction(async (tx) => {
|
||||
const current = await tx.chat.findUnique({ where: { id: chatId } });
|
||||
if (!current) return null;
|
||||
const provider = body.provider ?? current.lastUsedProvider;
|
||||
const model = body.model ?? current.lastUsedModel;
|
||||
if (body.provider !== undefined) {
|
||||
data.lastUsedProvider = provider;
|
||||
data.lastUsedModel = model;
|
||||
}
|
||||
if (body.provider !== undefined || body.reasoningEffort !== undefined) {
|
||||
const sameModel = provider === current.lastUsedProvider && model === current.lastUsedModel;
|
||||
data.reasoningEffort = normalizeReasoningEffort(provider, model, body.reasoningEffort !== undefined
|
||||
? body.reasoningEffort : sameModel ? current.reasoningEffort : null);
|
||||
}
|
||||
return tx.chat.update({ where: { id: chatId }, data: data as any });
|
||||
});
|
||||
|
||||
if (updated.count === 0) return app.httpErrors.notFound("chat not found");
|
||||
if (!updated) return app.httpErrors.notFound("chat not found");
|
||||
|
||||
const chat = await getChatSummary(chatId);
|
||||
if (!chat) return app.httpErrors.notFound("chat not found");
|
||||
@@ -1738,6 +1767,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin
|
||||
userLocation: z.string().trim().min(1).max(200).optional(),
|
||||
temperature: z.number().min(0).max(2).optional(),
|
||||
maxTokens: z.number().int().positive().optional(),
|
||||
reasoningEffort: ReasoningEffortSchema.optional(),
|
||||
});
|
||||
|
||||
const parsed = Body.safeParse(req.body);
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import test from "node:test";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Fastify from "fastify";
|
||||
import sensible from "@fastify/sensible";
|
||||
import { getReasoningCatalog, normalizeReasoningEffort } from "../src/llm/reasoning-effort.js";
|
||||
|
||||
const serverRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const databaseDir = mkdtempSync(join(tmpdir(), "sybil-reasoning-"));
|
||||
process.env.DATABASE_URL = `file:${join(databaseDir, "test.db")}`;
|
||||
writeFileSync(join(databaseDir, "test.db"), "");
|
||||
process.env.OPENAI_API_KEY = "test-key";
|
||||
process.env.XAI_API_KEY = "test-key";
|
||||
delete process.env.ADMIN_TOKEN;
|
||||
execFileSync(process.execPath, [join(serverRoot, "node_modules/prisma/build/index.js"), "migrate", "deploy"], {
|
||||
cwd: serverRoot, env: { ...process.env, PRISMA_HIDE_UPDATE_MESSAGE: "1" }, stdio: "pipe",
|
||||
});
|
||||
const [{ registerRoutes }, { prisma }, { getProviderChatAdapter }] = await Promise.all([
|
||||
import("../src/routes.js"), import("../src/db.js"), import("../src/llm/provider-adapters.js"),
|
||||
]);
|
||||
const app = Fastify({ logger: false });
|
||||
await app.register(sensible);
|
||||
await registerRoutes(app);
|
||||
await app.ready();
|
||||
|
||||
const requests: { path: string; body: any }[] = [];
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = async (input, init) => {
|
||||
const url = new URL(input instanceof Request ? input.url : String(input));
|
||||
assert.ok(["api.openai.com", "api.x.ai"].includes(url.hostname), `Unexpected network request: ${url.origin}`);
|
||||
const body = JSON.parse(String(init?.body));
|
||||
requests.push({ path: url.pathname, body });
|
||||
const message = { id: "message", type: "message", role: "assistant", content: [{ type: "output_text", text: "Mock answer." }] };
|
||||
const response = { id: "response", status: "completed", output: [message], output_text: "Mock answer." };
|
||||
if (!body.stream) return Response.json(url.pathname.endsWith("/responses") ? response : {
|
||||
id: "completion", choices: [{ message: { role: "assistant", content: "Mock answer." }, finish_reason: "stop" }],
|
||||
});
|
||||
const events = url.pathname.endsWith("/responses") ? [
|
||||
{ type: "response.output_item.added", item: message, output_index: 0 },
|
||||
{ type: "response.output_text.delta", delta: "Mock answer.", output_index: 0 },
|
||||
{ type: "response.completed", response },
|
||||
] : [{ choices: [{ index: 0, delta: { content: "Mock answer." }, finish_reason: "stop" }] }];
|
||||
return new Response(events.map((event) => `data: ${JSON.stringify(event)}\n\n`).join("") + "data: [DONE]\n\n", {
|
||||
headers: { "Content-Type": "text/event-stream" },
|
||||
});
|
||||
};
|
||||
test.after(async () => {
|
||||
globalThis.fetch = originalFetch;
|
||||
await app.close();
|
||||
await prisma.$disconnect();
|
||||
rmSync(databaseDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("capabilities require an exact verified provider/model/effort combination", () => {
|
||||
assert.equal(normalizeReasoningEffort("openai", "gpt-5.2", "xhigh"), "xhigh");
|
||||
assert.equal(normalizeReasoningEffort("openai", "gpt-5.2-2025-12-11", "none"), "none");
|
||||
assert.equal(normalizeReasoningEffort("openai", "gpt-5", "minimal"), "minimal");
|
||||
for (const [provider, model, effort] of [
|
||||
["openai", "gpt-5.1", "xhigh"], ["openai", "gpt-6.1-sol", "none"],
|
||||
["openai", "gpt-5.2-future", "high"], ["openai", "gpt-5.2-pro", "high"],
|
||||
["openai", "ft:gpt-5.2", "high"], ["openai", "gpt-4.1-mini", "high"],
|
||||
["openai", "chat-latest", "high"], ["xai", "gpt-5.2", "high"],
|
||||
]) assert.equal(normalizeReasoningEffort(provider, model, effort), null);
|
||||
const catalog = getReasoningCatalog("openai");
|
||||
assert.equal(catalog["gpt-5.2"].defaultEffort, "none");
|
||||
assert.equal(catalog["gpt-5.5"].defaultEffort, "medium");
|
||||
assert.deepEqual(getReasoningCatalog("anthropic"), {});
|
||||
catalog["gpt-5.2"].efforts.push("max");
|
||||
assert.equal(normalizeReasoningEffort("openai", "gpt-5.2", "max"), null);
|
||||
});
|
||||
|
||||
test("real SDK serialization uses the right effort field for every OpenAI protocol and stream mode", async () => {
|
||||
for (const enabledTools of [[], ["web_search"]]) {
|
||||
for (const streaming of [false, true]) {
|
||||
for (const reasoningEffort of ["high", "none", null] as const) {
|
||||
const adapter = getProviderChatAdapter("openai");
|
||||
const params = { model: "gpt-5.2", messages: [{ role: "user" as const, content: "Hello" }], enabledTools, reasoningEffort, temperature: 0.2, maxTokens: 1000 };
|
||||
if (streaming) {
|
||||
const events = [];
|
||||
for await (const event of adapter.stream(params)) events.push(event);
|
||||
assert.equal(events.at(-1)?.type, "done");
|
||||
} else assert.equal((await adapter.complete(params)).text, "Mock answer.");
|
||||
const { path, body } = requests.at(-1)!;
|
||||
assert.equal(path, enabledTools.length ? "/v1/responses" : "/v1/chat/completions");
|
||||
if (enabledTools.length) {
|
||||
assert.deepEqual(body.reasoning, reasoningEffort ? { effort: reasoningEffort } : undefined);
|
||||
assert.equal(body.reasoning_effort, undefined);
|
||||
assert.equal(body.max_output_tokens, 1000);
|
||||
} else {
|
||||
assert.equal(body.reasoning_effort, reasoningEffort ?? undefined);
|
||||
assert.equal(body.reasoning, undefined);
|
||||
assert.equal(body.max_completion_tokens, 1000);
|
||||
assert.equal(body.max_tokens, undefined);
|
||||
}
|
||||
assert.equal(body.temperature, reasoningEffort === "high" ? undefined : 0.2);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("unsupported models, unsupported efforts and other providers never receive an effort field", async () => {
|
||||
for (const provider of ["openai", "xai"] as const) {
|
||||
for (const model of ["gpt-4.1-mini", "chat-latest", "gpt-5.1", "gpt-5.2-unknown"]) {
|
||||
for (const enabledTools of [[], ["web_search"]]) {
|
||||
const adapter = getProviderChatAdapter(provider);
|
||||
await adapter.complete({ model, messages: [{ role: "user", content: "Hi" }], enabledTools, reasoningEffort: "xhigh" });
|
||||
const { body } = requests.at(-1)!;
|
||||
assert.equal("reasoning" in body, false);
|
||||
assert.equal("reasoning_effort" in body, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("chat settings persist, survive all summary/detail reads and forks, and reset on a model change", async () => {
|
||||
const created = await app.inject({ method: "POST", url: "/v1/chats", payload: { provider: "openai", model: "gpt-5.2", reasoningEffort: "high" } });
|
||||
assert.equal(created.statusCode, 200, created.body);
|
||||
const chat = created.json().chat;
|
||||
assert.equal(chat.reasoningEffort, "high");
|
||||
for (const url of [`/v1/chats/${chat.id}`, "/v1/chats", "/v1/chats?limit=10", "/v1/workspace-items"]) {
|
||||
const body = (await app.inject({ method: "GET", url })).json();
|
||||
const stored = body.chat ?? (body.chats ?? body.items).find((item: any) => item.id === chat.id);
|
||||
assert.equal(stored.reasoningEffort, "high", url);
|
||||
}
|
||||
const fork = await app.inject({ method: "POST", url: `/v1/chats/${chat.id}/fork`, payload: {} });
|
||||
assert.equal(fork.json().chat.reasoningEffort, "high");
|
||||
const patch = async (payload: object) => app.inject({ method: "PATCH", url: `/v1/chats/${chat.id}`, payload });
|
||||
assert.equal((await patch({ title: "Renamed" })).json().chat.reasoningEffort, "high");
|
||||
const switched = (await patch({ provider: "openai", model: "gpt-5.1" })).json().chat;
|
||||
assert.equal(switched.lastUsedModel, "gpt-5.1");
|
||||
assert.equal(switched.reasoningEffort, null);
|
||||
assert.equal((await patch({ reasoningEffort: "xhigh" })).json().chat.reasoningEffort, null);
|
||||
assert.equal((await patch({ reasoningEffort: "low" })).json().chat.reasoningEffort, "low");
|
||||
assert.equal((await patch({ reasoningEffort: null })).json().chat.reasoningEffort, null);
|
||||
assert.equal((await patch({ model: "gpt-5.2" })).statusCode, 400);
|
||||
assert.equal((await patch({ reasoningEffort: "bogus" })).statusCode, 400);
|
||||
const catalog = (await app.inject({ method: "GET", url: "/v1/models" })).json();
|
||||
assert.deepEqual(catalog.providers.openai.reasoningCapabilities["gpt-5.2"].efforts, ["none", "low", "medium", "high", "xhigh"]);
|
||||
});
|
||||
|
||||
test("saved effort is inherited by older clients, overridden/reset explicitly, and never carried to another model", async () => {
|
||||
for (const streaming of [false, true]) {
|
||||
const chat = await prisma.chat.create({ data: { lastUsedProvider: "openai", lastUsedModel: "gpt-5.2", reasoningEffort: "high", enabledTools: [] } });
|
||||
const send = async (extra: object) => {
|
||||
const response = await app.inject({ method: "POST", url: `/v1/chat-completions${streaming ? "/stream" : ""}`,
|
||||
payload: { chatId: chat.id, provider: "openai", model: "gpt-5.2", messages: [{ role: "user", content: "Hello" }], ...extra } });
|
||||
assert.equal(response.statusCode, 200, response.body);
|
||||
assert.doesNotMatch(response.body, /event: error/);
|
||||
return requests.at(-1)!.body;
|
||||
};
|
||||
assert.equal((await send({})).reasoning_effort, "high");
|
||||
assert.equal((await send({ reasoningEffort: "low" })).reasoning_effort, "low");
|
||||
assert.equal((await prisma.chat.findUniqueOrThrow({ where: { id: chat.id } })).reasoningEffort, "low");
|
||||
assert.equal((await send({ reasoningEffort: null })).reasoning_effort, undefined);
|
||||
await send({ reasoningEffort: "high" });
|
||||
assert.equal((await send({ model: "gpt-5.1" })).reasoning_effort, undefined);
|
||||
assert.equal((await prisma.chat.findUniqueOrThrow({ where: { id: chat.id } })).reasoningEffort, null);
|
||||
}
|
||||
});
|
||||
|
||||
test("Quick Questions accept validated effort without inheriting or persisting chat settings", async () => {
|
||||
const count = await prisma.chat.count();
|
||||
for (const [model, effort, expected] of [["gpt-5.2", "high", "high"], ["gpt-5.1", "xhigh", undefined], ["chat-latest", "high", undefined], ["gpt-5.2", undefined, undefined]] as const) {
|
||||
const response = await app.inject({ method: "POST", url: "/v1/quick-questions/stream", payload: { provider: "openai", model, question: "Hello", enabledTools: [], reasoningEffort: effort } });
|
||||
assert.equal(response.statusCode, 200, response.body);
|
||||
assert.doesNotMatch(response.body, /event: error/);
|
||||
assert.equal(requests.at(-1)!.body.reasoning_effort, expected);
|
||||
}
|
||||
assert.equal(await prisma.chat.count(), count);
|
||||
});
|
||||
+52
-9
@@ -7,6 +7,8 @@ import {
|
||||
type ProviderModelPreferences,
|
||||
} from "@/lib/workspace-domain";
|
||||
import { useEffect, useLayoutEffect, useMemo, useRef, useState } from "preact/hooks";
|
||||
import { ReasoningEffortControl } from "@/components/ReasoningEffortControl";
|
||||
import { getReasoningCapabilities, loadReasoningPreferences, normalizeReasoningEffort, reasoningPreferenceKey, saveReasoningPreferences, type ReasoningEffort } from "@/lib/reasoning-effort";
|
||||
import type { TargetedTouchEvent } from "preact";
|
||||
import { getProviderLabel } from "@/lib/message-generation";
|
||||
import {
|
||||
@@ -690,6 +692,9 @@ export default function App() {
|
||||
return stored.openai ?? PROVIDER_FALLBACK_MODELS.openai[0];
|
||||
});
|
||||
const [quickProvider, setQuickProvider] = useState<Provider>(() => loadStoredQuickQuestionModelSelection().provider);
|
||||
const [reasoningPreferences, setReasoningPreferences] = useState(loadReasoningPreferences);
|
||||
const [reasoningPreferencesSaved, setReasoningPreferencesSaved] = useState(true);
|
||||
const [reasoningEffort, setReasoningEffort] = useState<ReasoningEffort | null>(() => loadReasoningPreferences()[reasoningPreferenceKey(provider, model)] ?? null);
|
||||
const [quickProviderModelPreferences, setQuickProviderModelPreferences] = useState<ProviderModelPreferences>(
|
||||
() => loadStoredQuickQuestionModelSelection().modelPreferences
|
||||
);
|
||||
@@ -719,6 +724,7 @@ export default function App() {
|
||||
const [chatSettingsTitleDraft, setChatSettingsTitleDraft] = useState("");
|
||||
const [chatSettingsProviderDraft, setChatSettingsProviderDraft] = useState<Provider>("openai");
|
||||
const [chatSettingsModelDraft, setChatSettingsModelDraft] = useState("");
|
||||
const [chatSettingsReasoningDraft, setChatSettingsReasoningDraft] = useState<ReasoningEffort | null>(null);
|
||||
const [chatSettingsPromptDraft, setChatSettingsPromptDraft] = useState("");
|
||||
const [chatSettingsEnabledToolsDraft, setChatSettingsEnabledToolsDraft] = useState<string[]>([]);
|
||||
const [additionalSystemPrompt, setAdditionalSystemPrompt] = useState("");
|
||||
@@ -883,6 +889,8 @@ export default function App() {
|
||||
setChatSettingsTitleDraft("");
|
||||
setChatSettingsProviderDraft("openai");
|
||||
setChatSettingsModelDraft("");
|
||||
setChatSettingsReasoningDraft(null);
|
||||
setReasoningEffort(null);
|
||||
setChatSettingsPromptDraft("");
|
||||
setChatSettingsEnabledToolsDraft([]);
|
||||
setAdditionalSystemPrompt("");
|
||||
@@ -1148,6 +1156,11 @@ export default function App() {
|
||||
[chatSettingsProviderDraft, modelCatalog]
|
||||
);
|
||||
const providerOptions = PROVIDERS;
|
||||
const chatSettingsReasoningCapabilities = getReasoningCapabilities(modelCatalog, chatSettingsProviderDraft, chatSettingsModelDraft);
|
||||
|
||||
useEffect(() => {
|
||||
setReasoningPreferencesSaved(saveReasoningPreferences(reasoningPreferences));
|
||||
}, [reasoningPreferences]);
|
||||
|
||||
useEffect(() => {
|
||||
if (model.trim()) return;
|
||||
@@ -1443,6 +1456,7 @@ export default function App() {
|
||||
if (!chat) return;
|
||||
setAdditionalSystemPrompt(chat.additionalSystemPrompt ?? "");
|
||||
setEnabledTools(normalizeEnabledTools(chat.enabledTools, availableChatTools));
|
||||
setReasoningEffort(chat.reasoningEffort ?? null);
|
||||
}, [availableChatTools, draftKind, selectedChat, selectedChatSummary, selectedItem]);
|
||||
|
||||
const selectedTitle = useMemo(() => {
|
||||
@@ -1494,7 +1508,9 @@ export default function App() {
|
||||
setSelectedSearch(null);
|
||||
setPendingAttachments([]);
|
||||
setDraftChatTitle("");
|
||||
setModel(pickProviderModel(getModelOptions(modelCatalog, provider), providerModelPreferences[provider]));
|
||||
const nextModel = pickProviderModel(getModelOptions(modelCatalog, provider), providerModelPreferences[provider]);
|
||||
setModel(nextModel);
|
||||
setReasoningEffort(reasoningPreferences[reasoningPreferenceKey(provider, nextModel)] ?? null);
|
||||
setAdditionalSystemPrompt("");
|
||||
setEnabledTools(getDefaultEnabledTools(availableChatTools));
|
||||
setIsChatSettingsOpen(false);
|
||||
@@ -1751,6 +1767,7 @@ export default function App() {
|
||||
initiatedModel: updatedChat.initiatedModel,
|
||||
lastUsedProvider: updatedChat.lastUsedProvider,
|
||||
lastUsedModel: updatedChat.lastUsedModel,
|
||||
reasoningEffort: updatedChat.reasoningEffort,
|
||||
additionalSystemPrompt: updatedChat.additionalSystemPrompt,
|
||||
enabledTools: updatedChat.enabledTools,
|
||||
};
|
||||
@@ -1810,6 +1827,7 @@ export default function App() {
|
||||
setChatSettingsTitleDraft(getChatSettingsSeedTitle());
|
||||
setChatSettingsProviderDraft(provider);
|
||||
setChatSettingsModelDraft(model);
|
||||
setChatSettingsReasoningDraft(reasoningEffort);
|
||||
setChatSettingsPromptDraft(additionalSystemPrompt);
|
||||
setChatSettingsEnabledToolsDraft(normalizeEnabledTools(enabledTools, availableChatTools));
|
||||
setIsChatSettingsOpen(true);
|
||||
@@ -1822,9 +1840,19 @@ export default function App() {
|
||||
});
|
||||
};
|
||||
|
||||
const commitLocalChatSettings = (nextProvider: Provider, nextModel: string, nextPrompt: string, nextTools: string[], nextTitle: string) => {
|
||||
const changeChatSettingsModel = (nextProvider: Provider, nextModel: string) => {
|
||||
setChatSettingsProviderDraft(nextProvider);
|
||||
setChatSettingsModelDraft(nextModel.trim());
|
||||
setChatSettingsReasoningDraft(nextProvider === provider && nextModel.trim() === model.trim()
|
||||
? reasoningEffort : reasoningPreferences[reasoningPreferenceKey(nextProvider, nextModel)] ?? null);
|
||||
setChatSettingsError(null);
|
||||
};
|
||||
|
||||
const commitLocalChatSettings = (nextProvider: Provider, nextModel: string, nextPrompt: string, nextTools: string[], nextTitle: string, nextEffort: ReasoningEffort | null) => {
|
||||
setProvider(nextProvider);
|
||||
setModel(nextModel);
|
||||
setReasoningEffort(nextEffort);
|
||||
setReasoningPreferences((current) => ({ ...current, [reasoningPreferenceKey(nextProvider, nextModel)]: nextEffort }));
|
||||
setProviderModelPreferences((current) => ({
|
||||
...current,
|
||||
[nextProvider]: nextModel || null,
|
||||
@@ -1853,6 +1881,7 @@ export default function App() {
|
||||
}
|
||||
|
||||
const nextPrompt = chatSettingsPromptDraft.trim();
|
||||
const nextEffort = normalizeReasoningEffort(chatSettingsReasoningCapabilities, chatSettingsReasoningDraft);
|
||||
const nextTools = availableChatTools.length
|
||||
? normalizeEnabledTools(chatSettingsEnabledToolsDraft, availableChatTools)
|
||||
: chatSettingsEnabledToolsDraft;
|
||||
@@ -1864,6 +1893,9 @@ export default function App() {
|
||||
if (isExistingChat) {
|
||||
const updatedChat = await updateChatSettings(existingChatId, {
|
||||
title: nextTitle,
|
||||
provider: chatSettingsProviderDraft,
|
||||
model: nextModel,
|
||||
reasoningEffort: nextEffort,
|
||||
additionalSystemPrompt: nextPrompt || null,
|
||||
...(availableChatTools.length ? { enabledTools: nextTools } : {}),
|
||||
});
|
||||
@@ -1873,7 +1905,7 @@ export default function App() {
|
||||
setDraftKind("chat");
|
||||
}
|
||||
|
||||
commitLocalChatSettings(chatSettingsProviderDraft, nextModel, nextPrompt, nextTools, nextTitle);
|
||||
commitLocalChatSettings(chatSettingsProviderDraft, nextModel, nextPrompt, nextTools, nextTitle, nextEffort);
|
||||
setIsChatSettingsOpen(false);
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
@@ -2219,6 +2251,7 @@ export default function App() {
|
||||
|
||||
const handleSendChat = async (content: string, attachments: ChatAttachment[]): Promise<SidebarSelection> => {
|
||||
const selectedModel = model.trim();
|
||||
const selectedEffort = normalizeReasoningEffort(getReasoningCapabilities(modelCatalog, provider, selectedModel), reasoningEffort);
|
||||
if (!selectedModel) {
|
||||
throw new Error("No model available for selected provider");
|
||||
}
|
||||
@@ -2252,6 +2285,7 @@ export default function App() {
|
||||
...(draftChatTitle.trim() ? { title: draftChatTitle.trim() } : {}),
|
||||
provider,
|
||||
model: selectedModel,
|
||||
reasoningEffort: selectedEffort,
|
||||
...(additionalSystemPrompt.trim() ? { additionalSystemPrompt: additionalSystemPrompt.trim() } : {}),
|
||||
...(initialEnabledTools !== undefined ? { enabledTools: initialEnabledTools } : {}),
|
||||
});
|
||||
@@ -2281,6 +2315,7 @@ export default function App() {
|
||||
initiatedModel: chat.initiatedModel,
|
||||
lastUsedProvider: chat.lastUsedProvider,
|
||||
lastUsedModel: chat.lastUsedModel,
|
||||
reasoningEffort: chat.reasoningEffort,
|
||||
additionalSystemPrompt: chat.additionalSystemPrompt,
|
||||
enabledTools: chat.enabledTools,
|
||||
messages: [],
|
||||
@@ -2343,6 +2378,7 @@ export default function App() {
|
||||
clientRequestId,
|
||||
provider,
|
||||
model: selectedModel,
|
||||
reasoningEffort: selectedEffort,
|
||||
messages: requestMessages,
|
||||
},
|
||||
{
|
||||
@@ -2918,6 +2954,7 @@ export default function App() {
|
||||
initiatedModel: chat.initiatedModel,
|
||||
lastUsedProvider: chat.lastUsedProvider,
|
||||
lastUsedModel: chat.lastUsedModel,
|
||||
reasoningEffort: chat.reasoningEffort,
|
||||
additionalSystemPrompt: chat.additionalSystemPrompt,
|
||||
enabledTools: chat.enabledTools,
|
||||
messages: [],
|
||||
@@ -3080,6 +3117,7 @@ export default function App() {
|
||||
initiatedModel: chat.initiatedModel,
|
||||
lastUsedProvider: chat.lastUsedProvider,
|
||||
lastUsedModel: chat.lastUsedModel,
|
||||
reasoningEffort: chat.reasoningEffort,
|
||||
additionalSystemPrompt: chat.additionalSystemPrompt,
|
||||
enabledTools: chat.enabledTools,
|
||||
messages: [],
|
||||
@@ -3646,9 +3684,10 @@ export default function App() {
|
||||
quickModelsDisabled={isQuickQuestionSending || isConvertingQuickQuestion}
|
||||
onQuickProviderChange={selectQuickProvider}
|
||||
onQuickModelChange={selectQuickModel}
|
||||
preferencesSaved={modelPreferencesSaved && quickPreferencesSaved}
|
||||
preferencesSaved={modelPreferencesSaved && quickPreferencesSaved && reasoningPreferencesSaved}
|
||||
onRetrySave={() => {
|
||||
setModelPreferencesSaved(saveModelPreferences(providerModelPreferences));
|
||||
setReasoningPreferencesSaved(saveReasoningPreferences(reasoningPreferences));
|
||||
setQuickPreferencesSaved(saveQuickQuestionModelSelection({ provider: quickProvider, modelPreferences: quickProviderModelPreferences }));
|
||||
}}
|
||||
/> : null}
|
||||
@@ -3752,10 +3791,8 @@ export default function App() {
|
||||
value={chatSettingsProviderDraft}
|
||||
onChange={(event) => {
|
||||
const nextProvider = event.currentTarget.value as Provider;
|
||||
setChatSettingsProviderDraft(nextProvider);
|
||||
const options = getModelOptions(modelCatalog, nextProvider);
|
||||
setChatSettingsModelDraft(pickProviderModel(options, providerModelPreferences[nextProvider]));
|
||||
setChatSettingsError(null);
|
||||
changeChatSettingsModel(nextProvider, pickProviderModel(options, providerModelPreferences[nextProvider]));
|
||||
}}
|
||||
disabled={isSavingChatSettings}
|
||||
>
|
||||
@@ -3775,13 +3812,19 @@ export default function App() {
|
||||
value={chatSettingsModelDraft}
|
||||
disabled={isSavingChatSettings}
|
||||
onChange={(nextModel) => {
|
||||
setChatSettingsModelDraft(nextModel.trim());
|
||||
setChatSettingsError(null);
|
||||
changeChatSettingsModel(chatSettingsProviderDraft, nextModel);
|
||||
}}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<ReasoningEffortControl
|
||||
capabilities={chatSettingsReasoningCapabilities}
|
||||
value={chatSettingsReasoningDraft}
|
||||
onChange={setChatSettingsReasoningDraft}
|
||||
disabled={isSavingChatSettings}
|
||||
/>
|
||||
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-xs font-semibold text-violet-100/72">Additional system prompt</span>
|
||||
<Textarea
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { normalizeReasoningEffort, reasoningEffortLabel, type ReasoningCapabilities, type ReasoningEffort } from "@/lib/reasoning-effort";
|
||||
|
||||
export function ReasoningEffortControl({ capabilities, value, onChange, disabled }: {
|
||||
capabilities: ReasoningCapabilities | null;
|
||||
value: ReasoningEffort | null;
|
||||
onChange: (effort: ReasoningEffort | null) => void;
|
||||
disabled?: boolean;
|
||||
}) {
|
||||
const effort = normalizeReasoningEffort(capabilities, value);
|
||||
const label = effort ? reasoningEffortLabel(effort)
|
||||
: `Model default${capabilities?.defaultEffort ? ` (${reasoningEffortLabel(capabilities.defaultEffort)})` : ""}`;
|
||||
return (
|
||||
<section className="rounded-lg border border-violet-300/18 bg-background/44 px-3 py-3">
|
||||
<div className="flex flex-wrap items-center justify-between gap-2">
|
||||
<label htmlFor="chat-reasoning-effort" className="text-xs font-semibold text-violet-100/72">Reasoning effort</label>
|
||||
<output htmlFor="chat-reasoning-effort" className="text-sm text-violet-50">{label}</output>
|
||||
</div>
|
||||
{capabilities ? (
|
||||
<>
|
||||
<input
|
||||
id="chat-reasoning-effort"
|
||||
type="range"
|
||||
min={0}
|
||||
max={capabilities.efforts.length}
|
||||
step={1}
|
||||
value={effort ? capabilities.efforts.indexOf(effort) + 1 : 0}
|
||||
onInput={(event) => onChange(capabilities.efforts[Number(event.currentTarget.value) - 1] ?? null)}
|
||||
aria-valuetext={label}
|
||||
aria-describedby="chat-reasoning-help"
|
||||
disabled={disabled}
|
||||
className="mt-2 h-8 w-full cursor-pointer accent-violet-400 disabled:cursor-default disabled:opacity-50"
|
||||
/>
|
||||
<div className="flex justify-between text-xs text-muted-foreground" aria-hidden="true">
|
||||
<span>Model default</span>
|
||||
<span>{reasoningEffortLabel(capabilities.efforts[capabilities.efforts.length - 1])}</span>
|
||||
</div>
|
||||
<p id="chat-reasoning-help" className="mt-2 text-xs text-muted-foreground">Higher effort gives the model more time to reason and can increase response time and cost.</p>
|
||||
</>
|
||||
) : (
|
||||
<p className="mt-2 text-xs text-muted-foreground">Reasoning settings aren’t available for this model. It will use its own defaults.</p>
|
||||
)}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
+10
-1
@@ -1,3 +1,5 @@
|
||||
import type { ReasoningCapabilities, ReasoningEffort } from "./reasoning-effort";
|
||||
|
||||
export type ChatSummary = {
|
||||
id: string;
|
||||
title: string | null;
|
||||
@@ -11,6 +13,7 @@ export type ChatSummary = {
|
||||
initiatedModel: string | null;
|
||||
lastUsedProvider: Provider | null;
|
||||
lastUsedModel: string | null;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
additionalSystemPrompt: string | null;
|
||||
enabledTools: string[] | null;
|
||||
};
|
||||
@@ -72,6 +75,7 @@ export type ChatDetail = {
|
||||
initiatedModel: string | null;
|
||||
lastUsedProvider: Provider | null;
|
||||
lastUsedModel: string | null;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
additionalSystemPrompt: string | null;
|
||||
enabledTools: string[] | null;
|
||||
messages: Message[];
|
||||
@@ -163,6 +167,7 @@ export type ProviderModelInfo = {
|
||||
models: string[];
|
||||
loadedAt: string | null;
|
||||
error: string | null;
|
||||
reasoningCapabilities?: Record<string, ReasoningCapabilities>;
|
||||
};
|
||||
|
||||
export type ModelCatalogResponse = {
|
||||
@@ -207,6 +212,7 @@ type CreateChatRequest = {
|
||||
title?: string;
|
||||
provider?: Provider;
|
||||
model?: string;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
additionalSystemPrompt?: string;
|
||||
enabledTools?: string[];
|
||||
messages?: CompletionRequestMessage[];
|
||||
@@ -345,7 +351,7 @@ export async function updateChatStar(chatId: string, starred: boolean) {
|
||||
|
||||
export async function updateChatSettings(
|
||||
chatId: string,
|
||||
body: { title?: string; additionalSystemPrompt?: string | null; enabledTools?: string[] }
|
||||
body: { title?: string; provider?: Provider; model?: string; reasoningEffort?: ReasoningEffort | null; additionalSystemPrompt?: string | null; enabledTools?: string[] }
|
||||
) {
|
||||
const data = await api<{ chat: ChatSummary }>(`/v1/chats/${chatId}`, {
|
||||
method: "PATCH",
|
||||
@@ -569,6 +575,7 @@ export async function runCompletion(body: {
|
||||
chatId: string;
|
||||
provider: Provider;
|
||||
model: string;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
messages: CompletionRequestMessage[];
|
||||
additionalSystemPrompt?: string;
|
||||
enabledTools?: string[];
|
||||
@@ -587,6 +594,7 @@ export async function runCompletionStream(
|
||||
clientRequestId?: string;
|
||||
provider: Provider;
|
||||
model: string;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
messages: CompletionRequestMessage[];
|
||||
additionalSystemPrompt?: string;
|
||||
enabledTools?: string[];
|
||||
@@ -617,6 +625,7 @@ export async function runQuickQuestionStream(
|
||||
body: {
|
||||
provider: Provider;
|
||||
model: string;
|
||||
reasoningEffort?: ReasoningEffort | null;
|
||||
question: string;
|
||||
},
|
||||
handlers: CompletionStreamHandlers,
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import type { ModelCatalogResponse, Provider } from "./api";
|
||||
|
||||
export const REASONING_EFFORTS = ["none", "minimal", "low", "medium", "high", "xhigh", "max"] as const;
|
||||
export type ReasoningEffort = (typeof REASONING_EFFORTS)[number];
|
||||
export type ReasoningCapabilities = { efforts: ReasoningEffort[]; defaultEffort: ReasoningEffort | null };
|
||||
export type ReasoningPreferences = Record<string, ReasoningEffort | null>;
|
||||
export const REASONING_PREFERENCES_STORAGE_KEY = "sybil:reasoningEffortByModel";
|
||||
|
||||
export function reasoningPreferenceKey(provider: Provider, model: string) {
|
||||
return JSON.stringify([provider, model.trim()]);
|
||||
}
|
||||
|
||||
export function getReasoningCapabilities(catalog: ModelCatalogResponse["providers"], provider: Provider, model: string): ReasoningCapabilities | null {
|
||||
// Missing/old catalogs and unrecognized values must fail closed.
|
||||
const entry = catalog[provider]?.reasoningCapabilities?.[model.trim()];
|
||||
if (!entry || !Array.isArray(entry.efforts) || !entry.efforts.length
|
||||
|| !entry.efforts.every((effort) => REASONING_EFFORTS.includes(effort))
|
||||
|| new Set(entry.efforts).size !== entry.efforts.length
|
||||
|| (entry.defaultEffort !== null && !entry.efforts.includes(entry.defaultEffort))) return null;
|
||||
return entry;
|
||||
}
|
||||
|
||||
export function normalizeReasoningEffort(capabilities: ReasoningCapabilities | null, effort: unknown): ReasoningEffort | null {
|
||||
return capabilities?.efforts.includes(effort as ReasoningEffort) ? effort as ReasoningEffort : null;
|
||||
}
|
||||
|
||||
export function reasoningEffortLabel(effort: ReasoningEffort) {
|
||||
if (effort === "xhigh") return "Extra high";
|
||||
return effort.charAt(0).toUpperCase() + effort.slice(1);
|
||||
}
|
||||
|
||||
export function loadReasoningPreferences(): ReasoningPreferences {
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(window.localStorage.getItem(REASONING_PREFERENCES_STORAGE_KEY) ?? "{}");
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return {};
|
||||
return Object.fromEntries(Object.entries(parsed).filter(([, value]) => value === null || REASONING_EFFORTS.includes(value)));
|
||||
} catch { return {}; }
|
||||
}
|
||||
|
||||
export function saveReasoningPreferences(preferences: ReasoningPreferences) {
|
||||
try { window.localStorage.setItem(REASONING_PREFERENCES_STORAGE_KEY, JSON.stringify(preferences)); return true; }
|
||||
catch { return false; }
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// Run against a local production preview; all API/socket traffic is mocked.
|
||||
const { chromium } = await import(process.env.PLAYWRIGHT_MODULE || "playwright-core");
|
||||
const target = new URL(process.env.WEB_TEST_URL || "http://127.0.0.1:5197/");
|
||||
assert.ok(target.protocol === "http:" && ["127.0.0.1", "localhost"].includes(target.hostname));
|
||||
const browser = await chromium.launch({ headless: true, executablePath: process.env.CHROME_EXECUTABLE || undefined,
|
||||
...(process.env.BROWSER_NO_SANDBOX === "1" ? { args: ["--no-sandbox"] } : {}) });
|
||||
const date = "2026-11-15T12:00:00Z";
|
||||
const capabilities = {
|
||||
"gpt-5.2": { efforts: ["none", "low", "medium", "high", "xhigh"], defaultEffort: "none" },
|
||||
"gpt-6.1-sol": { efforts: ["low", "medium", "high", "xhigh", "max"], defaultEffort: "medium" },
|
||||
};
|
||||
|
||||
async function fixture(viewport, withCapabilities = true) {
|
||||
const context = await browser.newContext({ viewport, isMobile: viewport.width < 768, hasTouch: viewport.width < 768 });
|
||||
await context.addInitScript(() => {
|
||||
localStorage.setItem("sybil_admin_token", "fixture-token");
|
||||
localStorage.setItem("sybil:ui-mode", "full");
|
||||
});
|
||||
const chat = { id: "reasoning-chat", title: "Reasoning fixture", createdAt: date, updatedAt: date,
|
||||
parentChatId: null, titleGenerationPending: false, starred: false, starredAt: null,
|
||||
initiatedProvider: "openai", initiatedModel: "gpt-5.2", lastUsedProvider: "openai", lastUsedModel: "gpt-5.2",
|
||||
additionalSystemPrompt: null, enabledTools: [], reasoningEffort: null, messages: [] };
|
||||
const chats = new Map([[chat.id, chat]]);
|
||||
const writes = [], errors = [];
|
||||
await context.route("**/*", async route => {
|
||||
const request = route.request(), url = new URL(request.url()), path = url.pathname;
|
||||
if (url.origin !== target.origin) return route.abort();
|
||||
if (!path.startsWith("/api/")) return route.continue();
|
||||
const body = request.method() === "GET" ? null : request.postDataJSON();
|
||||
if (body) writes.push({ path, body });
|
||||
let value = {};
|
||||
const id = path.match(/\/chats\/([^/]+)$/)?.[1];
|
||||
if (path.endsWith("/auth/session")) value = { mode: "token", authenticated: true };
|
||||
else if (path.endsWith("/workspace-items")) value = { items: [...chats.values()].map(chat => ({ ...chat, type: "chat" })) };
|
||||
else if (path.endsWith("/models")) value = { providers: { openai: { models: Object.keys(capabilities), loadedAt: date, error: null,
|
||||
...(withCapabilities ? { reasoningCapabilities: capabilities } : {}) }, anthropic: { models: ["claude-fixture"], reasoningCapabilities: {} } } };
|
||||
else if (path.endsWith("/chat-tools") || path.endsWith("/agent/tools")) value = { tools: [] };
|
||||
else if (path.endsWith("/active-runs")) value = { chats: [], searches: [] };
|
||||
else if (path.endsWith("/agent/messages")) value = { conversationId: "1", messages: [], cursor: "0", beforeCursor: null, hasMore: false };
|
||||
else if (path.endsWith("/agent")) value = { id: "1", title: "Sybil Agent", activity: { state: "idle", isThinking: false, tools: [], queuedMessages: [] }, protocol: "pi-room.v1", websocket: "/v1/agent/ws" };
|
||||
else if (path.endsWith("/read-state")) value = { sourceId: "fixture", cursor: "0", updatedAt: null };
|
||||
else if (path.endsWith("/push/config")) value = { enabled: false };
|
||||
else if (path.endsWith("/chats") && body) {
|
||||
const created = { ...chat, id: `new-${chats.size}`, title: body.title || "New chat", initiatedProvider: body.provider,
|
||||
initiatedModel: body.model, lastUsedProvider: body.provider, lastUsedModel: body.model, reasoningEffort: body.reasoningEffort ?? null };
|
||||
chats.set(created.id, created);
|
||||
value = { chat: created };
|
||||
} else if (id && chats.has(id)) {
|
||||
const current = chats.get(id);
|
||||
if (request.method() === "PATCH") Object.assign(current, body, { lastUsedProvider: body.provider ?? current.lastUsedProvider, lastUsedModel: body.model ?? current.lastUsedModel });
|
||||
value = { chat: current };
|
||||
} else if (path.endsWith("/chat-completions/stream")) {
|
||||
const current = chats.get(body.chatId);
|
||||
current.messages = [{ id: "answer", role: "assistant", content: "Fixture answer.", createdAt: date, metadata: null }];
|
||||
return route.fulfill({ contentType: "text/event-stream", body: [
|
||||
["meta", { chatId: current.id, callId: "call", provider: body.provider, model: body.model }],
|
||||
["delta", { text: "Fixture answer." }], ["done", { text: "Fixture answer." }],
|
||||
].map(([event, data]) => `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`).join("") });
|
||||
}
|
||||
await route.fulfill({ json: value });
|
||||
});
|
||||
await context.routeWebSocket("**/api/v1/agent/ws", socket => socket.onMessage(raw => {
|
||||
if (JSON.parse(String(raw)).type === "auth") socket.send(JSON.stringify({ type: "ready", conversationId: "1" }));
|
||||
}));
|
||||
const page = await context.newPage();
|
||||
page.on("pageerror", error => errors.push(error.message));
|
||||
await page.goto(new URL(`/?chat=${chat.id}`, target).href, { waitUntil: "networkidle" });
|
||||
await page.locator("main h1").filter({ hasText: chat.title }).waitFor();
|
||||
const dialog = page.getByRole("dialog", { name: "Chat settings", exact: true });
|
||||
const slider = dialog.getByRole("slider", { name: "Reasoning effort" });
|
||||
const open = async () => { await page.getByRole("button", { name: "Open chat settings", exact: true }).click(); await dialog.waitFor(); };
|
||||
const changeModel = async model => { const input = dialog.getByRole("combobox", { name: "Model", exact: true }); await input.fill(model); await input.press("Enter"); };
|
||||
const save = async () => { await dialog.getByRole("button", { name: "Save", exact: true }).click(); await dialog.waitFor({ state: "hidden" }); };
|
||||
return { context, page, dialog, slider, open, changeModel, save, chat, writes, errors };
|
||||
}
|
||||
|
||||
try {
|
||||
for (const viewport of [{ width: 1280, height: 900 }, { width: 390, height: 844 }]) {
|
||||
const f = await fixture(viewport);
|
||||
const { page, dialog, slider, chat, writes } = f;
|
||||
await f.open();
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "Model default (None)");
|
||||
await slider.press("End");
|
||||
await slider.press("ArrowLeft");
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "High");
|
||||
await dialog.getByRole("button", { name: "Cancel", exact: true }).click();
|
||||
assert.equal(writes.filter(write => write.path.endsWith(`/chats/${chat.id}`)).length, 0);
|
||||
await f.open();
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "Model default (None)");
|
||||
await slider.press("End");
|
||||
await slider.press("ArrowLeft");
|
||||
await f.save();
|
||||
assert.equal(chat.reasoningEffort, "high");
|
||||
assert.equal(writes.at(-1).body.model, "gpt-5.2");
|
||||
await page.reload({ waitUntil: "networkidle" });
|
||||
await f.open();
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "High");
|
||||
await f.changeModel("gpt-6.1-sol");
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "Model default (Medium)");
|
||||
await slider.press("Home");
|
||||
await slider.press("ArrowRight");
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "Low"); // no unsupported None/Minimal
|
||||
await f.changeModel("gpt-5.2-pro");
|
||||
assert.equal(await slider.count(), 0);
|
||||
await dialog.getByText(/Reasoning settings aren’t available/).waitFor();
|
||||
await f.changeModel("gpt-5.2");
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "High");
|
||||
await dialog.getByRole("combobox", { name: "Provider", exact: true }).selectOption("anthropic");
|
||||
assert.equal(await slider.count(), 0);
|
||||
await dialog.getByRole("button", { name: "Cancel", exact: true }).click();
|
||||
await page.keyboard.press("Control+j");
|
||||
await f.open();
|
||||
assert.equal(await slider.getAttribute("aria-valuetext"), "High");
|
||||
const bounds = await dialog.boundingBox();
|
||||
assert.ok(bounds.x >= 0 && bounds.x + bounds.width <= viewport.width + 1);
|
||||
if (process.env.REASONING_SCREENSHOT_DIR) await page.screenshot({ path: `${process.env.REASONING_SCREENSHOT_DIR}/reasoning-${viewport.width}.png` });
|
||||
await f.save();
|
||||
const composer = page.locator("#composer-input");
|
||||
await composer.fill("Test remembered effort");
|
||||
const submitted = page.waitForResponse(response => response.url().endsWith("/chat-completions/stream"));
|
||||
await composer.press("Enter");
|
||||
await submitted;
|
||||
assert.equal(writes.findLast(write => write.path.endsWith("/chats")).body.reasoningEffort, "high");
|
||||
assert.equal(writes.findLast(write => write.path.endsWith("/chat-completions/stream")).body.reasoningEffort, "high");
|
||||
await page.getByText("Fixture answer.", { exact: true }).waitFor();
|
||||
await f.open();
|
||||
await slider.press("Home");
|
||||
await f.save();
|
||||
assert.equal(writes.findLast(write => /\/chats\/[^/]+$/.test(write.path)).body.reasoningEffort, null);
|
||||
assert.deepEqual(f.errors, []);
|
||||
await f.context.close();
|
||||
}
|
||||
const oldServer = await fixture({ width: 1280, height: 900 }, false);
|
||||
await oldServer.open();
|
||||
assert.equal(await oldServer.slider.count(), 0);
|
||||
await oldServer.dialog.getByText(/Reasoning settings aren’t available/).waitFor();
|
||||
await oldServer.save();
|
||||
assert.equal(oldServer.chat.reasoningEffort, null);
|
||||
assert.deepEqual(oldServer.errors, []);
|
||||
await oldServer.context.close();
|
||||
console.log("Reasoning settings browser checks passed (desktop, mobile, missing capabilities).");
|
||||
} finally { await browser.close(); }
|
||||
@@ -0,0 +1,39 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { getReasoningCapabilities, loadReasoningPreferences, normalizeReasoningEffort, reasoningPreferenceKey, saveReasoningPreferences } from "../src/lib/reasoning-effort.ts";
|
||||
|
||||
const capabilities = { efforts: ["none", "low", "medium", "high", "xhigh"], defaultEffort: "none" };
|
||||
const catalog = { openai: { reasoningCapabilities: { "gpt-5.2": capabilities } } };
|
||||
|
||||
test("capabilities come only from the selected provider/model and a valid catalog entry", () => {
|
||||
assert.deepEqual(getReasoningCapabilities(catalog, "openai", " gpt-5.2 "), capabilities);
|
||||
for (const [data, provider, model] of [[{}, "openai", "gpt-5.2"], [catalog, "xai", "gpt-5.2"], [catalog, "openai", "gpt-5.2-new"]]) {
|
||||
assert.equal(getReasoningCapabilities(data, provider, model), null);
|
||||
}
|
||||
for (const entry of [{}, { efforts: [] }, { efforts: ["future"], defaultEffort: "future" }, { efforts: ["high"], defaultEffort: "none" }]) {
|
||||
assert.equal(getReasoningCapabilities({ openai: { reasoningCapabilities: { model: entry } } }, "openai", "model"), null);
|
||||
}
|
||||
assert.equal(normalizeReasoningEffort(capabilities, "xhigh"), "xhigh");
|
||||
assert.equal(normalizeReasoningEffort(capabilities, "max"), null);
|
||||
assert.equal(normalizeReasoningEffort(null, "high"), null);
|
||||
assert.equal(normalizeReasoningEffort(capabilities, null), null);
|
||||
});
|
||||
|
||||
test("browser memory is per provider/model and survives corrupt or unavailable storage", (t) => {
|
||||
const original = globalThis.window;
|
||||
t.after(() => { globalThis.window = original; });
|
||||
let stored = "{}";
|
||||
globalThis.window = { localStorage: { getItem: () => stored, setItem: (_, value) => { stored = value; } } };
|
||||
const key = reasoningPreferenceKey("openai", "gpt-5.2");
|
||||
assert.notEqual(key, reasoningPreferenceKey("xai", "gpt-5.2"));
|
||||
assert.notEqual(key, reasoningPreferenceKey("openai", "gpt-5.1"));
|
||||
assert.equal(saveReasoningPreferences({ [key]: "high" }), true);
|
||||
assert.deepEqual(loadReasoningPreferences(), { [key]: "high" });
|
||||
stored = JSON.stringify({ [key]: "future", other: null });
|
||||
assert.deepEqual(loadReasoningPreferences(), { other: null });
|
||||
stored = "broken JSON";
|
||||
assert.deepEqual(loadReasoningPreferences(), {});
|
||||
globalThis.window = { get localStorage() { throw new Error("blocked"); } };
|
||||
assert.deepEqual(loadReasoningPreferences(), {});
|
||||
assert.equal(saveReasoningPreferences({ [key]: "high" }), false);
|
||||
});
|
||||
Reference in New Issue
Block a user