Compare commits

...
40 Commits
Author SHA1 Message Date
buzzert cb5a973ac7 Preserve provider changes before sending 2026-07-26 17:44:45 -07:00
buzzert e2443162b0 Preserve thread selection during background updates 2026-07-26 17:37:39 -07:00
buzzert 048456b8a5 Preserve draft views during collection refreshes 2026-07-25 17:14:39 -07:00
buzzert 211fad972a avoid mobile focus on conversation selection 2026-07-23 20:29:53 -07:00
buzzert a69c641481 add mobile workspace gestures 2026-07-23 18:05:57 -07:00
buzzert 9229896ad7 harden PWA resume and title fallback 2026-07-23 17:29:02 -07:00
buzzert c5217b2710 make PWA chat resume idempotent 2026-07-23 17:12:11 -07:00
buzzert abc1124d27 fix for PWA background/foreground 2026-07-23 16:32:20 -07:00
buzzert 4721717022 server: respect Brave search rate limits 2026-07-19 21:56:35 -07:00
buzzert 87b7d9502f server: add Brave search provider 2026-07-19 21:35:47 -07:00
buzzert 1952f4f358 web: open links in chat in new tab 2026-07-19 16:08:38 -07:00
buzzert 622659f6ca ios: preserve chat scroll position on resume
TestFlight / testflight (push) Successful in 1m46s
2026-07-12 13:57:45 -07:00
buzzert 93ca8a76c3 adds gemini support
TestFlight / testflight (push) Successful in 1m53s
2026-07-11 14:16:21 -07:00
buzzertandClaude Fable 5 69f50064a3 ci: restore Ruby PATH export for fastlane step
TestFlight / testflight (push) Successful in 1m58s
act_runner does not carry setup-ruby's PATH changes into later steps,
so without this the step runs the toolcache default Ruby (3.3.11) and
bundler cannot find the gems installed for 3.1.7.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 13:04:47 -07:00
buzzertandClaude Fable 5 e7b81f24bd ci: put CI keychain first in codesign search list
TestFlight / testflight (push) Failing after 19s
codesign resolves signing identities through the user keychain search
list (first match wins) and ignores --keychain for the lookup. This
runner hosts another project whose keychain holds the same Apple
Distribution identity, so when that keychain is locked, codesign fails
with errSecInternalComponent no matter how correctly our own keychain
is set up. Prepend the fresh CI keychain to the search list for the
build and always delete it afterward, which restores the original list.

Also drop the runner-diagnostics step, the GitHub-hosted Ruby PATH
export, and the duplicate CODE_SIGN_IDENTITY param.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 13:01:42 -07:00
buzzert 20a310a4b1 ci: use Ruby 3.1 for TestFlight upload
TestFlight / testflight (push) Failing after 46s
2026-07-11 12:01:27 -07:00
buzzert b6859706db ios: simplify CI signing keychain setup
TestFlight / testflight (push) Failing after 25s
2026-07-11 11:55:04 -07:00
buzzert ea148839d3 Revert "ios: pass match keychain to codesign"
This reverts commit a0e410155d.
2026-07-11 11:32:00 -07:00
buzzert a0e410155d ios: pass match keychain to codesign
TestFlight / testflight (push) Failing after 55s
2026-07-11 11:29:10 -07:00
buzzert b5a5d21767 docs: document iOS release workflow 2026-07-11 11:17:05 -07:00
buzzert 0ad96f4f99 ios: center transcript content on iPad
TestFlight / testflight (push) Failing after 52s
2026-07-11 11:15:01 -07:00
buzzert b0c2b74c16 remove temporary keychain (unneeded)
TestFlight / testflight (push) Successful in 1m46s
2026-07-06 09:25:24 -07:00
buzzert 14d2d8cb28 fix spacing between lists and paragraphs 2026-07-05 19:54:34 -07:00
buzzert 03836bbc4c Raise Anthropic default max tokens 2026-07-05 11:53:17 -07:00
buzzert 76ce62025a bullets in ul 2026-07-04 20:22:13 -07:00
buzzert b15473d24e ios: stamp release version into XcodeGen spec
TestFlight / testflight (push) Successful in 1m43s
2026-06-26 01:29:54 -07:00
buzzert a512a65844 ios: accept release tag namespace in Fastfile
TestFlight / testflight (push) Successful in 1m55s
2026-06-26 01:18:02 -07:00
buzzert 207b44f67f ios: ci: actually release/ios/v*.
TestFlight / testflight (push) Failing after 19s
2026-06-26 01:15:08 -07:00
buzzert c5ccd212c9 ios: ci: testflight only on release/v*
TestFlight / testflight (push) Successful in 1m55s
2026-06-26 01:08:47 -07:00
buzzertandClaude Opus 4.8 ee990dde5d ios: simplify Fastfile signing now that the runner has a real session
With SessionCreate on the runner's launchd job, standard fastlane keychain
handling works, so drop the debugging-era workarounds: the manual
default-keychain / list-keychains search-list juggling, the login-keychain
restoration in cleanup, the verify_ci_signing re-unlock/partition/find-identity
step (match already imports the cert and sets the key partition list), and the
CODE_SIGN_KEYCHAIN / OTHER_CODE_SIGN_FLAGS xcargs. CI signing is now a single
create_keychain + match. No behavior change; validated end-to-end on TestFlight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:52:36 -07:00
buzzertandClaude Opus 4.8 036743fe41 ios: source TestFlight build number from CI run number
TestFlight / testflight (push) Successful in 1m53s
App Store Connect requires CFBundleVersion to be unique and strictly
increasing app-wide. latest_testflight_build_number could return a stale
value (it missed an existing build 13) and produced colliding build
numbers, 409-ing every upload. Use the monotonic Gitea run number
(github.run_number -> SYBIL_BUILD_NUMBER) instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:41:37 -07:00
buzzertandClaude Opus 4.8 4e7ea7ab68 ios: get TestFlight CI signing working
TestFlight / testflight (push) Failing after 57s
Replace the old testflight-release workflow with a single
`testflight.yml` Gitea Actions workflow and rework the fastlane `beta`
lane: match-based app-store signing into a disposable CI keychain,
XcodeGen project generation, version/build-number bumping, and upload
to TestFlight. Pin Ruby to 3.1.7 for the runner.

Squashes the iterative CI signing debugging history into one commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:28:27 -07:00
buzzert f232013e5a ios: ci: deploy via fastlane
TestFlight Release / testflight (push) Failing after 9s
2026-06-25 19:30:58 -07:00
buzzert 27c425f664 supposedly better tool call animation 2026-06-14 19:10:56 -07:00
buzzert 297b053a91 big backend refactor 2026-06-13 12:02:22 -07:00
buzzert 7436544a69 ios: add tool call stacking 2026-06-12 00:26:21 -07:00
buzzert 95796646b1 web: tool stacking ui 2026-06-12 00:09:44 -07:00
buzzert d7214c88ad fix most web_fetches from getting blocked using a real user agent 2026-06-11 23:36:19 -07:00
buzzert 22aa652257 Fix iOS chat scroll pinning 2026-06-07 19:58:04 -07:00
buzzert 8f6e8c17a5 ios: add fastlane 2026-06-05 23:19:14 -07:00
63 changed files with 5767 additions and 1581 deletions
+44
View File
@@ -0,0 +1,44 @@
name: TestFlight
on:
workflow_dispatch:
push:
tags:
- "release/ios/v*"
jobs:
testflight:
runs-on: macos-arm64
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: "3.1.7"
bundler-cache: true
working-directory: ios
- name: Install XcodeGen
run: command -v xcodegen >/dev/null 2>&1 || brew install xcodegen
- name: Upload to TestFlight
working-directory: ios
env:
APP_STORE_CONNECT_KEY_ID: ${{ secrets.APP_STORE_CONNECT_KEY_ID }}
APP_STORE_CONNECT_ISSUER_ID: ${{ secrets.APP_STORE_CONNECT_ISSUER_ID }}
APP_STORE_CONNECT_KEY_CONTENT: ${{ secrets.APP_STORE_CONNECT_KEY_CONTENT }}
MATCH_PASSWORD: ${{ secrets.MATCH_PASSWORD }}
MATCH_GIT_URL: ${{ secrets.MATCH_GIT_URL }}
MATCH_GIT_BASIC_AUTHORIZATION: ${{ secrets.MATCH_GIT_BASIC_AUTHORIZATION }}
SYBIL_BUILD_NUMBER: ${{ github.run_number }}
FASTLANE_SKIP_UPDATE_CHECK: "1"
FASTLANE_XCODEBUILD_SETTINGS_TIMEOUT: "120"
# act_runner does not propagate setup-ruby's PATH changes into later
# steps, so put the selected Ruby back on PATH or `bundle` resolves to
# the toolcache default and misses the gems installed above.
run: |
export PATH="/Users/runner/hostedtoolcache/Ruby/3.1.7/arm64/bin:${PATH}"
bundle exec fastlane ios beta
+2 -1
View File
@@ -1,2 +1,3 @@
.env .env
ios/fastlane/README.md
ios/fastlane/report.xml
+26
View File
@@ -12,17 +12,43 @@ server {
location /api/ { location /api/ {
proxy_pass http://server:8787/; proxy_pass http://server:8787/;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Connection "";
}
location = /sw.js {
add_header Cache-Control "no-store, no-cache, must-revalidate" always;
expires -1;
try_files $uri =404;
} }
location = /manifest.webmanifest { location = /manifest.webmanifest {
default_type application/manifest+json; default_type application/manifest+json;
add_header Cache-Control "no-store, no-cache, must-revalidate" always;
expires -1;
try_files $uri =404;
}
location = /index.html {
add_header Cache-Control "no-store, no-cache, must-revalidate" always;
expires -1;
try_files $uri =404;
}
location /assets/ {
add_header Cache-Control "public, max-age=31536000, immutable" always;
try_files $uri =404; try_files $uri =404;
} }
location / { location / {
add_header Cache-Control "no-store, no-cache, must-revalidate" always;
expires -1;
try_files $uri $uri/ /index.html; try_files $uri $uri/ /index.html;
} }
} }
+2
View File
@@ -12,10 +12,12 @@ services:
OPENAI_API_KEY: ${OPENAI_API_KEY:-} OPENAI_API_KEY: ${OPENAI_API_KEY:-}
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-} ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
XAI_API_KEY: ${XAI_API_KEY:-} XAI_API_KEY: ${XAI_API_KEY:-}
GEMINI_API_KEY: ${GEMINI_API_KEY:-}
HERMES_AGENT_API_BASE_URL: ${HERMES_AGENT_API_BASE_URL:-http://127.0.0.1:8642/v1} HERMES_AGENT_API_BASE_URL: ${HERMES_AGENT_API_BASE_URL:-http://127.0.0.1:8642/v1}
HERMES_AGENT_API_KEY: ${HERMES_AGENT_API_KEY:-} HERMES_AGENT_API_KEY: ${HERMES_AGENT_API_KEY:-}
HERMES_AGENT_MODEL: ${HERMES_AGENT_MODEL:-} HERMES_AGENT_MODEL: ${HERMES_AGENT_MODEL:-}
EXA_API_KEY: ${EXA_API_KEY:-} EXA_API_KEY: ${EXA_API_KEY:-}
BRAVE_SEARCH_API_KEY: ${BRAVE_SEARCH_API_KEY:-}
CHAT_WEB_SEARCH_ENGINE: ${CHAT_WEB_SEARCH_ENGINE:-exa} CHAT_WEB_SEARCH_ENGINE: ${CHAT_WEB_SEARCH_ENGINE:-exa}
SEARXNG_BASE_URL: ${SEARXNG_BASE_URL:-} SEARXNG_BASE_URL: ${SEARXNG_BASE_URL:-}
CHAT_MAX_TOOL_ROUNDS: ${CHAT_MAX_TOOL_ROUNDS:-100} CHAT_MAX_TOOL_ROUNDS: ${CHAT_MAX_TOOL_ROUNDS:-100}
+28 -11
View File
@@ -34,11 +34,13 @@ Chat upload limits:
"openai": { "models": ["gpt-4.1-mini"], "loadedAt": "2026-02-14T00:00:00.000Z", "error": null }, "openai": { "models": ["gpt-4.1-mini"], "loadedAt": "2026-02-14T00:00:00.000Z", "error": null },
"anthropic": { "models": ["claude-3-5-sonnet-latest"], "loadedAt": null, "error": null }, "anthropic": { "models": ["claude-3-5-sonnet-latest"], "loadedAt": null, "error": null },
"xai": { "models": ["grok-3-mini"], "loadedAt": null, "error": null }, "xai": { "models": ["grok-3-mini"], "loadedAt": null, "error": null },
"gemini": { "models": ["gemini-3.5-flash"], "loadedAt": null, "error": null },
"hermes-agent": { "models": ["hermes-agent"], "loadedAt": null, "error": null } "hermes-agent": { "models": ["hermes-agent"], "loadedAt": null, "error": null }
} }
} }
``` ```
- OpenAI model lists are filtered to models that are expected to work with the backend's Responses API implementation. - OpenAI model lists are filtered to models that are expected to work with the backend's Responses API implementation.
- Gemini model lists are loaded from Google's native Models API and filtered to Gemini `generateContent` model ids.
- `hermes-agent` is included only when `HERMES_AGENT_API_KEY` is configured. Set it to Hermes `API_SERVER_KEY`, or any non-empty value if that local server does not require auth. `HERMES_AGENT_API_BASE_URL` defaults to `http://127.0.0.1:8642/v1`; set `HERMES_AGENT_MODEL` only when you need an additional fallback/override model id. - `hermes-agent` is included only when `HERMES_AGENT_API_KEY` is configured. Set it to Hermes `API_SERVER_KEY`, or any non-empty value if that local server does not require auth. `HERMES_AGENT_API_BASE_URL` defaults to `http://127.0.0.1:8642/v1`; set `HERMES_AGENT_MODEL` only when you need an additional fallback/override model id.
- The backend loads provider model lists at startup and refreshes them about once every 24 hours. If a later provider refresh fails, the response keeps the last loaded model list for that provider and sets `error` to the latest failure message. - The backend loads provider model lists at startup and refreshes them about once every 24 hours. If a later provider refresh fails, the response keeps the last loaded model list for that provider and sets `error` to the latest failure message.
@@ -56,7 +58,7 @@ Chat upload limits:
``` ```
Behavior notes: Behavior notes:
- Lists Sybil-managed chat tools that can be enabled for `openai` and `xai` chat completions. - Lists Sybil-managed chat tools that can be enabled for `openai`, `anthropic`, `xai`, and `gemini` chat completions.
- Optional tools such as `codex_exec` and `shell_exec` appear only when enabled by server environment configuration. - Optional tools such as `codex_exec` and `shell_exec` appear only when enabled by server environment configuration.
## Active Runs ## Active Runs
@@ -128,7 +130,7 @@ Behavior notes:
```json ```json
{ {
"title": "optional title", "title": "optional title",
"provider": "optional openai|anthropic|xai|hermes-agent", "provider": "optional openai|anthropic|xai|gemini|hermes-agent",
"model": "optional model id", "model": "optional model id",
"additionalSystemPrompt": "optional stored system prompt", "additionalSystemPrompt": "optional stored system prompt",
"enabledTools": ["web_search", "fetch_url"], "enabledTools": ["web_search", "fetch_url"],
@@ -184,6 +186,7 @@ Behavior notes:
- If the chat already has a non-empty title, server returns the existing chat unchanged. - If the chat already has a non-empty title, server returns the existing chat unchanged.
- If a title is set while suggestion generation is in flight, server returns the current chat instead of overwriting that title. - If a title is set while suggestion generation is in flight, server returns the current chat instead of overwriting that title.
- When no title exists at write time, server uses OpenAI `gpt-4.1-mini` to generate a one-line title (up to ~4 words), updates the chat title, and returns the updated chat. - When no title exists at write time, server uses OpenAI `gpt-4.1-mini` to generate a one-line title (up to ~4 words), updates the chat title, and returns the updated chat.
- If the title provider is unavailable or rejects the request, server still persists a deterministic title derived from the first line of `content` instead of leaving the chat untitled.
### `DELETE /v1/chats/:chatId` ### `DELETE /v1/chats/:chatId`
- Response: `{ "deleted": true }` - Response: `{ "deleted": true }`
@@ -234,7 +237,7 @@ Notes:
```json ```json
{ {
"chatId": "optional-chat-id", "chatId": "optional-chat-id",
"provider": "openai|anthropic|xai|hermes-agent", "provider": "openai|anthropic|xai|gemini|hermes-agent",
"model": "string", "model": "string",
"messages": [ "messages": [
{ {
@@ -285,21 +288,26 @@ Behavior notes:
- For `chatId` calls, server stores only *new* non-assistant messages from provided history to avoid duplicates. - For `chatId` calls, server stores only *new* non-assistant messages from provided history to avoid duplicates.
- `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint. - `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint.
- `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools. - `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
- `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap.
- Server persists final assistant output and call metadata (`LlmCall`) in DB. - Server persists final assistant output and call metadata (`LlmCall`) in DB.
- Server updates chat-level model metadata on each call: `lastUsedProvider`/`lastUsedModel`; first successful/failed call also initializes `initiatedProvider`/`initiatedModel` if unset. - Server updates chat-level model metadata on each call: `lastUsedProvider`/`lastUsedModel`; first successful/failed call also initializes `initiatedProvider`/`initiatedModel` if unset.
- Attachments are optional and currently apply to `user` messages. Persisted chat history stores them under `message.metadata.attachments`. - Attachments are optional and currently apply to `user` messages. Persisted chat history stores them under `message.metadata.attachments`.
- Images are forwarded inline to providers as multimodal image parts. Use PNG or JPEG for cross-provider compatibility. - Images are forwarded inline to providers as multimodal image parts. Use PNG or JPEG for cross-provider compatibility.
- Text files are forwarded as explicit text blocks rather than provider-managed file references. Large text attachments should already be truncated client-side before submission. - Text files are forwarded as explicit text blocks rather than provider-managed file references. Large text attachments should already be truncated client-side before submission.
- For `openai`, backend calls OpenAI's Responses API and enables internal tool use with an internal system instruction. - For `openai`, backend calls OpenAI's Responses API and enables internal tool use with an internal system instruction.
- For `anthropic`, backend calls Anthropic's Messages API and enables internal tool use with Anthropic `tool_use`/`tool_result` content blocks.
- For `xai`, backend calls xAI's OpenAI-compatible Chat Completions API and enables internal tool use with the same internal system instruction. - For `xai`, backend calls xAI's OpenAI-compatible Chat Completions API and enables internal tool use with the same internal system instruction.
- For `gemini`, backend calls Google's native Gemini `generateContent` API and enables internal tool use with Gemini function calling.
- For `hermes-agent`, backend calls the configured Hermes Agent OpenAI-compatible Chat Completions API without adding Sybil-managed tool definitions; Hermes Agent handles its own tools server-side. - For `hermes-agent`, backend calls the configured Hermes Agent OpenAI-compatible Chat Completions API without adding Sybil-managed tool definitions; Hermes Agent handles its own tools server-side.
- For `openai`, image attachments are sent as Responses `input_image` items and text attachments are sent as `input_text` items. - For `openai`, image attachments are sent as Responses `input_image` items and text attachments are sent as `input_text` items.
- For `gemini`, image attachments are sent as native Gemini `inlineData` parts and text attachments are sent as text parts.
- For `xai` and `hermes-agent`, image attachments are sent as Chat Completions content parts alongside text. - For `xai` and `hermes-agent`, image attachments are sent as Chat Completions content parts alongside text.
- For `openai`, Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds. - For `openai`, Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds.
- For `anthropic`, image attachments are sent as Messages API `image` blocks using base64 source data; text attachments are added as `text` blocks. - For `anthropic`, image attachments are sent as Messages API `image` blocks using base64 source data; text attachments are added as `text` blocks.
- Available Sybil-managed tool calls for `openai` and `xai`: `web_search` and `fetch_url`. When `CHAT_CODEX_TOOL_ENABLED=true`, `codex_exec` is also available. When `CHAT_SHELL_TOOL_ENABLED=true`, `shell_exec` is also available. - Available Sybil-managed tool calls for `openai`, `anthropic`, `xai`, and `gemini`: `web_search` and `fetch_url`. When `CHAT_CODEX_TOOL_ENABLED=true`, `codex_exec` is also available. When `CHAT_SHELL_TOOL_ENABLED=true`, `shell_exec` is also available.
- `web_search` returns ranked results with per-result summaries/snippets. Its backend engine is selected by `CHAT_WEB_SEARCH_ENGINE` (`exa` default, or `searxng` with `SEARXNG_BASE_URL` set). SearXNG mode requires the instance to allow `format=json`. - `web_search` returns ranked results with per-result summaries/snippets. Its backend engine is selected by `CHAT_WEB_SEARCH_ENGINE`: `exa` (default), `brave` (requires `BRAVE_SEARCH_API_KEY`), or `searxng` (requires `SEARXNG_BASE_URL`; the instance must allow `format=json`).
- `fetch_url` fetches a URL and returns plaintext page content (HTML converted to text server-side). - Brave searches are queued and evenly paced according to the shortest window in Brave's `X-RateLimit-Policy` response header. The backend also honors `X-RateLimit-Remaining`/`X-RateLimit-Reset` and retries `429` responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately.
- `fetch_url` fetches a URL with browser-like navigation headers and returns plaintext page content (HTML converted to text server-side).
- `codex_exec` delegates coding, shell, repository inspection, and other complex software tasks to a persistent remote Codex CLI workspace over SSH. The server runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt>` on the configured devbox inside `CHAT_CODEX_REMOTE_WORKDIR`, with SSH stdin closed. - `codex_exec` delegates coding, shell, repository inspection, and other complex software tasks to a persistent remote Codex CLI workspace over SSH. The server runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt>` on the configured devbox inside `CHAT_CODEX_REMOTE_WORKDIR`, with SSH stdin closed.
- `shell_exec` runs arbitrary non-interactive shell commands on the same configured devbox, starting in `CHAT_CODEX_REMOTE_WORKDIR`. It uses `bash -lc` when bash exists, otherwise `sh -lc`, closes SSH stdin, and does not run inside the Sybil server container. - `shell_exec` runs arbitrary non-interactive shell commands on the same configured devbox, starting in `CHAT_CODEX_REMOTE_WORKDIR`. It uses `bash -lc` when bash exists, otherwise `sh -lc`, closes SSH stdin, and does not run inside the Sybil server container.
- Devbox tool configuration: - Devbox tool configuration:
@@ -315,7 +323,16 @@ Behavior notes:
- `CHAT_CODEX_EXEC_TIMEOUT_MS=600000` (optional) - `CHAT_CODEX_EXEC_TIMEOUT_MS=600000` (optional)
- `CHAT_SHELL_EXEC_TIMEOUT_MS=120000` (optional) - `CHAT_SHELL_EXEC_TIMEOUT_MS=120000` (optional)
- When a tool call is executed, backend stores a chat `Message` with `role: "tool"` and tool metadata (`metadata.kind = "tool_call"`). Streaming requests emit an initiated SSE `tool_call` event before execution, then persist each completed or failed tool call as its terminal SSE `tool_call` event is emitted, then store the assistant output when the completion finishes. - When a tool call is executed, backend stores a chat `Message` with `role: "tool"` and tool metadata (`metadata.kind = "tool_call"`). Streaming requests emit an initiated SSE `tool_call` event before execution, then persist each completed or failed tool call as its terminal SSE `tool_call` event is emitted, then store the assistant output when the completion finishes.
- `anthropic` currently runs without server-managed tool calls.
## Streaming Chat
### `POST /v1/chat-completions/stream`
- The request accepts the chat-completion fields above plus optional `persist` and `clientRequestId` fields.
- `clientRequestId` is only valid for a persisted request with a `chatId`, may be up to 128 characters, and should be a stable unique value generated once per user submission.
- Retrying with the same `chatId` and `clientRequestId` replays the matching active or completed stream rather than starting a duplicate provider call.
- The server persists the ID in `metadata.clientRequestId` on the submitted user message and completed assistant message.
- The complete request, SSE event, persistence, retry, and attach contracts are defined in `docs/api/streaming-chat.md`.
## Searches ## Searches
@@ -416,9 +433,9 @@ Behavior notes:
"updatedAt": "...", "updatedAt": "...",
"starred": false, "starred": false,
"starredAt": null, "starredAt": null,
"initiatedProvider": "openai|anthropic|xai|hermes-agent|null", "initiatedProvider": "openai|anthropic|xai|gemini|hermes-agent|null",
"initiatedModel": "string|null", "initiatedModel": "string|null",
"lastUsedProvider": "openai|anthropic|xai|hermes-agent|null", "lastUsedProvider": "openai|anthropic|xai|gemini|hermes-agent|null",
"lastUsedModel": "string|null", "lastUsedModel": "string|null",
"additionalSystemPrompt": null, "additionalSystemPrompt": null,
"enabledTools": ["web_search", "fetch_url"] "enabledTools": ["web_search", "fetch_url"]
@@ -468,9 +485,9 @@ Behavior notes:
"updatedAt": "...", "updatedAt": "...",
"starred": false, "starred": false,
"starredAt": null, "starredAt": null,
"initiatedProvider": "openai|anthropic|xai|hermes-agent|null", "initiatedProvider": "openai|anthropic|xai|gemini|hermes-agent|null",
"initiatedModel": "string|null", "initiatedModel": "string|null",
"lastUsedProvider": "openai|anthropic|xai|hermes-agent|null", "lastUsedProvider": "openai|anthropic|xai|gemini|hermes-agent|null",
"lastUsedModel": "string|null", "lastUsedModel": "string|null",
"additionalSystemPrompt": null, "additionalSystemPrompt": null,
"enabledTools": ["web_search", "fetch_url"], "enabledTools": ["web_search", "fetch_url"],
+15 -5
View File
@@ -21,7 +21,8 @@ Authentication:
{ {
"chatId": "optional-chat-id", "chatId": "optional-chat-id",
"persist": true, "persist": true,
"provider": "openai|anthropic|xai|hermes-agent", "clientRequestId": "optional-client-generated-id",
"provider": "openai|anthropic|xai|gemini|hermes-agent",
"model": "string", "model": "string",
"messages": [ "messages": [
{ {
@@ -61,15 +62,19 @@ Notes:
- If `persist` is `true` and `chatId` is omitted, backend creates a new chat. - If `persist` is `true` and `chatId` is omitted, backend creates a new chat.
- If `chatId` is provided, backend validates it exists. - If `chatId` is provided, backend validates it exists.
- If `persist` is `false`, `chatId` must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or `LlmCall` metadata. - If `persist` is `false`, `chatId` must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or `LlmCall` metadata.
- `clientRequestId` is optional and is only valid for a persisted stream with a `chatId`. Clients should generate one stable, unique value per user submission and reuse it when retrying a disconnected request.
- A retry with the same `chatId` and `clientRequestId` attaches to and replays the matching active stream. If that submission already completed, the endpoint replays `meta` and `done` without invoking the provider again. This makes retrying the initial streaming `POST` idempotent.
- `clientRequestId` values may be up to 128 characters. The server stores the value in `metadata.clientRequestId` on the submitted user message and completed assistant message.
- For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates. - For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates.
- `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint. - `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint.
- `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools. - `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
- `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap.
- Attachments are optional and are persisted under `message.metadata.attachments` on stored user messages when `persist` is `true`. - Attachments are optional and are persisted under `message.metadata.attachments` on stored user messages when `persist` is `true`.
Persisted chat streams with a `chatId` are backend-owned active runs: Persisted chat streams with a `chatId` are backend-owned active runs:
- Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes. - Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes.
- While running, `GET /v1/active-runs` includes the `chatId`. - While running, `GET /v1/active-runs` includes the `chatId`.
- Starting a second persisted stream for the same active `chatId` returns `409`. - Starting a second persisted stream for the same active `chatId` returns `409`, unless its `clientRequestId` matches the active submission, in which case the existing stream is replayed.
- Clients can reattach with `POST /v1/chats/:chatId/stream/attach`. - Clients can reattach with `POST /v1/chats/:chatId/stream/attach`.
## Attach Endpoint ## Attach Endpoint
@@ -171,18 +176,23 @@ Terminal tool-call event:
## Provider Streaming Behavior ## Provider Streaming Behavior
- `openai`: backend uses OpenAI's Responses API and may execute internal function tool calls (`web_search`, `fetch_url`, optional `codex_exec`, and optional `shell_exec`) before producing final text. - `openai`: backend uses OpenAI's Responses API and may execute internal function tool calls (`web_search`, `fetch_url`, optional `codex_exec`, and optional `shell_exec`) before producing final text.
- `anthropic`: backend uses Anthropic's Messages API and may execute the same internal tools with `tool_use`/`tool_result` content blocks before producing final text.
- `xai`: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text. - `xai`: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text.
- `gemini`: backend uses Google's native Gemini `streamGenerateContent` API and may execute the same internal tool calls before producing final text.
- `fetch_url` sends browser-like navigation headers for outbound URL requests to reduce false 403s from sites that reject generic server clients.
- `hermes-agent`: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract. - `hermes-agent`: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract.
- `openai`: image attachments are sent as Responses `input_image` items; text attachments are sent as `input_text` items. - `openai`: image attachments are sent as Responses `input_image` items; text attachments are sent as `input_text` items.
- `gemini`: image attachments are sent as native Gemini `inlineData` parts; text attachments are inlined as text parts.
- `xai` and `hermes-agent`: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts. - `xai` and `hermes-agent`: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts.
- `openai`: Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds. - `openai`: Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds.
- `anthropic`: streamed via event stream; emits `delta` from `content_block_delta` with `text_delta`. Image attachments are sent as base64 `image` blocks and text attachments are appended as `text` blocks. - `anthropic`: streamed via event stream; emits `delta` from `content_block_delta` with `text_delta`, and emits normalized `tool_call` SSE events when Anthropic `tool_use` blocks are executed. Image attachments are sent as base64 `image` blocks and text attachments are appended as `text` blocks.
- `web_search` uses `CHAT_WEB_SEARCH_ENGINE` (`exa` default, or `searxng` with `SEARXNG_BASE_URL` set). SearXNG mode requires the instance to allow `format=json`. This only affects chat-mode tool calls, not search-mode endpoints. - `web_search` uses `CHAT_WEB_SEARCH_ENGINE`: `exa` (default), `brave` (requires `BRAVE_SEARCH_API_KEY`), or `searxng` (requires `SEARXNG_BASE_URL`; the instance must allow `format=json`). This only affects chat-mode tool calls, not search-mode endpoints.
- Brave searches are queued and evenly paced according to the shortest window in Brave's `X-RateLimit-Policy` response header. The backend also honors `X-RateLimit-Remaining`/`X-RateLimit-Reset` and retries `429` responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately.
- `codex_exec` is available only when `CHAT_CODEX_TOOL_ENABLED=true`. It SSHes to `CHAT_CODEX_REMOTE_HOST`, creates/uses `CHAT_CODEX_REMOTE_WORKDIR`, and runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt>` there with SSH stdin closed. Prefer `CHAT_CODEX_SSH_KEY_PATH` with a read-only mounted private key; `CHAT_CODEX_SSH_PRIVATE_KEY_B64` is also supported. - `codex_exec` is available only when `CHAT_CODEX_TOOL_ENABLED=true`. It SSHes to `CHAT_CODEX_REMOTE_HOST`, creates/uses `CHAT_CODEX_REMOTE_WORKDIR`, and runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt>` there with SSH stdin closed. Prefer `CHAT_CODEX_SSH_KEY_PATH` with a read-only mounted private key; `CHAT_CODEX_SSH_PRIVATE_KEY_B64` is also supported.
- `shell_exec` is available only when `CHAT_SHELL_TOOL_ENABLED=true`. It uses the same devbox SSH configuration, starts in `CHAT_CODEX_REMOTE_WORKDIR`, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container. - `shell_exec` is available only when `CHAT_SHELL_TOOL_ENABLED=true`. It uses the same devbox SSH configuration, starts in `CHAT_CODEX_REMOTE_WORKDIR`, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container.
- `CHAT_MAX_TOOL_ROUNDS` controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100. - `CHAT_MAX_TOOL_ROUNDS` controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100.
Tool-enabled streaming notes (`openai`/`xai`): Tool-enabled streaming notes (`openai`/`anthropic`/`xai`/`gemini`):
- Stream still emits standard `meta`, `delta`, `done|error` events. - Stream still emits standard `meta`, `delta`, `done|error` events.
- Stream may emit `tool_call` events while tool calls are executed. - Stream may emit `tool_call` events while tool calls are executed.
- `delta` events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls. - `delta` events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls.
+24
View File
@@ -0,0 +1,24 @@
FASTLANE_APP_IDENTIFIER=net.buzzert.sybil2
FASTLANE_TEAM_ID=DQQH5H6GBD
FASTLANE_SKIP_UPDATE_CHECK=1
FASTLANE_HIDE_CHANGELOG=1
SYBIL_APP_STORE_APPLE_ID=6759442828
SYBIL_PROVIDER_PUBLIC_ID=c043d167-ad88-4036-84ea-76c223f1b1b2
SYBIL_PROVISIONING_PROFILE_SPECIFIER=Sybil AppStore CI
SYBIL_PROVISIONING_PROFILE_UUID=
SYBIL_CODE_SIGN_IDENTITY=Apple Distribution: James Magahern (DQQH5H6GBD)
SYBIL_XCODE_CODE_SIGN_IDENTITY=6B74B268C4761720FB2051D01D8BB3E47B55D9F5
SYBIL_EXPORT_SIGNING_CERTIFICATE=Apple Distribution
SYBIL_SIGNING_CERTIFICATE_ID=
SYBIL_SIGNING_KEYCHAIN=
# App Store Connect API key settings for TestFlight upload and signing setup.
APP_STORE_CONNECT_API_KEY_ID=
APP_STORE_CONNECT_API_ISSUER_ID=
APP_STORE_CONNECT_API_KEY_PATH=
APP_STORE_CONNECT_API_KEY_CONTENT=
APP_STORE_CONNECT_API_KEY_CONTENT_BASE64=false
# Optional deployment overrides.
SYBIL_BUILD_NUMBER=
SYBIL_VERSION_TAG=
+10 -1
View File
@@ -1,2 +1,11 @@
*.xcodeproj *.xcodeproj
.env
.env.*
!.env.example
build/
*.ipa
*.dSYM.zip
fastlane/report.xml
fastlane/Preview.html
fastlane/screenshots/
fastlane/test_output/
+7
View File
@@ -21,6 +21,12 @@ Instructions for work under `/Users/buzzert/src/sybil-2/ios`.
- To choose a screenshot path, run `just screenshot path=build/name.png`. - To choose a screenshot path, run `just screenshot path=build/name.png`.
- The underlying screenshot command is `xcrun simctl io booted screenshot <path>` and requires a booted simulator. - The underlying screenshot command is `xcrun simctl io booted screenshot <path>` and requires a booted simulator.
## Release Workflow
- iOS release tags use the annotated tag namespace `release/ios/vX.Y.Z`; increment from the latest existing `release/ios/v*` tag.
- Tag message convention is `ios: X.Y.Z`, for example `git tag -a release/ios/v1.13.5 -m "ios: 1.13.5"`.
- Push the release commit and tag together with `git push origin <branch> release/ios/vX.Y.Z`.
- Fastlane derives the marketing version from the release tag and stamps `ios/Apps/Sybil/project.yml` during CI, so do not manually bump `MARKETING_VERSION` for normal tagged releases unless explicitly requested.
## App Structure ## App Structure
- App target entry: `/Users/buzzert/src/sybil-2/ios/Apps/Sybil/Sources/SybilApp.swift` - App target entry: `/Users/buzzert/src/sybil-2/ios/Apps/Sybil/Sources/SybilApp.swift`
- Shared iOS app code lives in Swift package: - Shared iOS app code lives in Swift package:
@@ -51,4 +57,5 @@ Instructions for work under `/Users/buzzert/src/sybil-2/ios`.
- OpenAI: `gpt-4.1-mini` - OpenAI: `gpt-4.1-mini`
- Anthropic: `claude-3-5-sonnet-latest` - Anthropic: `claude-3-5-sonnet-latest`
- xAI: `grok-3-mini` - xAI: `grok-3-mini`
- Gemini: `gemini-3.5-flash`
- Hermes Agent: `hermes-agent` - Hermes Agent: `hermes-agent`
+8 -2
View File
@@ -24,14 +24,20 @@ targets:
GENERATE_INFOPLIST_FILE: YES GENERATE_INFOPLIST_FILE: YES
INFOPLIST_FILE: Apps/Sybil/Info.plist INFOPLIST_FILE: Apps/Sybil/Info.plist
ASSETCATALOG_COMPILER_APPICON_NAME: AppIcon ASSETCATALOG_COMPILER_APPICON_NAME: AppIcon
MARKETING_VERSION: 1.9 MARKETING_VERSION: "1.13.2"
CURRENT_PROJECT_VERSION: 10 CURRENT_PROJECT_VERSION: 11
INFOPLIST_KEY_CFBundleDisplayName: Sybil INFOPLIST_KEY_CFBundleDisplayName: Sybil
INFOPLIST_KEY_ITSAppUsesNonExemptEncryption: NO INFOPLIST_KEY_ITSAppUsesNonExemptEncryption: NO
INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents: YES INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents: YES
INFOPLIST_KEY_UILaunchScreen_Generation: YES INFOPLIST_KEY_UILaunchScreen_Generation: YES
INFOPLIST_KEY_UISupportedInterfaceOrientations_iPhone: UIInterfaceOrientationPortrait INFOPLIST_KEY_UISupportedInterfaceOrientations_iPhone: UIInterfaceOrientationPortrait
INFOPLIST_KEY_UISupportedInterfaceOrientations_iPad: UIInterfaceOrientationPortrait UIInterfaceOrientationPortraitUpsideDown UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight INFOPLIST_KEY_UISupportedInterfaceOrientations_iPad: UIInterfaceOrientationPortrait UIInterfaceOrientationPortraitUpsideDown UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight
configs:
Release:
CODE_SIGN_STYLE: Manual
CODE_SIGN_IDENTITY: Apple Distribution
"CODE_SIGN_IDENTITY[sdk=iphoneos*]": Apple Distribution
PROVISIONING_PROFILE_SPECIFIER: Sybil AppStore CI
schemes: schemes:
Sybil: Sybil:
+3
View File
@@ -0,0 +1,3 @@
source "https://rubygems.org"
gem "fastlane"
+231
View File
@@ -0,0 +1,231 @@
GEM
remote: https://rubygems.org/
specs:
CFPropertyList (3.0.9)
abbrev (0.1.2)
addressable (2.9.0)
public_suffix (>= 2.0.2, < 8.0)
artifactory (3.0.17)
atomos (0.1.3)
aws-eventstream (1.3.2)
aws-partitions (1.1109.0)
aws-sdk-core (3.224.1)
aws-eventstream (~> 1, >= 1.3.0)
aws-partitions (~> 1, >= 1.992.0)
aws-sigv4 (~> 1.9)
base64
jmespath (~> 1, >= 1.6.1)
logger
aws-sdk-kms (1.101.0)
aws-sdk-core (~> 3, >= 3.216.0)
aws-sigv4 (~> 1.5)
aws-sdk-s3 (1.188.0)
aws-sdk-core (~> 3, >= 3.224.1)
aws-sdk-kms (~> 1)
aws-sigv4 (~> 1.5)
aws-sigv4 (1.11.0)
aws-eventstream (~> 1, >= 1.0.2)
babosa (1.0.4)
base64 (0.2.0)
claide (1.1.0)
colored (1.2)
colored2 (3.1.2)
commander (4.6.0)
highline (~> 2.0.0)
csv (3.3.5)
declarative (0.0.20)
digest-crc (0.7.0)
rake (>= 12.0.0, < 14.0.0)
domain_name (0.5.20190701)
unf (>= 0.0.5, < 1.0.0)
dotenv (2.8.1)
emoji_regex (3.2.3)
excon (0.109.0)
faraday (1.10.6)
faraday-em_http (~> 1.0)
faraday-em_synchrony (~> 1.0)
faraday-excon (~> 1.1)
faraday-httpclient (~> 1.0)
faraday-multipart (~> 1.0)
faraday-net_http (~> 1.0)
faraday-net_http_persistent (~> 1.0)
faraday-patron (~> 1.0)
faraday-rack (~> 1.0)
faraday-retry (~> 1.0)
ruby2_keywords (>= 0.0.4)
faraday-cookie_jar (0.0.8)
faraday (>= 0.8.0)
http-cookie (>= 1.0.0)
faraday-em_http (1.0.0)
faraday-em_synchrony (1.0.1)
faraday-excon (1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.2.0)
multipart-post (~> 2.0)
faraday-net_http (1.0.2)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
faraday-rack (1.0.0)
faraday-retry (1.0.4)
faraday_middleware (1.2.1)
faraday (~> 1.0)
fastimage (2.4.1)
fastlane (2.230.0)
CFPropertyList (>= 2.3, < 4.0.0)
abbrev (~> 0.1.2)
addressable (>= 2.8, < 3.0.0)
artifactory (~> 3.0)
aws-sdk-s3 (~> 1.0)
babosa (>= 1.0.3, < 2.0.0)
base64 (~> 0.2.0)
bundler (>= 1.12.0, < 3.0.0)
colored (~> 1.2)
commander (~> 4.6)
csv (~> 3.3)
dotenv (>= 2.1.1, < 3.0.0)
emoji_regex (>= 0.1, < 4.0)
excon (>= 0.71.0, < 1.0.0)
faraday (~> 1.0)
faraday-cookie_jar (~> 0.0.6)
faraday_middleware (~> 1.0)
fastimage (>= 2.1.0, < 3.0.0)
fastlane-sirp (>= 1.0.0)
gh_inspector (>= 1.1.2, < 2.0.0)
google-apis-androidpublisher_v3 (~> 0.3)
google-apis-playcustomapp_v1 (~> 0.1)
google-cloud-env (>= 1.6.0, < 2.0.0)
google-cloud-storage (~> 1.31)
highline (~> 2.0)
http-cookie (~> 1.0.5)
json (< 3.0.0)
jwt (>= 2.1.0, < 3)
logger (>= 1.6, < 2.0)
mini_magick (>= 4.9.4, < 5.0.0)
multipart-post (>= 2.0.0, < 3.0.0)
mutex_m (~> 0.3.0)
naturally (~> 2.2)
nkf (~> 0.2.0)
optparse (>= 0.1.1, < 1.0.0)
plist (>= 3.1.0, < 4.0.0)
rubyzip (>= 2.0.0, < 3.0.0)
security (= 0.1.5)
simctl (~> 1.6.3)
terminal-notifier (>= 2.0.0, < 3.0.0)
terminal-table (~> 3)
tty-screen (>= 0.6.3, < 1.0.0)
tty-spinner (>= 0.8.0, < 1.0.0)
word_wrap (~> 1.0.0)
xcodeproj (>= 1.13.0, < 2.0.0)
xcpretty (~> 0.4.1)
xcpretty-travis-formatter (>= 0.0.3, < 2.0.0)
fastlane-sirp (1.1.0)
gh_inspector (1.1.3)
google-apis-androidpublisher_v3 (0.54.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-core (0.11.3)
addressable (~> 2.5, >= 2.5.1)
googleauth (>= 0.16.2, < 2.a)
httpclient (>= 2.8.1, < 3.a)
mini_mime (~> 1.0)
representable (~> 3.0)
retriable (>= 2.0, < 4.a)
rexml
google-apis-iamcredentials_v1 (0.17.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-playcustomapp_v1 (0.13.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-storage_v1 (0.29.0)
google-apis-core (>= 0.11.0, < 2.a)
google-cloud-core (1.6.1)
google-cloud-env (>= 1.0, < 3.a)
google-cloud-errors (~> 1.0)
google-cloud-env (1.6.0)
faraday (>= 0.17.3, < 3.0)
google-cloud-errors (1.3.1)
google-cloud-storage (1.45.0)
addressable (~> 2.8)
digest-crc (~> 0.4)
google-apis-iamcredentials_v1 (~> 0.1)
google-apis-storage_v1 (~> 0.29.0)
google-cloud-core (~> 1.6)
googleauth (>= 0.16.2, < 2.a)
mini_mime (~> 1.0)
googleauth (1.8.1)
faraday (>= 0.17.3, < 3.a)
jwt (>= 1.4, < 3.0)
multi_json (~> 1.11)
os (>= 0.9, < 2.0)
signet (>= 0.16, < 2.a)
highline (2.0.3)
http-cookie (1.0.8)
domain_name (~> 0.5)
httpclient (2.9.0)
mutex_m
jmespath (1.6.2)
json (2.7.6)
jwt (2.10.3)
base64
logger (1.7.0)
mini_magick (4.13.2)
mini_mime (1.1.5)
multi_json (1.15.0)
multipart-post (2.4.1)
mutex_m (0.3.0)
nanaimo (0.4.0)
naturally (2.3.0)
nkf (0.2.0)
optparse (0.8.1)
os (1.1.4)
plist (3.7.2)
public_suffix (5.1.1)
rake (13.4.2)
representable (3.2.0)
declarative (< 0.1.0)
trailblazer-option (>= 0.1.1, < 0.2.0)
uber (< 0.2.0)
retriable (3.8.0)
rexml (3.4.4)
rouge (3.28.0)
ruby2_keywords (0.0.5)
rubyzip (2.4.1)
security (0.1.5)
signet (0.18.0)
addressable (~> 2.8)
faraday (>= 0.17.5, < 3.a)
jwt (>= 1.5, < 3.0)
multi_json (~> 1.10)
simctl (1.6.10)
CFPropertyList
naturally
terminal-notifier (2.0.0)
terminal-table (3.0.2)
unicode-display_width (>= 1.1.1, < 3)
trailblazer-option (0.1.2)
tty-cursor (0.7.1)
tty-screen (0.8.2)
tty-spinner (0.9.3)
tty-cursor (~> 0.7)
uber (0.1.0)
unf (0.2.0)
unicode-display_width (2.6.0)
word_wrap (1.0.0)
xcodeproj (1.27.0)
CFPropertyList (>= 2.3.3, < 4.0)
atomos (~> 0.1.3)
claide (>= 1.0.2, < 2.0)
colored2 (~> 3.1)
nanaimo (~> 0.4.0)
rexml (>= 3.3.6, < 4.0)
xcpretty (0.4.1)
rouge (~> 3.28.0)
xcpretty-travis-formatter (1.0.1)
xcpretty (~> 0.2, >= 0.0.7)
PLATFORMS
ruby
DEPENDENCIES
fastlane
BUNDLED WITH
2.5.23
@@ -7,39 +7,134 @@ struct SybilChatTranscriptView: View {
var isSending: Bool var isSending: Bool
var topContentInset: CGFloat = 0 var topContentInset: CGFloat = 0
var bottomContentInset: CGFloat = 0 var bottomContentInset: CGFloat = 0
var bottomPinRequestID: Int = 0
private var hasPendingAssistant: Bool { @State private var hasTrackedToolCallMessages = false
messages.contains { message in @State private var knownToolCallMessageIDs: Set<String> = []
message.id.hasPrefix("temp-assistant-") && message.content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
private let bottomAnchorID = "sybil-chat-transcript-bottom-anchor"
private var renderItems: [TranscriptRenderItem] {
buildTranscriptRenderItems(from: messages)
} }
private var toolCallMessageIDs: Set<String> {
Set(messages.compactMap { $0.toolCallMetadata == nil ? nil : $0.id })
}
private var enteringToolCallMessageIDs: Set<String> {
guard hasTrackedToolCallMessages else { return [] }
return toolCallMessageIDs.subtracting(knownToolCallMessageIDs)
}
private var toolCallMessageIDSignature: String {
toolCallMessageIDs.sorted().joined(separator: "|")
} }
var body: some View { var body: some View {
ScrollViewReader { proxy in
ScrollView { ScrollView {
LazyVStack(alignment: .leading, spacing: 26) { LazyVStack(alignment: .leading, spacing: 26) {
ForEach(messages.reversed()) { message in
MessageBubble(message: message, isSending: isSending)
.frame(maxWidth: .infinity)
.scaleEffect(x: 1, y: -1)
}
if isLoading && messages.isEmpty { if isLoading && messages.isEmpty {
Text("Loading messages…") Text("Loading messages…")
.font(.sybil(.footnote)) .font(.sybil(.footnote))
.foregroundStyle(SybilTheme.textMuted) .foregroundStyle(SybilTheme.textMuted)
.padding(.top, 24) .padding(.top, 24)
.scaleEffect(x: 1, y: -1) }
ForEach(renderItems) { item in
switch item {
case let .message(message):
MessageBubble(message: message, isSending: isSending)
.frame(maxWidth: .infinity)
case let .toolGroup(id, messages):
ToolCallStackView(
groupID: id,
messages: messages,
entryAnimationIDs: enteringToolCallMessageIDs
)
.frame(maxWidth: .infinity)
.id(id)
} }
} }
.frame(maxWidth: .infinity, alignment: .leading)
Color.clear
.frame(height: 18 + bottomContentInset)
.id(bottomAnchorID)
}
.frame(maxWidth: SybilLayout.webContentMaxWidth, alignment: .leading)
.frame(maxWidth: .infinity, alignment: .center)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.top, 18 + bottomContentInset) .padding(.top, 18 + topContentInset)
.padding(.bottom, 18 + topContentInset)
} }
.frame(maxWidth: .infinity, alignment: .leading) .frame(maxWidth: .infinity, alignment: .leading)
.scrollDismissesKeyboard(.interactively) .scrollDismissesKeyboard(.interactively)
.scaleEffect(x: 1, y: -1) .onAppear {
syncKnownToolCallMessageIDs()
} }
.onChange(of: toolCallMessageIDSignature) { _, _ in
syncKnownToolCallMessageIDs()
}
.onChange(of: bottomPinRequestID) { _, _ in
scrollToBottom(with: proxy, animated: true)
}
}
}
private func scrollToBottom(with proxy: ScrollViewProxy, animated: Bool) {
let action = {
proxy.scrollTo(bottomAnchorID, anchor: .bottom)
}
if animated {
withAnimation(.easeOut(duration: 0.18), action)
} else {
action()
}
}
private func syncKnownToolCallMessageIDs() {
guard !toolCallMessageIDs.isEmpty else { return }
knownToolCallMessageIDs.formUnion(toolCallMessageIDs)
hasTrackedToolCallMessages = true
}
}
enum TranscriptRenderItem: Identifiable {
case message(Message)
case toolGroup(id: String, messages: [Message])
var id: String {
switch self {
case let .message(message):
return message.id
case let .toolGroup(id, _):
return "tool-group-\(id)"
}
}
}
func buildTranscriptRenderItems(from messages: [Message]) -> [TranscriptRenderItem] {
var items: [TranscriptRenderItem] = []
var toolRun: [Message] = []
func flushToolRun() {
guard !toolRun.isEmpty else { return }
if toolRun.count == 1, let message = toolRun.first {
items.append(.message(message))
} else if let first = toolRun.first {
items.append(.toolGroup(id: first.id, messages: toolRun))
}
toolRun.removeAll(keepingCapacity: true)
}
for message in messages {
if message.toolCallMetadata != nil {
toolRun.append(message)
} else {
flushToolRun()
items.append(.message(message))
}
}
flushToolRun()
return items
} }
private struct MessageBubble: View { private struct MessageBubble: View {
@@ -137,6 +232,214 @@ private struct MessageBubble: View {
} }
} }
private struct ToolCallStackView: View {
private struct CardLayout {
var x: CGFloat
var y: CGFloat
var scale: CGFloat
var opacity: Double
var zIndex: Double
}
var groupID: String
var messages: [Message]
var entryAnimationIDs: Set<String>
@Environment(\.accessibilityReduceMotion) private var reduceMotion
@State private var isExpanded = false
private let visibleCollapsedLimit = 4
private let cardHeight: CGFloat = 62
private let expandedGap: CGFloat = 10
private let collapsedStepX: CGFloat = 11
private let collapsedStepY: CGFloat = 10
private let toggleSize: CGFloat = 32
private let toggleGap: CGFloat = 12
private var animation: Animation? {
reduceMotion ? nil : .easeInOut(duration: 0.34)
}
private var visibleCollapsedCount: Int {
min(messages.count, visibleCollapsedLimit)
}
private var hiddenCount: Int {
max(0, messages.count - visibleCollapsedLimit)
}
private var containerHeight: CGFloat {
if isExpanded {
return cardHeight + CGFloat(max(0, messages.count - 1)) * (cardHeight + expandedGap)
}
return cardHeight + CGFloat(max(0, visibleCollapsedCount - 1)) * collapsedStepY
}
private var accessibilityLabel: String {
"\(messages.count) tool \(messages.count == 1 ? "call" : "calls")"
}
var body: some View {
HStack(alignment: .top, spacing: 0) {
GeometryReader { geometry in
let cardWidth = max(220, min(520, geometry.size.width - toggleSize - toggleGap))
let toggleX = cardWidth + toggleGap
ZStack(alignment: .topLeading) {
ForEach(Array(messages.enumerated()), id: \.element.id) { index, message in
let layout = layout(for: index)
let depth = messages.count - index - 1
let isHidden = !isExpanded && depth >= visibleCollapsedLimit
let shouldAnimateEntry = entryAnimationIDs.contains(message.id) && !isHidden
ToolCallStackCard(
message: message,
cardHeight: cardHeight,
compactLayout: true,
animateEntry: shouldAnimateEntry
)
.frame(width: cardWidth, height: cardHeight, alignment: .topLeading)
.scaleEffect(layout.scale, anchor: .topLeading)
.opacity(layout.opacity)
.offset(x: layout.x, y: layout.y)
.zIndex(layout.zIndex)
.allowsHitTesting(!isHidden)
.accessibilityHidden(isHidden)
}
if !isExpanded && hiddenCount > 0 {
Text("+\(hiddenCount)")
.font(.sybil(.caption2, weight: .semibold))
.foregroundStyle(SybilTheme.accent.opacity(0.95))
.padding(.horizontal, 7)
.padding(.vertical, 3)
.background(
Capsule()
.fill(Color.black.opacity(0.58))
.overlay(
Capsule()
.stroke(SybilTheme.accent.opacity(0.34), lineWidth: 1)
)
)
.offset(x: max(0, cardWidth - 56), y: containerHeight - 13)
.transition(.opacity)
}
Button {
withAnimation(animation) {
isExpanded.toggle()
}
} label: {
Image(systemName: isExpanded ? "chevron.up" : "chevron.down")
.font(.system(size: 14, weight: .bold))
.foregroundStyle(SybilTheme.accent.opacity(0.95))
.frame(width: toggleSize, height: toggleSize)
.background(
Circle()
.fill(
LinearGradient(
colors: [
Color(red: 0.06, green: 0.08, blue: 0.15).opacity(0.96),
Color(red: 0.03, green: 0.04, blue: 0.10).opacity(0.96)
],
startPoint: .top,
endPoint: .bottom
)
)
.overlay(
Circle()
.stroke(SybilTheme.accent.opacity(0.38), lineWidth: 1)
)
.shadow(color: Color.black.opacity(0.30), radius: 10, x: 0, y: 6)
)
}
.buttonStyle(.plain)
.accessibilityLabel("\(isExpanded ? "Collapse" : "Expand") \(accessibilityLabel)")
.offset(x: toggleX, y: 8)
.zIndex(Double(messages.count + 2))
}
.frame(width: cardWidth + toggleSize + toggleGap, height: containerHeight, alignment: .topLeading)
.animation(animation, value: isExpanded)
}
.frame(height: containerHeight)
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
private func layout(for index: Int) -> CardLayout {
if isExpanded {
return CardLayout(
x: 0,
y: CGFloat(index) * (cardHeight + expandedGap),
scale: 1,
opacity: 1,
zIndex: Double(messages.count - index)
)
}
let depth = messages.count - index - 1
let visibleDepth = min(depth, visibleCollapsedLimit - 1)
let isHidden = depth >= visibleCollapsedLimit
return CardLayout(
x: CGFloat(visibleDepth) * collapsedStepX,
y: CGFloat(visibleDepth) * collapsedStepY,
scale: max(0.88, 1 - CGFloat(visibleDepth) * 0.035),
opacity: isHidden ? 0 : max(0.34, 1 - Double(visibleDepth) * 0.22),
zIndex: isHidden ? 0 : Double(visibleCollapsedCount - visibleDepth)
)
}
}
private struct ToolCallStackCard: View {
var message: Message
var cardHeight: CGFloat
var compactLayout: Bool
var animateEntry: Bool
@Environment(\.accessibilityReduceMotion) private var reduceMotion
@State private var entryAnimationArmed = false
@State private var didEnter = false
private var isPreparingEntry: Bool {
(animateEntry || entryAnimationArmed) && !didEnter
}
var body: some View {
Group {
if let metadata = message.toolCallMetadata {
ToolCallActivityChip(
metadata: metadata,
fallbackContent: message.content,
createdAt: message.createdAt,
compactLayout: compactLayout
)
}
}
.frame(height: cardHeight, alignment: .top)
.scaleEffect(isPreparingEntry ? 1.025 : 1, anchor: .topLeading)
.offset(y: isPreparingEntry ? -8 : 0)
.rotation3DEffect(.degrees(isPreparingEntry ? 3 : 0), axis: (x: 1, y: 0, z: 0), anchor: .top)
.opacity(isPreparingEntry ? 0.72 : 1)
.onAppear {
guard !didEnter, !entryAnimationArmed else { return }
guard animateEntry else {
didEnter = true
return
}
entryAnimationArmed = true
if reduceMotion {
didEnter = true
} else {
withAnimation(.easeOut(duration: 0.32).delay(0.03)) {
didEnter = true
}
}
}
}
}
private struct ToolCallActivityChip: View { private struct ToolCallActivityChip: View {
enum VisualState { enum VisualState {
case initiated case initiated
@@ -147,6 +450,7 @@ private struct ToolCallActivityChip: View {
var metadata: ToolCallMetadata var metadata: ToolCallMetadata
var fallbackContent: String var fallbackContent: String
var createdAt: Date var createdAt: Date
var compactLayout: Bool = false
private var summary: String { private var summary: String {
if let text = metadata.summary?.trimmingCharacters(in: .whitespacesAndNewlines), !text.isEmpty { if let text = metadata.summary?.trimmingCharacters(in: .whitespacesAndNewlines), !text.isEmpty {
@@ -233,7 +537,9 @@ private struct ToolCallActivityChip: View {
.font(.sybil(.subheadline)) .font(.sybil(.subheadline))
.foregroundStyle(isFailed ? SybilTheme.danger.opacity(0.96) : SybilTheme.text.opacity(0.94)) .foregroundStyle(isFailed ? SybilTheme.danger.opacity(0.96) : SybilTheme.text.opacity(0.94))
.lineSpacing(3) .lineSpacing(3)
.fixedSize(horizontal: false, vertical: true) .lineLimit(compactLayout ? 1 : nil)
.truncationMode(.tail)
.fixedSize(horizontal: false, vertical: !compactLayout)
HStack(spacing: 6) { HStack(spacing: 6) {
Text(toolLabel) Text(toolLabel)
@@ -4,6 +4,7 @@ public enum Provider: String, Codable, CaseIterable, Hashable, Sendable {
case openai case openai
case anthropic case anthropic
case xai case xai
case gemini
case hermesAgent = "hermes-agent" case hermesAgent = "hermes-agent"
public var displayName: String { public var displayName: String {
@@ -11,6 +12,7 @@ public enum Provider: String, Codable, CaseIterable, Hashable, Sendable {
case .openai: return "OpenAI" case .openai: return "OpenAI"
case .anthropic: return "Anthropic" case .anthropic: return "Anthropic"
case .xai: return "xAI" case .xai: return "xAI"
case .gemini: return "Gemini"
case .hermesAgent: return "Hermes Agent" case .hermesAgent: return "Hermes Agent"
} }
} }
@@ -98,7 +98,8 @@ struct SybilSearchResultsView: View {
.foregroundStyle(SybilTheme.danger) .foregroundStyle(SybilTheme.danger)
} }
} }
.frame(maxWidth: .infinity, alignment: .leading) .frame(maxWidth: SybilLayout.webContentMaxWidth, alignment: .leading)
.frame(maxWidth: .infinity, alignment: .center)
.padding(.horizontal, 14) .padding(.horizontal, 14)
.padding(.top, 20 + topContentInset) .padding(.top, 20 + topContentInset)
.padding(.bottom, 20 + bottomContentInset) .padding(.bottom, 20 + bottomContentInset)
@@ -11,11 +11,13 @@ final class SybilSettingsStore {
static let preferredOpenAIModel = "sybil.ios.preferredOpenAIModel" static let preferredOpenAIModel = "sybil.ios.preferredOpenAIModel"
static let preferredAnthropicModel = "sybil.ios.preferredAnthropicModel" static let preferredAnthropicModel = "sybil.ios.preferredAnthropicModel"
static let preferredXAIModel = "sybil.ios.preferredXAIModel" static let preferredXAIModel = "sybil.ios.preferredXAIModel"
static let preferredGeminiModel = "sybil.ios.preferredGeminiModel"
static let preferredHermesAgentModel = "sybil.ios.preferredHermesAgentModel" static let preferredHermesAgentModel = "sybil.ios.preferredHermesAgentModel"
static let quickQuestionPreferredProvider = "sybil.ios.quickQuestionPreferredProvider" static let quickQuestionPreferredProvider = "sybil.ios.quickQuestionPreferredProvider"
static let quickQuestionPreferredOpenAIModel = "sybil.ios.quickQuestionPreferredOpenAIModel" static let quickQuestionPreferredOpenAIModel = "sybil.ios.quickQuestionPreferredOpenAIModel"
static let quickQuestionPreferredAnthropicModel = "sybil.ios.quickQuestionPreferredAnthropicModel" static let quickQuestionPreferredAnthropicModel = "sybil.ios.quickQuestionPreferredAnthropicModel"
static let quickQuestionPreferredXAIModel = "sybil.ios.quickQuestionPreferredXAIModel" static let quickQuestionPreferredXAIModel = "sybil.ios.quickQuestionPreferredXAIModel"
static let quickQuestionPreferredGeminiModel = "sybil.ios.quickQuestionPreferredGeminiModel"
static let quickQuestionPreferredHermesAgentModel = "sybil.ios.quickQuestionPreferredHermesAgentModel" static let quickQuestionPreferredHermesAgentModel = "sybil.ios.quickQuestionPreferredHermesAgentModel"
} }
@@ -44,6 +46,7 @@ final class SybilSettingsStore {
.openai: defaults.string(forKey: Keys.preferredOpenAIModel) ?? "gpt-4.1-mini", .openai: defaults.string(forKey: Keys.preferredOpenAIModel) ?? "gpt-4.1-mini",
.anthropic: defaults.string(forKey: Keys.preferredAnthropicModel) ?? "claude-3-5-sonnet-latest", .anthropic: defaults.string(forKey: Keys.preferredAnthropicModel) ?? "claude-3-5-sonnet-latest",
.xai: defaults.string(forKey: Keys.preferredXAIModel) ?? "grok-3-mini", .xai: defaults.string(forKey: Keys.preferredXAIModel) ?? "grok-3-mini",
.gemini: defaults.string(forKey: Keys.preferredGeminiModel) ?? "gemini-3.5-flash",
.hermesAgent: defaults.string(forKey: Keys.preferredHermesAgentModel) ?? "hermes-agent" .hermesAgent: defaults.string(forKey: Keys.preferredHermesAgentModel) ?? "hermes-agent"
] ]
self.preferredModelByProvider = preferredModels self.preferredModelByProvider = preferredModels
@@ -54,6 +57,7 @@ final class SybilSettingsStore {
.openai: defaults.string(forKey: Keys.quickQuestionPreferredOpenAIModel) ?? preferredModels[.openai] ?? "gpt-4.1-mini", .openai: defaults.string(forKey: Keys.quickQuestionPreferredOpenAIModel) ?? preferredModels[.openai] ?? "gpt-4.1-mini",
.anthropic: defaults.string(forKey: Keys.quickQuestionPreferredAnthropicModel) ?? preferredModels[.anthropic] ?? "claude-3-5-sonnet-latest", .anthropic: defaults.string(forKey: Keys.quickQuestionPreferredAnthropicModel) ?? preferredModels[.anthropic] ?? "claude-3-5-sonnet-latest",
.xai: defaults.string(forKey: Keys.quickQuestionPreferredXAIModel) ?? preferredModels[.xai] ?? "grok-3-mini", .xai: defaults.string(forKey: Keys.quickQuestionPreferredXAIModel) ?? preferredModels[.xai] ?? "grok-3-mini",
.gemini: defaults.string(forKey: Keys.quickQuestionPreferredGeminiModel) ?? preferredModels[.gemini] ?? "gemini-3.5-flash",
.hermesAgent: defaults.string(forKey: Keys.quickQuestionPreferredHermesAgentModel) ?? preferredModels[.hermesAgent] ?? "hermes-agent" .hermesAgent: defaults.string(forKey: Keys.quickQuestionPreferredHermesAgentModel) ?? preferredModels[.hermesAgent] ?? "hermes-agent"
] ]
} }
@@ -72,12 +76,14 @@ final class SybilSettingsStore {
defaults.set(preferredModelByProvider[.openai], forKey: Keys.preferredOpenAIModel) defaults.set(preferredModelByProvider[.openai], forKey: Keys.preferredOpenAIModel)
defaults.set(preferredModelByProvider[.anthropic], forKey: Keys.preferredAnthropicModel) defaults.set(preferredModelByProvider[.anthropic], forKey: Keys.preferredAnthropicModel)
defaults.set(preferredModelByProvider[.xai], forKey: Keys.preferredXAIModel) defaults.set(preferredModelByProvider[.xai], forKey: Keys.preferredXAIModel)
defaults.set(preferredModelByProvider[.gemini], forKey: Keys.preferredGeminiModel)
defaults.set(preferredModelByProvider[.hermesAgent], forKey: Keys.preferredHermesAgentModel) defaults.set(preferredModelByProvider[.hermesAgent], forKey: Keys.preferredHermesAgentModel)
defaults.set(quickQuestionPreferredProvider.rawValue, forKey: Keys.quickQuestionPreferredProvider) defaults.set(quickQuestionPreferredProvider.rawValue, forKey: Keys.quickQuestionPreferredProvider)
defaults.set(quickQuestionPreferredModelByProvider[.openai], forKey: Keys.quickQuestionPreferredOpenAIModel) defaults.set(quickQuestionPreferredModelByProvider[.openai], forKey: Keys.quickQuestionPreferredOpenAIModel)
defaults.set(quickQuestionPreferredModelByProvider[.anthropic], forKey: Keys.quickQuestionPreferredAnthropicModel) defaults.set(quickQuestionPreferredModelByProvider[.anthropic], forKey: Keys.quickQuestionPreferredAnthropicModel)
defaults.set(quickQuestionPreferredModelByProvider[.xai], forKey: Keys.quickQuestionPreferredXAIModel) defaults.set(quickQuestionPreferredModelByProvider[.xai], forKey: Keys.quickQuestionPreferredXAIModel)
defaults.set(quickQuestionPreferredModelByProvider[.gemini], forKey: Keys.quickQuestionPreferredGeminiModel)
defaults.set(quickQuestionPreferredModelByProvider[.hermesAgent], forKey: Keys.quickQuestionPreferredHermesAgentModel) defaults.set(quickQuestionPreferredModelByProvider[.hermesAgent], forKey: Keys.quickQuestionPreferredHermesAgentModel)
} }
@@ -64,6 +64,10 @@ extension Font {
} }
} }
enum SybilLayout {
static let webContentMaxWidth: CGFloat = 896
}
enum SybilTheme { enum SybilTheme {
static let background = Color(red: 0.02, green: 0.02, blue: 0.05) static let background = Color(red: 0.02, green: 0.02, blue: 0.05)
static let surface = Color(red: 0.05, green: 0.04, blue: 0.10) static let surface = Color(red: 0.05, green: 0.04, blue: 0.10)
@@ -179,8 +183,8 @@ enum SybilTheme {
static var toolCallGradient: LinearGradient { static var toolCallGradient: LinearGradient {
LinearGradient( LinearGradient(
colors: [ colors: [
Color(red: 0.01, green: 0.15, blue: 0.17).opacity(0.70), Color(red: 0.01, green: 0.15, blue: 0.17),
Color(red: 0.03, green: 0.09, blue: 0.15).opacity(0.78) Color(red: 0.03, green: 0.09, blue: 0.15)
], ],
startPoint: .leading, startPoint: .leading,
endPoint: .trailing endPoint: .trailing
@@ -190,8 +194,8 @@ enum SybilTheme {
static var runningToolCallGradient: LinearGradient { static var runningToolCallGradient: LinearGradient {
LinearGradient( LinearGradient(
colors: [ colors: [
Color(red: 0.30, green: 0.19, blue: 0.04).opacity(0.72), Color(red: 0.30, green: 0.19, blue: 0.04),
Color(red: 0.09, green: 0.05, blue: 0.17).opacity(0.78) Color(red: 0.09, green: 0.05, blue: 0.17)
], ],
startPoint: .leading, startPoint: .leading,
endPoint: .trailing endPoint: .trailing
@@ -201,8 +205,8 @@ enum SybilTheme {
static var failedToolCallGradient: LinearGradient { static var failedToolCallGradient: LinearGradient {
LinearGradient( LinearGradient(
colors: [ colors: [
danger.opacity(0.18), Color(red: 0.27, green: 0.04, blue: 0.10),
Color(red: 0.15, green: 0.03, blue: 0.07).opacity(0.72) Color(red: 0.15, green: 0.03, blue: 0.07)
], ],
startPoint: .leading, startPoint: .leading,
endPoint: .trailing endPoint: .trailing
@@ -107,6 +107,7 @@ final class SybilViewModel {
var isLoadingCollections = false var isLoadingCollections = false
var isLoadingSelection = false var isLoadingSelection = false
var isCreatingSearchChat = false var isCreatingSearchChat = false
var chatBottomPinRequestID = 0
var errorMessage: String? var errorMessage: String?
var composer = "" var composer = ""
@@ -159,6 +160,7 @@ final class SybilViewModel {
.openai: ["gpt-4.1-mini"], .openai: ["gpt-4.1-mini"],
.anthropic: ["claude-3-5-sonnet-latest"], .anthropic: ["claude-3-5-sonnet-latest"],
.xai: ["grok-3-mini"], .xai: ["grok-3-mini"],
.gemini: ["gemini-3.5-flash", "gemini-flash-latest"],
.hermesAgent: ["hermes-agent"] .hermesAgent: ["hermes-agent"]
] ]
@@ -1699,6 +1701,10 @@ final class SybilViewModel {
isLoadingSelection = false isLoadingSelection = false
} }
private func requestChatBottomPin() {
chatBottomPinRequestID += 1
}
private func startSelectionRefreshTask() -> Task<Void, Never> { private func startSelectionRefreshTask() -> Task<Void, Never> {
isLoadingSelection = true isLoadingSelection = true
let task = Task { [weak self] in let task = Task { [weak self] in
@@ -1746,12 +1752,16 @@ final class SybilViewModel {
switch target { switch target {
case let .chat(chatID): case let .chat(chatID):
SybilLog.debug(SybilLog.app, "Refreshing chat \(chatID)") SybilLog.debug(SybilLog.app, "Refreshing chat \(chatID)")
let isSelectingDifferentChat = selectedChat?.id != chatID
let chat = try await client.getChat(chatID: chatID) let chat = try await client.getChat(chatID: chatID)
guard selectedItem == target, draftKind == nil else { guard selectedItem == target, draftKind == nil else {
return return
} }
selectedChat = chat selectedChat = chat
selectedSearch = nil selectedSearch = nil
if isSelectingDifferentChat {
requestChatBottomPin()
}
if let provider = chat.lastUsedProvider, if let provider = chat.lastUsedProvider,
let model = chat.lastUsedModel, let model = chat.lastUsedModel,
@@ -1824,6 +1834,7 @@ final class SybilViewModel {
} else { } else {
pendingDraftChatState = PendingChatState(chatID: nil, messages: optimisticMessages) pendingDraftChatState = PendingChatState(chatID: nil, messages: optimisticMessages)
} }
requestChatBottomPin()
if chatID == nil { if chatID == nil {
let created = try await client.createChat(title: nil) let created = try await client.createChat(title: nil)
@@ -1871,6 +1882,7 @@ final class SybilViewModel {
if let draftPending = pendingDraftChatState { if let draftPending = pendingDraftChatState {
pendingDraftChatState = nil pendingDraftChatState = nil
pendingChatStates[chatID] = PendingChatState(chatID: chatID, messages: draftPending.messages) pendingChatStates[chatID] = PendingChatState(chatID: chatID, messages: draftPending.messages)
requestChatBottomPin()
} else if pendingChatStates[chatID] == nil { } else if pendingChatStates[chatID] == nil {
pendingChatStates[chatID] = PendingChatState(chatID: chatID, messages: optimisticMessages) pendingChatStates[chatID] = PendingChatState(chatID: chatID, messages: optimisticMessages)
} else { } else {
@@ -194,7 +194,8 @@ struct SybilWorkspaceView: View {
isLoading: viewModel.isLoadingSelection, isLoading: viewModel.isLoadingSelection,
isSending: viewModel.isSendingVisibleChat, isSending: viewModel.isSendingVisibleChat,
topContentInset: showsCustomWorkspaceNavigation ? customWorkspaceNavigationContentInset : 0, topContentInset: showsCustomWorkspaceNavigation ? customWorkspaceNavigationContentInset : 0,
bottomContentInset: viewModel.showsComposer ? composerOverlayContentInset : 0 bottomContentInset: viewModel.showsComposer ? composerOverlayContentInset : 0,
bottomPinRequestID: viewModel.chatBottomPinRequestID
) )
.id(transcriptScrollContextID) .id(transcriptScrollContextID)
} }
@@ -402,6 +402,70 @@ private func makeSearchDetail(id: String, date: Date, answer: String) -> SearchD
) )
} }
private func makeToolCallMessage(id: String, date: Date, summary: String = "Ran a tool") -> Message {
Message(
id: id,
createdAt: date,
role: .tool,
content: summary,
name: "web_search",
metadata: .object([
"kind": .string("tool_call"),
"toolCallId": .string("call-\(id)"),
"toolName": .string("web_search"),
"status": .string("completed"),
"summary": .string(summary),
"durationMs": .number(120)
])
)
}
@Test func transcriptRenderItemsGroupAdjacentToolCalls() async throws {
let date = Date(timeIntervalSince1970: 1_700_000_000)
let user = Message(id: "user-1", createdAt: date, role: .user, content: "Search this", name: nil)
let toolA = makeToolCallMessage(id: "tool-a", date: date, summary: "Search A")
let toolB = makeToolCallMessage(id: "tool-b", date: date, summary: "Search B")
let assistant = Message(id: "assistant-1", createdAt: date, role: .assistant, content: "Answer", name: nil)
let items = buildTranscriptRenderItems(from: [user, toolA, toolB, assistant])
#expect(items.count == 3)
guard case let .message(firstMessage) = items[0] else {
Issue.record("Expected the first item to remain a normal message")
return
}
#expect(firstMessage.id == "user-1")
guard case let .toolGroup(groupID, groupedMessages) = items[1] else {
Issue.record("Expected adjacent tool calls to be grouped")
return
}
#expect(groupID == "tool-a")
#expect(groupedMessages.map(\.id) == ["tool-a", "tool-b"])
guard case let .message(lastMessage) = items[2] else {
Issue.record("Expected the assistant response to remain a normal message")
return
}
#expect(lastMessage.id == "assistant-1")
}
@Test func transcriptRenderItemsKeepSingleToolCallsInline() async throws {
let date = Date(timeIntervalSince1970: 1_700_000_000)
let user = Message(id: "user-1", createdAt: date, role: .user, content: "Search this", name: nil)
let tool = makeToolCallMessage(id: "tool-a", date: date)
let assistant = Message(id: "assistant-1", createdAt: date, role: .assistant, content: "Answer", name: nil)
let items = buildTranscriptRenderItems(from: [user, tool, assistant])
#expect(items.count == 3)
guard case let .message(toolMessage) = items[1] else {
Issue.record("Expected a single tool call to use the existing inline chip")
return
}
#expect(toolMessage.id == "tool-a")
}
@MainActor @MainActor
@Test func normalizedAPIBaseURLPreservesExplicitAPIPath() async throws { @Test func normalizedAPIBaseURLPreservesExplicitAPIPath() async throws {
let defaults = UserDefaults(suiteName: #function)! let defaults = UserDefaults(suiteName: #function)!
@@ -480,12 +544,14 @@ private func makeSearchDetail(id: String, date: Date, answer: String) -> SearchD
@MainActor @MainActor
@Test func foregroundChatRefreshReloadsSelectedTranscript() async throws { @Test func foregroundChatRefreshReloadsSelectedTranscript() async throws {
let date = Date(timeIntervalSince1970: 1_700_000_100) let date = Date(timeIntervalSince1970: 1_700_000_100)
let staleDetail = makeChatDetail(id: "chat-2", date: date, body: "stale transcript")
let detail = makeChatDetail(id: "chat-2", date: date, body: "refreshed transcript") let detail = makeChatDetail(id: "chat-2", date: date, body: "refreshed transcript")
let client = MockSybilClient(chatDetails: ["chat-2": detail]) let client = MockSybilClient(chatDetails: ["chat-2": detail])
let viewModel = SybilViewModel(settings: testSettings(named: #function)) { _ in client } let viewModel = SybilViewModel(settings: testSettings(named: #function)) { _ in client }
viewModel.isAuthenticated = true viewModel.isAuthenticated = true
viewModel.isCheckingSession = false viewModel.isCheckingSession = false
viewModel.selectedItem = .chat("chat-2") viewModel.selectedItem = .chat("chat-2")
viewModel.selectedChat = staleDetail
await viewModel.refreshVisibleContent(refreshCollections: false, refreshSelection: true) await viewModel.refreshVisibleContent(refreshCollections: false, refreshSelection: true)
@@ -495,6 +561,7 @@ private func makeSearchDetail(id: String, date: Date, answer: String) -> SearchD
#expect(snapshot.listSearches == 0) #expect(snapshot.listSearches == 0)
#expect(snapshot.getChat == 1) #expect(snapshot.getChat == 1)
#expect(viewModel.selectedChat?.messages.first?.content == "refreshed transcript") #expect(viewModel.selectedChat?.messages.first?.content == "refreshed transcript")
#expect(viewModel.chatBottomPinRequestID == 0)
} }
@MainActor @MainActor
@@ -610,6 +677,7 @@ private func makeSearchDetail(id: String, date: Date, answer: String) -> SearchD
#expect(viewModel.displayedMessages.first?.content == "fresh transcript") #expect(viewModel.displayedMessages.first?.content == "fresh transcript")
#expect(!viewModel.isLoadingSelection) #expect(!viewModel.isLoadingSelection)
#expect(viewModel.chatBottomPinRequestID == 1)
} }
@MainActor @MainActor
@@ -682,6 +750,37 @@ private func makeSearchDetail(id: String, date: Date, answer: String) -> SearchD
await sendTask.value await sendTask.value
} }
@MainActor
@Test func chatBottomPinRequestDoesNotFollowAssistantStreaming() async throws {
let date = Date(timeIntervalSince1970: 1_700_000_245)
let chat = makeChatSummary(id: "chat-pin", date: date)
let detail = makeChatDetail(id: "chat-pin", date: date, body: "existing transcript")
let client = MockSybilClient(
chatsResponse: [chat],
chatDetails: ["chat-pin": detail]
)
await client.setCompletionStreamEvents([
.delta(CompletionStreamDelta(text: "partial ")),
.delta(CompletionStreamDelta(text: "response")),
.done(CompletionStreamDone(text: "partial response"))
])
let viewModel = SybilViewModel(settings: testSettings(named: #function)) { _ in client }
viewModel.isAuthenticated = true
viewModel.isCheckingSession = false
viewModel.chats = [chat]
viewModel.workspaceItems = [WorkspaceItem(chat: chat)]
viewModel.selectedItem = .chat("chat-pin")
viewModel.selectedChat = detail
viewModel.composer = "continue"
let initialPinRequestID = viewModel.chatBottomPinRequestID
await viewModel.sendComposer()
let snapshot = await client.currentSnapshot()
#expect(snapshot.runCompletionStream == 1)
#expect(viewModel.chatBottomPinRequestID == initialPinRequestID + 1)
}
@MainActor @MainActor
@Test func quickQuestionRunsNonPersistentCompletionStream() async throws { @Test func quickQuestionRunsNonPersistentCompletionStream() async throws {
let client = MockSybilClient() let client = MockSybilClient()
+164
View File
@@ -0,0 +1,164 @@
require "shellwords"
default_platform(:ios)
APP_IDENTIFIER = "net.buzzert.sybil2"
SCHEME = "Sybil"
TEAM_ID = "DQQH5H6GBD"
PROFILE_NAME = "Sybil AppStore CI"
CI_KEYCHAIN_NAME = "sybil_ci_keychain"
CI_KEYCHAIN_PASSWORD = "sybil-ci-keychain-password"
CI_KEYCHAIN_DB_PATH = File.expand_path("~/Library/Keychains/#{CI_KEYCHAIN_NAME}-db")
IOS_ROOT = File.expand_path("..", __dir__)
PROJECT_FILE = File.join(IOS_ROOT, "Sybil.xcodeproj")
PROJECT_SPEC = File.join(IOS_ROOT, "project.yml")
APP_PROJECT_SPEC = File.join(IOS_ROOT, "Apps/Sybil/project.yml")
def present?(value)
!value.to_s.strip.empty?
end
def ci?
present?(ENV["CI"])
end
def release_version
tag = ENV["SYBIL_VERSION_TAG"]
tag = ENV["GITHUB_REF_NAME"] if !present?(tag)
tag = ENV["GITHUB_REF"].to_s.sub(%r{\Arefs/tags/}, "") if !present?(tag)
tag = sh("git describe --tags --abbrev=0").strip if !present?(tag)
match = tag.to_s.match(%r{\Arelease/ios/v(\d+\.\d+\.\d+)\z})
unless match
UI.user_error!("Release tag must look like release/ios/v1.2.3; got #{tag.inspect}")
end
match[1]
end
# App Store Connect requires CFBundleVersion to be unique and strictly
# increasing app-wide (not just per marketing version), so we derive it from
# the monotonic CI run number rather than querying TestFlight (that query can
# lag behind builds still processing and hand back a colliding value).
def build_number
value = present?(ENV["SYBIL_BUILD_NUMBER"]) ? ENV["SYBIL_BUILD_NUMBER"] : ENV["GITHUB_RUN_NUMBER"]
unless value.to_s.match?(/\A\d+\z/)
UI.user_error!("Build number must come from SYBIL_BUILD_NUMBER/GITHUB_RUN_NUMBER; got #{value.inspect}")
end
value.to_i
end
def stamp_marketing_version(version)
contents = File.read(APP_PROJECT_SPEC)
updated = contents.sub(/^(\s*MARKETING_VERSION:\s*).*/, "\\1\"#{version}\"")
if updated == contents
UI.user_error!("Could not find MARKETING_VERSION in #{APP_PROJECT_SPEC}")
end
File.write(APP_PROJECT_SPEC, updated)
end
platform :ios do
private_lane :app_store_api_key do
app_store_connect_api_key(
key_id: ENV.fetch("APP_STORE_CONNECT_KEY_ID"),
issuer_id: ENV.fetch("APP_STORE_CONNECT_ISSUER_ID"),
key_content: ENV.fetch("APP_STORE_CONNECT_KEY_CONTENT"),
is_key_content_base64: true
)
end
# CI signs headlessly, so match needs a fresh unlocked keychain to import
# into. codesign resolves identities through the user keychain *search list*
# (first match wins; the --keychain flag does not restrict the lookup), and
# other projects' keychains on this runner hold the same identity but are
# usually locked — so ours must come first. delete_keychain in the beta
# lane's ensure removes both the keychain and its search-list entry, which
# also keeps our (later locked) copy from shadowing those other projects.
private_lane :prepare_ci_keychain do
next unless ci?
delete_keychain(name: CI_KEYCHAIN_NAME) if File.file?(CI_KEYCHAIN_DB_PATH)
create_keychain(
name: CI_KEYCHAIN_NAME,
password: CI_KEYCHAIN_PASSWORD,
unlock: true,
timeout: 3600,
add_to_search_list: false
)
others = sh("security list-keychains -d user", log: false)
.scan(/"([^"]+)"/)
.flatten
.reject { |path| path.include?(CI_KEYCHAIN_NAME) }
sh("security list-keychains -d user -s #{([CI_KEYCHAIN_DB_PATH] + others).shelljoin}")
ENV["MATCH_KEYCHAIN_NAME"] = CI_KEYCHAIN_NAME
ENV["MATCH_KEYCHAIN_PASSWORD"] = CI_KEYCHAIN_PASSWORD
end
private_lane :sync_signing do |options|
match(
type: "appstore",
readonly: options.fetch(:readonly),
app_identifier: APP_IDENTIFIER,
team_id: TEAM_ID,
profile_name: PROFILE_NAME,
git_url: ENV.fetch("MATCH_GIT_URL"),
git_branch: "master",
git_full_name: "Sybil Release Bot",
git_user_email: "james.magahern@me.com",
api_key: options.fetch(:api_key)
)
end
desc "Create or update match signing assets"
lane :setup_signing do
sync_signing(api_key: app_store_api_key, readonly: false)
end
desc "Build and upload to TestFlight"
lane :beta do
prepare_ci_keychain
api_key = app_store_api_key
version = release_version
stamp_marketing_version(version)
sh("xcodegen", "--spec", PROJECT_SPEC)
increment_version_number(version_number: version, xcodeproj: PROJECT_FILE)
increment_build_number(build_number: build_number, xcodeproj: PROJECT_FILE)
sync_signing(api_key: api_key, readonly: true)
build_app(
project: PROJECT_FILE,
scheme: SCHEME,
export_method: "app-store",
xcargs: [
"DEVELOPMENT_TEAM=#{TEAM_ID.shellescape}",
"CODE_SIGN_STYLE=Manual",
"CODE_SIGN_IDENTITY=Apple\\ Distribution",
"PROVISIONING_PROFILE_SPECIFIER=#{PROFILE_NAME.shellescape}"
].join(" "),
export_options: {
signingStyle: "manual",
teamID: TEAM_ID,
provisioningProfiles: {
APP_IDENTIFIER => PROFILE_NAME
}
}
)
upload_to_testflight(
api_key: api_key,
skip_waiting_for_build_processing: true
)
ensure
delete_keychain(name: CI_KEYCHAIN_NAME) if ci? && File.file?(CI_KEYCHAIN_DB_PATH)
end
end
+8 -4
View File
@@ -5,8 +5,10 @@ derived_data := "build/DerivedData"
default: default:
@just build @just build
build: generate:
if [ ! -d "Sybil.xcodeproj" ]; then xcodegen --spec project.yml; fi xcodegen --spec project.yml
build: generate
if command -v xcbeautify >/dev/null 2>&1; then \ if command -v xcbeautify >/dev/null 2>&1; then \
xcodebuild -scheme Sybil -destination '{{simulator}}' | xcbeautify; \ xcodebuild -scheme Sybil -destination '{{simulator}}' | xcbeautify; \
else \ else \
@@ -16,13 +18,15 @@ build:
test: test:
cd Packages/Sybil && xcodebuild test -scheme Sybil -destination '{{simulator}}' -parallel-testing-enabled NO cd Packages/Sybil && xcodebuild test -scheme Sybil -destination '{{simulator}}' -parallel-testing-enabled NO
run: run: generate
if [ ! -d "Sybil.xcodeproj" ]; then xcodegen --spec project.yml; fi
xcrun simctl boot '{{simulator_name}}' 2>/dev/null || true xcrun simctl boot '{{simulator_name}}' 2>/dev/null || true
xcodebuild -scheme Sybil -destination '{{simulator}}' -derivedDataPath '{{derived_data}}' xcodebuild -scheme Sybil -destination '{{simulator}}' -derivedDataPath '{{derived_data}}'
xcrun simctl install booted '{{derived_data}}/Build/Products/Debug-iphonesimulator/Sybil.app' xcrun simctl install booted '{{derived_data}}/Build/Products/Debug-iphonesimulator/Sybil.app'
xcrun simctl launch booted net.buzzert.sybil2 xcrun simctl launch booted net.buzzert.sybil2
beta:
fastlane ios beta
screenshot path="build/sybil-screenshot.png": screenshot path="build/sybil-screenshot.png":
mkdir -p "$(dirname '{{path}}')" mkdir -p "$(dirname '{{path}}')"
xcrun simctl io booted screenshot '{{path}}' xcrun simctl io booted screenshot '{{path}}'
+6 -4
View File
@@ -1,7 +1,7 @@
# Sybil Server # Sybil Server
Backend API for: Backend API for:
- LLM multiplexer (OpenAI Responses / Anthropic / xAI Chat Completions-compatible Grok / Hermes Agent) - LLM multiplexer (OpenAI Responses / Anthropic / xAI Chat Completions-compatible Grok / Gemini / Hermes Agent)
- Personal chat database (chats/messages + LLM call log) - Personal chat database (chats/messages + LLM call log)
## Stack ## Stack
@@ -43,14 +43,16 @@ If `ADMIN_TOKEN` is not set, the server runs in open mode (dev).
- `OPENAI_API_KEY` - `OPENAI_API_KEY`
- `ANTHROPIC_API_KEY` - `ANTHROPIC_API_KEY`
- `XAI_API_KEY` - `XAI_API_KEY`
- `GEMINI_API_KEY`
- `HERMES_AGENT_API_BASE_URL` (`http://127.0.0.1:8642/v1` by default; include the `/v1` suffix) - `HERMES_AGENT_API_BASE_URL` (`http://127.0.0.1:8642/v1` by default; include the `/v1` suffix)
- `HERMES_AGENT_API_KEY` (enables the Hermes Agent provider; set to Hermes `API_SERVER_KEY`, or any non-empty value if that local server does not require auth) - `HERMES_AGENT_API_KEY` (enables the Hermes Agent provider; set to Hermes `API_SERVER_KEY`, or any non-empty value if that local server does not require auth)
- `HERMES_AGENT_MODEL` (optional fallback/override model id; defaults client-side to `hermes-agent`) - `HERMES_AGENT_MODEL` (optional fallback/override model id; defaults client-side to `hermes-agent`)
- `EXA_API_KEY` - `EXA_API_KEY`
- `CHAT_WEB_SEARCH_ENGINE` (`exa` by default, or `searxng` for chat tool calls only) - `BRAVE_SEARCH_API_KEY` (required when `CHAT_WEB_SEARCH_ENGINE=brave`)
- `CHAT_WEB_SEARCH_ENGINE` (`exa` by default; `brave` and `searxng` are also supported for chat tool calls only)
- `SEARXNG_BASE_URL` (required when `CHAT_WEB_SEARCH_ENGINE=searxng`; instance must allow `format=json`) - `SEARXNG_BASE_URL` (required when `CHAT_WEB_SEARCH_ENGINE=searxng`; instance must allow `format=json`)
- `CHAT_MAX_TOOL_ROUNDS` (`100` by default; maximum model/tool result cycles per chat completion) - `CHAT_MAX_TOOL_ROUNDS` (`100` by default; maximum model/tool result cycles per chat completion)
- `CHAT_CODEX_TOOL_ENABLED` (`false` by default; enables the `codex_exec` chat tool for OpenAI/xAI) - `CHAT_CODEX_TOOL_ENABLED` (`false` by default; enables the `codex_exec` chat tool for managed-tool providers)
- `CHAT_CODEX_REMOTE_HOST` (required when Codex tool is enabled; SSH host/IP or `user@host`) - `CHAT_CODEX_REMOTE_HOST` (required when Codex tool is enabled; SSH host/IP or `user@host`)
- `CHAT_CODEX_REMOTE_USER` (optional SSH user when host does not include one) - `CHAT_CODEX_REMOTE_USER` (optional SSH user when host does not include one)
- `CHAT_CODEX_REMOTE_PORT` (`22` by default) - `CHAT_CODEX_REMOTE_PORT` (`22` by default)
@@ -58,7 +60,7 @@ If `ADMIN_TOKEN` is not set, the server runs in open mode (dev).
- `CHAT_CODEX_SSH_KEY_PATH` (recommended: path to a read-only mounted private key) - `CHAT_CODEX_SSH_KEY_PATH` (recommended: path to a read-only mounted private key)
- `CHAT_CODEX_SSH_PRIVATE_KEY_B64` (optional fallback private key delivery) - `CHAT_CODEX_SSH_PRIVATE_KEY_B64` (optional fallback private key delivery)
- `CHAT_CODEX_EXEC_TIMEOUT_MS` (`600000` by default) - `CHAT_CODEX_EXEC_TIMEOUT_MS` (`600000` by default)
- `CHAT_SHELL_TOOL_ENABLED` (`false` by default; enables the `shell_exec` chat tool for OpenAI/xAI on the same devbox) - `CHAT_SHELL_TOOL_ENABLED` (`false` by default; enables the `shell_exec` chat tool for managed-tool providers on the same devbox)
- `CHAT_SHELL_EXEC_TIMEOUT_MS` (`120000` by default) - `CHAT_SHELL_EXEC_TIMEOUT_MS` (`120000` by default)
## API ## API
+1
View File
@@ -13,6 +13,7 @@ enum Provider {
openai openai
anthropic anthropic
xai xai
gemini
hermes_agent @map("hermes-agent") hermes_agent @map("hermes-agent")
} }
+26
View File
@@ -0,0 +1,26 @@
export const CHROMIUM_USER_AGENT =
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36";
export const BROWSER_ACCEPT_LANGUAGE = "en-US,en;q=0.9";
export const FETCH_URL_ACCEPT =
"text/html,application/xhtml+xml,application/xml;q=0.9,application/pdf;q=0.9,*/*;q=0.8";
export function buildBrowserLikeRequestHeaders(accept: string): Record<string, string> {
return {
"User-Agent": CHROMIUM_USER_AGENT,
Accept: accept,
"Accept-Language": BROWSER_ACCEPT_LANGUAGE,
};
}
export function buildBrowserLikeNavigationHeaders(accept = FETCH_URL_ACCEPT): Record<string, string> {
return {
...buildBrowserLikeRequestHeaders(accept),
"Upgrade-Insecure-Requests": "1",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
};
}
+11 -1
View File
@@ -24,7 +24,7 @@ const ChatWebSearchEngineSchema = z.preprocess(
const trimmed = value.trim(); const trimmed = value.trim();
return trimmed ? trimmed.toLowerCase() : undefined; return trimmed ? trimmed.toLowerCase() : undefined;
}, },
z.enum(["exa", "searxng"]).default("exa") z.enum(["exa", "searxng", "brave"]).default("exa")
); );
const BooleanFlagSchema = z.preprocess((value) => { const BooleanFlagSchema = z.preprocess((value) => {
@@ -66,10 +66,12 @@ const EnvSchema = z.object({
OPENAI_API_KEY: z.string().optional(), OPENAI_API_KEY: z.string().optional(),
ANTHROPIC_API_KEY: z.string().optional(), ANTHROPIC_API_KEY: z.string().optional(),
XAI_API_KEY: z.string().optional(), XAI_API_KEY: z.string().optional(),
GEMINI_API_KEY: z.string().optional(),
HERMES_AGENT_API_BASE_URL: HermesAgentApiBaseUrlSchema, HERMES_AGENT_API_BASE_URL: HermesAgentApiBaseUrlSchema,
HERMES_AGENT_API_KEY: OptionalTrimmedStringSchema, HERMES_AGENT_API_KEY: OptionalTrimmedStringSchema,
HERMES_AGENT_MODEL: OptionalTrimmedStringSchema, HERMES_AGENT_MODEL: OptionalTrimmedStringSchema,
EXA_API_KEY: z.string().optional(), EXA_API_KEY: z.string().optional(),
BRAVE_SEARCH_API_KEY: OptionalTrimmedStringSchema,
// Chat-mode web_search tool configuration. Search mode remains Exa-only for now. // Chat-mode web_search tool configuration. Search mode remains Exa-only for now.
CHAT_WEB_SEARCH_ENGINE: ChatWebSearchEngineSchema, CHAT_WEB_SEARCH_ENGINE: ChatWebSearchEngineSchema,
@@ -99,6 +101,14 @@ const EnvSchema = z.object({
}); });
} }
if (value.CHAT_WEB_SEARCH_ENGINE === "brave" && !value.BRAVE_SEARCH_API_KEY) {
ctx.addIssue({
code: "custom",
path: ["BRAVE_SEARCH_API_KEY"],
message: "BRAVE_SEARCH_API_KEY is required when CHAT_WEB_SEARCH_ENGINE=brave",
});
}
if ((value.CHAT_CODEX_TOOL_ENABLED || value.CHAT_SHELL_TOOL_ENABLED) && !value.CHAT_CODEX_REMOTE_HOST) { if ((value.CHAT_CODEX_TOOL_ENABLED || value.CHAT_SHELL_TOOL_ENABLED) && !value.CHAT_CODEX_REMOTE_HOST) {
ctx.addIssue({ ctx.addIssue({
code: "custom", code: "custom",
+44 -603
View File
@@ -4,19 +4,15 @@ import os from "node:os";
import path from "node:path"; import path from "node:path";
import { promisify } from "node:util"; import { promisify } from "node:util";
import { convert as htmlToText } from "html-to-text"; import { convert as htmlToText } from "html-to-text";
import type OpenAI from "openai";
import { z } from "zod"; import { z } from "zod";
import { buildBrowserLikeNavigationHeaders } from "../browser-fetch-headers.js";
import { env } from "../env.js"; import { env } from "../env.js";
import { searchBrave } from "../search/brave.js";
import { exaClient } from "../search/exa.js"; import { exaClient } from "../search/exa.js";
import { searchSearxng } from "../search/searxng.js"; import { searchSearxng } from "../search/searxng.js";
import {
buildOpenAIConversationMessage,
buildOpenAIResponsesInputMessage,
buildSystemPromptAugmentationMessage,
} from "./message-content.js";
import type { ChatMessage } from "./types.js"; import type { ChatMessage } from "./types.js";
const MAX_TOOL_ROUNDS = env.CHAT_MAX_TOOL_ROUNDS; export const MAX_TOOL_ROUNDS = env.CHAT_MAX_TOOL_ROUNDS;
const DEFAULT_WEB_RESULTS = 5; const DEFAULT_WEB_RESULTS = 5;
const MAX_WEB_RESULTS = 10; const MAX_WEB_RESULTS = 10;
const DEFAULT_FETCH_MAX_CHARACTERS = 12_000; const DEFAULT_FETCH_MAX_CHARACTERS = 12_000;
@@ -29,7 +25,7 @@ const MAX_SHELL_COMMAND_CHARACTERS = 20_000;
const DEFAULT_SHELL_MAX_OUTPUT_CHARACTERS = 24_000; const DEFAULT_SHELL_MAX_OUTPUT_CHARACTERS = 24_000;
const MAX_SHELL_MAX_OUTPUT_CHARACTERS = 80_000; const MAX_SHELL_MAX_OUTPUT_CHARACTERS = 80_000;
const REMOTE_EXEC_MAX_BUFFER_BYTES = 1_000_000; const REMOTE_EXEC_MAX_BUFFER_BYTES = 1_000_000;
const MAX_DANGLING_TOOL_INTENT_RETRIES = 1; export const MAX_DANGLING_TOOL_INTENT_RETRIES = 1;
const execFileAsync = promisify(execFile); const execFileAsync = promisify(execFile);
@@ -219,7 +215,7 @@ function getEnabledToolSet(params: Pick<ToolAwareCompletionParams, "enabledTools
return new Set(normalizeEnabledChatTools(params.enabledTools)); return new Set(normalizeEnabledChatTools(params.enabledTools));
} }
function getEnabledChatTools(params: Pick<ToolAwareCompletionParams, "enabledTools">) { export function getEnabledChatTools(params: Pick<ToolAwareCompletionParams, "enabledTools">) {
const enabled = getEnabledToolSet(params); const enabled = getEnabledToolSet(params);
return CHAT_TOOLS.filter((tool) => { return CHAT_TOOLS.filter((tool) => {
const name = getToolName(tool); const name = getToolName(tool);
@@ -227,19 +223,6 @@ function getEnabledChatTools(params: Pick<ToolAwareCompletionParams, "enabledToo
}); });
} }
function toResponsesChatTools(tools: any[]) {
return tools.map((tool) => {
if (tool?.type !== "function") return tool;
return {
type: "function",
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: false,
};
});
}
export const CHAT_TOOL_SYSTEM_PROMPT = export const CHAT_TOOL_SYSTEM_PROMPT =
"You can use tools to gather up-to-date web information when needed. " + "You can use tools to gather up-to-date web information when needed. " +
"Use web_search for discovery and recent facts, and fetch_url to read the full content of a specific page. " + "Use web_search for discovery and recent facts, and fetch_url to read the full content of a specific page. " +
@@ -253,18 +236,18 @@ export const CHAT_TOOL_SYSTEM_PROMPT =
: "") + : "") +
"Do not fabricate tool outputs; reason only from provided tool results."; "Do not fabricate tool outputs; reason only from provided tool results.";
type ToolRunOutcome = { export type ToolRunOutcome = {
ok: boolean; ok: boolean;
[key: string]: unknown; [key: string]: unknown;
}; };
type ToolAwareUsage = { export type ToolAwareUsage = {
inputTokens?: number; inputTokens?: number;
outputTokens?: number; outputTokens?: number;
totalTokens?: number; totalTokens?: number;
}; };
type ToolAwareCompletionResult = { export type ToolAwareCompletionResult = {
text: string; text: string;
usage?: ToolAwareUsage; usage?: ToolAwareUsage;
raw: unknown; raw: unknown;
@@ -276,8 +259,8 @@ export type ToolAwareStreamingEvent =
| { type: "tool_call"; event: ToolExecutionEvent } | { type: "tool_call"; event: ToolExecutionEvent }
| { type: "done"; result: ToolAwareCompletionResult }; | { type: "done"; result: ToolAwareCompletionResult };
type ToolAwareCompletionParams = { export type ToolAwareCompletionParams = {
client: OpenAI; client: any;
model: string; model: string;
messages: ChatMessage[]; messages: ChatMessage[];
enabledTools?: string[]; enabledTools?: string[];
@@ -439,7 +422,7 @@ function extractHtmlTitle(html: string) {
); );
} }
function buildChatToolSystemPrompt(params: Pick<ToolAwareCompletionParams, "enabledTools">) { export function buildChatToolSystemPrompt(params: Pick<ToolAwareCompletionParams, "enabledTools">) {
const enabled = getEnabledToolSet(params); const enabled = getEnabledToolSet(params);
return ( return (
"You can use tools to gather up-to-date web information when needed. " + "You can use tools to gather up-to-date web information when needed. " +
@@ -457,22 +440,6 @@ function buildChatToolSystemPrompt(params: Pick<ToolAwareCompletionParams, "enab
); );
} }
function normalizeIncomingMessages(messages: ChatMessage[], userLocation?: string, params: Pick<ToolAwareCompletionParams, "enabledTools"> = {}) {
const normalized = messages.map((message) => buildOpenAIConversationMessage(message));
return [{ role: "system", content: buildChatToolSystemPrompt(params) }, buildSystemPromptAugmentationMessage(userLocation), ...normalized];
}
function normalizePlainIncomingMessages(messages: ChatMessage[], userLocation?: string) {
return [buildSystemPromptAugmentationMessage(userLocation), ...messages.map((message) => buildOpenAIConversationMessage(message))];
}
function normalizeIncomingResponsesInput(messages: ChatMessage[], userLocation?: string, params: Pick<ToolAwareCompletionParams, "enabledTools"> = {}) {
const normalized = messages.map((message) => buildOpenAIResponsesInputMessage(message));
return [{ role: "system", content: buildChatToolSystemPrompt(params) }, buildSystemPromptAugmentationMessage(userLocation), ...normalized];
}
async function runExaWebSearchTool(args: WebSearchArgs): Promise<ToolRunOutcome> { async function runExaWebSearchTool(args: WebSearchArgs): Promise<ToolRunOutcome> {
const exa = exaClient(); const exa = exaClient();
const response = await exa.search(args.query, { const response = await exa.search(args.query, {
@@ -541,11 +508,33 @@ async function runSearxngWebSearchTool(args: WebSearchArgs): Promise<ToolRunOutc
}; };
} }
async function runBraveWebSearchTool(args: WebSearchArgs): Promise<ToolRunOutcome> {
const response = await searchBrave(args.query, {
numResults: args.numResults ?? DEFAULT_WEB_RESULTS,
includeDomains: args.includeDomains,
excludeDomains: args.excludeDomains,
});
return {
ok: true,
searchEngine: "brave",
query: args.query,
requestId: response.requestId,
results: response.results.map((result, index) => ({
rank: index + 1,
...result,
})),
};
}
async function runWebSearchTool(input: unknown): Promise<ToolRunOutcome> { async function runWebSearchTool(input: unknown): Promise<ToolRunOutcome> {
const args = WebSearchArgsSchema.parse(input); const args = WebSearchArgsSchema.parse(input);
if (env.CHAT_WEB_SEARCH_ENGINE === "searxng") { if (env.CHAT_WEB_SEARCH_ENGINE === "searxng") {
return runSearxngWebSearchTool(args); return runSearxngWebSearchTool(args);
} }
if (env.CHAT_WEB_SEARCH_ENGINE === "brave") {
return runBraveWebSearchTool(args);
}
return runExaWebSearchTool(args); return runExaWebSearchTool(args);
} }
@@ -570,10 +559,7 @@ async function runFetchUrlTool(input: unknown): Promise<ToolRunOutcome> {
response = await fetch(parsed.toString(), { response = await fetch(parsed.toString(), {
redirect: "follow", redirect: "follow",
signal: controller.signal, signal: controller.signal,
headers: { headers: buildBrowserLikeNavigationHeaders(),
"User-Agent": "SybilBot/1.0 (+https://sybil.local)",
Accept: "text/html, text/plain, application/json;q=0.9, */*;q=0.5",
},
}); });
} finally { } finally {
clearTimeout(timeout); clearTimeout(timeout);
@@ -844,7 +830,7 @@ async function executeTool(name: string, args: unknown): Promise<ToolRunOutcome>
return { ok: false, error: `Unknown tool: ${name}` }; return { ok: false, error: `Unknown tool: ${name}` };
} }
function parseToolArgs(raw: unknown) { export function parseToolArgs(raw: unknown) {
if (typeof raw !== "string") return {}; if (typeof raw !== "string") return {};
const trimmed = raw.trim(); const trimmed = raw.trim();
if (!trimmed) return {}; if (!trimmed) return {};
@@ -873,7 +859,7 @@ function buildEventArgs(name: string, args: Record<string, unknown>) {
return args; return args;
} }
function looksLikeDanglingToolIntent(text: string) { export function looksLikeDanglingToolIntent(text: string) {
const normalized = text const normalized = text
.toLowerCase() .toLowerCase()
.replace(/[`*_>#-]/g, " ") .replace(/[`*_>#-]/g, " ")
@@ -889,7 +875,7 @@ function looksLikeDanglingToolIntent(text: string) {
); );
} }
function appendDanglingToolIntentCorrection(conversation: any[], text: string) { export function appendDanglingToolIntentCorrection(conversation: any[], text: string) {
conversation.push({ role: "assistant", content: text }); conversation.push({ role: "assistant", content: text });
conversation.push({ conversation.push({
role: "system", role: "system",
@@ -898,7 +884,7 @@ function appendDanglingToolIntentCorrection(conversation: any[], text: string) {
}); });
} }
function mergeUsage(acc: Required<ToolAwareUsage>, usage: any) { export function mergeUsage(acc: Required<ToolAwareUsage>, usage: any) {
if (!usage) return false; if (!usage) return false;
acc.inputTokens += usage.prompt_tokens ?? 0; acc.inputTokens += usage.prompt_tokens ?? 0;
acc.outputTokens += usage.completion_tokens ?? 0; acc.outputTokens += usage.completion_tokens ?? 0;
@@ -906,79 +892,19 @@ function mergeUsage(acc: Required<ToolAwareUsage>, usage: any) {
return true; return true;
} }
function mergeResponsesUsage(acc: Required<ToolAwareUsage>, usage: any) { export function getUnstreamedText(finalText: string, streamedText: string) {
if (!usage) return false;
acc.inputTokens += usage.input_tokens ?? 0;
acc.outputTokens += usage.output_tokens ?? 0;
acc.totalTokens += usage.total_tokens ?? 0;
return true;
}
function getResponseOutputItems(response: any) {
return Array.isArray(response?.output) ? response.output : [];
}
function extractResponsesText(response: any, fallback = "") {
if (typeof response?.output_text === "string") return response.output_text;
const parts: string[] = [];
for (const item of getResponseOutputItems(response)) {
if (item?.type !== "message" || !Array.isArray(item.content)) continue;
for (const content of item.content) {
if (content?.type === "output_text" && typeof content.text === "string") {
parts.push(content.text);
} else if (content?.type === "refusal" && typeof content.refusal === "string") {
parts.push(content.refusal);
}
}
}
return parts.join("") || fallback;
}
function extractChatCompletionContent(message: any) {
if (typeof message?.content === "string") return message.content;
if (!Array.isArray(message?.content)) return "";
return message.content
.map((part: any) => {
if (typeof part === "string") return part;
if (typeof part?.text === "string") return part.text;
if (typeof part?.content === "string") return part.content;
return "";
})
.join("");
}
function getUnstreamedText(finalText: string, streamedText: string) {
if (!finalText) return ""; if (!finalText) return "";
if (!streamedText) return finalText; if (!streamedText) return finalText;
return finalText.startsWith(streamedText) ? finalText.slice(streamedText.length) : ""; return finalText.startsWith(streamedText) ? finalText.slice(streamedText.length) : "";
} }
function getResponseFailureMessage(response: any) { export type NormalizedToolCall = {
if (response?.status !== "failed" && response?.status !== "incomplete") return null;
const errorMessage = typeof response?.error?.message === "string" ? response.error.message : null;
const incompleteReason = typeof response?.incomplete_details?.reason === "string" ? response.incomplete_details.reason : null;
return errorMessage ?? (incompleteReason ? `Response incomplete: ${incompleteReason}` : `Response ${response.status}.`);
}
function normalizeResponsesToolCalls(outputItems: any[], round: number): NormalizedToolCall[] {
return outputItems
.filter((item) => item?.type === "function_call")
.map((call: any, index: number) => ({
id: call.call_id ?? call.id ?? `tool_call_${round}_${index}`,
name: call.name ?? "unknown_tool",
arguments: call.arguments ?? "{}",
}));
}
type NormalizedToolCall = {
id: string; id: string;
name: string; name: string;
arguments: string; arguments: string;
}; };
function normalizeModelToolCalls(toolCalls: any[], round: number): NormalizedToolCall[] { export function normalizeModelToolCalls(toolCalls: any[], round: number): NormalizedToolCall[] {
return toolCalls.map((call: any, index: number) => ({ return toolCalls.map((call: any, index: number) => ({
id: call?.id ?? `tool_call_${round}_${index}`, id: call?.id ?? `tool_call_${round}_${index}`,
name: call?.function?.name ?? "unknown_tool", name: call?.function?.name ?? "unknown_tool",
@@ -986,7 +912,7 @@ function normalizeModelToolCalls(toolCalls: any[], round: number): NormalizedToo
})); }));
} }
type PreparedToolCallExecution = { export type PreparedToolCallExecution = {
startedAtMs: number; startedAtMs: number;
startedAt: string; startedAt: string;
parsedArgs: Record<string, unknown>; parsedArgs: Record<string, unknown>;
@@ -994,7 +920,7 @@ type PreparedToolCallExecution = {
parseError?: unknown; parseError?: unknown;
}; };
function prepareToolCallExecution(call: NormalizedToolCall): { event: ToolExecutionEvent; execution: PreparedToolCallExecution } { export function prepareToolCallExecution(call: NormalizedToolCall): { event: ToolExecutionEvent; execution: PreparedToolCallExecution } {
const startedAtMs = Date.now(); const startedAtMs = Date.now();
const startedAt = new Date(startedAtMs).toISOString(); const startedAt = new Date(startedAtMs).toISOString();
let parsedArgs: Record<string, unknown> = {}; let parsedArgs: Record<string, unknown> = {};
@@ -1026,7 +952,7 @@ function prepareToolCallExecution(call: NormalizedToolCall): { event: ToolExecut
}; };
} }
async function executeToolCallAndBuildEvent( export async function executeToolCallAndBuildEvent(
call: NormalizedToolCall, call: NormalizedToolCall,
execution: PreparedToolCallExecution, execution: PreparedToolCallExecution,
params: ToolAwareCompletionParams params: ToolAwareCompletionParams
@@ -1070,488 +996,3 @@ async function executeToolCallAndBuildEvent(
return { event, toolResult }; return { event, toolResult };
} }
export async function runToolAwareOpenAIChat(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const enabledTools = getEnabledChatTools(params);
const input: any[] = normalizeIncomingResponsesInput(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const response = await params.client.responses.create({
model: params.model,
input,
temperature: params.temperature,
max_output_tokens: params.maxTokens,
tools: toResponsesChatTools(enabledTools),
tool_choice: "auto",
parallel_tool_calls: true,
// Tool loops pass response output items back as input; reasoning items need persistence.
store: true,
} as any);
rawResponses.push(response);
sawUsage = mergeResponsesUsage(usageAcc, response?.usage) || sawUsage;
const failureMessage = getResponseFailureMessage(response);
if (failureMessage) {
throw new Error(failureMessage);
}
const outputItems = getResponseOutputItems(response);
const normalizedToolCalls = normalizeResponsesToolCalls(outputItems, round);
if (!normalizedToolCalls.length) {
const text = extractResponsesText(response);
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(input, text);
continue;
}
return {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, api: "responses" },
toolEvents,
};
}
totalToolCalls += normalizedToolCalls.length;
input.push(...outputItems);
for (const call of normalizedToolCalls) {
const { execution } = prepareToolCallExecution(call);
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
input.push({
type: "function_call_output",
call_id: call.id,
output: JSON.stringify(toolResult),
});
}
}
return {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "responses" },
toolEvents,
};
}
export async function runToolAwareChatCompletions(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const enabledTools = getEnabledChatTools(params);
const conversation: any[] = normalizeIncomingMessages(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const completion = await params.client.chat.completions.create({
model: params.model,
messages: conversation,
temperature: params.temperature,
max_tokens: params.maxTokens,
tools: enabledTools,
tool_choice: "auto",
} as any);
rawResponses.push(completion);
sawUsage = mergeUsage(usageAcc, completion?.usage) || sawUsage;
const message = completion?.choices?.[0]?.message;
if (!message) {
return {
text: "",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, missingMessage: true },
toolEvents,
};
}
const toolCalls = Array.isArray(message.tool_calls) ? message.tool_calls : [];
if (!toolCalls.length) {
const text = typeof message.content === "string" ? message.content : "";
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(conversation, text);
continue;
}
return {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls },
toolEvents,
};
}
const normalizedToolCalls = normalizeModelToolCalls(toolCalls, round);
totalToolCalls += normalizedToolCalls.length;
const assistantToolCallMessage: any = {
role: "assistant",
tool_calls: normalizedToolCalls.map((call) => ({
id: call.id,
type: "function",
function: {
name: call.name,
arguments: call.arguments,
},
})),
};
if (typeof message.content === "string" && message.content.length) {
assistantToolCallMessage.content = message.content;
}
conversation.push(assistantToolCallMessage);
for (const call of normalizedToolCalls) {
const { execution } = prepareToolCallExecution(call);
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
conversation.push({
role: "tool",
tool_call_id: call.id,
content: JSON.stringify(toolResult),
});
}
}
return {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true },
toolEvents,
};
}
export async function runPlainChatCompletions(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const completion = await params.client.chat.completions.create({
model: params.model,
messages: normalizePlainIncomingMessages(params.messages, params.userLocation),
temperature: params.temperature,
max_tokens: params.maxTokens,
} as any);
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
const sawUsage = mergeUsage(usageAcc, completion?.usage);
const message = completion?.choices?.[0]?.message;
return {
text: extractChatCompletionContent(message),
usage: sawUsage ? usageAcc : undefined,
raw: { response: completion, api: "chat.completions" },
toolEvents: [],
};
}
export async function* runToolAwareOpenAIChatStream(
params: ToolAwareCompletionParams
): AsyncGenerator<ToolAwareStreamingEvent> {
const enabledTools = getEnabledChatTools(params);
const input: any[] = normalizeIncomingResponsesInput(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const stream = await params.client.responses.create({
model: params.model,
input,
temperature: params.temperature,
max_output_tokens: params.maxTokens,
tools: toResponsesChatTools(enabledTools),
tool_choice: "auto",
parallel_tool_calls: true,
// Tool loops pass response output items back as input; reasoning items need persistence.
store: true,
stream: true,
} as any);
let roundText = "";
let streamedRoundText = "";
let roundHasToolCalls = false;
let canStreamRoundText = false;
let completedResponse: any | null = null;
const completedOutputItems: any[] = [];
for await (const event of stream as any as AsyncIterable<any>) {
rawResponses.push(event);
if (event?.type === "response.output_text.delta" && typeof event.delta === "string") {
roundText += event.delta;
if (canStreamRoundText && !roundHasToolCalls && event.delta.length) {
streamedRoundText += event.delta;
yield { type: "delta", text: event.delta };
}
} else if (event?.type === "response.output_item.added" && event.item) {
if (event.item.type === "function_call") {
roundHasToolCalls = true;
canStreamRoundText = false;
} else if (event.item.type === "message" && !roundHasToolCalls) {
canStreamRoundText = true;
}
} else if (event?.type === "response.output_item.done" && event.item) {
completedOutputItems[event.output_index ?? completedOutputItems.length] = event.item;
if (event.item.type === "function_call") {
roundHasToolCalls = true;
canStreamRoundText = false;
}
} else if (event?.type === "response.completed") {
completedResponse = event.response;
sawUsage = mergeResponsesUsage(usageAcc, event.response?.usage) || sawUsage;
} else if (event?.type === "response.failed" || event?.type === "response.incomplete") {
completedResponse = event.response;
sawUsage = mergeResponsesUsage(usageAcc, event.response?.usage) || sawUsage;
} else if (event?.type === "error") {
throw new Error(event.message ?? "OpenAI Responses stream failed.");
}
}
const failureMessage = getResponseFailureMessage(completedResponse);
if (failureMessage) {
throw new Error(failureMessage);
}
const outputItems = getResponseOutputItems(completedResponse);
const responseOutputItems = outputItems.length ? outputItems : completedOutputItems.filter(Boolean);
const normalizedToolCalls = normalizeResponsesToolCalls(responseOutputItems, round);
if (!normalizedToolCalls.length) {
const text = extractResponsesText(completedResponse, roundText);
if (
!streamedRoundText &&
danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES &&
looksLikeDanglingToolIntent(text)
) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(input, text);
continue;
}
const unstreamedText = getUnstreamedText(text, streamedRoundText);
if (unstreamedText) {
yield { type: "delta", text: unstreamedText };
}
yield {
type: "done",
result: {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, api: "responses" },
toolEvents,
},
};
return;
}
totalToolCalls += normalizedToolCalls.length;
input.push(...responseOutputItems);
for (const call of normalizedToolCalls) {
const { event: initiatedEvent, execution } = prepareToolCallExecution(call);
yield { type: "tool_call", event: initiatedEvent };
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
yield { type: "tool_call", event };
input.push({
type: "function_call_output",
call_id: call.id,
output: JSON.stringify(toolResult),
});
}
}
yield {
type: "done",
result: {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "responses" },
toolEvents,
},
};
}
export async function* runToolAwareChatCompletionsStream(
params: ToolAwareCompletionParams
): AsyncGenerator<ToolAwareStreamingEvent> {
const enabledTools = getEnabledChatTools(params);
const conversation: any[] = normalizeIncomingMessages(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const stream = await params.client.chat.completions.create({
model: params.model,
messages: conversation,
temperature: params.temperature,
max_tokens: params.maxTokens,
tools: enabledTools,
tool_choice: "auto",
stream: true,
stream_options: { include_usage: true },
} as any);
let roundText = "";
let streamedRoundText = "";
let roundHasToolCalls = false;
const roundToolCalls = new Map<number, { id?: string; name?: string; arguments: string }>();
for await (const chunk of stream as any as AsyncIterable<any>) {
rawResponses.push(chunk);
sawUsage = mergeUsage(usageAcc, chunk?.usage) || sawUsage;
const choice = chunk?.choices?.[0];
const deltaText = choice?.delta?.content ?? "";
if (typeof deltaText === "string" && deltaText.length) {
roundText += deltaText;
if (!roundHasToolCalls) {
streamedRoundText += deltaText;
yield { type: "delta", text: deltaText };
}
}
const deltaToolCalls = Array.isArray(choice?.delta?.tool_calls) ? choice.delta.tool_calls : [];
if (deltaToolCalls.length) {
roundHasToolCalls = true;
}
for (const toolCall of deltaToolCalls) {
const idx = typeof toolCall?.index === "number" ? toolCall.index : 0;
const entry = roundToolCalls.get(idx) ?? { arguments: "" };
if (typeof toolCall?.id === "string" && toolCall.id.length) {
entry.id = toolCall.id;
}
if (typeof toolCall?.function?.name === "string" && toolCall.function.name.length) {
entry.name = toolCall.function.name;
}
if (typeof toolCall?.function?.arguments === "string" && toolCall.function.arguments.length) {
entry.arguments += toolCall.function.arguments;
}
roundToolCalls.set(idx, entry);
}
}
const normalizedToolCalls: NormalizedToolCall[] = [...roundToolCalls.entries()]
.sort((a, b) => a[0] - b[0])
.map(([_, call], index) => ({
id: call.id ?? `tool_call_${round}_${index}`,
name: call.name ?? "unknown_tool",
arguments: call.arguments || "{}",
}));
if (!normalizedToolCalls.length) {
if (
!streamedRoundText &&
danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES &&
looksLikeDanglingToolIntent(roundText)
) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(conversation, roundText);
continue;
}
const unstreamedText = getUnstreamedText(roundText, streamedRoundText);
if (unstreamedText) {
yield { type: "delta", text: unstreamedText };
}
yield {
type: "done",
result: {
text: roundText,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls },
toolEvents,
},
};
return;
}
totalToolCalls += normalizedToolCalls.length;
const assistantToolCallMessage: any = {
role: "assistant",
tool_calls: normalizedToolCalls.map((call) => ({
id: call.id,
type: "function",
function: {
name: call.name,
arguments: call.arguments,
},
})),
};
if (roundText) {
assistantToolCallMessage.content = roundText;
}
conversation.push(assistantToolCallMessage);
for (const call of normalizedToolCalls) {
const { event: initiatedEvent, execution } = prepareToolCallExecution(call);
yield { type: "tool_call", event: initiatedEvent };
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
yield { type: "tool_call", event };
conversation.push({
role: "tool",
tool_call_id: call.id,
content: JSON.stringify(toolResult),
});
}
}
yield {
type: "done",
result: {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true },
toolEvents,
},
};
}
export async function* runPlainChatCompletionsStream(
params: ToolAwareCompletionParams
): AsyncGenerator<ToolAwareStreamingEvent> {
const rawResponses: unknown[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let text = "";
const stream = await params.client.chat.completions.create({
model: params.model,
messages: normalizePlainIncomingMessages(params.messages, params.userLocation),
temperature: params.temperature,
max_tokens: params.maxTokens,
stream: true,
} as any);
for await (const chunk of stream as any as AsyncIterable<any>) {
rawResponses.push(chunk);
sawUsage = mergeUsage(usageAcc, chunk?.usage) || sawUsage;
const deltaText = chunk?.choices?.[0]?.delta?.content ?? "";
if (typeof deltaText === "string" && deltaText.length) {
text += deltaText;
yield { type: "delta", text: deltaText };
}
}
yield {
type: "done",
result: {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, api: "chat.completions" },
toolEvents: [],
},
};
}
+7 -182
View File
@@ -18,21 +18,21 @@ function escapeAttribute(value: string) {
return value.replace(/"/g, "&quot;"); return value.replace(/"/g, "&quot;");
} }
function getImageAttachments(message: ChatMessage) { export function getImageAttachments(message: ChatMessage) {
return (message.attachments ?? []).filter((attachment): attachment is ChatImageAttachment => attachment.kind === "image"); return (message.attachments ?? []).filter((attachment): attachment is ChatImageAttachment => attachment.kind === "image");
} }
function getTextAttachments(message: ChatMessage) { export function getTextAttachments(message: ChatMessage) {
return (message.attachments ?? []).filter((attachment): attachment is ChatTextAttachment => attachment.kind === "text"); return (message.attachments ?? []).filter((attachment): attachment is ChatTextAttachment => attachment.kind === "text");
} }
function buildImageSummaryText(attachments: ChatImageAttachment[]) { export function buildImageSummaryText(attachments: ChatImageAttachment[]) {
if (!attachments.length) return null; if (!attachments.length) return null;
const label = attachments.length === 1 ? "Attached image" : "Attached images"; const label = attachments.length === 1 ? "Attached image" : "Attached images";
return `${label}: ${attachments.map((attachment) => attachment.filename).join(", ")}.`; return `${label}: ${attachments.map((attachment) => attachment.filename).join(", ")}.`;
} }
function buildTextAttachmentPrompt(attachment: ChatTextAttachment) { export function buildTextAttachmentPrompt(attachment: ChatTextAttachment) {
const truncationNote = attachment.truncated ? ' truncated="true"' : ""; const truncationNote = attachment.truncated ? ' truncated="true"' : "";
return [ return [
`Attached text file: ${attachment.filename}${attachment.truncated ? " (content truncated)" : ""}`, `Attached text file: ${attachment.filename}${attachment.truncated ? " (content truncated)" : ""}`,
@@ -42,83 +42,7 @@ function buildTextAttachmentPrompt(attachment: ChatTextAttachment) {
].join("\n"); ].join("\n");
} }
function toOpenAIContent(message: ChatMessage) { export function parseImageDataUrl(attachment: ChatImageAttachment) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
if (!imageAttachments.length && !textAttachments.length) {
return message.content;
}
const parts: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
parts.push({
type: "image_url",
image_url: {
url: attachment.dataUrl,
detail: "auto",
},
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
parts.push({ type: "text", text: imageSummary });
}
for (const attachment of textAttachments) {
parts.push({ type: "text", text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
parts.push({ type: "text", text: message.content });
}
if (parts.length === 1 && parts[0]?.type === "text" && typeof parts[0].text === "string") {
return parts[0].text;
}
return parts;
}
function toOpenAIResponsesContent(message: ChatMessage) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
if (!imageAttachments.length && !textAttachments.length) {
return message.content;
}
const parts: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
parts.push({
type: "input_image",
image_url: attachment.dataUrl,
detail: "auto",
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
parts.push({ type: "input_text", text: imageSummary });
}
for (const attachment of textAttachments) {
parts.push({ type: "input_text", text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
parts.push({ type: "input_text", text: message.content });
}
if (parts.length === 1 && parts[0]?.type === "input_text" && typeof parts[0].text === "string") {
return parts[0].text;
}
return parts;
}
function parseImageDataUrl(attachment: ChatImageAttachment) {
const match = attachment.dataUrl.match(/^data:(image\/(?:png|jpeg));base64,([a-z0-9+/=\s]+)$/i); const match = attachment.dataUrl.match(/^data:(image\/(?:png|jpeg));base64,([a-z0-9+/=\s]+)$/i);
if (!match) { if (!match) {
throw new Error(`Invalid image attachment data URL for '${attachment.filename}'.`); throw new Error(`Invalid image attachment data URL for '${attachment.filename}'.`);
@@ -135,83 +59,6 @@ function parseImageDataUrl(attachment: ChatImageAttachment) {
}; };
} }
function toAnthropicContent(message: ChatMessage) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
if (!imageAttachments.length && !textAttachments.length) {
return message.content;
}
const blocks: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
const source = parseImageDataUrl(attachment);
blocks.push({
type: "image",
source: {
type: "base64",
media_type: source.mediaType,
data: source.data,
},
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
blocks.push({ type: "text", text: imageSummary });
}
for (const attachment of textAttachments) {
blocks.push({ type: "text", text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
blocks.push({ type: "text", text: message.content });
}
if (blocks.length === 1 && blocks[0]?.type === "text" && typeof blocks[0].text === "string") {
return blocks[0].text;
}
return blocks;
}
export function buildOpenAIConversationMessage(message: ChatMessage) {
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
content: `Tool output (${name}):\n${message.content}`,
};
}
const out: Record<string, unknown> = {
role: message.role,
content: toOpenAIContent(message),
};
if (message.name && (message.role === "assistant" || message.role === "user")) {
out.name = message.name;
}
return out;
}
export function buildOpenAIResponsesInputMessage(message: ChatMessage) {
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
content: `Tool output (${name}):\n${message.content}`,
};
}
return {
role: message.role,
content: toOpenAIResponsesContent(message),
};
}
export function buildSystemPromptAugmentationMessage(userLocation?: string) { export function buildSystemPromptAugmentationMessage(userLocation?: string) {
return { return {
role: "system", role: "system",
@@ -219,34 +66,12 @@ export function buildSystemPromptAugmentationMessage(userLocation?: string) {
}; };
} }
const ANTHROPIC_NO_SERVER_TOOLS_PROMPT = export function buildTopLevelSystemPrompt(messages: ChatMessage[], userLocation?: string, toolSystemPrompt?: string) {
"This Anthropic backend path does not have server-managed tool calls. Do not claim to run shell commands, Codex tasks, web searches, or fetch URLs. If the user asks for tool execution, explain that they should switch to OpenAI or xAI in this app for tool-enabled chat."; return [toolSystemPrompt, buildSystemPromptAugmentation(userLocation), messages.find((message) => message.role === "system")?.content]
export function getAnthropicSystemPrompt(messages: ChatMessage[], userLocation?: string) {
return [ANTHROPIC_NO_SERVER_TOOLS_PROMPT, buildSystemPromptAugmentation(userLocation), messages.find((message) => message.role === "system")?.content]
.filter(Boolean) .filter(Boolean)
.join("\n\n"); .join("\n\n");
} }
export function buildAnthropicConversationMessage(message: ChatMessage) {
if (message.role === "system") {
throw new Error("System messages must be handled separately for Anthropic.");
}
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
content: `Tool output (${name}):\n${message.content}`,
};
}
return {
role: message.role === "assistant" ? "assistant" : "user",
content: toAnthropicContent(message),
};
}
export function buildComparableAttachments(input: unknown): ChatAttachment[] { export function buildComparableAttachments(input: unknown): ChatAttachment[] {
if (!Array.isArray(input)) return []; if (!Array.isArray(input)) return [];
+10 -51
View File
@@ -1,6 +1,9 @@
import type { FastifyBaseLogger } from "fastify"; import type { FastifyBaseLogger } from "fastify";
import { env } from "../env.js"; import {
import { anthropicClient, hermesAgentClient, isHermesAgentConfigured, openaiClient, xaiClient } from "./providers.js"; fetchProviderCatalogModels,
getProviderCatalogFallbackModels,
listModelCatalogProviders,
} from "./provider-adapters.js";
import type { Provider } from "./types.js"; import type { Provider } from "./types.js";
export type ProviderModelSnapshot = { export type ProviderModelSnapshot = {
@@ -11,35 +14,13 @@ export type ProviderModelSnapshot = {
export type ModelCatalogSnapshot = Partial<Record<Provider, ProviderModelSnapshot>>; export type ModelCatalogSnapshot = Partial<Record<Provider, ProviderModelSnapshot>>;
const baseProviders: Provider[] = ["openai", "anthropic", "xai"];
const MODEL_FETCH_TIMEOUT_MS = 15000; const MODEL_FETCH_TIMEOUT_MS = 15000;
const MODEL_CATALOG_REFRESH_INTERVAL_MS = 24 * 60 * 60 * 1000; const MODEL_CATALOG_REFRESH_INTERVAL_MS = 24 * 60 * 60 * 1000;
const modelCatalog: ModelCatalogSnapshot = { const modelCatalog: ModelCatalogSnapshot = {};
openai: { models: [], loadedAt: null, error: null },
anthropic: { models: [], loadedAt: null, error: null },
xai: { models: [], loadedAt: null, error: null },
};
let catalogRefreshPromise: Promise<void> | null = null; let catalogRefreshPromise: Promise<void> | null = null;
function getCatalogProviders(): Provider[] {
return isHermesAgentConfigured() ? [...baseProviders, "hermes-agent"] : baseProviders;
}
function uniqSorted(models: string[]) {
return [...new Set(models.map((value) => value.trim()).filter(Boolean))].sort((a, b) => a.localeCompare(b));
}
function isLikelyOpenAIResponsesModel(model: string) {
const id = model.toLowerCase();
if (id.includes("embedding") || id.includes("moderation")) return false;
if (id.includes("audio") || id.includes("realtime") || id.includes("transcribe") || id.includes("tts")) return false;
if (id.includes("image") || id.includes("dall-e") || id.includes("sora")) return false;
if (id.includes("search") || id.includes("computer-use")) return false;
return /^(gpt-|o\d|chatgpt-)/.test(id);
}
async function withTimeout<T>(promise: Promise<T>, timeoutMs: number, label: string) { async function withTimeout<T>(promise: Promise<T>, timeoutMs: number, label: string) {
let timeoutId: NodeJS.Timeout | null = null; let timeoutId: NodeJS.Timeout | null = null;
try { try {
@@ -56,31 +37,9 @@ async function withTimeout<T>(promise: Promise<T>, timeoutMs: number, label: str
} }
} }
async function fetchProviderModels(provider: Provider) {
if (provider === "openai") {
const page = await openaiClient().models.list();
return uniqSorted(page.data.map((model) => model.id).filter(isLikelyOpenAIResponsesModel));
}
if (provider === "anthropic") {
const page = await anthropicClient().models.list({ limit: 200 });
return uniqSorted(page.data.map((model) => model.id));
}
if (provider === "xai") {
const page = await xaiClient().models.list();
return uniqSorted(page.data.map((model) => model.id));
}
const page = await hermesAgentClient().models.list();
const models = page.data.map((model) => model.id);
if (env.HERMES_AGENT_MODEL) models.push(env.HERMES_AGENT_MODEL);
return uniqSorted(models);
}
async function refreshProviderModels(provider: Provider, logger?: FastifyBaseLogger) { async function refreshProviderModels(provider: Provider, logger?: FastifyBaseLogger) {
try { try {
const models = await withTimeout(fetchProviderModels(provider), MODEL_FETCH_TIMEOUT_MS, `${provider} model fetch`); const models = await withTimeout(fetchProviderCatalogModels(provider), MODEL_FETCH_TIMEOUT_MS, `${provider} model fetch`);
modelCatalog[provider] = { modelCatalog[provider] = {
models, models,
loadedAt: new Date().toISOString(), loadedAt: new Date().toISOString(),
@@ -90,7 +49,7 @@ async function refreshProviderModels(provider: Provider, logger?: FastifyBaseLog
} catch (err: any) { } catch (err: any) {
const message = err?.message ?? String(err); const message = err?.message ?? String(err);
const previous = modelCatalog[provider]; const previous = modelCatalog[provider];
const fallbackModels = provider === "hermes-agent" && env.HERMES_AGENT_MODEL ? [env.HERMES_AGENT_MODEL] : []; const fallbackModels = getProviderCatalogFallbackModels(provider);
modelCatalog[provider] = { modelCatalog[provider] = {
models: previous?.models.length ? previous.models : fallbackModels, models: previous?.models.length ? previous.models : fallbackModels,
loadedAt: previous?.loadedAt ?? null, loadedAt: previous?.loadedAt ?? null,
@@ -103,7 +62,7 @@ async function refreshProviderModels(provider: Provider, logger?: FastifyBaseLog
export async function refreshModelCatalog(logger?: FastifyBaseLogger) { export async function refreshModelCatalog(logger?: FastifyBaseLogger) {
if (catalogRefreshPromise) return catalogRefreshPromise; if (catalogRefreshPromise) return catalogRefreshPromise;
catalogRefreshPromise = Promise.all(getCatalogProviders().map((provider) => refreshProviderModels(provider, logger))) catalogRefreshPromise = Promise.all(listModelCatalogProviders().map((provider) => refreshProviderModels(provider, logger)))
.then(() => undefined) .then(() => undefined)
.finally(() => { .finally(() => {
catalogRefreshPromise = null; catalogRefreshPromise = null;
@@ -129,7 +88,7 @@ export function startModelCatalogRefreshLoop(logger?: FastifyBaseLogger) {
export function getModelCatalogSnapshot(): ModelCatalogSnapshot { export function getModelCatalogSnapshot(): ModelCatalogSnapshot {
const snapshot: ModelCatalogSnapshot = {}; const snapshot: ModelCatalogSnapshot = {};
for (const provider of getCatalogProviders()) { for (const provider of listModelCatalogProviders()) {
const entry = modelCatalog[provider] ?? { models: [], loadedAt: null, error: null }; const entry = modelCatalog[provider] ?? { models: [], loadedAt: null, error: null };
snapshot[provider] = { snapshot[provider] = {
models: [...entry.models], models: [...entry.models],
+5 -79
View File
@@ -1,8 +1,7 @@
import { performance } from "node:perf_hooks"; import { performance } from "node:perf_hooks";
import { prisma } from "../db.js"; import { prisma } from "../db.js";
import { anthropicClient, hermesAgentClient, openaiClient, xaiClient } from "./providers.js"; import { buildToolLogMessageData } from "./chat-tools.js";
import { buildToolLogMessageData, normalizeEnabledChatTools, runPlainChatCompletions, runToolAwareChatCompletions, runToolAwareOpenAIChat } from "./chat-tools.js"; import { getProviderChatAdapter } from "./provider-adapters.js";
import { buildAnthropicConversationMessage, getAnthropicSystemPrompt } from "./message-content.js";
import { toPrismaProvider } from "./provider-ids.js"; import { toPrismaProvider } from "./provider-ids.js";
import type { MultiplexRequest, MultiplexResponse, Provider } from "./types.js"; import type { MultiplexRequest, MultiplexResponse, Provider } from "./types.js";
@@ -47,15 +46,11 @@ export async function runMultiplex(req: MultiplexRequest): Promise<MultiplexResp
let usage: MultiplexResponse["usage"] | undefined; let usage: MultiplexResponse["usage"] | undefined;
let raw: unknown; let raw: unknown;
let toolMessages: ReturnType<typeof buildToolLogMessageData>[] = []; let toolMessages: ReturnType<typeof buildToolLogMessageData>[] = [];
const enabledTools = normalizeEnabledChatTools(req.enabledTools); const adapter = getProviderChatAdapter(req.provider);
const r = await adapter.complete({
if (req.provider === "openai" && enabledTools.length > 0) {
const client = openaiClient();
const r = await runToolAwareOpenAIChat({
client,
model: req.model, model: req.model,
messages: req.messages, messages: req.messages,
enabledTools, enabledTools: req.enabledTools,
userLocation: req.userLocation, userLocation: req.userLocation,
temperature: req.temperature, temperature: req.temperature,
maxTokens: req.maxTokens, maxTokens: req.maxTokens,
@@ -69,75 +64,6 @@ export async function runMultiplex(req: MultiplexRequest): Promise<MultiplexResp
outText = r.text; outText = r.text;
usage = r.usage; usage = r.usage;
toolMessages = r.toolEvents.map((event) => buildToolLogMessageData(call.chatId, event)); toolMessages = r.toolEvents.map((event) => buildToolLogMessageData(call.chatId, event));
} else if (req.provider === "xai" && enabledTools.length > 0) {
const client = xaiClient();
const r = await runToolAwareChatCompletions({
client,
model: req.model,
messages: req.messages,
enabledTools,
userLocation: req.userLocation,
temperature: req.temperature,
maxTokens: req.maxTokens,
logContext: {
provider: req.provider,
model: req.model,
chatId,
},
});
raw = r.raw;
outText = r.text;
usage = r.usage;
toolMessages = r.toolEvents.map((event) => buildToolLogMessageData(call.chatId, event));
} else if (req.provider === "openai" || req.provider === "xai" || req.provider === "hermes-agent") {
const client = req.provider === "openai" ? openaiClient() : req.provider === "xai" ? xaiClient() : hermesAgentClient();
const r = await runPlainChatCompletions({
client,
model: req.model,
messages: req.messages,
userLocation: req.userLocation,
temperature: req.temperature,
maxTokens: req.maxTokens,
logContext: {
provider: req.provider,
model: req.model,
chatId,
},
});
raw = r.raw;
outText = r.text;
usage = r.usage;
} else if (req.provider === "anthropic") {
const client = anthropicClient();
const system = getAnthropicSystemPrompt(req.messages, req.userLocation);
const msgs = req.messages.filter((message) => message.role !== "system").map((message) => buildAnthropicConversationMessage(message));
const r = await client.messages.create({
model: req.model,
system,
max_tokens: req.maxTokens ?? 1024,
temperature: req.temperature,
messages: msgs as any,
});
raw = r;
outText = r.content
.map((c: any) => (c.type === "text" ? c.text : ""))
.join("")
.trim();
// Anthropic usage (SDK typing varies by version)
const ru: any = (r as any).usage;
if (ru) {
usage = {
inputTokens: ru.input_tokens,
outputTokens: ru.output_tokens,
totalTokens: (ru.input_tokens ?? 0) + (ru.output_tokens ?? 0),
};
}
} else {
throw new Error(`unknown provider: ${req.provider}`);
}
const latencyMs = Math.round(performance.now() - t0); const latencyMs = Math.round(performance.now() - t0);
@@ -0,0 +1,386 @@
import {
appendDanglingToolIntentCorrection,
buildChatToolSystemPrompt,
executeToolCallAndBuildEvent,
getEnabledChatTools,
getUnstreamedText,
looksLikeDanglingToolIntent,
MAX_DANGLING_TOOL_INTENT_RETRIES,
MAX_TOOL_ROUNDS,
mergeUsage,
normalizeModelToolCalls,
prepareToolCallExecution,
type NormalizedToolCall,
type ToolAwareCompletionParams,
type ToolAwareCompletionResult,
type ToolAwareStreamingEvent,
type ToolExecutionEvent,
} from "../chat-tools.js";
import {
buildImageSummaryText,
buildSystemPromptAugmentationMessage,
buildTextAttachmentPrompt,
getImageAttachments,
getTextAttachments,
} from "../message-content.js";
import type { ChatMessage } from "../types.js";
function toContentParts(message: ChatMessage) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
if (!imageAttachments.length && !textAttachments.length) {
return message.content;
}
const parts: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
parts.push({
type: "image_url",
image_url: {
url: attachment.dataUrl,
detail: "auto",
},
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
parts.push({ type: "text", text: imageSummary });
}
for (const attachment of textAttachments) {
parts.push({ type: "text", text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
parts.push({ type: "text", text: message.content });
}
if (parts.length === 1 && parts[0]?.type === "text" && typeof parts[0].text === "string") {
return parts[0].text;
}
return parts;
}
function buildConversationMessage(message: ChatMessage) {
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
content: `Tool output (${name}):\n${message.content}`,
};
}
const out: Record<string, unknown> = {
role: message.role,
content: toContentParts(message),
};
if (message.name && (message.role === "assistant" || message.role === "user")) {
out.name = message.name;
}
return out;
}
function normalizeMessages(messages: ChatMessage[], userLocation?: string, params: Pick<ToolAwareCompletionParams, "enabledTools"> = {}) {
const normalized = messages.map((message) => buildConversationMessage(message));
return [{ role: "system", content: buildChatToolSystemPrompt(params) }, buildSystemPromptAugmentationMessage(userLocation), ...normalized];
}
function normalizePlainMessages(messages: ChatMessage[], userLocation?: string) {
return [buildSystemPromptAugmentationMessage(userLocation), ...messages.map((message) => buildConversationMessage(message))];
}
function extractContent(message: any) {
if (typeof message?.content === "string") return message.content;
if (!Array.isArray(message?.content)) return "";
return message.content
.map((part: any) => {
if (typeof part === "string") return part;
if (typeof part?.text === "string") return part.text;
if (typeof part?.content === "string") return part.content;
return "";
})
.join("");
}
export async function completeWithChatCompletionsApi(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const enabledTools = getEnabledChatTools(params);
if (!enabledTools.length) {
const completion = await params.client.chat.completions.create({
model: params.model,
messages: normalizePlainMessages(params.messages, params.userLocation),
temperature: params.temperature,
max_tokens: params.maxTokens,
} as any);
const usageAcc: Required<NonNullable<ToolAwareCompletionResult["usage"]>> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
const sawUsage = mergeUsage(usageAcc, completion?.usage);
const message = completion?.choices?.[0]?.message;
return {
text: extractContent(message),
usage: sawUsage ? usageAcc : undefined,
raw: { response: completion, api: "chat.completions" },
toolEvents: [],
};
}
const conversation: any[] = normalizeMessages(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<NonNullable<ToolAwareCompletionResult["usage"]>> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const completion = await params.client.chat.completions.create({
model: params.model,
messages: conversation,
temperature: params.temperature,
max_tokens: params.maxTokens,
tools: enabledTools,
tool_choice: "auto",
} as any);
rawResponses.push(completion);
sawUsage = mergeUsage(usageAcc, completion?.usage) || sawUsage;
const message = completion?.choices?.[0]?.message;
if (!message) {
return {
text: "",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, missingMessage: true },
toolEvents,
};
}
const toolCalls = Array.isArray(message.tool_calls) ? message.tool_calls : [];
if (!toolCalls.length) {
const text = typeof message.content === "string" ? message.content : "";
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(conversation, text);
continue;
}
return {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls },
toolEvents,
};
}
const normalizedToolCalls = normalizeModelToolCalls(toolCalls, round);
totalToolCalls += normalizedToolCalls.length;
const assistantToolCallMessage: any = {
role: "assistant",
tool_calls: normalizedToolCalls.map((call) => ({
id: call.id,
type: "function",
function: {
name: call.name,
arguments: call.arguments,
},
})),
};
if (typeof message.content === "string" && message.content.length) {
assistantToolCallMessage.content = message.content;
}
conversation.push(assistantToolCallMessage);
for (const call of normalizedToolCalls) {
const { execution } = prepareToolCallExecution(call);
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
conversation.push({
role: "tool",
tool_call_id: call.id,
content: JSON.stringify(toolResult),
});
}
}
return {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true },
toolEvents,
};
}
export async function* streamWithChatCompletionsApi(params: ToolAwareCompletionParams): AsyncGenerator<ToolAwareStreamingEvent> {
const enabledTools = getEnabledChatTools(params);
if (!enabledTools.length) {
const rawResponses: unknown[] = [];
const usageAcc: Required<NonNullable<ToolAwareCompletionResult["usage"]>> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let text = "";
const stream = await params.client.chat.completions.create({
model: params.model,
messages: normalizePlainMessages(params.messages, params.userLocation),
temperature: params.temperature,
max_tokens: params.maxTokens,
stream: true,
} as any);
for await (const chunk of stream as any as AsyncIterable<any>) {
rawResponses.push(chunk);
sawUsage = mergeUsage(usageAcc, chunk?.usage) || sawUsage;
const deltaText = chunk?.choices?.[0]?.delta?.content ?? "";
if (typeof deltaText === "string" && deltaText.length) {
text += deltaText;
yield { type: "delta", text: deltaText };
}
}
yield {
type: "done",
result: {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, api: "chat.completions" },
toolEvents: [],
},
};
return;
}
const conversation: any[] = normalizeMessages(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<NonNullable<ToolAwareCompletionResult["usage"]>> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const stream = await params.client.chat.completions.create({
model: params.model,
messages: conversation,
temperature: params.temperature,
max_tokens: params.maxTokens,
tools: enabledTools,
tool_choice: "auto",
stream: true,
stream_options: { include_usage: true },
} as any);
let roundText = "";
let streamedRoundText = "";
let roundHasToolCalls = false;
const roundToolCalls = new Map<number, { id?: string; name?: string; arguments: string }>();
for await (const chunk of stream as any as AsyncIterable<any>) {
rawResponses.push(chunk);
sawUsage = mergeUsage(usageAcc, chunk?.usage) || sawUsage;
const choice = chunk?.choices?.[0];
const deltaText = choice?.delta?.content ?? "";
if (typeof deltaText === "string" && deltaText.length) {
roundText += deltaText;
if (!roundHasToolCalls) {
streamedRoundText += deltaText;
yield { type: "delta", text: deltaText };
}
}
const deltaToolCalls = Array.isArray(choice?.delta?.tool_calls) ? choice.delta.tool_calls : [];
if (deltaToolCalls.length) {
roundHasToolCalls = true;
}
for (const toolCall of deltaToolCalls) {
const idx = typeof toolCall?.index === "number" ? toolCall.index : 0;
const entry = roundToolCalls.get(idx) ?? { arguments: "" };
if (typeof toolCall?.id === "string" && toolCall.id.length) {
entry.id = toolCall.id;
}
if (typeof toolCall?.function?.name === "string" && toolCall.function.name.length) {
entry.name = toolCall.function.name;
}
if (typeof toolCall?.function?.arguments === "string" && toolCall.function.arguments.length) {
entry.arguments += toolCall.function.arguments;
}
roundToolCalls.set(idx, entry);
}
}
const normalizedToolCalls: NormalizedToolCall[] = [...roundToolCalls.entries()]
.sort((a, b) => a[0] - b[0])
.map(([_, call], index) => ({
id: call.id ?? `tool_call_${round}_${index}`,
name: call.name ?? "unknown_tool",
arguments: call.arguments || "{}",
}));
if (!normalizedToolCalls.length) {
if (!streamedRoundText && danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(roundText)) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(conversation, roundText);
continue;
}
const unstreamedText = getUnstreamedText(roundText, streamedRoundText);
if (unstreamedText) {
yield { type: "delta", text: unstreamedText };
}
yield {
type: "done",
result: {
text: roundText,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls },
toolEvents,
},
};
return;
}
totalToolCalls += normalizedToolCalls.length;
const assistantToolCallMessage: any = {
role: "assistant",
tool_calls: normalizedToolCalls.map((call) => ({
id: call.id,
type: "function",
function: {
name: call.name,
arguments: call.arguments,
},
})),
};
if (roundText) {
assistantToolCallMessage.content = roundText;
}
conversation.push(assistantToolCallMessage);
for (const call of normalizedToolCalls) {
const { event: initiatedEvent, execution } = prepareToolCallExecution(call);
yield { type: "tool_call", event: initiatedEvent };
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
yield { type: "tool_call", event };
conversation.push({
role: "tool",
tool_call_id: call.id,
content: JSON.stringify(toolResult),
});
}
}
yield {
type: "done",
result: {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true },
toolEvents,
},
};
}
+501
View File
@@ -0,0 +1,501 @@
import {
buildChatToolSystemPrompt,
executeToolCallAndBuildEvent,
getEnabledChatTools,
getUnstreamedText,
looksLikeDanglingToolIntent,
MAX_DANGLING_TOOL_INTENT_RETRIES,
MAX_TOOL_ROUNDS,
prepareToolCallExecution,
type NormalizedToolCall,
type ToolAwareCompletionParams,
type ToolAwareCompletionResult,
type ToolAwareStreamingEvent,
type ToolAwareUsage,
type ToolExecutionEvent,
} from "../chat-tools.js";
import {
buildImageSummaryText,
buildTextAttachmentPrompt,
buildTopLevelSystemPrompt,
getImageAttachments,
getTextAttachments,
parseImageDataUrl,
} from "../message-content.js";
import type { ChatMessage } from "../types.js";
type GeminiClient = {
apiKey: string;
baseURL: string;
};
const INTERNAL_CORRECTION =
"Internal correction: the previous assistant message claimed it would run a tool, but no tool call was made. If the task needs an available tool, call it now. Otherwise provide the final answer directly without saying you will run a tool.";
function normalizeModelResourceName(model: string) {
const trimmed = model.trim().replace(/^\/+/, "");
return trimmed.startsWith("models/") || trimmed.startsWith("tunedModels/") ? trimmed : `models/${trimmed}`;
}
function geminiUrl(client: GeminiClient, model: string, method: "generateContent" | "streamGenerateContent", extraParams: Record<string, string> = {}) {
const url = new URL(`${client.baseURL.replace(/\/+$/, "")}/${normalizeModelResourceName(model)}:${method}`);
url.searchParams.set("key", client.apiKey);
for (const [key, value] of Object.entries(extraParams)) {
url.searchParams.set(key, value);
}
return url;
}
function generationConfig(params: Pick<ToolAwareCompletionParams, "temperature" | "maxTokens">) {
const config: Record<string, unknown> = {};
if (params.temperature !== undefined) config.temperature = params.temperature;
if (params.maxTokens !== undefined) config.maxOutputTokens = params.maxTokens;
return Object.keys(config).length ? config : undefined;
}
function toGeminiJsonSchema(schema: unknown): Record<string, unknown> | undefined {
if (!schema || typeof schema !== "object" || Array.isArray(schema)) return undefined;
const input = schema as Record<string, unknown>;
const output: Record<string, unknown> = {};
if (typeof input.type === "string") output.type = input.type;
if (typeof input.description === "string") output.description = input.description;
if (typeof input.format === "string") output.format = input.format;
if (typeof input.nullable === "boolean") output.nullable = input.nullable;
if (Array.isArray(input.enum)) output.enum = input.enum.filter((value) => typeof value === "string");
if (Array.isArray(input.required)) output.required = input.required.filter((value) => typeof value === "string");
const items = toGeminiJsonSchema(input.items);
if (items) output.items = items;
if (input.properties && typeof input.properties === "object" && !Array.isArray(input.properties)) {
const properties: Record<string, unknown> = {};
for (const [key, value] of Object.entries(input.properties)) {
const propertySchema = toGeminiJsonSchema(value);
if (propertySchema) properties[key] = propertySchema;
}
if (Object.keys(properties).length) output.properties = properties;
}
return Object.keys(output).length ? output : undefined;
}
function toGeminiTools(tools: any[]) {
const functionDeclarations = tools
.map((tool) => {
if (tool?.type !== "function") return null;
const declaration: Record<string, unknown> = {
name: tool.function.name,
description: tool.function.description,
};
const parameters = toGeminiJsonSchema(tool.function.parameters);
if (parameters) declaration.parameters = parameters;
return declaration;
})
.filter(Boolean);
return functionDeclarations.length ? [{ functionDeclarations }] : undefined;
}
function toContentParts(message: ChatMessage) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
const parts: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
const source = parseImageDataUrl(attachment);
parts.push({
inlineData: {
mimeType: source.mediaType,
data: source.data,
},
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
parts.push({ text: imageSummary });
}
for (const attachment of textAttachments) {
parts.push({ text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
parts.push({ text: message.content });
}
return parts.length ? parts : [{ text: "" }];
}
function buildConversationContent(message: ChatMessage) {
if (message.role === "system") {
throw new Error("System messages must be handled separately for Gemini.");
}
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
parts: [{ text: `Tool output (${name}):\n${message.content}` }],
};
}
return {
role: message.role === "assistant" ? "model" : "user",
parts: toContentParts(message),
};
}
function buildBaseContents(messages: ChatMessage[]) {
return messages.filter((message) => message.role !== "system").map((message) => buildConversationContent(message));
}
function buildSystemInstruction(params: ToolAwareCompletionParams, toolSystemPrompt?: string) {
const text = buildTopLevelSystemPrompt(params.messages, params.userLocation, toolSystemPrompt);
return text ? { parts: [{ text }] } : undefined;
}
function mergeUsage(acc: Required<ToolAwareUsage>, usage: any) {
const normalized = normalizeUsage(usage);
if (!normalized) return false;
acc.inputTokens += normalized.inputTokens;
acc.outputTokens += normalized.outputTokens;
acc.totalTokens += normalized.totalTokens;
return true;
}
function normalizeUsage(usage: any) {
if (!usage) return null;
const inputTokens = usage.promptTokenCount ?? 0;
const outputTokens = usage.candidatesTokenCount ?? 0;
const totalTokens = usage.totalTokenCount ?? inputTokens + outputTokens;
return { inputTokens, outputTokens, totalTokens };
}
function getCandidate(response: any) {
return Array.isArray(response?.candidates) ? response.candidates[0] : null;
}
function getParts(response: any) {
const parts = getCandidate(response)?.content?.parts;
return Array.isArray(parts) ? parts : [];
}
function extractText(response: any) {
return getParts(response)
.map((part: any) => (typeof part?.text === "string" ? part.text : ""))
.join("");
}
function stringifyToolArgs(args: unknown) {
try {
return JSON.stringify(args ?? {});
} catch {
return "{}";
}
}
function normalizeToolCallsFromParts(parts: any[], round: number): NormalizedToolCall[] {
return parts
.filter((part) => part?.functionCall)
.map((part, index) => ({
id: part.functionCall.id ?? `tool_call_${round}_${index}`,
name: part.functionCall.name ?? "unknown_tool",
arguments: stringifyToolArgs(part.functionCall.args),
}));
}
function buildFunctionResponsePart(call: NormalizedToolCall, toolResult: unknown) {
return {
functionResponse: {
id: call.id,
name: call.name,
response: toolResult,
},
};
}
function appendCorrection(conversation: any[], text: string) {
conversation.push({ role: "model", parts: [{ text }] });
conversation.push({ role: "user", parts: [{ text: INTERNAL_CORRECTION }] });
}
async function parseGeminiResponse(response: Response) {
const bodyText = await response.text();
let body: any = null;
try {
body = bodyText ? JSON.parse(bodyText) : null;
} catch {
body = { raw: bodyText };
}
if (!response.ok) {
throw new Error(body?.error?.message ?? `Gemini API request failed with status ${response.status}.`);
}
return body;
}
async function generateContent(params: ToolAwareCompletionParams, body: Record<string, unknown>) {
const response = await fetch(geminiUrl(params.client, params.model, "generateContent"), {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
return parseGeminiResponse(response);
}
function getFailureMessage(response: any, text: string, toolCallCount: number) {
const promptBlockReason = response?.promptFeedback?.blockReason;
if (promptBlockReason) return `Gemini prompt blocked: ${promptBlockReason}.`;
const candidate = getCandidate(response);
const finishReason = candidate?.finishReason;
if (!finishReason || finishReason === "STOP" || finishReason === "MAX_TOKENS") return null;
if (text || toolCallCount > 0) return null;
return candidate?.finishMessage ?? `Gemini response stopped: ${finishReason}.`;
}
function buildRequest(params: ToolAwareCompletionParams, conversation: any[], enabledTools: any[] = []) {
const tools = toGeminiTools(enabledTools);
return {
contents: conversation,
systemInstruction: buildSystemInstruction(params, enabledTools.length ? buildChatToolSystemPrompt(params) : undefined),
generationConfig: generationConfig(params),
tools,
toolConfig: tools ? { functionCallingConfig: { mode: "AUTO" } } : undefined,
};
}
export async function completeWithGeminiApi(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const enabledTools = getEnabledChatTools(params);
const conversation = buildBaseContents(params.messages);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const response = await generateContent(params, buildRequest(params, conversation, enabledTools));
rawResponses.push(response);
sawUsage = mergeUsage(usageAcc, response?.usageMetadata) || sawUsage;
const parts = getParts(response);
const text = extractText(response);
const normalizedToolCalls = normalizeToolCallsFromParts(parts, round);
const failureMessage = getFailureMessage(response, text, normalizedToolCalls.length);
if (failureMessage) throw new Error(failureMessage);
if (!normalizedToolCalls.length) {
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendCorrection(conversation, text);
continue;
}
return {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, api: "gemini.generateContent" },
toolEvents,
};
}
totalToolCalls += normalizedToolCalls.length;
conversation.push({ role: "model", parts });
const toolResultParts: any[] = [];
for (const call of normalizedToolCalls) {
const { execution } = prepareToolCallExecution(call);
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
toolResultParts.push(buildFunctionResponsePart(call, toolResult));
}
conversation.push({ role: "user", parts: toolResultParts });
}
return {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "gemini.generateContent" },
toolEvents,
};
}
function findSseBoundary(buffer: string) {
const crlf = buffer.indexOf("\r\n\r\n");
const lf = buffer.indexOf("\n\n");
if (crlf === -1) return lf === -1 ? null : { index: lf, length: 2 };
if (lf === -1) return { index: crlf, length: 4 };
return crlf < lf ? { index: crlf, length: 4 } : { index: lf, length: 2 };
}
function parseSseEvent(rawEvent: string) {
const data = rawEvent
.split(/\r?\n/)
.filter((line) => line.startsWith("data:"))
.map((line) => line.slice("data:".length).trimStart())
.join("\n")
.trim();
if (!data || data === "[DONE]") return null;
return JSON.parse(data);
}
async function* streamGeminiResponses(params: ToolAwareCompletionParams, body: Record<string, unknown>) {
const response = await fetch(geminiUrl(params.client, params.model, "streamGenerateContent", { alt: "sse" }), {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
if (!response.ok) {
await parseGeminiResponse(response);
return;
}
if (!response.body) {
throw new Error("Gemini stream response did not include a body.");
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
let boundary = findSseBoundary(buffer);
while (boundary) {
const rawEvent = buffer.slice(0, boundary.index);
buffer = buffer.slice(boundary.index + boundary.length);
const event = parseSseEvent(rawEvent);
if (event) yield event;
boundary = findSseBoundary(buffer);
}
}
buffer += decoder.decode();
const tail = buffer.trim();
if (tail) {
const event = parseSseEvent(tail);
if (event) yield event;
}
}
export async function* streamWithGeminiApi(params: ToolAwareCompletionParams): AsyncGenerator<ToolAwareStreamingEvent> {
const enabledTools = getEnabledChatTools(params);
const conversation = buildBaseContents(params.messages);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
if (!enabledTools.length) {
let text = "";
let latestUsage: any = null;
for await (const response of streamGeminiResponses(params, buildRequest(params, conversation))) {
rawResponses.push(response);
if (response?.usageMetadata) latestUsage = response.usageMetadata;
const failureMessage = getFailureMessage(response, extractText(response), 0);
if (failureMessage) throw new Error(failureMessage);
const delta = extractText(response);
if (delta) {
text += delta;
yield { type: "delta", text: delta };
}
}
sawUsage = mergeUsage(usageAcc, latestUsage) || sawUsage;
yield {
type: "done",
result: {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: 0, api: "gemini.streamGenerateContent" },
toolEvents: [],
},
};
return;
}
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const roundParts: any[] = [];
let roundText = "";
let latestRoundResponse: any = null;
let latestRoundUsage: any = null;
for await (const response of streamGeminiResponses(params, buildRequest(params, conversation, enabledTools))) {
rawResponses.push(response);
latestRoundResponse = response;
if (response?.usageMetadata) latestRoundUsage = response.usageMetadata;
roundParts.push(...getParts(response));
roundText += extractText(response);
}
sawUsage = mergeUsage(usageAcc, latestRoundUsage) || sawUsage;
const normalizedToolCalls = normalizeToolCallsFromParts(roundParts, round);
const failureMessage = getFailureMessage(latestRoundResponse ?? { candidates: [{ content: { parts: roundParts } }] }, roundText, normalizedToolCalls.length);
if (failureMessage) throw new Error(failureMessage);
if (!normalizedToolCalls.length) {
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(roundText)) {
danglingToolIntentRetries += 1;
appendCorrection(conversation, roundText);
continue;
}
const unstreamedText = getUnstreamedText(roundText, "");
if (unstreamedText) {
yield { type: "delta", text: unstreamedText };
}
yield {
type: "done",
result: {
text: roundText,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, api: "gemini.streamGenerateContent" },
toolEvents,
},
};
return;
}
totalToolCalls += normalizedToolCalls.length;
conversation.push({ role: "model", parts: roundParts });
const toolResultParts: any[] = [];
for (const call of normalizedToolCalls) {
const { event: initiatedEvent, execution } = prepareToolCallExecution(call);
yield { type: "tool_call", event: initiatedEvent };
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
yield { type: "tool_call", event };
toolResultParts.push(buildFunctionResponsePart(call, toolResult));
}
conversation.push({ role: "user", parts: toolResultParts });
}
yield {
type: "done",
result: {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: {
streamed: true,
responses: rawResponses,
toolCallsUsed: totalToolCalls,
toolCallLimitReached: true,
api: "gemini.streamGenerateContent",
},
toolEvents,
},
};
}
+511
View File
@@ -0,0 +1,511 @@
import {
buildChatToolSystemPrompt,
executeToolCallAndBuildEvent,
getEnabledChatTools,
looksLikeDanglingToolIntent,
MAX_DANGLING_TOOL_INTENT_RETRIES,
MAX_TOOL_ROUNDS,
parseToolArgs,
prepareToolCallExecution,
type NormalizedToolCall,
type ToolAwareCompletionParams,
type ToolAwareCompletionResult,
type ToolAwareStreamingEvent,
type ToolAwareUsage,
type ToolExecutionEvent,
type ToolRunOutcome,
} from "../chat-tools.js";
import {
buildImageSummaryText,
buildTextAttachmentPrompt,
buildTopLevelSystemPrompt,
getImageAttachments,
getTextAttachments,
parseImageDataUrl,
} from "../message-content.js";
import type { ChatMessage } from "../types.js";
const INTERNAL_CORRECTION =
"Internal correction: the previous assistant message claimed it would run a tool, but no tool call was made. If the task needs an available tool, call it now. Otherwise provide the final answer directly without saying you will run a tool.";
const DEFAULT_ANTHROPIC_MAX_TOKENS = 128_000;
const MODEL_MAX_TOKENS_CACHE_MS = 24 * 60 * 60 * 1000;
const modelMaxTokensCache = new Map<string, { maxTokens: number; expiresAt: number }>();
function readMaxTokens(value: unknown) {
return Number.isSafeInteger(value) && (value as number) > 0 ? (value as number) : undefined;
}
function getModelInfoMaxTokens(modelInfo: any) {
return readMaxTokens(modelInfo?.max_tokens) ?? readMaxTokens(modelInfo?.maxTokens);
}
async function getMessagesMaxTokens(params: ToolAwareCompletionParams) {
if (params.maxTokens) return params.maxTokens;
const cached = modelMaxTokensCache.get(params.model);
if (cached && cached.expiresAt > Date.now()) return cached.maxTokens;
try {
const retrieve = params.client?.models?.retrieve;
if (typeof retrieve === "function") {
const modelInfo = await retrieve.call(params.client.models, params.model);
const maxTokens = getModelInfoMaxTokens(modelInfo);
if (maxTokens) {
modelMaxTokensCache.set(params.model, {
maxTokens,
expiresAt: Date.now() + MODEL_MAX_TOKENS_CACHE_MS,
});
return maxTokens;
}
}
} catch {
// Fall back to the documented max for Claude Opus 4.8 and related high-output models.
}
return DEFAULT_ANTHROPIC_MAX_TOKENS;
}
function toTools(tools: any[]) {
return tools
.map((tool) => {
if (tool?.type !== "function") return null;
return {
name: tool.function.name,
description: tool.function.description,
input_schema: tool.function.parameters,
};
})
.filter(Boolean);
}
function toContentBlocks(message: ChatMessage) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
if (!imageAttachments.length && !textAttachments.length) {
return message.content;
}
const blocks: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
const source = parseImageDataUrl(attachment);
blocks.push({
type: "image",
source: {
type: "base64",
media_type: source.mediaType,
data: source.data,
},
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
blocks.push({ type: "text", text: imageSummary });
}
for (const attachment of textAttachments) {
blocks.push({ type: "text", text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
blocks.push({ type: "text", text: message.content });
}
if (blocks.length === 1 && blocks[0]?.type === "text" && typeof blocks[0].text === "string") {
return blocks[0].text;
}
return blocks;
}
function buildConversationMessage(message: ChatMessage) {
if (message.role === "system") {
throw new Error("System messages must be handled separately for top-level-system protocols.");
}
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
content: `Tool output (${name}):\n${message.content}`,
};
}
return {
role: message.role === "assistant" ? "assistant" : "user",
content: toContentBlocks(message),
};
}
function buildBaseMessages(params: ToolAwareCompletionParams) {
return params.messages.filter((message) => message.role !== "system").map((message) => buildConversationMessage(message));
}
function stringifyToolInput(input: unknown) {
if (typeof input === "string") return input;
try {
return JSON.stringify(input ?? {});
} catch {
return "{}";
}
}
function normalizeToolCalls(content: any[], round: number): NormalizedToolCall[] {
return content
.filter((item) => item?.type === "tool_use")
.map((call: any, index: number) => ({
id: call?.id ?? `tool_call_${round}_${index}`,
name: call?.name ?? "unknown_tool",
arguments: stringifyToolInput(call?.input),
}));
}
function extractText(response: any) {
if (!Array.isArray(response?.content)) return "";
return response.content
.map((content: any) => (content?.type === "text" && typeof content.text === "string" ? content.text : ""))
.join("")
.trim();
}
function buildToolResultBlock(call: NormalizedToolCall, toolResult: ToolRunOutcome) {
return {
type: "tool_result",
tool_use_id: call.id,
content: JSON.stringify(toolResult),
is_error: !toolResult.ok,
};
}
function appendCorrection(conversation: any[], text: string) {
conversation.push({ role: "assistant", content: text });
conversation.push({
role: "user",
content: INTERNAL_CORRECTION,
});
}
function mergeUsage(acc: Required<ToolAwareUsage>, usage: any) {
if (!usage) return false;
const inputTokens = usage.input_tokens ?? 0;
const outputTokens = usage.output_tokens ?? 0;
acc.inputTokens += inputTokens;
acc.outputTokens += outputTokens;
acc.totalTokens += inputTokens + outputTokens;
return true;
}
export async function completeWithMessagesApi(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const enabledTools = getEnabledChatTools(params);
const maxTokens = await getMessagesMaxTokens(params);
if (!enabledTools.length) {
const response = await params.client.messages.create({
model: params.model,
system: buildTopLevelSystemPrompt(params.messages, params.userLocation),
max_tokens: maxTokens,
temperature: params.temperature,
messages: buildBaseMessages(params),
} as any);
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
const sawUsage = mergeUsage(usageAcc, response?.usage);
return {
text: extractText(response),
usage: sawUsage ? usageAcc : undefined,
raw: { response, api: "messages" },
toolEvents: [],
};
}
const conversation: any[] = buildBaseMessages(params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const response = await params.client.messages.create({
model: params.model,
system: buildTopLevelSystemPrompt(params.messages, params.userLocation, buildChatToolSystemPrompt(params)),
max_tokens: maxTokens,
temperature: params.temperature,
messages: conversation,
tools: toTools(enabledTools),
tool_choice: { type: "auto" },
} as any);
rawResponses.push(response);
sawUsage = mergeUsage(usageAcc, response?.usage) || sawUsage;
const content = Array.isArray(response?.content) ? response.content : [];
const normalizedToolCalls = normalizeToolCalls(content, round);
if (!normalizedToolCalls.length) {
const text = extractText(response);
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendCorrection(conversation, text);
continue;
}
return {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, api: "messages" },
toolEvents,
};
}
totalToolCalls += normalizedToolCalls.length;
conversation.push({
role: "assistant",
content,
});
const toolResultBlocks: any[] = [];
for (const call of normalizedToolCalls) {
const { execution } = prepareToolCallExecution(call);
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
toolResultBlocks.push(buildToolResultBlock(call, toolResult));
}
conversation.push({
role: "user",
content: toolResultBlocks,
});
}
return {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "messages" },
toolEvents,
};
}
export async function* streamWithMessagesApi(params: ToolAwareCompletionParams): AsyncGenerator<ToolAwareStreamingEvent> {
const enabledTools = getEnabledChatTools(params);
const maxTokens = await getMessagesMaxTokens(params);
if (!enabledTools.length) {
const rawResponses: unknown[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let roundInputTokens = 0;
let roundOutputTokens = 0;
let text = "";
const stream = await params.client.messages.create({
model: params.model,
system: buildTopLevelSystemPrompt(params.messages, params.userLocation),
max_tokens: maxTokens,
temperature: params.temperature,
messages: buildBaseMessages(params),
stream: true,
} as any);
for await (const ev of stream as any as AsyncIterable<any>) {
rawResponses.push(ev);
if (ev?.type === "message_start" && ev?.message?.usage) {
roundInputTokens = ev.message.usage.input_tokens ?? roundInputTokens;
sawUsage = true;
}
if (ev?.type === "content_block_delta" && ev?.delta?.type === "text_delta") {
const delta = ev.delta.text ?? "";
if (delta) {
text += delta;
yield { type: "delta", text: delta };
}
}
if (ev?.type === "message_delta" && ev.usage) {
roundInputTokens = ev.usage.input_tokens ?? roundInputTokens;
roundOutputTokens = ev.usage.output_tokens ?? roundOutputTokens;
sawUsage = true;
}
}
if (sawUsage) {
usageAcc.inputTokens += roundInputTokens;
usageAcc.outputTokens += roundOutputTokens;
usageAcc.totalTokens += roundInputTokens + roundOutputTokens;
}
yield {
type: "done",
result: {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: 0, api: "messages" },
toolEvents: [],
},
};
return;
}
const conversation: any[] = buildBaseMessages(params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const stream = await params.client.messages.create({
model: params.model,
system: buildTopLevelSystemPrompt(params.messages, params.userLocation, buildChatToolSystemPrompt(params)),
max_tokens: maxTokens,
temperature: params.temperature,
messages: conversation,
tools: toTools(enabledTools),
tool_choice: { type: "auto" },
stream: true,
} as any);
const contentByIndex = new Map<number, any>();
const toolArgumentByIndex = new Map<number, string>();
let roundText = "";
let roundHasToolCalls = false;
let roundInputTokens = 0;
let roundOutputTokens = 0;
let sawRoundUsage = false;
for await (const ev of stream as any as AsyncIterable<any>) {
rawResponses.push(ev);
if (ev?.type === "message_start" && ev?.message?.usage) {
roundInputTokens = ev.message.usage.input_tokens ?? roundInputTokens;
sawRoundUsage = true;
}
if (ev?.type === "content_block_start" && typeof ev.index === "number") {
const block = ev.content_block ?? {};
if (block.type === "tool_use") {
roundHasToolCalls = true;
contentByIndex.set(ev.index, {
type: "tool_use",
id: block.id,
name: block.name,
input: block.input ?? {},
});
toolArgumentByIndex.set(ev.index, "");
} else if (block.type === "text") {
contentByIndex.set(ev.index, {
type: "text",
text: typeof block.text === "string" ? block.text : "",
});
} else if (block.type) {
contentByIndex.set(ev.index, block);
}
}
if (ev?.type === "content_block_delta" && typeof ev.index === "number") {
if (ev.delta?.type === "text_delta") {
const delta = typeof ev.delta.text === "string" ? ev.delta.text : "";
if (delta) {
const block = contentByIndex.get(ev.index) ?? { type: "text", text: "" };
if (block.type === "text") {
block.text = `${typeof block.text === "string" ? block.text : ""}${delta}`;
contentByIndex.set(ev.index, block);
}
roundText += delta;
}
} else if (ev.delta?.type === "input_json_delta") {
roundHasToolCalls = true;
const partialJson = typeof ev.delta.partial_json === "string" ? ev.delta.partial_json : "";
toolArgumentByIndex.set(ev.index, `${toolArgumentByIndex.get(ev.index) ?? ""}${partialJson}`);
}
}
if (ev?.type === "content_block_stop" && typeof ev.index === "number") {
const block = contentByIndex.get(ev.index);
if (block?.type === "tool_use") {
const rawArguments = toolArgumentByIndex.get(ev.index) || stringifyToolInput(block.input);
try {
block.input = parseToolArgs(rawArguments);
} catch {
block.input = {};
}
contentByIndex.set(ev.index, block);
}
}
if (ev?.type === "message_delta" && ev.usage) {
roundInputTokens = ev.usage.input_tokens ?? roundInputTokens;
roundOutputTokens = ev.usage.output_tokens ?? roundOutputTokens;
sawRoundUsage = true;
}
}
if (sawRoundUsage) {
usageAcc.inputTokens += roundInputTokens;
usageAcc.outputTokens += roundOutputTokens;
usageAcc.totalTokens += roundInputTokens + roundOutputTokens;
sawUsage = true;
}
const indexedContent = [...contentByIndex.entries()].sort((a, b) => a[0] - b[0]);
const assistantContent = indexedContent.map(([, block]) => block);
const normalizedToolCalls: NormalizedToolCall[] = indexedContent
.filter(([, block]) => block?.type === "tool_use")
.map(([index, block], callIndex) => ({
id: block.id ?? `tool_call_${round}_${callIndex}`,
name: block.name ?? "unknown_tool",
arguments: toolArgumentByIndex.get(index) || stringifyToolInput(block.input),
}));
if (!normalizedToolCalls.length) {
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(roundText)) {
danglingToolIntentRetries += 1;
appendCorrection(conversation, roundText);
continue;
}
if (roundText) {
yield { type: "delta", text: roundText };
}
yield {
type: "done",
result: {
text: roundText,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, api: "messages" },
toolEvents,
},
};
return;
}
totalToolCalls += normalizedToolCalls.length;
conversation.push({
role: "assistant",
content: assistantContent,
});
const toolResultBlocks: any[] = [];
for (const call of normalizedToolCalls) {
const { event: initiatedEvent, execution } = prepareToolCallExecution(call);
yield { type: "tool_call", event: initiatedEvent };
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
yield { type: "tool_call", event };
toolResultBlocks.push(buildToolResultBlock(call, toolResult));
}
conversation.push({
role: "user",
content: toolResultBlocks,
});
}
yield {
type: "done",
result: {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "messages" },
toolEvents,
},
};
}
+332
View File
@@ -0,0 +1,332 @@
import {
appendDanglingToolIntentCorrection,
buildChatToolSystemPrompt,
executeToolCallAndBuildEvent,
getEnabledChatTools,
getUnstreamedText,
looksLikeDanglingToolIntent,
MAX_DANGLING_TOOL_INTENT_RETRIES,
MAX_TOOL_ROUNDS,
prepareToolCallExecution,
type NormalizedToolCall,
type ToolAwareCompletionParams,
type ToolAwareCompletionResult,
type ToolAwareStreamingEvent,
type ToolAwareUsage,
type ToolExecutionEvent,
} from "../chat-tools.js";
import {
buildImageSummaryText,
buildSystemPromptAugmentationMessage,
buildTextAttachmentPrompt,
getImageAttachments,
getTextAttachments,
} from "../message-content.js";
import type { ChatMessage } from "../types.js";
function toResponsesTools(tools: any[]) {
return tools.map((tool) => {
if (tool?.type !== "function") return tool;
return {
type: "function",
name: tool.function.name,
description: tool.function.description,
parameters: tool.function.parameters,
strict: false,
};
});
}
function toContentParts(message: ChatMessage) {
const imageAttachments = getImageAttachments(message);
const textAttachments = getTextAttachments(message);
if (!imageAttachments.length && !textAttachments.length) {
return message.content;
}
const parts: Array<Record<string, unknown>> = [];
for (const attachment of imageAttachments) {
parts.push({
type: "input_image",
image_url: attachment.dataUrl,
detail: "auto",
});
}
const imageSummary = buildImageSummaryText(imageAttachments);
if (imageSummary) {
parts.push({ type: "input_text", text: imageSummary });
}
for (const attachment of textAttachments) {
parts.push({ type: "input_text", text: buildTextAttachmentPrompt(attachment) });
}
if (message.content.trim()) {
parts.push({ type: "input_text", text: message.content });
}
if (parts.length === 1 && parts[0]?.type === "input_text" && typeof parts[0].text === "string") {
return parts[0].text;
}
return parts;
}
function buildInputMessage(message: ChatMessage) {
if (message.role === "tool") {
const name = message.name?.trim() || "tool";
return {
role: "user",
content: `Tool output (${name}):\n${message.content}`,
};
}
return {
role: message.role,
content: toContentParts(message),
};
}
function normalizeInput(messages: ChatMessage[], userLocation?: string, params: Pick<ToolAwareCompletionParams, "enabledTools"> = {}) {
const normalized = messages.map((message) => buildInputMessage(message));
return [{ role: "system", content: buildChatToolSystemPrompt(params) }, buildSystemPromptAugmentationMessage(userLocation), ...normalized];
}
function mergeUsage(acc: Required<ToolAwareUsage>, usage: any) {
if (!usage) return false;
acc.inputTokens += usage.input_tokens ?? 0;
acc.outputTokens += usage.output_tokens ?? 0;
acc.totalTokens += usage.total_tokens ?? 0;
return true;
}
function getOutputItems(response: any) {
return Array.isArray(response?.output) ? response.output : [];
}
function extractText(response: any, fallback = "") {
if (typeof response?.output_text === "string") return response.output_text;
const parts: string[] = [];
for (const item of getOutputItems(response)) {
if (item?.type !== "message" || !Array.isArray(item.content)) continue;
for (const content of item.content) {
if (content?.type === "output_text" && typeof content.text === "string") {
parts.push(content.text);
} else if (content?.type === "refusal" && typeof content.refusal === "string") {
parts.push(content.refusal);
}
}
}
return parts.join("") || fallback;
}
function getFailureMessage(response: any) {
if (response?.status !== "failed" && response?.status !== "incomplete") return null;
const errorMessage = typeof response?.error?.message === "string" ? response.error.message : null;
const incompleteReason = typeof response?.incomplete_details?.reason === "string" ? response.incomplete_details.reason : null;
return errorMessage ?? (incompleteReason ? `Response incomplete: ${incompleteReason}` : `Response ${response.status}.`);
}
function normalizeToolCalls(outputItems: any[], round: number): NormalizedToolCall[] {
return outputItems
.filter((item) => item?.type === "function_call")
.map((call: any, index: number) => ({
id: call.call_id ?? call.id ?? `tool_call_${round}_${index}`,
name: call.name ?? "unknown_tool",
arguments: call.arguments ?? "{}",
}));
}
export async function completeWithResponsesApi(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult> {
const enabledTools = getEnabledChatTools(params);
const input: any[] = normalizeInput(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const response = await params.client.responses.create({
model: params.model,
input,
temperature: params.temperature,
max_output_tokens: params.maxTokens,
tools: toResponsesTools(enabledTools),
tool_choice: "auto",
parallel_tool_calls: true,
store: true,
} as any);
rawResponses.push(response);
sawUsage = mergeUsage(usageAcc, response?.usage) || sawUsage;
const failureMessage = getFailureMessage(response);
if (failureMessage) {
throw new Error(failureMessage);
}
const outputItems = getOutputItems(response);
const normalizedToolCalls = normalizeToolCalls(outputItems, round);
if (!normalizedToolCalls.length) {
const text = extractText(response);
if (danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(input, text);
continue;
}
return {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, api: "responses" },
toolEvents,
};
}
totalToolCalls += normalizedToolCalls.length;
input.push(...outputItems);
for (const call of normalizedToolCalls) {
const { execution } = prepareToolCallExecution(call);
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
input.push({
type: "function_call_output",
call_id: call.id,
output: JSON.stringify(toolResult),
});
}
}
return {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "responses" },
toolEvents,
};
}
export async function* streamWithResponsesApi(params: ToolAwareCompletionParams): AsyncGenerator<ToolAwareStreamingEvent> {
const enabledTools = getEnabledChatTools(params);
const input: any[] = normalizeInput(params.messages, params.userLocation, params);
const rawResponses: unknown[] = [];
const toolEvents: ToolExecutionEvent[] = [];
const usageAcc: Required<ToolAwareUsage> = { inputTokens: 0, outputTokens: 0, totalTokens: 0 };
let sawUsage = false;
let totalToolCalls = 0;
let danglingToolIntentRetries = 0;
for (let round = 0; round < MAX_TOOL_ROUNDS; round += 1) {
const stream = await params.client.responses.create({
model: params.model,
input,
temperature: params.temperature,
max_output_tokens: params.maxTokens,
tools: toResponsesTools(enabledTools),
tool_choice: "auto",
parallel_tool_calls: true,
store: true,
stream: true,
} as any);
let roundText = "";
let streamedRoundText = "";
let roundHasToolCalls = false;
let canStreamRoundText = false;
let completedResponse: any | null = null;
const completedOutputItems: any[] = [];
for await (const event of stream as any as AsyncIterable<any>) {
rawResponses.push(event);
if (event?.type === "response.output_text.delta" && typeof event.delta === "string") {
roundText += event.delta;
if (canStreamRoundText && !roundHasToolCalls && event.delta.length) {
streamedRoundText += event.delta;
yield { type: "delta", text: event.delta };
}
} else if (event?.type === "response.output_item.added" && event.item) {
if (event.item.type === "function_call") {
roundHasToolCalls = true;
canStreamRoundText = false;
} else if (event.item.type === "message" && !roundHasToolCalls) {
canStreamRoundText = true;
}
} else if (event?.type === "response.output_item.done" && event.item) {
completedOutputItems[event.output_index ?? completedOutputItems.length] = event.item;
if (event.item.type === "function_call") {
roundHasToolCalls = true;
canStreamRoundText = false;
}
} else if (event?.type === "response.completed") {
completedResponse = event.response;
sawUsage = mergeUsage(usageAcc, event.response?.usage) || sawUsage;
} else if (event?.type === "response.failed" || event?.type === "response.incomplete") {
completedResponse = event.response;
sawUsage = mergeUsage(usageAcc, event.response?.usage) || sawUsage;
} else if (event?.type === "error") {
throw new Error(event.message ?? "Responses stream failed.");
}
}
const failureMessage = getFailureMessage(completedResponse);
if (failureMessage) {
throw new Error(failureMessage);
}
const outputItems = getOutputItems(completedResponse);
const responseOutputItems = outputItems.length ? outputItems : completedOutputItems.filter(Boolean);
const normalizedToolCalls = normalizeToolCalls(responseOutputItems, round);
if (!normalizedToolCalls.length) {
const text = extractText(completedResponse, roundText);
if (!streamedRoundText && danglingToolIntentRetries < MAX_DANGLING_TOOL_INTENT_RETRIES && looksLikeDanglingToolIntent(text)) {
danglingToolIntentRetries += 1;
appendDanglingToolIntentCorrection(input, text);
continue;
}
const unstreamedText = getUnstreamedText(text, streamedRoundText);
if (unstreamedText) {
yield { type: "delta", text: unstreamedText };
}
yield {
type: "done",
result: {
text,
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, api: "responses" },
toolEvents,
},
};
return;
}
totalToolCalls += normalizedToolCalls.length;
input.push(...responseOutputItems);
for (const call of normalizedToolCalls) {
const { event: initiatedEvent, execution } = prepareToolCallExecution(call);
yield { type: "tool_call", event: initiatedEvent };
const { event, toolResult } = await executeToolCallAndBuildEvent(call, execution, params);
toolEvents.push(event);
yield { type: "tool_call", event };
input.push({
type: "function_call_output",
call_id: call.id,
output: JSON.stringify(toolResult),
});
}
}
yield {
type: "done",
result: {
text: "I reached the tool-call limit while gathering information. Please narrow the request and try again.",
usage: sawUsage ? usageAcc : undefined,
raw: { streamed: true, responses: rawResponses, toolCallsUsed: totalToolCalls, toolCallLimitReached: true, api: "responses" },
toolEvents,
},
};
}
+284
View File
@@ -0,0 +1,284 @@
import {
normalizeEnabledChatTools,
type ToolAwareCompletionParams,
type ToolAwareCompletionResult,
type ToolAwareStreamingEvent,
} from "./chat-tools.js";
import { completeWithChatCompletionsApi, streamWithChatCompletionsApi } from "./protocols/chat-completions-api.js";
import { completeWithGeminiApi, streamWithGeminiApi } from "./protocols/gemini-api.js";
import { completeWithMessagesApi, streamWithMessagesApi } from "./protocols/messages-api.js";
import { completeWithResponsesApi, streamWithResponsesApi } from "./protocols/responses-api.js";
import { env } from "../env.js";
import { anthropicClient, geminiClient, hermesAgentClient, isHermesAgentConfigured, openaiClient, xaiClient } from "./providers.js";
import type { ChatMessage, Provider } from "./types.js";
type ProviderAdapterParams = {
model: string;
messages: ChatMessage[];
enabledTools?: string[];
userLocation?: string;
temperature?: number;
maxTokens?: number;
logContext?: ToolAwareCompletionParams["logContext"];
};
export type ProviderChatAdapter = {
provider: Provider;
complete(params: ProviderAdapterParams): Promise<ToolAwareCompletionResult>;
stream(params: ProviderAdapterParams): AsyncGenerator<ToolAwareStreamingEvent>;
};
type ChatProtocolId = "chat-completions" | "gemini" | "messages" | "responses";
type ChatProtocol = {
id: ChatProtocolId;
complete(params: ToolAwareCompletionParams): Promise<ToolAwareCompletionResult>;
stream(params: ToolAwareCompletionParams): AsyncGenerator<ToolAwareStreamingEvent>;
};
type ModelCatalogSpec = {
enabled?: () => boolean;
fetchModels(client: any): Promise<string[]>;
fallbackModels?: () => string[];
sortModels?: (models: string[]) => string[];
};
type ProviderBackendSpec = {
createClient: () => any;
plainProtocol: ChatProtocol;
toolProtocol?: ChatProtocol;
managedTools?: boolean;
modelCatalog?: ModelCatalogSpec;
};
const chatCompletionsProtocol: ChatProtocol = {
id: "chat-completions",
complete: completeWithChatCompletionsApi,
stream: streamWithChatCompletionsApi,
};
const messagesProtocol: ChatProtocol = {
id: "messages",
complete: completeWithMessagesApi,
stream: streamWithMessagesApi,
};
const geminiProtocol: ChatProtocol = {
id: "gemini",
complete: completeWithGeminiApi,
stream: streamWithGeminiApi,
};
const responsesProtocol: ChatProtocol = {
id: "responses",
complete: completeWithResponsesApi,
stream: streamWithResponsesApi,
};
function uniqSorted(values: string[]) {
return [...new Set(values.map((value) => value.trim()).filter(Boolean))].sort((a, b) => a.localeCompare(b));
}
function modelIdsFromListResponse(page: any) {
return Array.isArray(page?.data)
? page.data.map((model: any) => model?.id).filter((id: unknown): id is string => typeof id === "string")
: [];
}
function stripModelResourcePrefix(model: string) {
return model.startsWith("models/") ? model.slice("models/".length) : model;
}
function isLikelyResponsesApiModel(model: string) {
const id = model.toLowerCase();
if (id.includes("embedding") || id.includes("moderation")) return false;
if (id.includes("audio") || id.includes("realtime") || id.includes("transcribe") || id.includes("tts")) return false;
if (id.includes("image") || id.includes("dall-e") || id.includes("sora")) return false;
if (id.includes("search") || id.includes("computer-use")) return false;
return /^(gpt-|o\d|chatgpt-)/.test(id);
}
function isLikelyGeminiChatModel(model: string) {
const id = model.toLowerCase();
if (!id.startsWith("gemini-")) return false;
if (id.includes("embedding") || id.includes("embed")) return false;
if (id.includes("image") || id.includes("imagen") || id.includes("veo")) return false;
if (id.includes("audio") || id.includes("tts") || id.includes("live")) return false;
if (id.includes("computer-use") || id.includes("robotics")) return false;
return true;
}
function preferGeminiModels(models: string[]) {
const preferred = [
"gemini-3.5-flash",
"gemini-flash-latest",
"gemini-3.1-flash-lite",
"gemini-3-flash-preview",
"gemini-pro-latest",
];
const modelSet = new Set(models);
return [...preferred.filter((model) => modelSet.delete(model)), ...[...modelSet].sort((a, b) => a.localeCompare(b))];
}
async function fetchJson(url: URL): Promise<any> {
const response = await fetch(url);
const body: any = await response.json().catch(() => null);
if (!response.ok) {
throw new Error(body?.error?.message ?? `Gemini model fetch failed with status ${response.status}.`);
}
return body;
}
function withClient(params: ProviderAdapterParams, client: any, enabledTools?: string[]): ToolAwareCompletionParams {
return {
client,
model: params.model,
messages: params.messages,
enabledTools,
userLocation: params.userLocation,
temperature: params.temperature,
maxTokens: params.maxTokens,
logContext: params.logContext,
};
}
function selectChatProtocol(spec: ProviderBackendSpec, params: Pick<ProviderAdapterParams, "enabledTools">) {
const enabledTools = normalizeEnabledChatTools(params.enabledTools);
const useManagedTools = spec.managedTools === true && spec.toolProtocol && enabledTools.length > 0;
return {
protocol: useManagedTools ? spec.toolProtocol! : spec.plainProtocol,
enabledTools: useManagedTools ? enabledTools : [],
managedTools: Boolean(useManagedTools),
};
}
function createProviderChatAdapter(provider: Provider, spec: ProviderBackendSpec): ProviderChatAdapter {
return {
provider,
complete(params) {
const selected = selectChatProtocol(spec, params);
return selected.protocol.complete(withClient(params, spec.createClient(), selected.enabledTools));
},
stream(params) {
const selected = selectChatProtocol(spec, params);
return selected.protocol.stream(withClient(params, spec.createClient(), selected.enabledTools));
},
};
}
const backendSpecs: Record<Provider, ProviderBackendSpec> = {
openai: {
createClient: openaiClient,
plainProtocol: chatCompletionsProtocol,
toolProtocol: responsesProtocol,
managedTools: true,
modelCatalog: {
async fetchModels(client) {
const page = await client.models.list();
return modelIdsFromListResponse(page).filter(isLikelyResponsesApiModel);
},
},
},
anthropic: {
createClient: anthropicClient,
plainProtocol: messagesProtocol,
toolProtocol: messagesProtocol,
managedTools: true,
modelCatalog: {
async fetchModels(client) {
const page = await client.models.list({ limit: 200 });
return modelIdsFromListResponse(page);
},
},
},
xai: {
createClient: xaiClient,
plainProtocol: chatCompletionsProtocol,
toolProtocol: chatCompletionsProtocol,
managedTools: true,
modelCatalog: {
async fetchModels(client) {
const page = await client.models.list();
return modelIdsFromListResponse(page);
},
},
},
gemini: {
createClient: geminiClient,
plainProtocol: geminiProtocol,
toolProtocol: geminiProtocol,
managedTools: true,
modelCatalog: {
async fetchModels(client) {
const url = new URL(`${client.baseURL.replace(/\/+$/, "")}/models`);
url.searchParams.set("key", client.apiKey);
url.searchParams.set("pageSize", "1000");
const page = await fetchJson(url);
return Array.isArray(page?.models)
? page.models
.filter((model: any) => Array.isArray(model?.supportedGenerationMethods) && model.supportedGenerationMethods.includes("generateContent"))
.map((model: any) => model?.name)
.filter((id: unknown): id is string => typeof id === "string")
.map(stripModelResourcePrefix)
.filter(isLikelyGeminiChatModel)
: [];
},
sortModels: preferGeminiModels,
},
},
"hermes-agent": {
createClient: hermesAgentClient,
plainProtocol: chatCompletionsProtocol,
managedTools: false,
modelCatalog: {
enabled: isHermesAgentConfigured,
async fetchModels(client) {
const page = await client.models.list();
const models = modelIdsFromListResponse(page);
if (env.HERMES_AGENT_MODEL) models.push(env.HERMES_AGENT_MODEL);
return models;
},
fallbackModels() {
return env.HERMES_AGENT_MODEL ? [env.HERMES_AGENT_MODEL] : [];
},
},
},
};
const providerChatAdapters: Record<Provider, ProviderChatAdapter> = Object.fromEntries(
Object.entries(backendSpecs).map(([provider, spec]) => [provider, createProviderChatAdapter(provider as Provider, spec)])
) as Record<Provider, ProviderChatAdapter>;
export function getProviderChatAdapter(provider: Provider) {
return providerChatAdapters[provider];
}
export function describeProviderChatBackend(provider: Provider, enabledTools?: string[]) {
const selected = selectChatProtocol(backendSpecs[provider], { enabledTools });
return {
provider,
protocol: selected.protocol.id,
managedTools: selected.managedTools,
enabledTools: selected.enabledTools,
};
}
export function listModelCatalogProviders(): Provider[] {
return (Object.entries(backendSpecs) as [Provider, ProviderBackendSpec][])
.filter(([, spec]) => {
const catalog = spec.modelCatalog;
return catalog !== undefined && catalog.enabled?.() !== false;
})
.map(([provider]) => provider);
}
export async function fetchProviderCatalogModels(provider: Provider) {
const spec = backendSpecs[provider].modelCatalog;
if (!spec) return [];
const models = uniqSorted(await spec.fetchModels(backendSpecs[provider].createClient()));
return spec.sortModels ? spec.sortModels(models) : models;
}
export function getProviderCatalogFallbackModels(provider: Provider) {
return uniqSorted(backendSpecs[provider].modelCatalog?.fallbackModels?.() ?? []);
}
+19 -4
View File
@@ -2,15 +2,30 @@ import type { Provider } from "./types.js";
type PrismaProvider = Exclude<Provider, "hermes-agent"> | "hermes_agent"; type PrismaProvider = Exclude<Provider, "hermes-agent"> | "hermes_agent";
const apiToPrismaProvider = {
openai: "openai",
anthropic: "anthropic",
xai: "xai",
gemini: "gemini",
"hermes-agent": "hermes_agent",
} as const satisfies Record<Provider, PrismaProvider>;
const prismaToApiProvider = {
openai: "openai",
anthropic: "anthropic",
xai: "xai",
gemini: "gemini",
hermes_agent: "hermes-agent",
"hermes-agent": "hermes-agent",
} as const satisfies Record<PrismaProvider | "hermes-agent", Provider>;
export function toPrismaProvider(provider: Provider): PrismaProvider { export function toPrismaProvider(provider: Provider): PrismaProvider {
return provider === "hermes-agent" ? "hermes_agent" : provider; return apiToPrismaProvider[provider];
} }
export function fromPrismaProvider(provider: unknown): Provider | null { export function fromPrismaProvider(provider: unknown): Provider | null {
if (provider === null || provider === undefined) return null; if (provider === null || provider === undefined) return null;
if (provider === "hermes_agent" || provider === "hermes-agent") return "hermes-agent"; return prismaToApiProvider[provider as keyof typeof prismaToApiProvider] ?? null;
if (provider === "openai" || provider === "anthropic" || provider === "xai") return provider;
return null;
} }
export function serializeProviderFields<T extends Record<string, any>>(value: T): T { export function serializeProviderFields<T extends Record<string, any>>(value: T): T {
+9 -1
View File
@@ -1,5 +1,5 @@
import OpenAI from "openai";
import Anthropic from "@anthropic-ai/sdk"; import Anthropic from "@anthropic-ai/sdk";
import OpenAI from "openai";
import { env } from "../env.js"; import { env } from "../env.js";
export function openaiClient() { export function openaiClient() {
@@ -13,6 +13,14 @@ export function xaiClient() {
return new OpenAI({ apiKey: env.XAI_API_KEY, baseURL: "https://api.x.ai/v1" }); return new OpenAI({ apiKey: env.XAI_API_KEY, baseURL: "https://api.x.ai/v1" });
} }
export function geminiClient() {
if (!env.GEMINI_API_KEY) throw new Error("GEMINI_API_KEY not set");
return {
apiKey: env.GEMINI_API_KEY,
baseURL: "https://generativelanguage.googleapis.com/v1beta",
};
}
export function isHermesAgentConfigured() { export function isHermesAgentConfigured() {
return Boolean(env.HERMES_AGENT_API_KEY); return Boolean(env.HERMES_AGENT_API_KEY);
} }
+11 -82
View File
@@ -1,15 +1,10 @@
import { performance } from "node:perf_hooks"; import { performance } from "node:perf_hooks";
import { prisma } from "../db.js"; import { prisma } from "../db.js";
import { anthropicClient, hermesAgentClient, openaiClient, xaiClient } from "./providers.js";
import { import {
buildToolLogMessageData, buildToolLogMessageData,
normalizeEnabledChatTools,
runPlainChatCompletionsStream,
runToolAwareChatCompletionsStream,
runToolAwareOpenAIChatStream,
type ToolExecutionEvent, type ToolExecutionEvent,
} from "./chat-tools.js"; } from "./chat-tools.js";
import { buildAnthropicConversationMessage, getAnthropicSystemPrompt } from "./message-content.js"; import { getProviderChatAdapter } from "./provider-adapters.js";
import { toPrismaProvider } from "./provider-ids.js"; import { toPrismaProvider } from "./provider-ids.js";
import type { MultiplexRequest, Provider } from "./types.js"; import type { MultiplexRequest, Provider } from "./types.js";
@@ -75,44 +70,11 @@ export async function* runMultiplexStream(req: MultiplexRequest): AsyncGenerator
let raw: unknown = { streamed: true }; let raw: unknown = { streamed: true };
try { try {
if (req.provider === "openai" || req.provider === "xai" || req.provider === "hermes-agent") { const adapter = getProviderChatAdapter(req.provider);
const client = req.provider === "openai" ? openaiClient() : req.provider === "xai" ? xaiClient() : hermesAgentClient(); const streamEvents = adapter.stream({
const enabledTools = normalizeEnabledChatTools(req.enabledTools);
const streamEvents =
req.provider === "openai" && enabledTools.length > 0
? runToolAwareOpenAIChatStream({
client,
model: req.model, model: req.model,
messages: req.messages, messages: req.messages,
enabledTools, enabledTools: req.enabledTools,
userLocation: req.userLocation,
temperature: req.temperature,
maxTokens: req.maxTokens,
logContext: {
provider: req.provider,
model: req.model,
chatId: chatId ?? undefined,
},
})
: req.provider === "hermes-agent" || enabledTools.length === 0
? runPlainChatCompletionsStream({
client,
model: req.model,
messages: req.messages,
userLocation: req.userLocation,
temperature: req.temperature,
maxTokens: req.maxTokens,
logContext: {
provider: req.provider,
model: req.model,
chatId: chatId ?? undefined,
},
})
: runToolAwareChatCompletionsStream({
client,
model: req.model,
messages: req.messages,
enabledTools,
userLocation: req.userLocation, userLocation: req.userLocation,
temperature: req.temperature, temperature: req.temperature,
maxTokens: req.maxTokens, maxTokens: req.maxTokens,
@@ -122,6 +84,7 @@ export async function* runMultiplexStream(req: MultiplexRequest): AsyncGenerator
chatId: chatId ?? undefined, chatId: chatId ?? undefined,
}, },
}); });
for await (const ev of streamEvents) { for await (const ev of streamEvents) {
if (ev.type === "delta") { if (ev.type === "delta") {
text += ev.text; text += ev.text;
@@ -150,52 +113,18 @@ export async function* runMultiplexStream(req: MultiplexRequest): AsyncGenerator
usage = ev.result.usage; usage = ev.result.usage;
text = ev.result.text; text = ev.result.text;
} }
} else if (req.provider === "anthropic") {
const client = anthropicClient();
const system = getAnthropicSystemPrompt(req.messages, req.userLocation);
const msgs = req.messages.filter((message) => message.role !== "system").map((message) => buildAnthropicConversationMessage(message));
const stream = await client.messages.create({
model: req.model,
system,
max_tokens: req.maxTokens ?? 1024,
temperature: req.temperature,
messages: msgs as any,
stream: true,
});
for await (const ev of stream as any as AsyncIterable<any>) {
// Anthropic streaming events include content_block_delta with text_delta
if (ev?.type === "content_block_delta" && ev?.delta?.type === "text_delta") {
const delta = ev.delta.text ?? "";
if (delta) {
text += delta;
yield { type: "delta", text: delta };
}
}
// capture usage if present on message_delta
if (ev?.type === "message_delta" && ev?.usage) {
usage = {
inputTokens: ev.usage.input_tokens,
outputTokens: ev.usage.output_tokens,
totalTokens:
(ev.usage.input_tokens ?? 0) + (ev.usage.output_tokens ?? 0),
};
}
// some streams end with message_stop
}
raw = { streamed: true, provider: "anthropic" };
} else {
throw new Error(`unknown provider: ${req.provider}`);
}
const latencyMs = Math.round(performance.now() - t0); const latencyMs = Math.round(performance.now() - t0);
if (shouldPersist && chatId && call) { if (shouldPersist && chatId && call) {
await prisma.$transaction(async (tx) => { await prisma.$transaction(async (tx) => {
await tx.message.create({ await tx.message.create({
data: { chatId, role: "assistant" as any, content: text }, data: {
chatId,
role: "assistant" as any,
content: text,
metadata: req.clientRequestId ? ({ clientRequestId: req.clientRequestId } as any) : undefined,
},
}); });
await tx.llmCall.update({ await tx.llmCall.update({
where: { id: call.id }, where: { id: call.id },
+2 -1
View File
@@ -1,4 +1,4 @@
export const PROVIDERS = ["openai", "anthropic", "xai", "hermes-agent"] as const; export const PROVIDERS = ["openai", "anthropic", "xai", "gemini", "hermes-agent"] as const;
export type Provider = (typeof PROVIDERS)[number]; export type Provider = (typeof PROVIDERS)[number];
@@ -33,6 +33,7 @@ export type ChatMessage = {
export type MultiplexRequest = { export type MultiplexRequest = {
chatId?: string; chatId?: string;
persist?: boolean; persist?: boolean;
clientRequestId?: string;
provider: Provider; provider: Provider;
model: string; model: string;
messages: ChatMessage[]; messages: ChatMessage[];
+120 -13
View File
@@ -16,7 +16,7 @@ import { exaClient } from "./search/exa.js";
import { isFreshSearchCacheHit, normalizeSearchQuery } from "./search-cache.js"; import { isFreshSearchCacheHit, normalizeSearchQuery } from "./search-cache.js";
import type { ChatAttachment } from "./llm/types.js"; import type { ChatAttachment } from "./llm/types.js";
const ProviderSchema = z.enum(["openai", "anthropic", "xai", "hermes-agent"]); const ProviderSchema = z.enum(["openai", "anthropic", "xai", "gemini", "hermes-agent"]);
const MAX_ADDITIONAL_SYSTEM_PROMPT_CHARS = 12_000; const MAX_ADDITIONAL_SYSTEM_PROMPT_CHARS = 12_000;
const EnabledToolsSchema = z.array(z.string().trim().min(1).max(80)).max(20).transform((value) => normalizeEnabledChatTools(value)); const EnabledToolsSchema = z.array(z.string().trim().min(1).max(80)).max(20).transform((value) => normalizeEnabledChatTools(value));
@@ -88,7 +88,7 @@ function withRequestUserLocation<T extends { userLocation?: string }>(body: T, r
return body.userLocation ? body : { ...body, userLocation: inferRequestUserLocation(req) }; return body.userLocation ? body : { ...body, userLocation: inferRequestUserLocation(req) };
} }
async function storeNonAssistantMessages(chatId: string, messages: IncomingChatMessage[]) { async function storeNonAssistantMessages(chatId: string, messages: IncomingChatMessage[], clientRequestId?: string) {
const incoming = messages.filter((m) => m.role !== "assistant"); const incoming = messages.filter((m) => m.role !== "assistant");
if (!incoming.length) return; if (!incoming.length) return;
@@ -109,14 +109,21 @@ async function storeNonAssistantMessages(chatId: string, messages: IncomingChatM
const toInsert = sharedPrefix === existingNonAssistant.length ? incoming.slice(existingNonAssistant.length) : incoming; const toInsert = sharedPrefix === existingNonAssistant.length ? incoming.slice(existingNonAssistant.length) : incoming;
if (!toInsert.length) return; if (!toInsert.length) return;
const finalUserMessageIndex = toInsert.map((message) => message.role).lastIndexOf("user");
await prisma.message.createMany({ await prisma.message.createMany({
data: toInsert.map((m) => ({ data: toInsert.map((m, index) => {
const metadata = {
...(m.attachments?.length ? { attachments: m.attachments } : {}),
...(clientRequestId && index === finalUserMessageIndex ? { clientRequestId } : {}),
};
return {
chatId, chatId,
role: m.role as any, role: m.role as any,
content: m.content, content: m.content,
name: m.name, name: m.name,
metadata: m.attachments?.length ? ({ attachments: m.attachments } as any) : undefined, metadata: Object.keys(metadata).length ? (metadata as any) : undefined,
})), };
}),
}); });
} }
@@ -169,6 +176,7 @@ const CompletionStreamBody = z
.object({ .object({
chatId: z.string().optional(), chatId: z.string().optional(),
persist: z.boolean().optional(), persist: z.boolean().optional(),
clientRequestId: z.string().trim().min(1).max(128).optional(),
provider: ProviderSchema, provider: ProviderSchema,
model: z.string().min(1), model: z.string().min(1),
messages: z.array(CompletionMessageSchema), messages: z.array(CompletionMessageSchema),
@@ -186,6 +194,13 @@ const CompletionStreamBody = z
path: ["chatId"], path: ["chatId"],
}); });
} }
if (value.clientRequestId && (value.persist === false || !value.chatId)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: "clientRequestId requires a persisted stream with chatId",
path: ["clientRequestId"],
});
}
}); });
function mergeAttachmentsIntoMetadata(metadata: unknown, attachments?: ChatAttachment[]) { function mergeAttachmentsIntoMetadata(metadata: unknown, attachments?: ChatAttachment[]) {
@@ -399,6 +414,7 @@ function buildSseHeaders(originHeader: string | undefined) {
type SearchRunRequest = z.infer<typeof SearchRunBody>; type SearchRunRequest = z.infer<typeof SearchRunBody>;
const activeChatStreams = new Map<string, ActiveSseStream>(); const activeChatStreams = new Map<string, ActiveSseStream>();
const activeChatStreamRequestIds = new Map<string, string>();
const activeSearchStreams = new Map<string, ActiveSseStream>(); const activeSearchStreams = new Map<string, ActiveSseStream>();
const STARRED_PROJECT_ID = "starred"; const STARRED_PROJECT_ID = "starred";
@@ -554,6 +570,7 @@ function writeSseEvent(reply: FastifyReply, event: SseStreamEvent) {
} }
async function streamActiveRun(req: FastifyRequest, reply: FastifyReply, stream: ActiveSseStream) { async function streamActiveRun(req: FastifyRequest, reply: FastifyReply, stream: ActiveSseStream) {
if (reply.raw.destroyed || reply.raw.writableEnded) return reply;
reply.raw.writeHead(200, buildSseHeaders(typeof req.headers.origin === "string" ? req.headers.origin : undefined)); reply.raw.writeHead(200, buildSseHeaders(typeof req.headers.origin === "string" ? req.headers.origin : undefined));
reply.raw.flushHeaders?.(); reply.raw.flushHeaders?.();
@@ -588,10 +605,24 @@ function mapChatStreamEvent(ev: StreamEvent): SseStreamEvent {
return { event: ev.type, data: ev }; return { event: ev.type, data: ev };
} }
function startActiveChatStream(chatId: string, body: z.infer<typeof CompletionStreamBody>) { function registerActiveChatStream(chatId: string, clientRequestId?: string) {
const stream = new ActiveSseStream(); const stream = new ActiveSseStream();
activeChatStreams.set(chatId, stream); activeChatStreams.set(chatId, stream);
if (clientRequestId) {
activeChatStreamRequestIds.set(chatId, clientRequestId);
} else {
activeChatStreamRequestIds.delete(chatId);
}
return stream;
}
function clearActiveChatStream(chatId: string, stream: ActiveSseStream) {
if (activeChatStreams.get(chatId) !== stream) return;
activeChatStreams.delete(chatId);
activeChatStreamRequestIds.delete(chatId);
}
function executeActiveChatStream(chatId: string, body: z.infer<typeof CompletionStreamBody>, stream: ActiveSseStream) {
void (async () => { void (async () => {
let sawTerminalEvent = false; let sawTerminalEvent = false;
try { try {
@@ -611,13 +642,54 @@ function startActiveChatStream(chatId: string, body: z.infer<typeof CompletionSt
} catch (err) { } catch (err) {
stream.complete({ event: "error", data: { message: getErrorMessage(err) } }); stream.complete({ event: "error", data: { message: getErrorMessage(err) } });
} finally { } finally {
activeChatStreams.delete(chatId); clearActiveChatStream(chatId, stream);
} }
})(); })();
}
function startActiveChatStream(chatId: string, body: z.infer<typeof CompletionStreamBody>) {
const stream = registerActiveChatStream(chatId, body.clientRequestId);
executeActiveChatStream(chatId, body, stream);
return stream; return stream;
} }
function getMetadataClientRequestId(metadata: unknown) {
if (!metadata || typeof metadata !== "object" || Array.isArray(metadata)) return null;
const clientRequestId = (metadata as Record<string, unknown>).clientRequestId;
return typeof clientRequestId === "string" ? clientRequestId : null;
}
async function findCompletedChatSubmission(chatId: string, clientRequestId: string) {
const assistantMessages = await prisma.message.findMany({
where: { chatId, role: "assistant" as any },
orderBy: { createdAt: "desc" },
select: { content: true, metadata: true },
});
return assistantMessages.find((message) => getMetadataClientRequestId(message.metadata) === clientRequestId) ?? null;
}
function completeChatSubmissionStream(
stream: ActiveSseStream,
chatId: string,
body: z.infer<typeof CompletionStreamBody>,
assistantText: string
) {
stream.emit("meta", {
type: "meta",
chatId,
callId: null,
provider: body.provider,
model: body.model,
});
stream.complete({
event: "done",
data: {
type: "done",
text: assistantText,
},
});
}
async function executeSearchRunStream(searchId: string, body: SearchRunRequest, stream: ActiveSseStream) { async function executeSearchRunStream(searchId: string, body: SearchRunRequest, stream: ActiveSseStream) {
const startedAt = performance.now(); const startedAt = performance.now();
const query = body.query?.trim(); const query = body.query?.trim();
@@ -935,7 +1007,18 @@ export async function registerRoutes(app: FastifyInstance) {
if (existing.title?.trim()) return { chat: serializeChatLike(existing) }; if (existing.title?.trim()) return { chat: serializeChatLike(existing) };
const fallback = body.content.split(/\r?\n/)[0]?.trim().slice(0, 48) || "New chat"; const fallback = body.content.split(/\r?\n/)[0]?.trim().slice(0, 48) || "New chat";
const suggestedRaw = await generateChatTitle(body.content); let suggestedRaw = "";
try {
suggestedRaw = await generateChatTitle(body.content);
} catch (err) {
req.log.warn(
{
chatId: body.chatId,
err: getErrorMessage(err),
},
"chat title generation failed; using fallback"
);
}
const title = normalizeSuggestedTitle(suggestedRaw, fallback); const title = normalizeSuggestedTitle(suggestedRaw, fallback);
await prisma.chat.updateMany({ await prisma.chat.updateMany({
@@ -1353,15 +1436,39 @@ export async function registerRoutes(app: FastifyInstance) {
if (!exists) return app.httpErrors.notFound("chat not found"); if (!exists) return app.httpErrors.notFound("chat not found");
} }
// Store only new non-assistant messages to avoid duplicate history entries.
if (body.persist !== false && body.chatId) { if (body.persist !== false && body.chatId) {
await storeNonAssistantMessages(body.chatId, body.messages); const activeStream = activeChatStreams.get(body.chatId);
if (activeStream) {
if (body.clientRequestId && activeChatStreamRequestIds.get(body.chatId) === body.clientRequestId) {
return streamActiveRun(req, reply, activeStream);
} }
if (body.persist !== false && body.chatId) {
if (activeChatStreams.has(body.chatId)) {
return app.httpErrors.conflict("chat completion already running"); return app.httpErrors.conflict("chat completion already running");
} }
if (body.clientRequestId) {
const reservedStream = registerActiveChatStream(body.chatId, body.clientRequestId);
try {
const completedSubmission = await findCompletedChatSubmission(body.chatId, body.clientRequestId);
if (completedSubmission) {
completeChatSubmissionStream(reservedStream, body.chatId, body, completedSubmission.content);
clearActiveChatStream(body.chatId, reservedStream);
return streamActiveRun(req, reply, reservedStream);
}
// Store only new non-assistant messages to avoid duplicate history entries.
await storeNonAssistantMessages(body.chatId, body.messages, body.clientRequestId);
const configuredBody = await applyStoredChatSettings(body);
executeActiveChatStream(body.chatId, configuredBody, reservedStream);
return streamActiveRun(req, reply, reservedStream);
} catch (err) {
reservedStream.complete({ event: "error", data: { message: getErrorMessage(err) } });
clearActiveChatStream(body.chatId, reservedStream);
throw err;
}
}
// Legacy requests without an idempotency key retain the original behavior.
await storeNonAssistantMessages(body.chatId, body.messages);
const stream = startActiveChatStream(body.chatId, await applyStoredChatSettings(body)); const stream = startActiveChatStream(body.chatId, await applyStoredChatSettings(body));
return streamActiveRun(req, reply, stream); return streamActiveRun(req, reply, stream);
} }
+305
View File
@@ -0,0 +1,305 @@
import { buildBrowserLikeRequestHeaders } from "../browser-fetch-headers.js";
import { env } from "../env.js";
const BRAVE_WEB_SEARCH_URL = "https://api.search.brave.com/res/v1/web/search";
const BRAVE_SEARCH_TIMEOUT_MS = 12_000;
const DEFAULT_BRAVE_REQUEST_INTERVAL_MS = 1_000;
const RATE_LIMIT_INTERVAL_SAFETY_RATIO = 0.05;
const MIN_RATE_LIMIT_INTERVAL_SAFETY_MS = 2;
const RATE_LIMIT_RESET_SAFETY_MS = 50;
const MAX_RATE_LIMIT_RETRIES = 3;
const MAX_RATE_LIMIT_RETRY_DELAY_MS = 8_000;
type RateLimitPolicy = {
limit: number;
windowSeconds: number;
};
let requestIntervalMs = addIntervalSafety(DEFAULT_BRAVE_REQUEST_INTERVAL_MS);
let lastRequestAtMs = 0;
let nextRequestAtMs = 0;
let quotaUnavailableUntilMs = 0;
let requestQueue = Promise.resolve();
export type BraveSearchOptions = {
numResults: number;
includeDomains?: string[];
excludeDomains?: string[];
};
export type BraveSearchResult = {
title: string | null;
url: string | null;
publishedDate: string | null;
author: string | null;
summary: string | null;
text: string | null;
highlights: string[];
};
export type BraveSearchResponse = {
query: string;
requestId: string | null;
results: BraveSearchResult[];
};
function clipText(input: string, maxCharacters: number) {
return input.length <= maxCharacters ? input : `${input.slice(0, maxCharacters)}...`;
}
function compactWhitespace(input: string) {
return input.replace(/\r/g, "").replace(/[ \t]+\n/g, "\n").replace(/\n{3,}/g, "\n\n").replace(/\s+/g, " ").trim();
}
function requireBraveSearchApiKey() {
if (!env.BRAVE_SEARCH_API_KEY) {
throw new Error("BRAVE_SEARCH_API_KEY not set");
}
return env.BRAVE_SEARCH_API_KEY;
}
function sleep(milliseconds: number) {
return new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
}
function addIntervalSafety(intervalMs: number) {
return intervalMs + Math.max(MIN_RATE_LIMIT_INTERVAL_SAFETY_MS, Math.ceil(intervalMs * RATE_LIMIT_INTERVAL_SAFETY_RATIO));
}
function parseCommaSeparatedNumbers(value: string | null) {
if (!value) return [];
return value.split(",").map((part) => Number(part.trim())).map((number) => (Number.isFinite(number) ? number : null));
}
function parseRateLimitPolicy(value: string | null): RateLimitPolicy[] {
if (!value) return [];
return value.split(",").flatMap((part) => {
const match = part.trim().match(/^(\d+)\s*;\s*w=(\d+)$/i);
if (!match) return [];
const limit = Number(match[1]);
const windowSeconds = Number(match[2]);
return limit > 0 && windowSeconds > 0 ? [{ limit, windowSeconds }] : [];
});
}
function getBurstPolicyIndex(policies: RateLimitPolicy[]) {
if (!policies.length) return null;
let burstIndex = 0;
for (let index = 1; index < policies.length; index += 1) {
if (policies[index]!.windowSeconds < policies[burstIndex]!.windowSeconds) burstIndex = index;
}
return burstIndex;
}
function updateRateLimitState(headers: Headers) {
const policies = parseRateLimitPolicy(headers.get("x-ratelimit-policy"));
const burstIndex = getBurstPolicyIndex(policies);
if (burstIndex === null) return;
const burstPolicy = policies[burstIndex]!;
const learnedIntervalMs = addIntervalSafety(Math.ceil((burstPolicy.windowSeconds * 1_000) / burstPolicy.limit));
if (learnedIntervalMs < requestIntervalMs && lastRequestAtMs > 0) {
nextRequestAtMs = Math.min(nextRequestAtMs, lastRequestAtMs + learnedIntervalMs);
}
requestIntervalMs = learnedIntervalMs;
const remaining = parseCommaSeparatedNumbers(headers.get("x-ratelimit-remaining"));
const resetSeconds = parseCommaSeparatedNumbers(headers.get("x-ratelimit-reset"));
for (let index = 0; index < policies.length; index += 1) {
if ((remaining[index] ?? null) === null || remaining[index]! >= 1 || (resetSeconds[index] ?? 0) <= 0) continue;
const unavailableUntilMs = Date.now() + resetSeconds[index]! * 1_000 + RATE_LIMIT_RESET_SAFETY_MS;
if (index === burstIndex) {
nextRequestAtMs = Math.max(nextRequestAtMs, unavailableUntilMs);
} else {
quotaUnavailableUntilMs = Math.max(quotaUnavailableUntilMs, unavailableUntilMs);
}
}
}
function assertLongTermQuotaAvailable() {
if (quotaUnavailableUntilMs <= Date.now()) {
quotaUnavailableUntilMs = 0;
return;
}
const resetSeconds = Math.ceil((quotaUnavailableUntilMs - Date.now()) / 1_000);
throw new Error(`Brave Search API long-term quota is exhausted; reset is expected in ${resetSeconds} seconds.`);
}
async function waitForRateLimitSlot() {
const reservation = requestQueue.then(async () => {
while (true) {
assertLongTermQuotaAvailable();
const waitMs = nextRequestAtMs - Date.now();
if (waitMs <= 0) break;
await sleep(waitMs);
}
lastRequestAtMs = Date.now();
nextRequestAtMs = lastRequestAtMs + requestIntervalMs;
});
requestQueue = reservation.catch(() => undefined);
await reservation;
}
function get429RetryDelayMs(headers: Headers, retryNumber: number) {
const remaining = parseCommaSeparatedNumbers(headers.get("x-ratelimit-remaining"));
const resetSeconds = parseCommaSeparatedNumbers(headers.get("x-ratelimit-reset"));
const exhaustedResetSeconds = resetSeconds.filter((reset, index): reset is number => reset !== null && (remaining[index] ?? 0) < 1);
const headerDelayMs = exhaustedResetSeconds.length ? Math.max(...exhaustedResetSeconds) * 1_000 : 0;
const exponentialDelayMs = 2 ** retryNumber * 1_000;
const delayMs = Math.max(headerDelayMs + RATE_LIMIT_RESET_SAFETY_MS, exponentialDelayMs);
return delayMs <= MAX_RATE_LIMIT_RETRY_DELAY_MS ? delayMs : null;
}
async function fetchBrave(url: URL) {
const apiKey = requireBraveSearchApiKey();
for (let attempt = 0; attempt <= MAX_RATE_LIMIT_RETRIES; attempt += 1) {
await waitForRateLimitSlot();
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), BRAVE_SEARCH_TIMEOUT_MS);
let response: Response;
try {
response = await fetch(url, {
signal: controller.signal,
headers: {
...buildBrowserLikeRequestHeaders("application/json"),
"X-Subscription-Token": apiKey,
},
});
} finally {
clearTimeout(timeout);
}
updateRateLimitState(response.headers);
if (response.status !== 429 || attempt === MAX_RATE_LIMIT_RETRIES) return response;
const retryDelayMs = get429RetryDelayMs(response.headers, attempt);
await response.arrayBuffer();
if (retryDelayMs === null) {
throw new Error("Brave Search API rate limit quota is exhausted beyond the retry window.");
}
await sleep(retryDelayMs);
}
throw new Error("Brave Search API request failed after rate-limit retries.");
}
function normalizeDomain(input: string) {
const trimmed = input.trim().toLowerCase();
if (!trimmed) return null;
try {
const parsed = new URL(trimmed.includes("://") ? trimmed : `https://${trimmed}`);
return parsed.hostname.replace(/^www\./, "");
} catch {
return trimmed.split(/[/?#]/, 1)[0]?.replace(/^www\./, "") || null;
}
}
function normalizeDomains(input: string[] | undefined) {
return Array.from(new Set((input ?? []).map(normalizeDomain).filter((domain): domain is string => Boolean(domain))));
}
function hostnameMatchesDomain(urlRaw: string | null, domain: string) {
if (!urlRaw) return false;
try {
const hostname = new URL(urlRaw).hostname.toLowerCase().replace(/^www\./, "");
return hostname === domain || hostname.endsWith(`.${domain}`);
} catch {
return false;
}
}
function filterResultsByDomains(results: BraveSearchResult[], options: BraveSearchOptions) {
const includeDomains = normalizeDomains(options.includeDomains);
const excludeDomains = normalizeDomains(options.excludeDomains);
return results.filter((result) => {
if (includeDomains.length && !includeDomains.some((domain) => hostnameMatchesDomain(result.url, domain))) return false;
if (excludeDomains.some((domain) => hostnameMatchesDomain(result.url, domain))) return false;
return true;
});
}
function buildBraveQuery(query: string, options: BraveSearchOptions) {
const includeDomains = normalizeDomains(options.includeDomains);
const excludeDomains = normalizeDomains(options.excludeDomains);
const includeClause =
includeDomains.length === 0
? ""
: includeDomains.length === 1
? `site:${includeDomains[0]}`
: `(${includeDomains.map((domain) => `site:${domain}`).join(" OR ")})`;
const excludeClause = excludeDomains.map((domain) => `-site:${domain}`).join(" ");
return [query, includeClause, excludeClause].filter(Boolean).join(" ");
}
function buildSearchUrl(query: string, options: BraveSearchOptions) {
const url = new URL(BRAVE_WEB_SEARCH_URL);
url.searchParams.set("q", buildBraveQuery(query, options));
url.searchParams.set("count", String(options.numResults));
url.searchParams.set("safesearch", "moderate");
url.searchParams.set("result_filter", "web");
url.searchParams.set("text_decorations", "false");
url.searchParams.set("extra_snippets", "true");
return url;
}
function stringOrNull(value: unknown) {
if (typeof value !== "string") return null;
const normalized = compactWhitespace(value);
return normalized || null;
}
function stringArray(value: unknown) {
if (!Array.isArray(value)) return [];
return value.filter((item): item is string => typeof item === "string").map(compactWhitespace).filter(Boolean);
}
function mapWebResult(result: any): BraveSearchResult {
const description = stringOrNull(result?.description);
const extraSnippets = stringArray(result?.extra_snippets);
const snippets = [description, ...extraSnippets].filter((snippet): snippet is string => Boolean(snippet));
const combinedText = snippets.join("\n\n");
return {
title: stringOrNull(result?.title),
url: stringOrNull(result?.url),
publishedDate: stringOrNull(result?.page_age),
author: stringOrNull(result?.profile?.name) ?? stringOrNull(result?.article?.author),
summary: description ? clipText(description, 1_400) : null,
text: combinedText ? clipText(combinedText, 700) : null,
highlights: snippets.slice(0, 3).map((snippet) => clipText(snippet, 280)),
};
}
export async function searchBrave(query: string, options: BraveSearchOptions): Promise<BraveSearchResponse> {
const url = buildSearchUrl(query, options);
const response = await fetchBrave(url);
if (!response.ok) {
await response.arrayBuffer();
throw new Error(`Brave Search API request failed with status ${response.status}.`);
}
const contentType = response.headers.get("content-type")?.toLowerCase() ?? "";
if (!contentType.includes("application/json")) {
await response.arrayBuffer();
throw new Error(`Brave Search API returned ${contentType || "unknown content type"}.`);
}
const data: any = await response.json();
const results = Array.isArray(data?.web?.results) ? data.web.results.map(mapWebResult) : [];
return {
query,
requestId: response.headers.get("x-request-id"),
results: filterResultsByDomains(results, options).slice(0, options.numResults),
};
}
export function resetBraveRateLimitStateForTests() {
requestIntervalMs = addIntervalSafety(DEFAULT_BRAVE_REQUEST_INTERVAL_MS);
lastRequestAtMs = 0;
nextRequestAtMs = 0;
quotaUnavailableUntilMs = 0;
requestQueue = Promise.resolve();
}
+2 -4
View File
@@ -1,3 +1,4 @@
import { buildBrowserLikeRequestHeaders } from "../browser-fetch-headers.js";
import { env } from "../env.js"; import { env } from "../env.js";
const SEARXNG_TIMEOUT_MS = 12_000; const SEARXNG_TIMEOUT_MS = 12_000;
@@ -106,10 +107,7 @@ async function fetchSearxng(url: URL, accept: string) {
return await fetch(url, { return await fetch(url, {
redirect: "follow", redirect: "follow",
signal: controller.signal, signal: controller.signal,
headers: { headers: buildBrowserLikeRequestHeaders(accept),
"User-Agent": "SybilBot/1.0 (+https://sybil.local)",
Accept: accept,
},
}); });
} finally { } finally {
clearTimeout(timeout); clearTimeout(timeout);
+284
View File
@@ -0,0 +1,284 @@
import assert from "node:assert/strict";
import test from "node:test";
import { env } from "../src/env.js";
import { resetBraveRateLimitStateForTests, searchBrave } from "../src/search/brave.js";
test("searchBrave authenticates, builds filters, and normalizes web results", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
const fetchCalls: Array<{ input: RequestInfo | URL; init?: RequestInit }> = [];
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
fetchCalls.push({ input, init });
return new Response(
JSON.stringify({
web: {
results: [
{
title: " Brave result ",
url: "https://docs.example.com/article",
description: "Main\n snippet",
extra_snippets: ["Extra snippet one", "Extra snippet two"],
page_age: "2026-07-18T12:00:00Z",
profile: { name: "Example Docs" },
},
{
title: "Excluded result",
url: "https://blocked.example.com/article",
description: "Should be filtered",
},
],
},
}),
{
status: 200,
headers: {
"content-type": "application/json; charset=utf-8",
"x-request-id": "brave-request-1",
},
}
);
}) as typeof fetch;
try {
const response = await searchBrave("latest docs", {
numResults: 5,
includeDomains: ["https://example.com/path"],
excludeDomains: ["blocked.example.com"],
});
assert.equal(fetchCalls.length, 1);
const requestUrl = new URL(String(fetchCalls[0]?.input));
assert.equal(requestUrl.origin + requestUrl.pathname, "https://api.search.brave.com/res/v1/web/search");
assert.equal(requestUrl.searchParams.get("q"), "latest docs site:example.com -site:blocked.example.com");
assert.equal(requestUrl.searchParams.get("count"), "5");
assert.equal(requestUrl.searchParams.get("safesearch"), "moderate");
assert.equal(requestUrl.searchParams.get("result_filter"), "web");
assert.equal(requestUrl.searchParams.get("text_decorations"), "false");
assert.equal(requestUrl.searchParams.get("extra_snippets"), "true");
assert.equal((fetchCalls[0]?.init?.headers as Record<string, string>)["X-Subscription-Token"], "test-brave-key");
assert.deepEqual(response, {
query: "latest docs",
requestId: "brave-request-1",
results: [
{
title: "Brave result",
url: "https://docs.example.com/article",
publishedDate: "2026-07-18T12:00:00Z",
author: "Example Docs",
summary: "Main snippet",
text: "Main snippet\n\nExtra snippet one\n\nExtra snippet two",
highlights: ["Main snippet", "Extra snippet one", "Extra snippet two"],
},
],
});
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
}
});
test("searchBrave rejects requests without an API key", async () => {
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = undefined;
try {
await assert.rejects(() => searchBrave("test", { numResults: 1 }), /BRAVE_SEARCH_API_KEY not set/);
} finally {
env.BRAVE_SEARCH_API_KEY = originalApiKey;
}
});
test("searchBrave reports non-JSON responses", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async () =>
new Response("upstream error", {
status: 200,
headers: { "content-type": "text/plain" },
})) as typeof fetch;
try {
await assert.rejects(
() => searchBrave("test", { numResults: 1 }),
/Brave Search API returned text\/plain/
);
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
}
});
test("searchBrave evenly paces concurrent bursts using Brave's shortest policy window", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
const requestStartedAt: number[] = [];
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async () => {
requestStartedAt.push(Date.now());
return new Response(JSON.stringify({ web: { results: [] } }), {
status: 200,
headers: {
"content-type": "application/json",
"x-ratelimit-policy": "1;w=1, 2000;w=2678400",
"x-ratelimit-remaining": "1, 1999",
"x-ratelimit-reset": "1, 2678400",
},
});
}) as typeof fetch;
try {
await Promise.all([
searchBrave("burst one", { numResults: 1 }),
searchBrave("burst two", { numResults: 1 }),
searchBrave("burst three", { numResults: 1 }),
]);
assert.equal(requestStartedAt.length, 3);
assert.ok(requestStartedAt[1]! - requestStartedAt[0]! >= 1_000);
assert.ok(requestStartedAt[2]! - requestStartedAt[1]! >= 1_000);
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
resetBraveRateLimitStateForTests();
}
});
test("searchBrave adapts its pacing to a 50 request-per-second Search plan", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
const requestStartedAt: number[] = [];
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async () => {
requestStartedAt.push(Date.now());
return new Response(JSON.stringify({ web: { results: [] } }), {
status: 200,
headers: {
"content-type": "application/json",
"x-ratelimit-policy": "50;w=1, 0;w=2678400",
"x-ratelimit-remaining": "49, 0",
"x-ratelimit-reset": "1, 2678400",
},
});
}) as typeof fetch;
try {
await searchBrave("learn upgraded policy", { numResults: 1 });
await Promise.all(Array.from({ length: 8 }, (_, index) => searchBrave(`fast burst ${index}`, { numResults: 1 })));
assert.equal(requestStartedAt.length, 9);
const burstStartedAt = requestStartedAt.slice(1);
for (let index = 1; index < burstStartedAt.length; index += 1) {
assert.ok(burstStartedAt[index]! - burstStartedAt[index - 1]! >= 18);
}
assert.ok(burstStartedAt.at(-1)! - burstStartedAt[0]! < 500);
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
resetBraveRateLimitStateForTests();
}
});
test("searchBrave retries 429 responses after the burst window resets", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
let fetchCount = 0;
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async () => {
fetchCount += 1;
const rateLimitHeaders = {
"content-type": "application/json",
"x-ratelimit-policy": "1;w=1, 2000;w=2678400",
"x-ratelimit-remaining": fetchCount === 1 ? "0, 1999" : "1, 1998",
"x-ratelimit-reset": "1, 2678400",
};
if (fetchCount === 1) {
return new Response(JSON.stringify({ error: { detail: "Rate limit exceeded" } }), {
status: 429,
headers: rateLimitHeaders,
});
}
return new Response(JSON.stringify({ web: { results: [] } }), { status: 200, headers: rateLimitHeaders });
}) as typeof fetch;
try {
const startedAt = Date.now();
await searchBrave("retry burst", { numResults: 1 });
assert.equal(fetchCount, 2);
assert.ok(Date.now() - startedAt >= 1_000);
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
resetBraveRateLimitStateForTests();
}
});
test("searchBrave does not wait for exhausted long-term quotas", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async () =>
new Response(JSON.stringify({ error: { detail: "Quota exceeded" } }), {
status: 429,
headers: {
"content-type": "application/json",
"x-ratelimit-policy": "1;w=1, 2000;w=2678400",
"x-ratelimit-remaining": "0, 0",
"x-ratelimit-reset": "1, 100000",
},
})) as typeof fetch;
try {
const startedAt = Date.now();
await assert.rejects(
() => searchBrave("quota exhausted", { numResults: 1 }),
/rate limit quota is exhausted beyond the retry window/
);
assert.ok(Date.now() - startedAt < 1_000);
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
resetBraveRateLimitStateForTests();
}
});
test("searchBrave blocks locally after a successful request exhausts the long-term quota", async () => {
const originalFetch = globalThis.fetch;
const originalApiKey = env.BRAVE_SEARCH_API_KEY;
let fetchCount = 0;
resetBraveRateLimitStateForTests();
env.BRAVE_SEARCH_API_KEY = "test-brave-key";
globalThis.fetch = (async () => {
fetchCount += 1;
return new Response(JSON.stringify({ web: { results: [] } }), {
status: 200,
headers: {
"content-type": "application/json",
"x-ratelimit-policy": "1;w=1, 2000;w=2678400",
"x-ratelimit-remaining": "0, 0",
"x-ratelimit-reset": "1, 100000",
},
});
}) as typeof fetch;
try {
await searchBrave("last allowed query", { numResults: 1 });
await assert.rejects(
() => searchBrave("over quota query", { numResults: 1 }),
/long-term quota is exhausted/
);
assert.equal(fetchCount, 1);
} finally {
globalThis.fetch = originalFetch;
env.BRAVE_SEARCH_API_KEY = originalApiKey;
resetBraveRateLimitStateForTests();
}
});
+366 -13
View File
@@ -1,11 +1,9 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import test from "node:test"; import test from "node:test";
import { import { type ToolAwareStreamingEvent } from "../src/llm/chat-tools.js";
runPlainChatCompletionsStream, import { completeWithChatCompletionsApi, streamWithChatCompletionsApi } from "../src/llm/protocols/chat-completions-api.js";
runToolAwareChatCompletionsStream, import { completeWithMessagesApi, streamWithMessagesApi } from "../src/llm/protocols/messages-api.js";
runToolAwareOpenAIChatStream, import { streamWithResponsesApi } from "../src/llm/protocols/responses-api.js";
type ToolAwareStreamingEvent,
} from "../src/llm/chat-tools.js";
async function* streamFrom(events: any[]) { async function* streamFrom(events: any[]) {
for (const event of events) { for (const event of events) {
@@ -22,7 +20,7 @@ async function collectEvents(iterable: AsyncIterable<ToolAwareStreamingEvent>) {
return events; return events;
} }
test("OpenAI Responses stream emits text deltas as they arrive", async () => { test("Responses API stream emits text deltas as they arrive", async () => {
const outputMessage = { const outputMessage = {
id: "msg_1", id: "msg_1",
type: "message", type: "message",
@@ -52,7 +50,7 @@ test("OpenAI Responses stream emits text deltas as they arrive", async () => {
}; };
const events = await collectEvents( const events = await collectEvents(
runToolAwareOpenAIChatStream({ streamWithResponsesApi({
client: client as any, client: client as any,
model: "gpt-test", model: "gpt-test",
messages: [{ role: "user", content: "Say hello" }], messages: [{ role: "user", content: "Say hello" }],
@@ -70,7 +68,7 @@ test("OpenAI Responses stream emits text deltas as they arrive", async () => {
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Hello"); assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Hello");
}); });
test("OpenAI-compatible Chat Completions stream emits text deltas as they arrive", async () => { test("Chat Completions API stream emits text deltas as they arrive", async () => {
const client = { const client = {
chat: { chat: {
completions: { completions: {
@@ -89,7 +87,7 @@ test("OpenAI-compatible Chat Completions stream emits text deltas as they arrive
}; };
const events = await collectEvents( const events = await collectEvents(
runToolAwareChatCompletionsStream({ streamWithChatCompletionsApi({
client: client as any, client: client as any,
model: "grok-test", model: "grok-test",
messages: [{ role: "user", content: "Say hello" }], messages: [{ role: "user", content: "Say hello" }],
@@ -124,10 +122,11 @@ test("plain Chat Completions stream does not send Sybil-managed tools", async ()
}; };
const events = await collectEvents( const events = await collectEvents(
runPlainChatCompletionsStream({ streamWithChatCompletionsApi({
client: client as any, client: client as any,
model: "hermes-agent", model: "hermes-agent",
messages: [{ role: "user", content: "Say hi" }], messages: [{ role: "user", content: "Say hi" }],
enabledTools: [],
}) })
); );
@@ -141,7 +140,242 @@ test("plain Chat Completions stream does not send Sybil-managed tools", async ()
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Hi"); assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Hi");
}); });
test("OpenAI-compatible Chat Completions stream emits initiated and terminal tool call updates", async () => { test("Messages API defaults max_tokens to the Anthropic model maximum", async () => {
let requestBody: any = null;
let retrievedModel: string | null = null;
const client = {
models: {
retrieve: async (model: string) => {
retrievedModel = model;
return { id: model, max_tokens: 128000 };
},
},
messages: {
create: async (body: any) => {
requestBody = body;
return {
content: [{ type: "text", text: "Done" }],
usage: { input_tokens: 1, output_tokens: 1 },
};
},
},
};
const result = await completeWithMessagesApi({
client: client as any,
model: "claude-max-default-test",
messages: [{ role: "user", content: "Say done" }],
});
assert.equal(retrievedModel, "claude-max-default-test");
assert.equal(requestBody?.max_tokens, 128000);
assert.equal(result.text, "Done");
});
test("Messages API preserves explicit maxTokens", async () => {
let requestBody: any = null;
let didRetrieveModel = false;
const client = {
models: {
retrieve: async () => {
didRetrieveModel = true;
return { max_tokens: 128000 };
},
},
messages: {
create: async (body: any) => {
requestBody = body;
return streamFrom([
{
type: "message_start",
message: {
usage: { input_tokens: 1, output_tokens: 0 },
},
},
{
type: "content_block_start",
index: 0,
content_block: { type: "text", text: "" },
},
{
type: "content_block_delta",
index: 0,
delta: { type: "text_delta", text: "Done" },
},
{ type: "content_block_stop", index: 0 },
{
type: "message_delta",
delta: { stop_reason: "end_turn", stop_sequence: null },
usage: { output_tokens: 1 },
},
{ type: "message_stop" },
]);
},
},
};
const events = await collectEvents(
streamWithMessagesApi({
client: client as any,
model: "claude-explicit-max-test",
messages: [{ role: "user", content: "Say done" }],
maxTokens: 4096,
})
);
assert.equal(didRetrieveModel, false);
assert.equal(requestBody?.max_tokens, 4096);
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Done");
});
test("fetch_url sends browser-like navigation headers", async () => {
const originalFetch = globalThis.fetch;
const fetchCalls: Array<{ input: RequestInfo | URL; init?: RequestInit }> = [];
globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
fetchCalls.push({ input, init });
return new Response("<!doctype html><title>CPI</title><main>Consumer price index</main>", {
status: 200,
headers: { "content-type": "text/html; charset=utf-8" },
});
}) as typeof fetch;
try {
let requestCount = 0;
const client = {
chat: {
completions: {
create: async () => {
requestCount += 1;
if (requestCount === 1) {
return {
choices: [
{
message: {
tool_calls: [
{
id: "call_1",
type: "function",
function: {
name: "fetch_url",
arguments: JSON.stringify({ url: "https://www.bls.gov/news.release/pdf/cpi.pdf" }),
},
},
],
},
},
],
};
}
return {
choices: [{ message: { content: "Fetched" } }],
};
},
},
},
};
const result = await completeWithChatCompletionsApi({
client: client as any,
model: "grok-test",
messages: [{ role: "user", content: "Fetch CPI PDF" }],
});
assert.equal(result.text, "Fetched");
assert.equal(fetchCalls.length, 1);
assert.equal(String(fetchCalls[0]?.input), "https://www.bls.gov/news.release/pdf/cpi.pdf");
assert.deepEqual(fetchCalls[0]?.init?.headers, {
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,application/pdf;q=0.9,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Upgrade-Insecure-Requests": "1",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
});
assert.equal(result.toolEvents[0]?.status, "completed");
} finally {
globalThis.fetch = originalFetch;
}
});
test("Messages API executes tool_use blocks and sends tool_result follow-up", async () => {
const originalFetch = globalThis.fetch;
const fetchCalls: Array<{ input: RequestInfo | URL; init?: RequestInit }> = [];
globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
fetchCalls.push({ input, init });
return new Response("<!doctype html><title>Example</title><main>Tool result body</main>", {
status: 200,
headers: { "content-type": "text/html; charset=utf-8" },
});
}) as typeof fetch;
try {
const requestBodies: any[] = [];
const client = {
messages: {
create: async (body: any) => {
requestBodies.push(body);
if (requestBodies.length === 1) {
return {
content: [
{
type: "tool_use",
id: "toolu_1",
name: "fetch_url",
input: { url: "https://example.com/article" },
},
],
usage: { input_tokens: 3, output_tokens: 2 },
};
}
return {
content: [{ type: "text", text: "Fetched" }],
usage: { input_tokens: 5, output_tokens: 1 },
};
},
},
};
const result = await completeWithMessagesApi({
client: client as any,
model: "claude-test",
messages: [{ role: "user", content: "Fetch the article" }],
});
assert.equal(result.text, "Fetched");
assert.equal(fetchCalls.length, 1);
assert.equal(String(fetchCalls[0]?.input), "https://example.com/article");
assert.equal(requestBodies.length, 2);
assert.equal(requestBodies[0]?.model, "claude-test");
assert.equal(requestBodies[0]?.tool_choice?.type, "auto");
const fetchTool = requestBodies[0]?.tools?.find((tool: any) => tool.name === "fetch_url");
assert.equal(fetchTool?.input_schema?.type, "object");
assert.equal(fetchTool?.input_schema?.properties?.url?.type, "string");
const secondMessages = requestBodies[1]?.messages ?? [];
assert.equal(secondMessages.at(-2)?.role, "assistant");
assert.equal(secondMessages.at(-2)?.content?.[0]?.type, "tool_use");
assert.equal(secondMessages.at(-1)?.role, "user");
const toolResult = secondMessages.at(-1)?.content?.[0];
assert.equal(toolResult?.type, "tool_result");
assert.equal(toolResult?.tool_use_id, "toolu_1");
assert.equal(toolResult?.is_error, false);
assert.equal(JSON.parse(toolResult?.content ?? "{}").ok, true);
assert.equal(result.toolEvents[0]?.toolCallId, "toolu_1");
assert.equal(result.toolEvents[0]?.status, "completed");
assert.equal(result.usage?.inputTokens, 8);
assert.equal(result.usage?.outputTokens, 3);
assert.equal(result.usage?.totalTokens, 11);
} finally {
globalThis.fetch = originalFetch;
}
});
test("Chat Completions API stream emits initiated and terminal tool call updates", async () => {
let requestCount = 0; let requestCount = 0;
const client = { const client = {
chat: { chat: {
@@ -182,7 +416,7 @@ test("OpenAI-compatible Chat Completions stream emits initiated and terminal too
}; };
const events = await collectEvents( const events = await collectEvents(
runToolAwareChatCompletionsStream({ streamWithChatCompletionsApi({
client: client as any, client: client as any,
model: "grok-test", model: "grok-test",
messages: [{ role: "user", content: "Use a tool" }], messages: [{ role: "user", content: "Use a tool" }],
@@ -206,3 +440,122 @@ test("OpenAI-compatible Chat Completions stream emits initiated and terminal too
assert.equal(typeof toolEvents[1]?.durationMs, "number"); assert.equal(typeof toolEvents[1]?.durationMs, "number");
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Done"); assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Done");
}); });
test("Messages API stream emits initiated and terminal tool call updates", async () => {
let requestCount = 0;
const requestBodies: any[] = [];
const client = {
messages: {
create: async (body: any) => {
requestCount += 1;
requestBodies.push(body);
if (requestCount === 1) {
return streamFrom([
{
type: "message_start",
message: {
usage: { input_tokens: 3, output_tokens: 0 },
},
},
{
type: "content_block_start",
index: 0,
content_block: { type: "text", text: "" },
},
{
type: "content_block_delta",
index: 0,
delta: { type: "text_delta", text: "I'll check that." },
},
{ type: "content_block_stop", index: 0 },
{
type: "content_block_start",
index: 1,
content_block: {
type: "tool_use",
id: "toolu_1",
name: "unknown_tool",
input: {},
},
},
{
type: "content_block_delta",
index: 1,
delta: { type: "input_json_delta", partial_json: "{\"query\":\"current weather\"}" },
},
{ type: "content_block_stop", index: 1 },
{
type: "message_delta",
delta: { stop_reason: "tool_use", stop_sequence: null },
usage: { output_tokens: 2 },
},
{ type: "message_stop" },
]);
}
return streamFrom([
{
type: "message_start",
message: {
usage: { input_tokens: 4, output_tokens: 0 },
},
},
{
type: "content_block_start",
index: 0,
content_block: { type: "text", text: "" },
},
{
type: "content_block_delta",
index: 0,
delta: { type: "text_delta", text: "Done" },
},
{ type: "content_block_stop", index: 0 },
{
type: "message_delta",
delta: { stop_reason: "end_turn", stop_sequence: null },
usage: { output_tokens: 1 },
},
{ type: "message_stop" },
]);
},
},
};
const events = await collectEvents(
streamWithMessagesApi({
client: client as any,
model: "claude-test",
messages: [{ role: "user", content: "Use a tool" }],
})
);
assert.deepEqual(
events.map((event) => event.type),
["tool_call", "tool_call", "delta", "done"]
);
assert.equal(requestBodies[0]?.stream, true);
assert.equal(requestBodies[0]?.tools?.some((tool: any) => tool.name === "fetch_url"), true);
const secondMessages = requestBodies[1]?.messages ?? [];
assert.equal(secondMessages.at(-2)?.role, "assistant");
assert.equal(secondMessages.at(-2)?.content?.[0]?.type, "text");
assert.equal(secondMessages.at(-2)?.content?.[0]?.text, "I'll check that.");
assert.equal(secondMessages.at(-2)?.content?.[1]?.type, "tool_use");
assert.deepEqual(secondMessages.at(-2)?.content?.[1]?.input, { query: "current weather" });
const toolResult = secondMessages.at(-1)?.content?.[0];
assert.equal(toolResult?.type, "tool_result");
assert.equal(toolResult?.tool_use_id, "toolu_1");
assert.equal(toolResult?.is_error, true);
assert.match(JSON.parse(toolResult?.content ?? "{}").error ?? "", /Unknown tool: unknown_tool/);
const toolEvents = events.flatMap((event) => (event.type === "tool_call" ? [event.event] : []));
assert.equal(toolEvents[0]?.toolCallId, "toolu_1");
assert.equal(toolEvents[0]?.status, "initiated");
assert.equal(toolEvents[1]?.toolCallId, "toolu_1");
assert.equal(toolEvents[1]?.status, "failed");
assert.match(toolEvents[1]?.error ?? "", /Unknown tool: unknown_tool/);
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.text : null, "Done");
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.usage?.inputTokens : null, 7);
assert.equal(events.at(-1)?.type === "done" ? events.at(-1)?.result.usage?.outputTokens : null, 3);
});
+3 -3
View File
@@ -1,6 +1,6 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import test from "node:test"; import test from "node:test";
import { buildSystemPromptAugmentation, getAnthropicSystemPrompt } from "../src/llm/message-content.js"; import { buildSystemPromptAugmentation, buildTopLevelSystemPrompt } from "../src/llm/message-content.js";
test("system prompt augmentation includes date and default location", () => { test("system prompt augmentation includes date and default location", () => {
const prompt = buildSystemPromptAugmentation(undefined, new Date("2026-05-24T15:30:00Z")); const prompt = buildSystemPromptAugmentation(undefined, new Date("2026-05-24T15:30:00Z"));
@@ -14,8 +14,8 @@ test("system prompt augmentation uses provided user location", () => {
assert.equal(prompt, "Current date: 2026-05-24.\nUser location: New York, NY."); assert.equal(prompt, "Current date: 2026-05-24.\nUser location: New York, NY.");
}); });
test("Anthropic system prompt includes runtime context with existing system messages", () => { test("top-level system prompt includes runtime context with existing system messages", () => {
const prompt = getAnthropicSystemPrompt( const prompt = buildTopLevelSystemPrompt(
[{ role: "system", content: "Use concise answers." }], [{ role: "system", content: "Use concise answers." }],
"Los Angeles, CA" "Los Angeles, CA"
); );
+42
View File
@@ -0,0 +1,42 @@
import assert from "node:assert/strict";
import test from "node:test";
import { describeProviderChatBackend } from "../src/llm/provider-adapters.js";
test("provider backend registry selects chat protocol and managed-tool mode", () => {
assert.deepEqual(describeProviderChatBackend("openai", []), {
provider: "openai",
protocol: "chat-completions",
managedTools: false,
enabledTools: [],
});
assert.deepEqual(describeProviderChatBackend("openai", ["web_search"]), {
provider: "openai",
protocol: "responses",
managedTools: true,
enabledTools: ["web_search"],
});
assert.deepEqual(describeProviderChatBackend("anthropic", ["web_search"]), {
provider: "anthropic",
protocol: "messages",
managedTools: true,
enabledTools: ["web_search"],
});
assert.deepEqual(describeProviderChatBackend("xai", ["web_search"]), {
provider: "xai",
protocol: "chat-completions",
managedTools: true,
enabledTools: ["web_search"],
});
assert.deepEqual(describeProviderChatBackend("gemini", ["web_search"]), {
provider: "gemini",
protocol: "gemini",
managedTools: true,
enabledTools: ["web_search"],
});
assert.deepEqual(describeProviderChatBackend("hermes-agent", ["web_search"]), {
provider: "hermes-agent",
protocol: "chat-completions",
managedTools: false,
enabledTools: [],
});
});
+4 -2
View File
@@ -5,8 +5,10 @@ import { fromPrismaProvider, serializeProviderFields, toPrismaProvider } from ".
test("Hermes Agent provider id maps between API and Prisma enum forms", () => { test("Hermes Agent provider id maps between API and Prisma enum forms", () => {
assert.equal(toPrismaProvider("hermes-agent"), "hermes_agent"); assert.equal(toPrismaProvider("hermes-agent"), "hermes_agent");
assert.equal(fromPrismaProvider("hermes_agent"), "hermes-agent"); assert.equal(fromPrismaProvider("hermes_agent"), "hermes-agent");
assert.deepEqual(serializeProviderFields({ initiatedProvider: "hermes_agent", lastUsedProvider: "xai" }), { assert.equal(toPrismaProvider("gemini"), "gemini");
assert.equal(fromPrismaProvider("gemini"), "gemini");
assert.deepEqual(serializeProviderFields({ initiatedProvider: "hermes_agent", lastUsedProvider: "gemini" }), {
initiatedProvider: "hermes-agent", initiatedProvider: "hermes-agent",
lastUsedProvider: "xai", lastUsedProvider: "gemini",
}); });
}); });
+1 -1
View File
@@ -1,6 +1,6 @@
import type { Provider } from "./types.js"; import type { Provider } from "./types.js";
const PROVIDERS: Provider[] = ["openai", "anthropic", "xai", "hermes-agent"]; const PROVIDERS: Provider[] = ["openai", "anthropic", "xai", "gemini", "hermes-agent"];
function normalizeBaseUrl(value: string) { function normalizeBaseUrl(value: string) {
const trimmed = value.trim(); const trimmed = value.trim();
+5 -1
View File
@@ -42,12 +42,13 @@ type ToolLogMetadata = {
resultPreview?: string | null; resultPreview?: string | null;
}; };
const BASE_PROVIDERS: Provider[] = ["openai", "anthropic", "xai"]; const BASE_PROVIDERS: Provider[] = ["openai", "anthropic", "xai", "gemini"];
const PROVIDERS: Provider[] = [...BASE_PROVIDERS, "hermes-agent"]; const PROVIDERS: Provider[] = [...BASE_PROVIDERS, "hermes-agent"];
const PROVIDER_FALLBACK_MODELS: Record<Provider, string[]> = { const PROVIDER_FALLBACK_MODELS: Record<Provider, string[]> = {
openai: ["gpt-4.1-mini"], openai: ["gpt-4.1-mini"],
anthropic: ["claude-3-5-sonnet-latest"], anthropic: ["claude-3-5-sonnet-latest"],
xai: ["grok-3-mini"], xai: ["grok-3-mini"],
gemini: ["gemini-3.5-flash", "gemini-flash-latest"],
"hermes-agent": ["hermes-agent"], "hermes-agent": ["hermes-agent"],
}; };
@@ -55,6 +56,7 @@ const EMPTY_MODEL_CATALOG: ModelCatalogResponse["providers"] = {
openai: { models: [], loadedAt: null, error: null }, openai: { models: [], loadedAt: null, error: null },
anthropic: { models: [], loadedAt: null, error: null }, anthropic: { models: [], loadedAt: null, error: null },
xai: { models: [], loadedAt: null, error: null }, xai: { models: [], loadedAt: null, error: null },
gemini: { models: [], loadedAt: null, error: null },
}; };
function escapeTags(value: string) { function escapeTags(value: string) {
@@ -79,6 +81,7 @@ function getProviderLabel(provider: Provider | null | undefined) {
if (provider === "openai") return "OpenAI"; if (provider === "openai") return "OpenAI";
if (provider === "anthropic") return "Anthropic"; if (provider === "anthropic") return "Anthropic";
if (provider === "xai") return "xAI"; if (provider === "xai") return "xAI";
if (provider === "gemini") return "Gemini";
if (provider === "hermes-agent") return "Hermes Agent"; if (provider === "hermes-agent") return "Hermes Agent";
return ""; return "";
} }
@@ -266,6 +269,7 @@ async function main() {
openai: null, openai: null,
anthropic: null, anthropic: null,
xai: null, xai: null,
gemini: null,
"hermes-agent": null, "hermes-agent": null,
}; };
let model: string = config.defaultModel ?? pickProviderModel(getModelOptions(modelCatalog, provider), null); let model: string = config.defaultModel ?? pickProviderModel(getModelOptions(modelCatalog, provider), null);
+1 -1
View File
@@ -1,4 +1,4 @@
export type Provider = "openai" | "anthropic" | "xai" | "hermes-agent"; export type Provider = "openai" | "anthropic" | "xai" | "gemini" | "hermes-agent";
export type ProviderModelInfo = { export type ProviderModelInfo = {
models: string[]; models: string[];
+1
View File
@@ -7,6 +7,7 @@
"dev": "vite", "dev": "vite",
"build": "tsc -b && vite build", "build": "tsc -b && vite build",
"preview": "vite preview", "preview": "vite preview",
"test": "node --test --experimental-strip-types tests/*.test.mjs",
"typecheck": "tsc --noEmit" "typecheck": "tsc --noEmit"
}, },
"dependencies": { "dependencies": {
+16 -2
View File
@@ -3,10 +3,24 @@ self.addEventListener("install", () => {
}); });
self.addEventListener("activate", (event) => { self.addEventListener("activate", (event) => {
event.waitUntil(self.clients.claim()); event.waitUntil(
(async () => {
await self.clients.claim();
const windows = await self.clients.matchAll({ type: "window", includeUncontrolled: true });
await Promise.all(
windows.map(async (client) => {
try {
await client.navigate(client.url);
} catch {
// The client may have closed while the new worker was activating.
}
})
);
})()
);
}); });
self.addEventListener("fetch", (event) => { self.addEventListener("fetch", (event) => {
if (event.request.mode !== "navigate") return; if (event.request.mode !== "navigate") return;
event.respondWith(fetch(event.request)); event.respondWith(fetch(new Request(event.request, { cache: "no-store" })));
}); });
+500 -68
View File
@@ -1,4 +1,5 @@
import { useEffect, useMemo, useRef, useState } from "preact/hooks"; import { useEffect, useMemo, useRef, useState } from "preact/hooks";
import type { TargetedTouchEvent } from "preact";
import { import {
Check, Check,
ChevronDown, ChevronDown,
@@ -62,9 +63,16 @@ import {
type WorkspaceItem, type WorkspaceItem,
} from "@/lib/api"; } from "@/lib/api";
import { useSessionAuth } from "@/hooks/use-session-auth"; import { useSessionAuth } from "@/hooks/use-session-auth";
import {
getChatModelSelection,
getChatModelSelectionSyncKey,
} from "@/lib/chat-model-selection";
import {
resolveSidebarSelectionAfterRefresh,
type SidebarSelection,
} from "@/lib/sidebar-selection";
import { cn } from "@/lib/utils"; import { cn } from "@/lib/utils";
type SidebarSelection = { kind: "chat" | "search"; id: string };
type DraftSelectionKind = "chat" | "search"; type DraftSelectionKind = "chat" | "search";
type SidebarItem = SidebarSelection & { type SidebarItem = SidebarSelection & {
title: string; title: string;
@@ -92,6 +100,11 @@ type ActiveRunsState = {
chats: Record<string, true>; chats: Record<string, true>;
searches: Record<string, true>; searches: Record<string, true>;
}; };
type RefreshCollectionsOptions = {
initialSelection?: SidebarSelection;
reportTransientError?: boolean;
selectFallback?: boolean;
};
function readSidebarSelectionFromUrl(): SidebarSelection | null { function readSidebarSelectionFromUrl(): SidebarSelection | null {
if (typeof window === "undefined") return null; if (typeof window === "undefined") return null;
@@ -123,6 +136,7 @@ const PROVIDER_FALLBACK_MODELS: Record<Provider, string[]> = {
openai: ["gpt-4.1-mini"], openai: ["gpt-4.1-mini"],
anthropic: ["claude-3-5-sonnet-latest"], anthropic: ["claude-3-5-sonnet-latest"],
xai: ["grok-3-mini"], xai: ["grok-3-mini"],
gemini: ["gemini-3.5-flash", "gemini-flash-latest"],
"hermes-agent": ["hermes-agent"], "hermes-agent": ["hermes-agent"],
}; };
@@ -130,13 +144,30 @@ const EMPTY_MODEL_CATALOG: ModelCatalogResponse["providers"] = {
openai: { models: [], loadedAt: null, error: null }, openai: { models: [], loadedAt: null, error: null },
anthropic: { models: [], loadedAt: null, error: null }, anthropic: { models: [], loadedAt: null, error: null },
xai: { models: [], loadedAt: null, error: null }, xai: { models: [], loadedAt: null, error: null },
gemini: { models: [], loadedAt: null, error: null },
}; };
const BASE_PROVIDERS: Provider[] = ["openai", "anthropic", "xai"]; const BASE_PROVIDERS: Provider[] = ["openai", "anthropic", "xai", "gemini"];
const ALL_PROVIDERS: Provider[] = [...BASE_PROVIDERS, "hermes-agent"]; const ALL_PROVIDERS: Provider[] = [...BASE_PROVIDERS, "hermes-agent"];
const MODEL_PREFERENCES_STORAGE_KEY = "sybil:modelPreferencesByProvider"; const MODEL_PREFERENCES_STORAGE_KEY = "sybil:modelPreferencesByProvider";
const QUICK_QUESTION_MODEL_SELECTION_STORAGE_KEY = "sybil:quickQuestionModelSelection"; const QUICK_QUESTION_MODEL_SELECTION_STORAGE_KEY = "sybil:quickQuestionModelSelection";
const STREAM_RESUME_RETRY_DELAYS_MS = [0, 250, 750, 1500];
const MOBILE_LAYOUT_MEDIA_QUERY = "(max-width: 767px)";
const MOBILE_SWIPE_ACTIVATION_DISTANCE = 18;
const MOBILE_SWIPE_DIRECTION_DOMINANCE = 1.22;
const MOBILE_SWIPE_VELOCITY_PROJECTION_SECONDS = 0.18;
const MOBILE_SWIPE_COMPLETION_VELOCITY = 620;
type MobileWorkspaceSwipe = {
touchIdentifier: number;
startX: number;
startY: number;
lastX: number;
lastTimestamp: number;
velocityX: number;
direction: -1 | 1 | null;
};
type ProviderModelPreferences = Record<Provider, string | null>; type ProviderModelPreferences = Record<Provider, string | null>;
@@ -149,6 +180,7 @@ const EMPTY_MODEL_PREFERENCES: ProviderModelPreferences = {
openai: null, openai: null,
anthropic: null, anthropic: null,
xai: null, xai: null,
gemini: null,
"hermes-agent": null, "hermes-agent": null,
}; };
const EMPTY_ACTIVE_RUNS: ActiveRunsState = { const EMPTY_ACTIVE_RUNS: ActiveRunsState = {
@@ -156,6 +188,95 @@ const EMPTY_ACTIVE_RUNS: ActiveRunsState = {
searches: {}, searches: {},
}; };
function isMobileLayout() {
return typeof window !== "undefined" && window.matchMedia(MOBILE_LAYOUT_MEDIA_QUERY).matches;
}
function getMobileSwipeLatchDistance() {
if (typeof window === "undefined") return 112;
return Math.min(Math.max(window.innerWidth * 0.28, 112), 152);
}
function findTouch(touches: TouchList, identifier: number) {
for (let index = 0; index < touches.length; index += 1) {
const touch = touches.item(index);
if (touch?.identifier === identifier) return touch;
}
return null;
}
function waitForAppForeground() {
if (typeof document === "undefined" || typeof window === "undefined") return Promise.resolve();
if (document.visibilityState !== "hidden") return Promise.resolve();
return new Promise<void>((resolve) => {
const finishIfReady = () => {
if (document.visibilityState === "hidden") return;
document.removeEventListener("visibilitychange", finishIfReady);
window.removeEventListener("pageshow", finishIfReady);
resolve();
};
document.addEventListener("visibilitychange", finishIfReady);
window.addEventListener("pageshow", finishIfReady);
});
}
function waitForRetry(delayMs: number) {
if (!delayMs) return Promise.resolve();
return new Promise<void>((resolve) => window.setTimeout(resolve, delayMs));
}
async function retryAfterAppResume<T>(operation: () => Promise<T>) {
await waitForAppForeground();
let lastError: unknown = new Error("Unable to reconnect");
for (const delayMs of STREAM_RESUME_RETRY_DELAYS_MS) {
await waitForRetry(delayMs);
try {
return await operation();
} catch (err) {
lastError = err;
const message = err instanceof Error ? err.message : String(err);
if (message.includes("bearer token")) throw err;
}
}
throw lastError;
}
function getActiveRunsAfterResume() {
return retryAfterAppResume(getActiveRuns);
}
function createClientRequestId() {
if (typeof crypto !== "undefined" && typeof crypto.randomUUID === "function") {
return crypto.randomUUID();
}
return `web-${Date.now()}-${Math.random().toString(36).slice(2)}`;
}
function isRecoverableStreamDisconnect(error: unknown) {
if (error instanceof TypeError) return true;
const message = (error instanceof Error ? error.message : String(error)).toLowerCase();
return (
message.includes("failed to fetch") ||
message.includes("load failed") ||
message.includes("network error") ||
message.includes("networkerror") ||
message.includes("no response stream") ||
message.includes("stream disconnected before completion")
);
}
function hasNewPersistedUserMessage(chat: ChatDetail, previousMessageIds: Set<string>, content: string, attachments: ChatAttachment[]) {
return chat.messages.some((message) => {
if (previousMessageIds.has(message.id) || message.role !== "user" || message.content !== content) return false;
const persistedAttachmentIds = new Set(getMessageAttachments(message.metadata).map((attachment) => attachment.id));
return attachments.length === persistedAttachmentIds.size && attachments.every((attachment) => persistedAttachmentIds.has(attachment.id));
});
}
const TRANSCRIPT_BOTTOM_GAP = 20; const TRANSCRIPT_BOTTOM_GAP = 20;
const REPLY_SCROLL_BUFFER_MIN = 288; const REPLY_SCROLL_BUFFER_MIN = 288;
const REPLY_SCROLL_BUFFER_MAX = 576; const REPLY_SCROLL_BUFFER_MAX = 576;
@@ -345,6 +466,7 @@ function loadStoredModelPreferences() {
openai: typeof parsed.openai === "string" && parsed.openai.trim() ? parsed.openai.trim() : null, openai: typeof parsed.openai === "string" && parsed.openai.trim() ? parsed.openai.trim() : null,
anthropic: typeof parsed.anthropic === "string" && parsed.anthropic.trim() ? parsed.anthropic.trim() : null, anthropic: typeof parsed.anthropic === "string" && parsed.anthropic.trim() ? parsed.anthropic.trim() : null,
xai: typeof parsed.xai === "string" && parsed.xai.trim() ? parsed.xai.trim() : null, xai: typeof parsed.xai === "string" && parsed.xai.trim() ? parsed.xai.trim() : null,
gemini: typeof parsed.gemini === "string" && parsed.gemini.trim() ? parsed.gemini.trim() : null,
"hermes-agent": "hermes-agent":
typeof parsed["hermes-agent"] === "string" && parsed["hermes-agent"].trim() ? parsed["hermes-agent"].trim() : null, typeof parsed["hermes-agent"] === "string" && parsed["hermes-agent"].trim() ? parsed["hermes-agent"].trim() : null,
}; };
@@ -354,7 +476,9 @@ function loadStoredModelPreferences() {
} }
function normalizeStoredProvider(value: unknown): Provider { function normalizeStoredProvider(value: unknown): Provider {
return value === "anthropic" || value === "xai" || value === "openai" || value === "hermes-agent" ? value : "openai"; return value === "anthropic" || value === "xai" || value === "gemini" || value === "openai" || value === "hermes-agent"
? value
: "openai";
} }
function normalizeStoredModelPreferences(value: unknown): ProviderModelPreferences { function normalizeStoredModelPreferences(value: unknown): ProviderModelPreferences {
@@ -364,6 +488,7 @@ function normalizeStoredModelPreferences(value: unknown): ProviderModelPreferenc
openai: typeof parsed.openai === "string" && parsed.openai.trim() ? parsed.openai.trim() : null, openai: typeof parsed.openai === "string" && parsed.openai.trim() ? parsed.openai.trim() : null,
anthropic: typeof parsed.anthropic === "string" && parsed.anthropic.trim() ? parsed.anthropic.trim() : null, anthropic: typeof parsed.anthropic === "string" && parsed.anthropic.trim() ? parsed.anthropic.trim() : null,
xai: typeof parsed.xai === "string" && parsed.xai.trim() ? parsed.xai.trim() : null, xai: typeof parsed.xai === "string" && parsed.xai.trim() ? parsed.xai.trim() : null,
gemini: typeof parsed.gemini === "string" && parsed.gemini.trim() ? parsed.gemini.trim() : null,
"hermes-agent": "hermes-agent":
typeof parsed["hermes-agent"] === "string" && parsed["hermes-agent"].trim() ? parsed["hermes-agent"].trim() : null, typeof parsed["hermes-agent"] === "string" && parsed["hermes-agent"].trim() ? parsed["hermes-agent"].trim() : null,
}; };
@@ -395,6 +520,7 @@ function getProviderLabel(provider: Provider | null | undefined) {
if (provider === "openai") return "OpenAI"; if (provider === "openai") return "OpenAI";
if (provider === "anthropic") return "Anthropic"; if (provider === "anthropic") return "Anthropic";
if (provider === "xai") return "xAI"; if (provider === "xai") return "xAI";
if (provider === "gemini") return "Gemini";
if (provider === "hermes-agent") return "Hermes Agent"; if (provider === "hermes-agent") return "Hermes Agent";
return ""; return "";
} }
@@ -423,14 +549,6 @@ function normalizeEnabledTools(value: unknown, availableTools: ChatToolInfo[]) {
); );
} }
function getChatModelSelection(chat: Pick<ChatSummary, "lastUsedProvider" | "lastUsedModel"> | Pick<ChatDetail, "lastUsedProvider" | "lastUsedModel"> | null) {
if (!chat?.lastUsedProvider || !chat.lastUsedModel?.trim()) return null;
return {
provider: chat.lastUsedProvider,
model: chat.lastUsedModel.trim(),
};
}
type ToolLogMetadata = { type ToolLogMetadata = {
kind: "tool_call"; kind: "tool_call";
toolCallId?: string; toolCallId?: string;
@@ -862,11 +980,14 @@ export default function App() {
const selectedItemRef = useRef<SidebarSelection | null>(null); const selectedItemRef = useRef<SidebarSelection | null>(null);
const pendingTitleGenerationRef = useRef<Set<string>>(new Set()); const pendingTitleGenerationRef = useRef<Set<string>>(new Set());
const chatStreamAbortRefs = useRef<Map<string, AbortController>>(new Map()); const chatStreamAbortRefs = useRef<Map<string, AbortController>>(new Map());
const backgroundSuspendedChatStreamsRef = useRef<Set<string>>(new Set());
const searchRunAbortRefs = useRef<Map<string, AbortController>>(new Map()); const searchRunAbortRefs = useRef<Map<string, AbortController>>(new Map());
const quickQuestionAbortRef = useRef<AbortController | null>(null); const quickQuestionAbortRef = useRef<AbortController | null>(null);
const searchRunCountersRef = useRef<Map<string, number>>(new Map()); const searchRunCountersRef = useRef<Map<string, number>>(new Map());
const shouldAutoScrollRef = useRef(true); const shouldAutoScrollRef = useRef(true);
const wasSendingRef = useRef(false); const wasSendingRef = useRef(false);
const suppressNextMobileComposerFocusRef = useRef(false);
const mobileWorkspaceSwipeRef = useRef<MobileWorkspaceSwipe | null>(null);
const pendingReplyScrollRef = useRef(false); const pendingReplyScrollRef = useRef(false);
const transcriptTailSpacerHeightRef = useRef(TRANSCRIPT_BOTTOM_GAP); const transcriptTailSpacerHeightRef = useRef(TRANSCRIPT_BOTTOM_GAP);
const transcriptTailSpacerSettleFrameRef = useRef<number | null>(null); const transcriptTailSpacerSettleFrameRef = useRef<number | null>(null);
@@ -969,6 +1090,7 @@ export default function App() {
controller.abort(); controller.abort();
} }
chatStreamAbortRefs.current.clear(); chatStreamAbortRefs.current.clear();
backgroundSuspendedChatStreamsRef.current.clear();
for (const controller of searchRunAbortRefs.current.values()) { for (const controller of searchRunAbortRefs.current.values()) {
controller.abort(); controller.abort();
} }
@@ -1007,7 +1129,11 @@ export default function App() {
resetWorkspaceState(); resetWorkspaceState();
}; };
const refreshCollections = async (preferredSelection?: SidebarSelection) => { const refreshCollections = async ({
initialSelection,
reportTransientError = true,
selectFallback = false,
}: RefreshCollectionsOptions = {}) => {
setIsLoadingCollections(true); setIsLoadingCollections(true);
try { try {
const nextWorkspaceItems = await listWorkspaceItems(); const nextWorkspaceItems = await listWorkspaceItems();
@@ -1016,26 +1142,17 @@ export default function App() {
setChats(nextChats); setChats(nextChats);
setSearches(nextSearches); setSearches(nextSearches);
setSelectedItem((current) => { setSelectedItem((current) =>
const hasItem = (candidate: SidebarSelection | null) => { resolveSidebarSelectionAfterRefresh(current, nextWorkspaceItems, {
if (!candidate) return false; initialSelection,
return nextWorkspaceItems.some((item) => item.type === candidate.kind && item.id === candidate.id); selectFallback,
}; })
);
if (preferredSelection && hasItem(preferredSelection)) {
return preferredSelection;
}
if (hasItem(current)) {
return current;
}
const first = nextWorkspaceItems[0];
return first ? { kind: first.type, id: first.id } : null;
});
} catch (err) { } catch (err) {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
if (message.includes("bearer token")) { if (message.includes("bearer token")) {
handleAuthFailure(message); handleAuthFailure(message);
} else { } else if (reportTransientError || !isRecoverableStreamDisconnect(err)) {
setError(message); setError(message);
} }
} finally { } finally {
@@ -1094,7 +1211,7 @@ export default function App() {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
if (message.includes("bearer token")) { if (message.includes("bearer token")) {
handleAuthFailure(message); handleAuthFailure(message);
} else { } else if (!isRecoverableStreamDisconnect(err)) {
setError(message); setError(message);
} }
} finally { } finally {
@@ -1112,7 +1229,7 @@ export default function App() {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
if (message.includes("bearer token")) { if (message.includes("bearer token")) {
handleAuthFailure(message); handleAuthFailure(message);
} else { } else if (!isRecoverableStreamDisconnect(err)) {
setError(message); setError(message);
} }
} finally { } finally {
@@ -1122,9 +1239,14 @@ export default function App() {
useEffect(() => { useEffect(() => {
if (!isAuthenticated) return; if (!isAuthenticated) return;
const preferredSelection = initialRouteSelectionRef.current; const initialSelection = initialRouteSelectionRef.current;
initialRouteSelectionRef.current = null; initialRouteSelectionRef.current = null;
void Promise.all([refreshCollections(preferredSelection ?? undefined), refreshModels(), refreshChatTools(), refreshActiveRuns()]); void Promise.all([
refreshCollections({ initialSelection: initialSelection ?? undefined, selectFallback: true }),
refreshModels(),
refreshChatTools(),
refreshActiveRuns(),
]);
}, [isAuthenticated]); }, [isAuthenticated]);
useEffect(() => { useEffect(() => {
@@ -1135,6 +1257,44 @@ export default function App() {
return () => window.clearInterval(interval); return () => window.clearInterval(interval);
}, [isAuthenticated]); }, [isAuthenticated]);
useEffect(() => {
if (!isAuthenticated) return;
const suspendChatStreams = () => {
for (const [chatId, controller] of chatStreamAbortRefs.current) {
backgroundSuspendedChatStreamsRef.current.add(chatId);
controller.abort();
}
};
const handleVisibilityChange = () => {
if (document.visibilityState === "hidden") {
suspendChatStreams();
return;
}
setError((current) => (current && isRecoverableStreamDisconnect(new Error(current)) ? null : current));
void refreshActiveRuns();
const currentSelection = selectedItemRef.current;
void refreshCollections({ reportTransientError: false });
if (currentSelection?.kind === "chat") {
void refreshChat(currentSelection.id);
} else if (currentSelection?.kind === "search") {
void refreshSearch(currentSelection.id);
}
};
const handlePageShow = () => {
void refreshActiveRuns();
};
document.addEventListener("visibilitychange", handleVisibilityChange);
window.addEventListener("pagehide", suspendChatStreams);
window.addEventListener("pageshow", handlePageShow);
return () => {
document.removeEventListener("visibilitychange", handleVisibilityChange);
window.removeEventListener("pagehide", suspendChatStreams);
window.removeEventListener("pageshow", handlePageShow);
};
}, [isAuthenticated]);
useEffect(() => { useEffect(() => {
const onPopState = () => { const onPopState = () => {
setContextMenu(null); setContextMenu(null);
@@ -1229,10 +1389,18 @@ export default function App() {
const selectedKey = selectedItem ? `${selectedItem.kind}:${selectedItem.id}` : null; const selectedKey = selectedItem ? `${selectedItem.kind}:${selectedItem.id}` : null;
const transcriptViewKey = draftKind ? `draft:${draftKind}` : selectedKey ?? "empty"; const transcriptViewKey = draftKind ? `draft:${draftKind}` : selectedKey ?? "empty";
const selectedChatPendingState = selectedItem?.kind === "chat" ? pendingChatStates[selectedItem.id] ?? null : null; const selectedChatPendingState =
const selectedSearchRunState = selectedItem?.kind === "search" ? runningSearchStates[selectedItem.id] ?? null : null; draftKind === null && selectedItem?.kind === "chat" ? pendingChatStates[selectedItem.id] ?? null : null;
const selectedChatIsActive = selectedItem?.kind === "chat" && (!!selectedChatPendingState || !!activeRuns.chats[selectedItem.id]); const selectedSearchRunState =
const selectedSearchIsActive = selectedItem?.kind === "search" && (!!selectedSearchRunState || !!activeRuns.searches[selectedItem.id]); draftKind === null && selectedItem?.kind === "search" ? runningSearchStates[selectedItem.id] ?? null : null;
const selectedChatIsActive =
draftKind === null &&
selectedItem?.kind === "chat" &&
(!!selectedChatPendingState || !!activeRuns.chats[selectedItem.id]);
const selectedSearchIsActive =
draftKind === null &&
selectedItem?.kind === "search" &&
(!!selectedSearchRunState || !!activeRuns.searches[selectedItem.id]);
const isSearchMode = draftKind ? draftKind === "search" : selectedItem?.kind === "search"; const isSearchMode = draftKind ? draftKind === "search" : selectedItem?.kind === "search";
const isSearchRunning = !!selectedSearchIsActive; const isSearchRunning = !!selectedSearchIsActive;
const isSendingActiveChat = draftKind !== "search" && !!selectedChatIsActive; const isSendingActiveChat = draftKind !== "search" && !!selectedChatIsActive;
@@ -1287,6 +1455,7 @@ export default function App() {
wasSendingRef.current = isSendingActiveChat; wasSendingRef.current = isSendingActiveChat;
if (isSendingActiveChat) return; if (isSendingActiveChat) return;
if (wasSending) { if (wasSending) {
suppressNextMobileComposerFocusRef.current = isMobileLayout();
shouldAutoScrollRef.current = false; shouldAutoScrollRef.current = false;
return; return;
} }
@@ -1310,6 +1479,12 @@ export default function App() {
if (isActiveSelectionSending) return; if (isActiveSelectionSending) return;
const hasWorkspaceSelection = Boolean(selectedItem) || draftKind !== null; const hasWorkspaceSelection = Boolean(selectedItem) || draftKind !== null;
if (!hasWorkspaceSelection) return; if (!hasWorkspaceSelection) return;
const mobileLayout = isMobileLayout();
if (mobileLayout && (Boolean(selectedItem) || suppressNextMobileComposerFocusRef.current)) {
suppressNextMobileComposerFocusRef.current = false;
return;
}
suppressNextMobileComposerFocusRef.current = false;
focusComposer(); focusComposer();
}, [draftKind, isActiveSelectionSending, selectedKey]); }, [draftKind, isActiveSelectionSending, selectedKey]);
@@ -1332,7 +1507,10 @@ export default function App() {
}; };
}, []); }, []);
const messages = selectedChat?.messages ?? []; const messages =
draftKind === null && selectedItem?.kind === "chat" && selectedChat?.id === selectedItem.id
? selectedChat.messages
: [];
useEffect(() => { useEffect(() => {
if (isSearchMode && pendingAttachments.length) { if (isSearchMode && pendingAttachments.length) {
@@ -1380,15 +1558,32 @@ export default function App() {
return searches.find((search) => search.id === selectedItem.id) ?? null; return searches.find((search) => search.id === selectedItem.id) ?? null;
}, [searches, selectedItem]); }, [searches, selectedItem]);
useEffect(() => { const selectedChatModelSelection = useMemo(() => {
if (draftKind || selectedItem?.kind !== "chat") return; if (draftKind || selectedItem?.kind !== "chat") return null;
const detailSelection = selectedChat?.id === selectedItem.id ? getChatModelSelection(selectedChat) : null; const detailSelection = selectedChat?.id === selectedItem.id ? getChatModelSelection(selectedChat) : null;
const summarySelection = getChatModelSelection(selectedChatSummary); const summarySelection = getChatModelSelection(selectedChatSummary);
const nextSelection = detailSelection ?? summarySelection; return detailSelection ?? summarySelection;
if (!nextSelection) return; }, [
setProvider(nextSelection.provider); draftKind,
setModel(nextSelection.model); selectedChat?.id,
}, [draftKind, selectedChat, selectedChatSummary, selectedItem]); selectedChat?.lastUsedModel,
selectedChat?.lastUsedProvider,
selectedChatSummary?.id,
selectedChatSummary?.lastUsedModel,
selectedChatSummary?.lastUsedProvider,
selectedItem?.id,
selectedItem?.kind,
]);
const selectedChatModelSelectionSyncKey = getChatModelSelectionSyncKey(
selectedItem?.kind === "chat" ? selectedItem.id : null,
selectedChatModelSelection
);
useEffect(() => {
if (!selectedChatModelSelection) return;
setProvider(selectedChatModelSelection.provider);
setModel(selectedChatModelSelection.model);
}, [selectedChatModelSelectionSyncKey]);
useEffect(() => { useEffect(() => {
if (draftKind === "chat") return; if (draftKind === "chat") return;
@@ -1439,6 +1634,7 @@ export default function App() {
setError(null); setError(null);
setContextMenu(null); setContextMenu(null);
setDraftKind("chat"); setDraftKind("chat");
selectedItemRef.current = null;
setSelectedItem(null); setSelectedItem(null);
setSelectedChat(null); setSelectedChat(null);
setSelectedSearch(null); setSelectedSearch(null);
@@ -1451,6 +1647,120 @@ export default function App() {
setIsMobileSidebarOpen(false); setIsMobileSidebarOpen(false);
}; };
const resetMobileWorkspaceSwipe = () => {
mobileWorkspaceSwipeRef.current = null;
};
const handleMobileWorkspaceTouchStart = (event: TargetedTouchEvent<HTMLElement>) => {
if (
event.touches.length !== 1 ||
!isMobileLayout() ||
isMobileSidebarOpen ||
isQuickQuestionOpen ||
isChatSettingsOpen ||
renameChatDialog !== null
) {
return;
}
const target = event.target;
if (
target instanceof Element &&
target.closest("input, textarea, select, button, a, [role='dialog'], [data-mobile-swipe-ignore]")
) {
return;
}
const touch = event.touches.item(0);
if (!touch) return;
mobileWorkspaceSwipeRef.current = {
touchIdentifier: touch.identifier,
startX: touch.clientX,
startY: touch.clientY,
lastX: touch.clientX,
lastTimestamp: event.timeStamp,
velocityX: 0,
direction: null,
};
};
const handleMobileWorkspaceTouchMove = (event: TargetedTouchEvent<HTMLElement>) => {
const swipe = mobileWorkspaceSwipeRef.current;
if (!swipe) return;
const touch = findTouch(event.touches, swipe.touchIdentifier);
if (!touch) return;
const deltaX = touch.clientX - swipe.startX;
const deltaY = touch.clientY - swipe.startY;
const horizontalTravel = Math.abs(deltaX);
const verticalTravel = Math.abs(deltaY);
if (swipe.direction === null) {
if (
verticalTravel >= MOBILE_SWIPE_ACTIVATION_DISTANCE &&
verticalTravel > horizontalTravel * MOBILE_SWIPE_DIRECTION_DOMINANCE
) {
resetMobileWorkspaceSwipe();
return;
}
if (
horizontalTravel < MOBILE_SWIPE_ACTIVATION_DISTANCE ||
horizontalTravel < verticalTravel * MOBILE_SWIPE_DIRECTION_DOMINANCE
) {
return;
}
swipe.direction = deltaX > 0 ? 1 : -1;
}
const elapsedMs = event.timeStamp - swipe.lastTimestamp;
if (elapsedMs > 0 && touch.clientX !== swipe.lastX) {
swipe.velocityX = ((touch.clientX - swipe.lastX) / elapsedMs) * 1000;
}
swipe.lastX = touch.clientX;
swipe.lastTimestamp = event.timeStamp;
if (event.cancelable) event.preventDefault();
};
const handleMobileWorkspaceTouchEnd = (event: TargetedTouchEvent<HTMLElement>) => {
const swipe = mobileWorkspaceSwipeRef.current;
if (!swipe) return;
const touch = findTouch(event.changedTouches, swipe.touchIdentifier);
if (!touch) return;
const direction = swipe.direction;
const deltaX = touch.clientX - swipe.startX;
const elapsedMs = event.timeStamp - swipe.lastTimestamp;
if (elapsedMs > 0 && touch.clientX !== swipe.lastX) {
swipe.velocityX = ((touch.clientX - swipe.lastX) / elapsedMs) * 1000;
}
resetMobileWorkspaceSwipe();
if (direction === null) return;
const directionalDistance = deltaX * direction;
const directionalVelocity = swipe.velocityX * direction;
const projectedDistance =
directionalDistance + directionalVelocity * MOBILE_SWIPE_VELOCITY_PROJECTION_SECONDS;
const shouldComplete =
directionalVelocity <= -MOBILE_SWIPE_COMPLETION_VELOCITY
? false
: directionalVelocity >= MOBILE_SWIPE_COMPLETION_VELOCITY ||
directionalDistance >= getMobileSwipeLatchDistance() ||
projectedDistance >= getMobileSwipeLatchDistance();
if (!shouldComplete) return;
if (direction > 0) {
setIsMobileSidebarOpen(true);
} else {
handleCreateChat();
}
};
const handleMobileWorkspaceTouchCancel = (event: TargetedTouchEvent<HTMLElement>) => {
const swipe = mobileWorkspaceSwipeRef.current;
if (!swipe || !findTouch(event.changedTouches, swipe.touchIdentifier)) return;
resetMobileWorkspaceSwipe();
};
const handleOpenQuickQuestion = () => { const handleOpenQuickQuestion = () => {
setQuickQuestionError(null); setQuickQuestionError(null);
setIsQuickQuestionOpen(true); setIsQuickQuestionOpen(true);
@@ -1461,6 +1771,7 @@ export default function App() {
setError(null); setError(null);
setContextMenu(null); setContextMenu(null);
setDraftKind("search"); setDraftKind("search");
selectedItemRef.current = null;
setSelectedItem(null); setSelectedItem(null);
setSelectedChat(null); setSelectedChat(null);
setSelectedSearch(null); setSelectedSearch(null);
@@ -1788,7 +2099,7 @@ export default function App() {
} else { } else {
await deleteSearch(target.id); await deleteSearch(target.id);
} }
await refreshCollections(); await refreshCollections({ selectFallback: true });
} catch (err) { } catch (err) {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
if (message.includes("bearer token")) { if (message.includes("bearer token")) {
@@ -2043,6 +2354,7 @@ export default function App() {
if (!baseChat || baseChat.id !== chatId) { if (!baseChat || baseChat.id !== chatId) {
baseChat = await getChat(chatId); baseChat = await getChat(chatId);
} }
const previousMessageIds = new Set(baseChat.messages.map((message) => message.id));
setPendingChatStates((current) => ({ setPendingChatStates((current) => ({
...current, ...current,
@@ -2095,12 +2407,20 @@ export default function App() {
}); });
} }
const target: SidebarSelection = { kind: "chat", id: chatId };
const clientRequestId = createClientRequestId();
while (true) {
let streamErrorMessage: string | null = null; let streamErrorMessage: string | null = null;
let replayedAssistantText = "";
const abortController = new AbortController();
chatStreamAbortRefs.current.set(chatId, abortController);
try { try {
await runCompletionStream( await runCompletionStream(
{ {
chatId, chatId,
clientRequestId,
provider, provider,
model: selectedModel, model: selectedModel,
messages: requestMessages, messages: requestMessages,
@@ -2123,6 +2443,7 @@ export default function App() {
}, },
onDelta: (payload) => { onDelta: (payload) => {
if (!payload.text) return; if (!payload.text) return;
replayedAssistantText += payload.text;
setPendingChatStates((current) => { setPendingChatStates((current) => {
const pendingState = current[chatId]; const pendingState = current[chatId];
if (!pendingState) return current; if (!pendingState) return current;
@@ -2131,7 +2452,7 @@ export default function App() {
const isTarget = index === all.length - 1 && message.id.startsWith("temp-assistant-"); const isTarget = index === all.length - 1 && message.id.startsWith("temp-assistant-");
if (!isTarget) return message; if (!isTarget) return message;
updated = true; updated = true;
return { ...message, content: message.content + payload.text }; return { ...message, content: replayedAssistantText };
}); });
return updated ? { ...current, [chatId]: { messages: nextMessages } } : current; return updated ? { ...current, [chatId]: { messages: nextMessages } } : current;
}); });
@@ -2153,30 +2474,90 @@ export default function App() {
onError: (payload) => { onError: (payload) => {
streamErrorMessage = payload.message; streamErrorMessage = payload.message;
}, },
} },
{ signal: abortController.signal }
); );
if (streamErrorMessage) { if (streamErrorMessage) {
throw new Error(streamErrorMessage); throw new Error(streamErrorMessage);
} }
await refreshCollections(); backgroundSuspendedChatStreamsRef.current.delete(chatId);
const currentSelection = selectedItemRef.current; const persistedChat = await retryAfterAppResume(() => getChat(chatId));
if (currentSelection?.kind === "chat" && currentSelection.id === chatId) { await refreshCollections({ reportTransientError: false });
await refreshChat(chatId); if (isCurrentSelection(target)) {
setSelectedChat(persistedChat);
setSelectedSearch(null);
} }
removePendingChatState(chatId); removePendingChatState(chatId);
removeActiveRun("chat", chatId); removeActiveRun("chat", chatId);
if (currentSelection?.kind === "chat" && currentSelection.id === chatId) { if (isCurrentSelection(target)) {
requestSettleTranscriptTailSpacer(); requestSettleTranscriptTailSpacer();
} }
return { kind: "chat", id: chatId }; return target;
} catch (err) { } catch (caughtError) {
let err: unknown = caughtError;
const wasBackgroundSuspended =
abortController.signal.aborted && backgroundSuspendedChatStreamsRef.current.delete(chatId);
const shouldResumeStream =
wasBackgroundSuspended ||
(!abortController.signal.aborted && streamErrorMessage === null && isRecoverableStreamDisconnect(err));
if (chatStreamAbortRefs.current.get(chatId) === abortController) {
chatStreamAbortRefs.current.delete(chatId);
}
if (shouldResumeStream) {
try {
const persistedChat = await retryAfterAppResume(() => getChat(chatId));
setError(null);
if (isCurrentSelection(target)) {
setSelectedChat(persistedChat);
setSelectedSearch(null);
}
// Retrying the same idempotent request either starts an unaccepted
// submission or replays the existing backend-owned stream. Keep
// the current optimistic transcript visible until replay begins.
continue;
} catch (resumeError) {
if (isRecoverableStreamDisconnect(resumeError)) {
setError(null);
await waitForAppForeground();
await waitForRetry(1000);
continue;
}
err = resumeError;
}
} else if (streamErrorMessage) {
try {
const persistedChat = await retryAfterAppResume(() => getChat(chatId));
if (hasNewPersistedUserMessage(persistedChat, previousMessageIds, content, attachments)) {
if (isCurrentSelection(target)) {
setSelectedChat(persistedChat);
setSelectedSearch(null);
setError(streamErrorMessage);
}
removePendingChatState(chatId);
removeActiveRun("chat", chatId);
requestSettleTranscriptTailSpacer();
return target;
}
} catch (reconcileError) {
err = reconcileError;
}
}
removePendingChatState(chatId); removePendingChatState(chatId);
removeActiveRun("chat", chatId); removeActiveRun("chat", chatId);
pendingReplyScrollRef.current = false; pendingReplyScrollRef.current = false;
setTranscriptTailSpacer(TRANSCRIPT_BOTTOM_GAP); setTranscriptTailSpacer(TRANSCRIPT_BOTTOM_GAP);
throw err; throw err;
} finally {
if (chatStreamAbortRefs.current.get(chatId) === abortController) {
chatStreamAbortRefs.current.delete(chatId);
}
}
} }
}; };
@@ -2317,23 +2698,28 @@ export default function App() {
} }
} }
await refreshCollections(target); await refreshCollections();
return target; return target;
}; };
const attachToActiveChatStream = async (chatId: string) => { async function attachToActiveChatStream(chatId: string, resetFromServer = false) {
if (chatStreamAbortRefs.current.has(chatId)) return; if (chatStreamAbortRefs.current.has(chatId)) return;
const target: SidebarSelection = { kind: "chat", id: chatId }; const target: SidebarSelection = { kind: "chat", id: chatId };
addActiveRun("chat", chatId);
let shouldResetFromServer = resetFromServer;
try {
while (true) {
const abortController = new AbortController(); const abortController = new AbortController();
chatStreamAbortRefs.current.set(chatId, abortController); chatStreamAbortRefs.current.set(chatId, abortController);
addActiveRun("chat", chatId);
let streamErrorMessage: string | null = null; let streamErrorMessage: string | null = null;
try { try {
const baseChat = selectedChat?.id === chatId ? selectedChat : await getChat(chatId); const baseChat = await getChat(chatId);
const resetPendingState = shouldResetFromServer;
shouldResetFromServer = false;
setPendingChatStates((current) => { setPendingChatStates((current) => {
if (current[chatId]) return current; if (!resetPendingState && current[chatId]) return current;
return { return {
...current, ...current,
[chatId]: { [chatId]: {
@@ -2404,12 +2790,41 @@ export default function App() {
throw new Error(streamErrorMessage); throw new Error(streamErrorMessage);
} }
backgroundSuspendedChatStreamsRef.current.delete(chatId);
await refreshCollections(); await refreshCollections();
if (isCurrentSelection(target)) { if (isCurrentSelection(target)) {
await refreshChat(chatId); await refreshChat(chatId);
} }
return;
} catch (err) { } catch (err) {
if (abortController.signal.aborted) return; const wasBackgroundSuspended =
abortController.signal.aborted && backgroundSuspendedChatStreamsRef.current.delete(chatId);
const shouldResumeStream =
wasBackgroundSuspended ||
(!abortController.signal.aborted && streamErrorMessage === null && isRecoverableStreamDisconnect(err));
if (shouldResumeStream) {
shouldResetFromServer = true;
try {
const resumedRuns = await getActiveRunsAfterResume();
setActiveRuns(buildActiveRunsState(resumedRuns));
if (resumedRuns.chats.includes(chatId)) {
continue;
}
const persistedChat = await retryAfterAppResume(() => getChat(chatId));
if (isCurrentSelection(target)) {
setSelectedChat(persistedChat);
setSelectedSearch(null);
}
void refreshCollections();
return;
} catch (resumeError) {
err = resumeError;
}
}
if (abortController.signal.aborted && !shouldResumeStream) return;
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
if (message.includes("active chat stream not found")) { if (message.includes("active chat stream not found")) {
await refreshActiveRuns(); await refreshActiveRuns();
@@ -2419,15 +2834,22 @@ export default function App() {
} else if (isCurrentSelection(target)) { } else if (isCurrentSelection(target)) {
setError(message); setError(message);
} }
return;
} finally { } finally {
if (chatStreamAbortRefs.current.get(chatId) === abortController) {
chatStreamAbortRefs.current.delete(chatId); chatStreamAbortRefs.current.delete(chatId);
}
}
}
} finally {
backgroundSuspendedChatStreamsRef.current.delete(chatId);
removePendingChatState(chatId); removePendingChatState(chatId);
removeActiveRun("chat", chatId); removeActiveRun("chat", chatId);
if (isCurrentSelection(target)) { if (isCurrentSelection(target)) {
requestSettleTranscriptTailSpacer(); requestSettleTranscriptTailSpacer();
} }
} }
}; }
const attachToActiveSearchStream = async (searchId: string) => { const attachToActiveSearchStream = async (searchId: string) => {
if (searchRunAbortRefs.current.has(searchId)) return; if (searchRunAbortRefs.current.has(searchId)) return;
@@ -2501,7 +2923,7 @@ export default function App() {
{ signal: abortController.signal } { signal: abortController.signal }
); );
await refreshCollections(target); await refreshCollections();
if (isCurrentSelection(target)) { if (isCurrentSelection(target)) {
await refreshSearch(searchId); await refreshSearch(searchId);
} }
@@ -2569,7 +2991,7 @@ export default function App() {
messages: [], messages: [],
}); });
setSelectedSearch(null); setSelectedSearch(null);
await refreshCollections({ kind: "chat", id: chat.id }); await refreshCollections();
await refreshChat(chat.id); await refreshChat(chat.id);
} catch (err) { } catch (err) {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
@@ -2731,7 +3153,7 @@ export default function App() {
messages: [], messages: [],
}); });
setSelectedSearch(null); setSelectedSearch(null);
await refreshCollections({ kind: "chat", id: chat.id }); await refreshCollections();
await refreshChat(chat.id); await refreshChat(chat.id);
} catch (err) { } catch (err) {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
@@ -2790,7 +3212,8 @@ export default function App() {
await refreshSearch(sentTarget.id); await refreshSearch(sentTarget.id);
} }
} finally { } finally {
if (!sentTarget || isCurrentSelection(sentTarget)) { const shouldSuppressMobileChatFocus = sentTarget?.kind === "chat" && isMobileLayout();
if ((!sentTarget || isCurrentSelection(sentTarget)) && !shouldSuppressMobileChatFocus) {
focusComposer(); focusComposer();
} }
} }
@@ -2977,7 +3400,13 @@ export default function App() {
</div> </div>
</aside> </aside>
<main className="glass-panel relative flex min-w-0 flex-1 flex-col overflow-hidden border-violet-300/18 md:rounded-2xl md:border"> <main
className="glass-panel relative flex min-w-0 flex-1 touch-pan-y flex-col overflow-hidden border-violet-300/18 md:touch-auto md:rounded-2xl md:border"
onTouchStart={handleMobileWorkspaceTouchStart}
onTouchMove={handleMobileWorkspaceTouchMove}
onTouchEnd={handleMobileWorkspaceTouchEnd}
onTouchCancel={handleMobileWorkspaceTouchCancel}
>
<header className="flex items-center justify-between gap-2 border-b border-violet-300/12 bg-[linear-gradient(180deg,hsl(243_48%_10%_/_0.86),hsl(236_48%_6%_/_0.66))] px-4 py-3 md:gap-3 md:px-7"> <header className="flex items-center justify-between gap-2 border-b border-violet-300/12 bg-[linear-gradient(180deg,hsl(243_48%_10%_/_0.86),hsl(236_48%_6%_/_0.66))] px-4 py-3 md:gap-3 md:px-7">
<div className="flex min-w-0 items-center gap-2"> <div className="flex min-w-0 items-center gap-2">
<Button <Button
@@ -3052,7 +3481,10 @@ export default function App() {
<div ref={transcriptEndRef} /> <div ref={transcriptEndRef} />
</div> </div>
<footer className="pointer-events-none absolute inset-x-0 bottom-0 z-10 bg-[linear-gradient(to_top,hsl(235_50%_4%)_0%,hsl(235_50%_4%_/_0.92)_58%,transparent)] p-3 pt-14 md:p-6 md:pt-20"> <footer
className="pointer-events-none absolute inset-x-0 bottom-0 z-10 bg-[linear-gradient(to_top,hsl(235_50%_4%)_0%,hsl(235_50%_4%_/_0.92)_58%,transparent)] p-3 pt-14 md:p-6 md:pt-20"
data-mobile-swipe-ignore
>
<div <div
className={cn( className={cn(
"pointer-events-auto mx-auto max-w-4xl rounded-2xl border bg-[linear-gradient(135deg,hsl(235_48%_7%_/_0.96),hsl(258_48%_11%_/_0.94))] p-2 shadow-lg shadow-black/20 transition", "pointer-events-auto mx-auto max-w-4xl rounded-2xl border bg-[linear-gradient(135deg,hsl(235_48%_7%_/_0.96),hsl(258_48%_11%_/_0.94))] p-2 shadow-lg shadow-black/20 transition",
+325 -17
View File
@@ -1,8 +1,10 @@
import { useEffect, useMemo, useRef, useState } from "preact/hooks";
import type { ComponentChildren, JSX } from "preact";
import { cn } from "@/lib/utils"; import { cn } from "@/lib/utils";
import { ChatAttachmentList } from "@/components/chat/chat-attachment-list"; import { ChatAttachmentList } from "@/components/chat/chat-attachment-list";
import { getMessageAttachments, type Message } from "@/lib/api"; import { getMessageAttachments, type Message } from "@/lib/api";
import { MarkdownContent } from "@/components/markdown/markdown-content"; import { MarkdownContent } from "@/components/markdown/markdown-content";
import { Globe2, Link2, Wrench } from "lucide-preact"; import { ChevronDown, ChevronUp, Globe2, Link2, Wrench } from "lucide-preact";
type Props = { type Props = {
messages: Message[]; messages: Message[];
@@ -72,6 +74,29 @@ function formatToolTimestamp(...values: Array<string | null | undefined>) {
} }
type ToolCallVisualState = "initiated" | "completed" | "failed"; type ToolCallVisualState = "initiated" | "completed" | "failed";
type MessageRenderItem = { kind: "message"; message: Message } | { kind: "tool_group"; key: string; messages: Message[] };
type ToolStackStyle = JSX.CSSProperties & {
"--tool-stack-x"?: string;
"--tool-stack-y"?: string;
"--tool-stack-z"?: string;
"--tool-stack-scale"?: string;
"--tool-stack-opacity"?: string;
"--tool-stack-delay"?: string;
"--tool-stack-from-transform"?: string;
"--tool-stack-to-transform"?: string;
"--tool-stack-from-opacity"?: string;
"--tool-stack-to-opacity"?: string;
};
type ToolStackContainerStyle = JSX.CSSProperties & {
"--tool-stack-from-height"?: string;
"--tool-stack-to-height"?: string;
};
type ToolStackMotionDirection = "expand" | "collapse" | null;
const COLLAPSED_TOOL_STACK_LIMIT = 4;
const TOOL_STACK_CARD_HEIGHT = 62;
const TOOL_STACK_CARD_GAP = 10;
const TOOL_STACK_LAYOUT_ANIMATION_MS = 340;
function getToolVisualState(metadata: ToolLogMetadata): ToolCallVisualState { function getToolVisualState(metadata: ToolLogMetadata): ToolCallVisualState {
if (metadata.status === "failed") return "failed"; if (metadata.status === "failed") return "failed";
@@ -89,16 +114,132 @@ function getToolDetailLabel(message: Message, metadata: ToolLogMetadata, state:
.join(" • "); .join(" • ");
} }
export function ChatMessagesPanel({ messages, isLoading, isSending }: Props) { function buildMessageRenderItems(messages: Message[]) {
const hasPendingAssistant = messages.some((message) => message.id.startsWith("temp-assistant-") && message.content.trim().length === 0); const items: MessageRenderItem[] = [];
let toolRun: Message[] = [];
return ( const flushToolRun = () => {
<> if (!toolRun.length) return;
{isLoading && messages.length === 0 ? <p className="text-sm text-muted-foreground">Loading messages...</p> : null} if (toolRun.length === 1) {
<div className="mx-auto max-w-4xl space-y-6"> items.push({ kind: "message", message: toolRun[0] });
{messages.map((message) => { } else {
items.push({ kind: "tool_group", key: toolRun[0].id, messages: toolRun });
}
toolRun = [];
};
for (const message of messages) {
if (message.role === "tool" && asToolLogMetadata(message.metadata)) {
toolRun.push(message);
continue;
}
flushToolRun();
items.push({ kind: "message", message });
}
flushToolRun();
return items;
}
function getToolCallMessageIDs(messages: Message[]) {
const ids = new Set<string>();
for (const message of messages) {
if (message.role === "tool" && asToolLogMetadata(message.metadata)) ids.add(message.id);
}
return ids;
}
function getToolStackHeight(messageCount: number, expanded: boolean) {
const visibleCount = Math.min(messageCount, COLLAPSED_TOOL_STACK_LIMIT);
return expanded
? `${TOOL_STACK_CARD_HEIGHT + Math.max(0, messageCount - 1) * (TOOL_STACK_CARD_HEIGHT + TOOL_STACK_CARD_GAP)}px`
: `${TOOL_STACK_CARD_HEIGHT + Math.max(0, visibleCount - 1) * TOOL_STACK_CARD_GAP}px`;
}
function getToolStackContainerStyle(messageCount: number, expanded: boolean, motionDirection: ToolStackMotionDirection): ToolStackContainerStyle {
const collapsedHeight = getToolStackHeight(messageCount, false);
const expandedHeight = getToolStackHeight(messageCount, true);
const targetHeight = expanded ? expandedHeight : collapsedHeight;
const fromHeight = motionDirection === "expand" ? collapsedHeight : motionDirection === "collapse" ? expandedHeight : targetHeight;
return {
"--tool-stack-from-height": fromHeight,
"--tool-stack-to-height": targetHeight,
height: targetHeight,
};
}
function getExpandedToolLayout(index: number, messageCount: number) {
const y = `${index * (TOOL_STACK_CARD_HEIGHT + TOOL_STACK_CARD_GAP)}px`;
return {
opacity: "1",
transform: `translate3d(0px, ${y}, 0px) scale(1)`,
x: "0px",
y,
z: "0px",
scale: "1",
zIndex: messageCount - index,
};
}
function getCollapsedToolLayout(index: number, messageCount: number) {
const depth = messageCount - index - 1;
const visibleDepth = Math.min(depth, COLLAPSED_TOOL_STACK_LIMIT - 1);
const isHidden = depth >= COLLAPSED_TOOL_STACK_LIMIT;
const visibleCount = Math.min(messageCount, COLLAPSED_TOOL_STACK_LIMIT);
const x = `${visibleDepth * 11}px`;
const y = `${visibleDepth * TOOL_STACK_CARD_GAP}px`;
const z = `${visibleDepth * -36}px`;
const scale = `${Math.max(0.88, 1 - visibleDepth * 0.035)}`;
const opacity = isHidden ? "0" : `${Math.max(0.34, 1 - visibleDepth * 0.22)}`;
return {
opacity,
transform: `translate3d(${x}, ${y}, ${z}) scale(${scale})`,
x,
y,
z,
scale,
zIndex: isHidden ? 0 : visibleCount - visibleDepth,
};
}
function getToolStackStyle(index: number, messageCount: number, expanded: boolean, motionDirection: ToolStackMotionDirection): ToolStackStyle {
const expandedLayout = getExpandedToolLayout(index, messageCount);
const collapsedLayout = getCollapsedToolLayout(index, messageCount);
const targetLayout = expanded ? expandedLayout : collapsedLayout;
const fromLayout = motionDirection === "expand" ? collapsedLayout : motionDirection === "collapse" ? expandedLayout : targetLayout;
return {
"--tool-stack-x": targetLayout.x,
"--tool-stack-y": targetLayout.y,
"--tool-stack-z": targetLayout.z,
"--tool-stack-scale": targetLayout.scale,
"--tool-stack-opacity": targetLayout.opacity,
"--tool-stack-delay": `${Math.min(messageCount - index - 1, COLLAPSED_TOOL_STACK_LIMIT - 1) * 34}ms`,
"--tool-stack-from-transform": fromLayout.transform,
"--tool-stack-to-transform": targetLayout.transform,
"--tool-stack-from-opacity": fromLayout.opacity,
"--tool-stack-to-opacity": targetLayout.opacity,
opacity: targetLayout.opacity,
transform: targetLayout.transform,
zIndex: targetLayout.zIndex,
};
}
function ToolCallCard({
message,
className,
style,
}: {
message: Message;
className?: string;
style?: JSX.CSSProperties;
}) {
const toolLogMetadata = asToolLogMetadata(message.metadata); const toolLogMetadata = asToolLogMetadata(message.metadata);
if (message.role === "tool" && toolLogMetadata) { if (!toolLogMetadata) return null;
const iconKind = getToolIconName(toolLogMetadata.toolName ?? message.name); const iconKind = getToolIconName(toolLogMetadata.toolName ?? message.name);
const Icon = iconKind === "search" ? Globe2 : iconKind === "fetch" ? Link2 : Wrench; const Icon = iconKind === "search" ? Globe2 : iconKind === "fetch" ? Link2 : Wrench;
const toolState = getToolVisualState(toolLogMetadata); const toolState = getToolVisualState(toolLogMetadata);
@@ -107,17 +248,19 @@ export function ChatMessagesPanel({ messages, isLoading, isSending }: Props) {
const toolSummary = getToolSummary(message, toolLogMetadata); const toolSummary = getToolSummary(message, toolLogMetadata);
const toolLabel = getToolLabel(message, toolLogMetadata); const toolLabel = getToolLabel(message, toolLogMetadata);
const toolDetailLabel = getToolDetailLabel(message, toolLogMetadata, toolState); const toolDetailLabel = getToolDetailLabel(message, toolLogMetadata, toolState);
return ( return (
<div key={message.id} className="flex justify-start">
<div <div
className={cn( className={cn(
"inline-flex max-w-[85%] min-w-0 items-start gap-3 overflow-hidden rounded-xl border px-3 py-2.5 shadow-[inset_0_1px_0_hsl(180_100%_88%_/_0.06)]", "inline-flex min-w-0 items-start gap-3 overflow-hidden rounded-xl border px-3 py-2.5 shadow-[inset_0_1px_0_hsl(180_100%_88%_/_0.06)]",
isFailed isFailed
? "border-rose-400/34 bg-[linear-gradient(90deg,hsl(350_72%_44%_/_0.18),hsl(342_66%_9%_/_0.72))]" ? "border-rose-400/44 bg-[linear-gradient(90deg,hsl(350_64%_20%),hsl(342_58%_9%))]"
: isInitiated : isInitiated
? "border-amber-300/34 bg-[linear-gradient(90deg,hsl(43_74%_30%_/_0.34),hsl(260_48%_13%_/_0.74))]" ? "border-amber-300/44 bg-[linear-gradient(90deg,hsl(43_72%_20%),hsl(260_48%_13%))]"
: "border-cyan-400/34 bg-[linear-gradient(90deg,hsl(184_89%_21%_/_0.70),hsl(208_66%_12%_/_0.78))]" : "border-cyan-400/44 bg-[linear-gradient(90deg,hsl(184_82%_14%),hsl(208_66%_10%))]",
className
)} )}
style={style}
title={`${toolSummary}\n${toolLabel}${toolDetailLabel}`} title={`${toolSummary}\n${toolLabel}${toolDetailLabel}`}
> >
<span <span
@@ -133,9 +276,7 @@ export function ChatMessagesPanel({ messages, isLoading, isSending }: Props) {
<Icon className="h-4 w-4" /> <Icon className="h-4 w-4" />
</span> </span>
<span className="min-w-0 flex-1 space-y-1"> <span className="min-w-0 flex-1 space-y-1">
<span className={cn("block truncate text-sm leading-5", isFailed ? "text-rose-200" : "text-violet-50/95")}> <span className={cn("block truncate text-sm leading-5", isFailed ? "text-rose-200" : "text-violet-50/95")}>{toolSummary}</span>
{toolSummary}
</span>
<span className="flex min-w-0 items-center gap-1.5 text-[11px] leading-4"> <span className="flex min-w-0 items-center gap-1.5 text-[11px] leading-4">
<span className={cn("min-w-0 truncate font-semibold", isFailed ? "text-rose-300/85" : isInitiated ? "text-amber-200/90" : "text-cyan-200/90")}> <span className={cn("min-w-0 truncate font-semibold", isFailed ? "text-rose-300/85" : isInitiated ? "text-amber-200/90" : "text-cyan-200/90")}>
{toolLabel} {toolLabel}
@@ -144,6 +285,172 @@ export function ChatMessagesPanel({ messages, isLoading, isSending }: Props) {
</span> </span>
</span> </span>
</div> </div>
);
}
function ToolCallStackCardSurface({
messageID,
animateEntry,
isHidden,
children,
}: {
messageID: string;
animateEntry: boolean;
isHidden: boolean;
children: ComponentChildren;
}) {
const [shouldAnimateEntry] = useState(() => animateEntry);
return (
<div
className={cn("tool-call-stack-card-surface", shouldAnimateEntry && !isHidden && "tool-call-stack-card-enter")}
data-tool-stack-card-id={messageID}
>
{children}
</div>
);
}
function ToolCallStack({
groupKey,
messages,
expanded,
entryMessageIDs,
onToggle,
}: {
groupKey: string;
messages: Message[];
expanded: boolean;
entryMessageIDs: Set<string>;
onToggle: (groupKey: string) => void;
}) {
const hiddenCount = Math.max(0, messages.length - COLLAPSED_TOOL_STACK_LIMIT);
const countLabel = `${messages.length} tool ${messages.length === 1 ? "call" : "calls"}`;
const [motionDirection, setMotionDirection] = useState<ToolStackMotionDirection>(null);
const [motionRevision, setMotionRevision] = useState(0);
const motionResetTimerRef = useRef<number | null>(null);
const handleToggle = () => {
setMotionDirection(expanded ? "collapse" : "expand");
setMotionRevision((current) => current + 1);
if (typeof window !== "undefined") {
if (motionResetTimerRef.current !== null) window.clearTimeout(motionResetTimerRef.current);
motionResetTimerRef.current = window.setTimeout(() => {
setMotionDirection(null);
motionResetTimerRef.current = null;
}, TOOL_STACK_LAYOUT_ANIMATION_MS + 60);
}
onToggle(groupKey);
};
return (
<div className="flex justify-start">
<div
className={cn(
"tool-call-stack-shell relative w-full max-w-[85%] min-w-0 pr-10",
motionDirection && (motionRevision % 2 === 0 ? "tool-call-stack-shell-layout-a" : "tool-call-stack-shell-layout-b")
)}
data-tool-stack-group={groupKey}
data-expanded={expanded ? "true" : "false"}
style={getToolStackContainerStyle(messages.length, expanded, motionDirection)}
>
{messages.map((message, index) => {
const depth = messages.length - index - 1;
const isHidden = !expanded && depth >= COLLAPSED_TOOL_STACK_LIMIT;
const shouldAnimateEntry = entryMessageIDs.has(message.id) && !isHidden;
return (
<div
key={message.id}
className={cn(
"tool-call-stack-card absolute left-0 right-10 top-0 w-auto max-w-none",
motionDirection && (motionRevision % 2 === 0 ? "tool-call-stack-card-layout-a" : "tool-call-stack-card-layout-b"),
isHidden && "pointer-events-none"
)}
style={getToolStackStyle(index, messages.length, expanded, motionDirection)}
aria-hidden={isHidden ? "true" : undefined}
>
<ToolCallStackCardSurface messageID={message.id} animateEntry={shouldAnimateEntry} isHidden={isHidden}>
<ToolCallCard message={message} className="tool-call-stack-card-glass w-full max-w-full" />
</ToolCallStackCardSurface>
</div>
);
})}
{!expanded && hiddenCount ? (
<span className="absolute bottom-1 right-10 z-20 rounded-full border border-cyan-300/30 bg-slate-950/86 px-2 py-0.5 text-[10px] font-semibold leading-none text-cyan-100 shadow-sm">
+{hiddenCount}
</span>
) : null}
<button
type="button"
className="tool-call-stack-toggle absolute right-0 top-2 z-20 flex h-8 w-8 items-center justify-center rounded-full"
aria-expanded={expanded ? "true" : "false"}
aria-label={`${expanded ? "Collapse" : "Expand"} ${countLabel}`}
title={`${expanded ? "Collapse" : "Expand"} ${countLabel}`}
onClick={handleToggle}
>
{expanded ? <ChevronUp className="h-4 w-4" /> : <ChevronDown className="h-4 w-4" />}
</button>
</div>
</div>
);
}
export function ChatMessagesPanel({ messages, isLoading, isSending }: Props) {
const hasPendingAssistant = messages.some((message) => message.id.startsWith("temp-assistant-") && message.content.trim().length === 0);
const renderItems = useMemo(() => buildMessageRenderItems(messages), [messages]);
const toolCallMessageIDs = useMemo(() => getToolCallMessageIDs(messages), [messages]);
const seenToolCallMessageIDsRef = useRef<Set<string> | null>(null);
const entryToolCallMessageIDs = useMemo(() => {
const seenIDs = seenToolCallMessageIDsRef.current;
if (!seenIDs) return new Set<string>();
const entryIDs = new Set<string>();
for (const id of toolCallMessageIDs) {
if (!seenIDs.has(id)) entryIDs.add(id);
}
return entryIDs;
}, [toolCallMessageIDs]);
const [expandedToolGroups, setExpandedToolGroups] = useState<Set<string>>(() => new Set());
useEffect(() => {
if (!toolCallMessageIDs.size) return;
const seenIDs = seenToolCallMessageIDsRef.current ?? new Set<string>();
for (const id of toolCallMessageIDs) seenIDs.add(id);
seenToolCallMessageIDsRef.current = seenIDs;
}, [toolCallMessageIDs]);
const toggleToolGroup = (groupKey: string) => {
setExpandedToolGroups((current) => {
const next = new Set(current);
if (next.has(groupKey)) next.delete(groupKey);
else next.add(groupKey);
return next;
});
};
return (
<>
{isLoading && messages.length === 0 ? <p className="text-sm text-muted-foreground">Loading messages...</p> : null}
<div className="mx-auto max-w-4xl space-y-6">
{renderItems.map((item) => {
if (item.kind === "tool_group") {
return (
<ToolCallStack
key={`tool-group-${item.key}`}
groupKey={item.key}
messages={item.messages}
expanded={expandedToolGroups.has(item.key)}
entryMessageIDs={entryToolCallMessageIDs}
onToggle={toggleToolGroup}
/>
);
}
const { message } = item;
const toolLogMetadata = asToolLogMetadata(message.metadata);
if (message.role === "tool" && toolLogMetadata) {
return (
<div key={message.id} className="flex justify-start">
<ToolCallCard message={message} className="max-w-[85%]" />
</div> </div>
); );
} }
@@ -171,6 +478,7 @@ export function ChatMessagesPanel({ messages, isLoading, isSending }: Props) {
) : message.content.trim() ? ( ) : message.content.trim() ? (
<MarkdownContent <MarkdownContent
markdown={message.content} markdown={message.content}
openLinksInNewTab
className={cn("[&_a]:text-inherit [&_a]:underline", isUser ? "leading-[1.78] text-fuchsia-50" : "leading-[1.82] text-violet-50")} className={cn("[&_a]:text-inherit [&_a]:underline", isUser ? "leading-[1.78] text-fuchsia-50" : "leading-[1.82] text-violet-50")}
/> />
) : null} ) : null}
@@ -10,6 +10,7 @@ type Props = {
className?: string; className?: string;
mode?: MarkdownMode; mode?: MarkdownMode;
resolveCitationIndex?: (href: string) => number | undefined; resolveCitationIndex?: (href: string) => number | undefined;
openLinksInNewTab?: boolean;
}; };
function replaceMarkdownLinksWithCitationTokens(markdown: string, resolveCitationIndex?: (href: string) => number | undefined) { function replaceMarkdownLinksWithCitationTokens(markdown: string, resolveCitationIndex?: (href: string) => number | undefined) {
@@ -28,17 +29,30 @@ markdownRenderer.table = (token) => {
return `<div class="md-table-scroll">${renderTable(token)}</div>`; return `<div class="md-table-scroll">${renderTable(token)}</div>`;
}; };
function renderMarkdown(markdown: string) { function setNewTabLinkAttributes(currentNode: Element) {
const rawHtml = marked.parse(markdown, { gfm: true, breaks: true, renderer: markdownRenderer }) as string; if (currentNode.tagName !== "A") return;
return DOMPurify.sanitize(rawHtml, { ADD_ATTR: ["class", "target", "rel"] }); currentNode.setAttribute("target", "_blank");
currentNode.setAttribute("rel", "noopener noreferrer");
} }
export function MarkdownContent({ markdown, className, mode = "default", resolveCitationIndex }: Props) { function renderMarkdown(markdown: string, openLinksInNewTab: boolean) {
const rawHtml = marked.parse(markdown, { gfm: true, breaks: true, renderer: markdownRenderer }) as string;
if (!openLinksInNewTab) return DOMPurify.sanitize(rawHtml, { ADD_ATTR: ["class", "target", "rel"] });
DOMPurify.addHook("afterSanitizeAttributes", setNewTabLinkAttributes);
try {
return DOMPurify.sanitize(rawHtml, { ADD_ATTR: ["class", "target", "rel"] });
} finally {
DOMPurify.removeHook("afterSanitizeAttributes", setNewTabLinkAttributes);
}
}
export function MarkdownContent({ markdown, className, mode = "default", resolveCitationIndex, openLinksInNewTab = false }: Props) {
const html = useMemo(() => { const html = useMemo(() => {
const prepared = const prepared =
mode === "citationTokens" ? replaceMarkdownLinksWithCitationTokens(markdown, resolveCitationIndex) : markdown; mode === "citationTokens" ? replaceMarkdownLinksWithCitationTokens(markdown, resolveCitationIndex) : markdown;
return renderMarkdown(prepared); return renderMarkdown(prepared, openLinksInNewTab);
}, [markdown, mode, resolveCitationIndex]); }, [markdown, mode, openLinksInNewTab, resolveCitationIndex]);
return <div className={cn("md-content", className)} dangerouslySetInnerHTML={{ __html: html }} />; return <div className={cn("md-content", className)} dangerouslySetInnerHTML={{ __html: html }} />;
} }
+172 -1
View File
@@ -140,10 +140,160 @@ textarea {
0 14px 36px hsl(240 80% 2% / 0.28); 0 14px 36px hsl(240 80% 2% / 0.28);
} }
.tool-call-stack-shell {
perspective: 900px;
transform-style: preserve-3d;
isolation: isolate;
}
.tool-call-stack-card {
transform: translate3d(var(--tool-stack-x, 0), var(--tool-stack-y, 0), var(--tool-stack-z, 0)) scale(var(--tool-stack-scale, 1));
transform-origin: top left;
opacity: var(--tool-stack-opacity, 1);
transition:
opacity 180ms ease,
transform 300ms cubic-bezier(0.2, 0.8, 0.22, 1);
will-change: transform, opacity;
}
.tool-call-stack-shell-layout-a {
animation: tool-call-stack-height-a 340ms cubic-bezier(0.22, 0.61, 0.36, 1) both;
}
.tool-call-stack-shell-layout-b {
animation: tool-call-stack-height-b 340ms cubic-bezier(0.22, 0.61, 0.36, 1) both;
}
.tool-call-stack-card-layout-a {
animation: tool-call-stack-layout-a 340ms cubic-bezier(0.22, 0.61, 0.36, 1) both;
}
.tool-call-stack-card-layout-b {
animation: tool-call-stack-layout-b 340ms cubic-bezier(0.22, 0.61, 0.36, 1) both;
}
.tool-call-stack-card-surface {
transform-origin: top left;
}
.tool-call-stack-card-glass {
backdrop-filter: none;
}
.tool-call-stack-card-enter {
animation: tool-call-stack-drop-in 320ms cubic-bezier(0.18, 0.95, 0.28, 1) backwards;
animation-delay: var(--tool-stack-delay, 0ms);
}
.tool-call-stack-toggle {
border: 1px solid hsl(188 82% 70% / 0.36);
background:
linear-gradient(180deg, hsl(230 36% 16% / 0.96), hsl(238 48% 7% / 0.96)),
hsl(236 48% 8%);
color: hsl(186 92% 86%);
box-shadow:
inset 0 1px 0 hsl(180 100% 88% / 0.08),
0 8px 22px hsl(235 72% 2% / 0.42);
transition:
border-color 160ms ease,
color 160ms ease,
transform 160ms ease,
filter 160ms ease;
}
.tool-call-stack-toggle:hover {
border-color: hsl(188 92% 74% / 0.62);
color: hsl(184 100% 92%);
filter: brightness(1.08);
}
.tool-call-stack-toggle:focus-visible {
outline: 2px solid hsl(188 92% 72% / 0.9);
outline-offset: 2px;
}
@keyframes tool-call-stack-height-a {
from {
height: var(--tool-stack-from-height);
}
to {
height: var(--tool-stack-to-height);
}
}
@keyframes tool-call-stack-height-b {
from {
height: var(--tool-stack-from-height);
}
to {
height: var(--tool-stack-to-height);
}
}
@keyframes tool-call-stack-layout-a {
from {
opacity: var(--tool-stack-from-opacity, 1);
transform: var(--tool-stack-from-transform);
}
to {
opacity: var(--tool-stack-to-opacity, 1);
transform: var(--tool-stack-to-transform);
}
}
@keyframes tool-call-stack-layout-b {
from {
opacity: var(--tool-stack-from-opacity, 1);
transform: var(--tool-stack-from-transform);
}
to {
opacity: var(--tool-stack-to-opacity, 1);
transform: var(--tool-stack-to-transform);
}
}
@keyframes tool-call-stack-drop-in {
from {
opacity: 0.72;
transform: translate3d(0, -0.65rem, 120px) scale(1.025) rotateX(3deg);
}
to {
opacity: 1;
transform: translate3d(0, 0, 0) scale(1) rotateX(0);
}
}
@media (prefers-reduced-motion: reduce) {
.tool-call-stack-card {
transition: none;
}
.tool-call-stack-shell-layout-a,
.tool-call-stack-shell-layout-b,
.tool-call-stack-card-layout-a,
.tool-call-stack-card-layout-b,
.tool-call-stack-card-enter {
animation: none;
}
}
.md-content { .md-content {
word-break: break-word; word-break: break-word;
} }
.md-content > :first-child {
margin-top: 0;
}
.md-content > :last-child {
margin-bottom: 0;
}
.md-table-scroll { .md-table-scroll {
max-width: 100%; max-width: 100%;
margin: 0.35rem 0 1rem; margin: 0.35rem 0 1rem;
@@ -242,7 +392,8 @@ textarea {
.md-content ul, .md-content ul,
.md-content ol { .md-content ol {
margin-top: 0.65rem; margin-top: 0.85rem;
margin-bottom: 0.85rem;
margin-left: 0; margin-left: 0;
padding-left: 0; padding-left: 0;
list-style: none; list-style: none;
@@ -254,6 +405,26 @@ textarea {
padding-left: 1.35rem; padding-left: 1.35rem;
} }
.md-content ul > li {
position: relative;
padding-left: 1.1rem;
}
.md-content ul > li::before {
content: "";
position: absolute;
left: 0;
top: 0.76em;
width: 0.36rem;
height: 0.36rem;
border-radius: 9999px;
background: hsl(188 86% 62%);
box-shadow:
0 0 0 2px hsl(188 86% 62% / 0.12),
0 0 10px hsl(188 86% 62% / 0.42);
transform: translateY(-50%);
}
.md-content li + li { .md-content li + li {
margin-top: 0.3rem; margin-top: 0.3rem;
} }
+14 -151
View File
@@ -149,7 +149,7 @@ export type CompletionRequestMessage = {
attachments?: ChatAttachment[]; attachments?: ChatAttachment[];
}; };
export type Provider = "openai" | "anthropic" | "xai" | "hermes-agent"; export type Provider = "openai" | "anthropic" | "xai" | "gemini" | "hermes-agent";
export type ProviderModelInfo = { export type ProviderModelInfo = {
models: string[]; models: string[];
@@ -450,6 +450,7 @@ async function readSseStream(response: Response, dispatch: (eventName: string, p
let buffer = ""; let buffer = "";
let eventName = "message"; let eventName = "message";
let dataLines: string[] = []; let dataLines: string[] = [];
let sawTerminalEvent = false;
const flushEvent = () => { const flushEvent = () => {
if (!dataLines.length) { if (!dataLines.length) {
@@ -466,6 +467,9 @@ async function readSseStream(response: Response, dispatch: (eventName: string, p
} }
dispatch(eventName, payload); dispatch(eventName, payload);
if (eventName === "done" || eventName === "error") {
sawTerminalEvent = true;
}
dataLines = []; dataLines = [];
eventName = "message"; eventName = "message";
@@ -505,6 +509,10 @@ async function readSseStream(response: Response, dispatch: (eventName: string, p
} }
} }
flushEvent(); flushEvent();
if (!sawTerminalEvent) {
throw new Error("Stream disconnected before completion");
}
} }
export async function runSearchStream( export async function runSearchStream(
@@ -528,87 +536,14 @@ export async function runSearchStream(
signal: options?.signal, signal: options?.signal,
}); });
if (!response.ok) { await readSseStream(response, (eventName, payload) => {
const fallback = `${response.status} ${response.statusText}`;
let message = fallback;
try {
const body = (await response.json()) as { message?: string };
if (body.message) message = body.message;
} catch {
// keep fallback message
}
throw new Error(message);
}
if (!response.body) {
throw new Error("No response stream");
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
let eventName = "message";
let dataLines: string[] = [];
const flushEvent = () => {
if (!dataLines.length) {
eventName = "message";
return;
}
const dataText = dataLines.join("\n");
let payload: any = null;
try {
payload = JSON.parse(dataText);
} catch {
payload = { message: dataText };
}
if (eventName === "search_results") handlers.onSearchResults?.(payload); if (eventName === "search_results") handlers.onSearchResults?.(payload);
else if (eventName === "search_error") handlers.onSearchError?.(payload); else if (eventName === "search_error") handlers.onSearchError?.(payload);
else if (eventName === "answer") handlers.onAnswer?.(payload); else if (eventName === "answer") handlers.onAnswer?.(payload);
else if (eventName === "answer_error") handlers.onAnswerError?.(payload); else if (eventName === "answer_error") handlers.onAnswerError?.(payload);
else if (eventName === "done") handlers.onDone?.(payload); else if (eventName === "done") handlers.onDone?.(payload);
else if (eventName === "error") handlers.onError?.(payload); else if (eventName === "error") handlers.onError?.(payload);
});
dataLines = [];
eventName = "message";
};
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
let newlineIndex = buffer.indexOf("\n");
while (newlineIndex >= 0) {
const rawLine = buffer.slice(0, newlineIndex);
buffer = buffer.slice(newlineIndex + 1);
const line = rawLine.endsWith("\r") ? rawLine.slice(0, -1) : rawLine;
if (!line) {
flushEvent();
} else if (line.startsWith("event:")) {
eventName = line.slice("event:".length).trim();
} else if (line.startsWith("data:")) {
dataLines.push(line.slice("data:".length).trimStart());
}
newlineIndex = buffer.indexOf("\n");
}
}
buffer += decoder.decode();
if (buffer.length) {
const line = buffer.endsWith("\r") ? buffer.slice(0, -1) : buffer;
if (line.startsWith("event:")) {
eventName = line.slice("event:".length).trim();
} else if (line.startsWith("data:")) {
dataLines.push(line.slice("data:".length).trimStart());
}
}
flushEvent();
} }
export async function attachSearchStream(searchId: string, handlers: RunSearchStreamHandlers, options?: { signal?: AbortSignal }) { export async function attachSearchStream(searchId: string, handlers: RunSearchStreamHandlers, options?: { signal?: AbortSignal }) {
@@ -654,6 +589,7 @@ export async function runCompletionStream(
body: { body: {
chatId?: string | null; chatId?: string | null;
persist?: boolean; persist?: boolean;
clientRequestId?: string;
provider: Provider; provider: Provider;
model: string; model: string;
messages: CompletionRequestMessage[]; messages: CompletionRequestMessage[];
@@ -679,86 +615,13 @@ export async function runCompletionStream(
signal: options?.signal, signal: options?.signal,
}); });
if (!response.ok) { await readSseStream(response, (eventName, payload) => {
const fallback = `${response.status} ${response.statusText}`;
let message = fallback;
try {
const body = (await response.json()) as { message?: string };
if (body.message) message = body.message;
} catch {
// keep fallback message
}
throw new Error(message);
}
if (!response.body) {
throw new Error("No response stream");
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
let eventName = "message";
let dataLines: string[] = [];
const flushEvent = () => {
if (!dataLines.length) {
eventName = "message";
return;
}
const dataText = dataLines.join("\n");
let payload: any = null;
try {
payload = JSON.parse(dataText);
} catch {
payload = { message: dataText };
}
if (eventName === "meta") handlers.onMeta?.(payload); if (eventName === "meta") handlers.onMeta?.(payload);
else if (eventName === "tool_call") handlers.onToolCall?.(payload); else if (eventName === "tool_call") handlers.onToolCall?.(payload);
else if (eventName === "delta") handlers.onDelta?.(payload); else if (eventName === "delta") handlers.onDelta?.(payload);
else if (eventName === "done") handlers.onDone?.(payload); else if (eventName === "done") handlers.onDone?.(payload);
else if (eventName === "error") handlers.onError?.(payload); else if (eventName === "error") handlers.onError?.(payload);
});
dataLines = [];
eventName = "message";
};
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
let newlineIndex = buffer.indexOf("\n");
while (newlineIndex >= 0) {
const rawLine = buffer.slice(0, newlineIndex);
buffer = buffer.slice(newlineIndex + 1);
const line = rawLine.endsWith("\r") ? rawLine.slice(0, -1) : rawLine;
if (!line) {
flushEvent();
} else if (line.startsWith("event:")) {
eventName = line.slice("event:".length).trim();
} else if (line.startsWith("data:")) {
dataLines.push(line.slice("data:".length).trimStart());
}
newlineIndex = buffer.indexOf("\n");
}
}
buffer += decoder.decode();
if (buffer.length) {
const line = buffer.endsWith("\r") ? buffer.slice(0, -1) : buffer;
if (line.startsWith("event:")) {
eventName = line.slice("event:".length).trim();
} else if (line.startsWith("data:")) {
dataLines.push(line.slice("data:".length).trimStart());
}
}
flushEvent();
} }
export async function attachCompletionStream(chatId: string, handlers: CompletionStreamHandlers, options?: { signal?: AbortSignal }) { export async function attachCompletionStream(chatId: string, handlers: CompletionStreamHandlers, options?: { signal?: AbortSignal }) {
+24
View File
@@ -0,0 +1,24 @@
import type { Provider } from "./api";
type PersistedChatModel = {
lastUsedProvider: Provider | null;
lastUsedModel: string | null;
};
export type ChatModelSelection = {
provider: Provider;
model: string;
};
export function getChatModelSelection(chat: PersistedChatModel | null): ChatModelSelection | null {
if (!chat?.lastUsedProvider || !chat.lastUsedModel?.trim()) return null;
return {
provider: chat.lastUsedProvider,
model: chat.lastUsedModel.trim(),
};
}
export function getChatModelSelectionSyncKey(chatId: string | null, selection: ChatModelSelection | null) {
if (!chatId || !selection) return null;
return JSON.stringify([chatId, selection.provider, selection.model]);
}
+31
View File
@@ -0,0 +1,31 @@
export type SidebarSelection = { kind: "chat" | "search"; id: string };
type WorkspaceSelectionItem = { type: SidebarSelection["kind"]; id: string };
type ResolveSidebarSelectionOptions = {
initialSelection?: SidebarSelection;
selectFallback?: boolean;
};
export function resolveSidebarSelectionAfterRefresh(
current: SidebarSelection | null,
workspaceItems: WorkspaceSelectionItem[],
{ initialSelection, selectFallback = false }: ResolveSidebarSelectionOptions = {}
): SidebarSelection | null {
const hasItem = (candidate: SidebarSelection | null | undefined) => {
if (!candidate) return false;
return workspaceItems.some((item) => item.type === candidate.kind && item.id === candidate.id);
};
if (hasItem(current)) {
return current;
}
if (hasItem(initialSelection)) {
return initialSelection ?? null;
}
if (!selectFallback) {
return null;
}
const first = workspaceItems[0];
return first ? { kind: first.type, id: first.id } : null;
}
+4 -1
View File
@@ -2,7 +2,10 @@ export function registerServiceWorker() {
if (!import.meta.env.PROD || !("serviceWorker" in navigator)) return; if (!import.meta.env.PROD || !("serviceWorker" in navigator)) return;
window.addEventListener("load", () => { window.addEventListener("load", () => {
void navigator.serviceWorker.register("/sw.js").catch((error: unknown) => { void navigator.serviceWorker
.register("/sw.js", { updateViaCache: "none" })
.then((registration) => registration.update())
.catch((error: unknown) => {
console.warn("Sybil service worker registration failed", error); console.warn("Sybil service worker registration failed", error);
}); });
}); });
+49
View File
@@ -0,0 +1,49 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
getChatModelSelection,
getChatModelSelectionSyncKey,
} from "../src/lib/chat-model-selection.ts";
test("chat model selections are normalized from persisted metadata", () => {
assert.deepEqual(
getChatModelSelection({
lastUsedProvider: "anthropic",
lastUsedModel: " claude-sonnet-4-5 ",
}),
{
provider: "anthropic",
model: "claude-sonnet-4-5",
}
);
});
test("unrelated chat updates do not change the model synchronization key", () => {
const beforeSettingsSave = getChatModelSelection({
lastUsedProvider: "openai",
lastUsedModel: "gpt-4.1-mini",
});
const afterSettingsSave = getChatModelSelection({
lastUsedProvider: "openai",
lastUsedModel: "gpt-4.1-mini",
});
assert.equal(
getChatModelSelectionSyncKey("chat-1", beforeSettingsSave),
getChatModelSelectionSyncKey("chat-1", afterSettingsSave)
);
});
test("switching chats or persisted models changes the synchronization key", () => {
const original = { provider: "openai", model: "gpt-4.1-mini" };
const updated = { provider: "gemini", model: "gemini-3.5-flash" };
assert.notEqual(
getChatModelSelectionSyncKey("chat-1", original),
getChatModelSelectionSyncKey("chat-2", original)
);
assert.notEqual(
getChatModelSelectionSyncKey("chat-1", original),
getChatModelSelectionSyncKey("chat-1", updated)
);
});
+45
View File
@@ -0,0 +1,45 @@
import assert from "node:assert/strict";
import test from "node:test";
import { resolveSidebarSelectionAfterRefresh } from "../src/lib/sidebar-selection.ts";
const workspaceItems = [
{ type: "chat", id: "completed-chat" },
{ type: "chat", id: "selected-chat" },
{ type: "search", id: "selected-search" },
];
test("a collection refresh preserves the current thread selection", () => {
assert.deepEqual(
resolveSidebarSelectionAfterRefresh({ kind: "chat", id: "selected-chat" }, workspaceItems),
{ kind: "chat", id: "selected-chat" }
);
});
test("an initial route selection cannot override a current thread selection", () => {
assert.deepEqual(
resolveSidebarSelectionAfterRefresh(
{ kind: "search", id: "selected-search" },
workspaceItems,
{ initialSelection: { kind: "chat", id: "completed-chat" }, selectFallback: true }
),
{ kind: "search", id: "selected-search" }
);
});
test("a collection refresh preserves an intentionally empty selection", () => {
assert.equal(resolveSidebarSelectionAfterRefresh(null, workspaceItems), null);
});
test("initial load can select the URL thread or fall back to the first item", () => {
assert.deepEqual(
resolveSidebarSelectionAfterRefresh(null, workspaceItems, {
initialSelection: { kind: "search", id: "selected-search" },
selectFallback: true,
}),
{ kind: "search", id: "selected-search" }
);
assert.deepEqual(resolveSidebarSelectionAfterRefresh(null, workspaceItems, { selectFallback: true }), {
kind: "chat",
id: "completed-chat",
});
});
+1 -1
View File
@@ -1 +1 @@
{"root":["./src/App.tsx","./src/main.tsx","./src/pwa.ts","./src/root-router.tsx","./src/vite-env.d.ts","./src/components/sybil-character.tsx","./src/components/auth/auth-screen.tsx","./src/components/chat/chat-attachment-list.tsx","./src/components/chat/chat-messages-panel.tsx","./src/components/markdown/markdown-content.tsx","./src/components/search/search-results-panel.tsx","./src/components/ui/button.tsx","./src/components/ui/input.tsx","./src/components/ui/scroll-area.tsx","./src/components/ui/separator.tsx","./src/components/ui/textarea.tsx","./src/hooks/use-session-auth.ts","./src/lib/api.ts","./src/lib/utils.ts","./src/pages/search-route-page.tsx"],"version":"5.9.3"} {"root":["./src/App.tsx","./src/main.tsx","./src/pwa.ts","./src/root-router.tsx","./src/vite-env.d.ts","./src/components/sybil-character.tsx","./src/components/auth/auth-screen.tsx","./src/components/chat/chat-attachment-list.tsx","./src/components/chat/chat-messages-panel.tsx","./src/components/markdown/markdown-content.tsx","./src/components/search/search-results-panel.tsx","./src/components/ui/button.tsx","./src/components/ui/input.tsx","./src/components/ui/scroll-area.tsx","./src/components/ui/separator.tsx","./src/components/ui/textarea.tsx","./src/hooks/use-session-auth.ts","./src/lib/api.ts","./src/lib/chat-model-selection.ts","./src/lib/sidebar-selection.ts","./src/lib/utils.ts","./src/pages/search-route-page.tsx"],"version":"5.9.3"}