adds attachment support
This commit is contained in:
@@ -10,6 +10,12 @@ Content type:
|
||||
- Requests with bodies use `application/json`.
|
||||
- Responses are JSON unless noted otherwise.
|
||||
|
||||
Chat upload limits:
|
||||
- Chat completion and direct message payloads support inline attachments up to a 32 MB request body.
|
||||
- Up to 8 attachments per message.
|
||||
- Image attachments: PNG or JPEG only, max 6 MB each.
|
||||
- Text attachments: up to 8 MB source size each; server accepts at most 200,000 characters of inlined text content per attachment.
|
||||
|
||||
## Health + Auth
|
||||
|
||||
### `GET /health`
|
||||
@@ -74,11 +80,34 @@ Behavior notes:
|
||||
"role": "system|user|assistant|tool",
|
||||
"content": "string",
|
||||
"name": "optional",
|
||||
"metadata": {}
|
||||
"metadata": {},
|
||||
"attachments": [
|
||||
{
|
||||
"kind": "image",
|
||||
"id": "attachment-id",
|
||||
"filename": "photo.jpg",
|
||||
"mimeType": "image/jpeg",
|
||||
"sizeBytes": 12345,
|
||||
"dataUrl": "data:image/jpeg;base64,..."
|
||||
},
|
||||
{
|
||||
"kind": "text",
|
||||
"id": "attachment-id",
|
||||
"filename": "notes.md",
|
||||
"mimeType": "text/markdown",
|
||||
"sizeBytes": 4567,
|
||||
"text": "# Notes\\n...",
|
||||
"truncated": false
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
- Response: `{ "message": Message }`
|
||||
|
||||
Notes:
|
||||
- `attachments` is optional and is merged into stored `message.metadata.attachments`.
|
||||
- Tool messages should not include attachments.
|
||||
|
||||
## Chat Completions (non-streaming)
|
||||
|
||||
### `POST /v1/chat-completions`
|
||||
@@ -89,7 +118,30 @@ Behavior notes:
|
||||
"provider": "openai|anthropic|xai",
|
||||
"model": "string",
|
||||
"messages": [
|
||||
{ "role": "system|user|assistant|tool", "content": "string", "name": "optional" }
|
||||
{
|
||||
"role": "system|user|assistant|tool",
|
||||
"content": "string",
|
||||
"name": "optional",
|
||||
"attachments": [
|
||||
{
|
||||
"kind": "image",
|
||||
"id": "attachment-id",
|
||||
"filename": "photo.jpg",
|
||||
"mimeType": "image/jpeg",
|
||||
"sizeBytes": 12345,
|
||||
"dataUrl": "data:image/jpeg;base64,..."
|
||||
},
|
||||
{
|
||||
"kind": "text",
|
||||
"id": "attachment-id",
|
||||
"filename": "notes.md",
|
||||
"mimeType": "text/markdown",
|
||||
"sizeBytes": 4567,
|
||||
"text": "# Notes\\n...",
|
||||
"truncated": false
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"temperature": 0.2,
|
||||
"maxTokens": 256
|
||||
@@ -112,7 +164,12 @@ Behavior notes:
|
||||
- For `chatId` calls, server stores only *new* non-assistant messages from provided history to avoid duplicates.
|
||||
- Server persists final assistant output and call metadata (`LlmCall`) in DB.
|
||||
- Server updates chat-level model metadata on each call: `lastUsedProvider`/`lastUsedModel`; first successful/failed call also initializes `initiatedProvider`/`initiatedModel` if unset.
|
||||
- Attachments are optional and currently apply to `user` messages. Persisted chat history stores them under `message.metadata.attachments`.
|
||||
- Images are forwarded inline to providers as multimodal image parts. Use PNG or JPEG for cross-provider compatibility.
|
||||
- Text files are forwarded as explicit text blocks rather than provider-managed file references. Large text attachments should already be truncated client-side before submission.
|
||||
- For `openai` and `xai`, backend enables tool use during chat completion with an internal system instruction.
|
||||
- For `openai` and `xai`, image attachments are sent as chat-completions content parts alongside text.
|
||||
- For `anthropic`, image attachments are sent as Messages API `image` blocks using base64 source data; text attachments are added as `text` blocks.
|
||||
- Available tool calls for chat: `web_search` and `fetch_url`.
|
||||
- `web_search` returns ranked results with per-result summaries/snippets. Its backend engine is selected by `CHAT_WEB_SEARCH_ENGINE` (`exa` default, or `searxng` with `SEARXNG_BASE_URL` set). SearXNG mode requires the instance to allow `format=json`.
|
||||
- `fetch_url` fetches a URL and returns plaintext page content (HTML converted to text server-side).
|
||||
@@ -189,10 +246,32 @@ Search run notes:
|
||||
"role": "system|user|assistant|tool",
|
||||
"content": "...",
|
||||
"name": null,
|
||||
"metadata": null
|
||||
"metadata": {
|
||||
"attachments": [
|
||||
{
|
||||
"kind": "image",
|
||||
"id": "attachment-id",
|
||||
"filename": "photo.jpg",
|
||||
"mimeType": "image/jpeg",
|
||||
"sizeBytes": 12345,
|
||||
"dataUrl": "data:image/jpeg;base64,..."
|
||||
},
|
||||
{
|
||||
"kind": "text",
|
||||
"id": "attachment-id",
|
||||
"filename": "notes.md",
|
||||
"mimeType": "text/markdown",
|
||||
"sizeBytes": 4567,
|
||||
"text": "# Notes\\n...",
|
||||
"truncated": false
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`metadata` remains nullable. Tool-call log messages still use `metadata.kind = "tool_call"`; regular user messages with attachments use `metadata.attachments`.
|
||||
|
||||
`ChatDetail`
|
||||
```json
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user