API · System & Operations¶
Settings, health, metrics, the model registry, LM Studio, the inference arbiter, the tool-schema and tool-security surfaces, webhooks, the OpenClaw bridge, and project file access. All endpoints are unauthenticated (see API conventions).
SettingsController¶
api/settings — read and upsert SystemSettings rows. POST is an upsert (there is no separate update verb). See Configuration · SystemSettings.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/settings |
— | List<SystemSetting> |
yes |
| GET | /api/settings/{key} |
route key:string |
SystemSetting / 404 |
yes |
| POST | /api/settings |
body SystemSetting (upsert) |
200 |
yes |
| GET | /api/settings/group/{group} |
route group:string |
Dictionary<string,string> |
yes |
HealthController¶
api/health — liveness and a detailed dependency report. The maintenance-mode middleware always lets these through.
The rag block reports whether the knowledge index is searchable in the space the Engine embeds queries in: status (ok / degraded / unavailable), activeGeneration, the Engine's stamp (<model>|<prefix scheme>) and the active global collection's indexStamp with stampMatch, fingerprintCosine (a fresh probe embedding against the fingerprint the global collection was built with; under 0.999 is degraded), collections and mismatched (active-generation collections a query cannot search: another dimension or stamp), legacy (collections of other generations), chromaImage (the pinned image) and build (the index job: idle, or the running or last build's generation, phase, progress, error and whether it went live). It is cached for a minute and, like embedding, never changes the overall status.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/health |
— | 200 { status, timestamp, version, database } / 503 |
yes |
| GET | /api/health/detailed |
— | DetailedHealthDto (200/503) — DB, Docker, LM Studio + instances, scheduler, builds, disk, embedding, rag |
yes |
MetricsController¶
api/metrics — analytics over a trailing window (hours query, default 24).
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/metrics |
— | SystemMetrics |
yes |
| GET | /api/metrics/dashboard |
query hours:int=24 |
DashboardMetrics |
yes |
| GET | /api/metrics/activity |
query hours:int=24 |
List<ActivityPoint> |
yes |
| GET | /api/metrics/tokens |
query hours:int=24 |
List<TokenDistribution> |
yes |
| GET | /api/metrics/agents |
query hours:int=24 |
List<AgentPerformance> |
yes |
| GET | /api/metrics/models |
query hours:int=24 |
List<ModelComparison> |
yes |
| GET | /api/metrics/errors |
query limit:int=10 |
List<RecentError> |
yes |
| GET | /api/metrics/interactions |
query projectId:int?, limit:int=20 |
List<InteractionLogDto> |
yes |
ModelRegistryController¶
api/model-registry — the benchmark-derived model registry and per-task-type routing.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/model-registry |
— | registry | no |
| GET | /api/model-registry/status |
— | { state, path, configured, modelCount, detail } |
no |
| POST | /api/model-registry/reload |
— | { message, models } |
no |
| GET | /api/model-registry/routing/{taskType} |
route taskType:string |
{ taskType, modelId, entry } / 404 |
no |
status answers why the registry looks the way it does, which the registry payload itself cannot: state is one of Loaded, NotConfigured (no path set — the Engine is simply not routing on benchmarks), Missing (a path was configured and the file is not there) or Invalid (the file is there and could not be parsed), and configured distinguishes a path that was asked for from the built-in default. reload reports what actually happened rather than always claiming success, so a reload that found nothing says so.
LMStudioController¶
api/lmstudio — LM Studio connection, model list, direct chat, and the multi-instance configuration.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/lmstudio/connection |
— | bool |
yes |
| GET | /api/lmstudio/models |
— | List<string> (normalized model ids) |
yes |
| POST | /api/lmstudio/chat |
body ChatRequest (ModelId, Messages, Temperature, MaxTokens, UseJsonFormat, JsonSchemaJson?, JsonSchemaStrict, Seed?) |
ChatDetailedResponse / 409 { error } when the model cannot be made resident on Primary or Primary's card is held / 500 |
yes |
| GET | /api/lmstudio/config |
— | LMStudioConfig |
yes |
| POST | /api/lmstudio/config |
body LMStudioConfig |
200 |
yes |
ChatRequest is declared inline. CallLLMAsync and the detailed-response contract are described in §06 — Key Services.
InferenceController¶
api/inference — the box-level inference arbiter: leases on each slot's VRAM. A lease names a consumer, a priority (run, interactive, batch), a slot and the models it needs; grant order is run > interactive > batch, a batch lease that cannot be granted queues, as does one that would start a GPU service or load a model while a run is live on the same card (the planner budgets memory, not compute), and the sweep restores the slot's standard set once nothing is live or queued on it. Request bodies bind PascalCase.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| POST | /api/inference/leases |
body LeaseRequestDto (Consumer, Priority, Slot, Models[{ModelId, Context?}], TtlSeconds?, ExpectedSeconds?, WaitSeconds? (capped at 60), ContextRequired, Services[]? (registered GPU services the lease keeps running; a lease may name services and no model), Card? (a card hold — see below)) |
200 LeaseDto held / 202 LeaseDto queued / 409 { error, lease } refused / 400 |
yes |
| GET | /api/inference/leases/{id} |
route id:guid |
LeaseDto / 404 (terminal leases are kept ten minutes) |
no |
| POST | /api/inference/leases/{id}/touch |
route id:guid |
LeaseDto / 409 when the lease is no longer live / 404 |
no |
| DELETE | /api/inference/leases/{id} |
route id:guid |
204 / 404 |
yes |
| GET | /api/inference/queue |
— | InferenceQueueDto — per slot, in SlotOrder: the machine its server runs on, its card, description and whether it is enabled, its planner budget and what its resident models and running GPU services are costed at (BudgetBytes, CostedBytes), backend, probe, resident models, standard set (and whether it is protected), active requests, live / queued / recent leases, registered GPU services with their unit state, holders and, for one kept warm with no lease naming it, WarmUntil (when the arbiter will stop it), foreign occupants held by a co-resident LM Studio, registered schedules (a card hold is listed on every slot of its card); plus LocalMachine (this host's name), Cards and HostMemory as GET /api/inference/cards reports them. Slots are probed in parallel, each bounded by Aetherium:LMStudio:ProbeTimeoutSeconds; a slot that is known down (below) is not probed at all — its probe reads failed with an offline: … note giving the time of its last health probe, its GPU services count as stopped and it lists no foreign occupants — so the view answers at once while that slot's machine is off |
yes |
| GET | /api/inference/cards |
— | GpuCardsDto — each physical GPU as sysfs reports it: every card in Aetherium:Gpus:Cards (Present: false when its UUID is not found; a card with a Host is read on that machine over SSH through the wrapper's cards verb, with the same fields except processes, refreshed in the background once its last answer is 5 s old and shown until it is 30 s old) and any unconfigured card with a ROCm UUID, named by its PCI address; VRAM total and used, the planner budget for the slot on it (MaxVramGB less the card's headroom), busy %, temperatures, power and cap, fan, performance level, the endpoint link and the real slot link, connected displays, the slots placed on it, and every process holding VRAM on it grouped by systemd unit with that unit's host memory; plus host RAM and swap. A value sysfs does not provide is null, never zero. No probes; cached for two seconds |
no |
| GET | /api/inference/schedules |
— | List<InferenceScheduleDto> — what timers and campaigns say they will need and when |
yes |
| PUT | /api/inference/schedules/{consumer} |
route consumer; body InferenceScheduleDto (Slot, Priority, Models[], Cadence (systemd OnCalendar text), NextRunAt?, ExpectedSeconds?, Note?) |
InferenceScheduleDto / 400 — one row per consumer, upserted; registration reserves nothing |
yes |
| DELETE | /api/inference/schedules/{consumer} |
route consumer |
204 / 404 |
yes |
A LeaseDto carries State (Queued, Acquiring, Held, Preempted, Released, Expired, Failed), ExpiresAt, LastTouched, a FailureReason or WaitingReason, and for a queued lease its Position. A holder that sees Preempted should finish or abandon its request and release; after PreemptGraceSeconds the model is unloaded regardless. Only a run preempts; an interactive lease is granted only when its models are already resident and nothing else holds the slot's server — while a run or batch lease that names a model is live there, even the same model, the interactive request fails with a "busy" reason — and it changes nothing on the card. A lease that names only GPU services (the speech relay, an image job) runs its own process beside the server, so it neither refuses nor evicts an interactive lease. A lease on a slot that is known down — its last two health probes failed and the latest is under two minutes old, as when its machine is off — fails at once (409, <slot> is offline: its last health probe failed …) rather than waiting out a residency probe; the health check probes every 30 s, and the slot takes leases again once it answers.
A request that names a Card (an Aetherium:Gpus:Cards key) is a card hold: it reserves the whole card for something outside the Engine. Slot, Models and Services are ignored and the priority is always run; LeaseDto.Card names the card. A hold never ends a run: while a run lease is live on any slot of the card it queues (202), then it ends interactive leases, preempts batch ones, unloads every model and stops every GPU service on the card's slots. A slot that is known down (two failed health probes in a row: its machine is off or booted into its other OS) has nothing on the card and is skipped; a slot that has failed only one probe fails the hold, because it may still be holding models. While it is live no lease is granted there (batch queues, run and interactive are refused), pre-warm, re-warm and restore skip those slots, and the Engine's own loads refuse them with the holder's name. Releasing it, or its TTL passing (default RunLeaseTtlHours), restores the slots' standard sets. A card no slot is placed on is a 400; a second hold on a held card is a 409.
/v1/chat/completions — the relay is deliberately outside the contract
The same host also answers POST /v1/chat/completions, POST /v1/responses, GET /v1/models and GET /models for an external OpenAI-compatible chat client: the request is forwarded byte-for-byte (streaming included) to the first slot, in slot order, where the model is resident, else the first whose catalog names it — a llama-server preset list or an LM Studio slot's downloaded models — under a resident-only interactive lease per client (X-Consumer header, else openclaw from the local host or openclaw@<ip> from another). GET /v1/models lists every slot's catalog with a load state and the slot on each entry; it asks every slot at once and leaves out a slot that has not answered within 1.5 s. Chat requests resolve their slot the same way, so a slot that has gone dark costs a turn at most 1.5 s rather than its whole probe timeout. When the model is resident on more than one slot and the first refuses because a run or a batch job holds it, the next slot with a resident copy takes the request; a request the first slot accepts never waits on a later slot's probe. The relay never loads a model: when the model is not resident, or a run or a batch job holds every slot where it is (whatever model each holds), it answers 503 {"error":{"code":503,"message":"model is not resident on any slot right now; retry after 300 seconds","type":"server_error"}} so a client with a failover chain advances to its next hop; a slot that holds the model but does not answer gets the same 503 with its own message. Both carry Retry-After: 300 and x-should-retry: false, so a client that retries a 5xx on the same model gives up at once instead of looping (the relay answered 400 until 2026-10-12). For each chat call the inference server timed, the relay writes one [RELAY-CACHE] line to the Engine's log — the client, slot and model, the prompt tokens reused from the prefix cache and processed, the processing time, and a short hash of each message and of the tool list — the same shape the aetherium.prompt.* events below give the Engine's own calls, for clients that never pass its scheduler. POST /v1/audio/speech speaks for the same clients in OpenAI's text-to-speech shape ({model, input, voice, response_format, speed?}): voice must name a reference in Aetherium:Tts:RelayVoices (otherwise 400), and the whole input, up to Aetherium:Tts:RelayMaxWords words, is rendered in one request by the Base clone service and returned as audio. The relay holds one batch lease on that service for every caller, starting it on the first request (about a minute) and releasing it after Aetherium:Tts:RelayIdleMinutes without a request; while a run holds the card a request waits up to Aetherium:Tts:RelayQueueWaitSeconds for the lease, then gets a 503 with the reason. A render that returns no audio within its budget (Aetherium:Tts:RelayRenderBaseSeconds plus Aetherium:Tts:RelayRenderSecondsPerWord per input word, never more than Aetherium:Tts:LineTimeoutSeconds) is cut off with a 504 that names the word count, the budget and how busy the card was; a caller that hangs up cancels its render. model and instructions are accepted and ignored. These routes are excluded from the OpenAPI contract on purpose.
ToolsController¶
api/tools — the MCP tool-schema discovery surface. Returns the same filtered catalog the router assembles, in OpenAI function-calling format.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/tools/schemas |
query servers:string? (CSV) |
{ tools, metadata } — alias of mcp-schemas |
no |
| GET | /api/tools/mcp-schemas |
query servers:string? (CSV) |
{ tools[], metadata: { total, filtered_by, generated_at, source } } |
no |
When ?servers= is omitted, the default server list is workspace, shell, research, knowledge, quality, planning, comms, orchestration, comfyui — note team (the send_message_to tool) is not in the default and must be requested explicitly.
See Tools for the full catalog and §07 — MCP Architecture for how the filter is applied.
ToolSecurityController¶
api/tools (shared namespace) — the Restricted-tool approval callback and the security audit log.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| POST | /api/tools/approval/{requestId} |
route requestId:string; body ApprovalRequest (Approved, ApprovedBy?) |
200 / 404 / 409 |
no |
| GET | /api/tools/security-log |
query projectId:int?, toolName:string?, level:string?, from:DateTime?, to:DateTime?, limit:int=50 |
List<SecurityLogEntryDto> |
yes |
| GET | /api/tools/security-log/{id:int} |
route id:int |
SecurityLogDetailDto / 404 |
yes |
| GET | /api/tools/approval/pending |
query projectId?:int |
{ pendingCount, pending[] } |
no |
| GET | /api/tools/approval/pending/{requestId} |
route requestId:string |
PendingApprovalDto / 404 |
no |
ApprovalRequest is declared inline. The approval callback is unauthenticated — there is no token check on it. The audit log it reads has no retention policy (Configuration).
Each entry in pending[] carries what a decision actually needs: requestId, toolName, the
args verbatim, agentId and agentName, projectId, teamName, securityLevel,
requestedAt, expiresAt, and secondsRemaining computed on read so a polling client shows a
live countdown rather than a stale figure. Reaching zero is an auto-deny and the tool call fails —
it is not "expired and harmless".
POST /api/tools/approval/{requestId} distinguishes three outcomes by status code, because they
are different facts: 200 decided, 409 already resolved by someone else or by the timeout, and
404 no such gate.
CapabilitiesController¶
api/capabilities — what this build can actually do, published so a client can discover it rather than assume it.
| Method | Path | Request | Response | Auth |
|---|---|---|---|---|
| GET | /api/capabilities |
— | registered tools with their JSON Schemas, topologies, card kinds, event types | no |
| GET | /api/capabilities/cards |
— | one real example per tool-result card kind, from the server's own builder | no |
| GET | /api/capabilities/live |
— | every SignalR hub, its join method, and each event it broadcasts with its payload type | no |
The point of publishing the vocabulary is that a gap becomes visible. Eleven of the twenty-six SignalR event types had no consumer at all, in either direction, until the catalog named them. Startup guards keep the document honest: the server source is scanned for broadcasts, and a mismatch in either direction — an event broadcast but not catalogued, or catalogued but never broadcast — fails the test suite rather than drifting quietly.
/cards is built by the same code that produces cards at runtime, so a kind the server starts or
stops producing appears or disappears here without anyone maintaining a list.
AgUiController¶
api/agui — the run event stream the web client is built against, additive to the SignalR hubs rather than a replacement for them.
| Method | Path | Request | Response | Auth |
|---|---|---|---|---|
| GET | /api/agui/{projectId}/stream |
query afterSeq?:long, or the Last-Event-ID header |
text/event-stream |
no |
| GET | /api/agui/{projectId}/events |
query afterSeq?:long |
events since a sequence number, from the buffer and the durable log (fromBuffer, fromStore say which) |
no |
| GET | /api/agui/{projectId}/head |
— | the latest sequence number, buffer occupancy, and persistence state (durable, persisted, persistFailures) |
no |
| GET | /api/agui/{projectId}/fork-points |
— | the run's aetherium.prompt.assembled events: every position a fork can start from, with its sequence number, agent, model and cassette key |
no |
| GET | /api/agui/tool-requests |
query projectId?:int, days?:int (1–365, default 30) |
ToolRequestReport: per agent id and tool, how often a tool outside the node's list was asked for and what the gate did (requested, granted, denied, reasons, projects, lastAt) |
no |
| POST | /api/agui/{projectId}/fork |
body ForkRequest — Seq (a fork point) or CassetteKey; optional CassettePath, ModelId, Instance, Temperature, MaxTokens, Seed, AppendToSystemPrompt, ReplaceLastMessage, Samples (1–20) |
the parent exchange and each re-issued answer (shape only); 400 unusable request, 409 the position was never recorded, 502 the model call failed |
no |
Server-sent events rather than WebSockets, deliberately: the stream is one-directional — actions go
over the normal REST surface — SSE resumes natively through Last-Event-ID, and it survives proxies
that mishandle upgrades.
Events are numbered, which is what the hubs cannot offer. A consumer that connects late or
reconnects replays from its last sequence number and can prove it has the whole trajectory. Every
event is also written, with the same sequence number, to the session_events table behind the
in-memory buffer, so a replay reaches past what the buffer has evicted and survives a restart:
numbering continues from the table, and a stream or events request is served from the table
for the part the buffer no longer holds. Only when neither holds the requested position is the
client told so up front with an aetherium.gap event, instead of being handed a stream with a
silent hole in it.
Six of the stream's events describe the Engine's own context decisions rather than agent
activity: aetherium.prompt.assembled (the shape of each prompt sent to a model — counts, token
estimate, window, and the same content-addressed key the model-exchange cassette uses; a short hash
and token estimate per message of the prompt as sent, and a hash of the tool list, so a prefix-cache
miss can be traced to the first message that changed; never the text), aetherium.llm.usage (what each call actually cost, as the inference server reported it —
prompt and completion tokens beside the Engine's own estimate, the requested context, max_tokens,
finish reason and latency), aetherium.context.fitted (the emergency trim or max_tokens clamp that fits a request to
the window), aetherium.context.compacted (mid-run history summarized or truncated),
aetherium.memory.injected (agent memory placed at the head of the agent's first user message), and
aetherium.prompt.cache (how many prompt tokens the inference server reused from its prefix cache
versus processed, and how long processing took — emitted only when the backend reports it).
A fork re-issues one recorded model exchange from a position in the log — the request the
cassette recorded at that aetherium.prompt.assembled event — through the normal scheduler with
overrides (another model or instance, a different temperature, max_tokens or seed, a line appended
to the system prompt, the last message replaced), up to twenty samples at once. It is one exchange,
not a resumed run, and it needs a run that was recorded: an unrecorded position answers 409. The
result is logged as aetherium.session.forked with the parent's and each child's shape.
Two more custom events describe agent work at a finer grain than the tool calls: the protocol's
REASONING_START … REASONING_END bracket precedes every agent reply and verifier verdict that
carried model reasoning, under the same message id, with reasoningTruncated on the start event;
and aetherium.step.round records each attempt at a plan step (round, cap, the Engine's verdict —
continue, complete or blocked — and what the round left behind) when AI_StepRounds allows
more than one. A third, aetherium.tool.requested, records every tool call the Engine refused
before dispatch, with the agent, the tool and a reason from a closed set — not-in-node-tools,
not-in-allowlist or catalog-miss — so the demand for a tool a role lacks, and for a tool the
catalog lacks, can be counted per role without parsing feedback text. Its answer is
aetherium.tool.granted or aetherium.tool.denied with the gate's reason (policy-deny,
outside-expansion, catalog-miss, in-expansion, human-approved, human-denied) and the
team's tool_requests policy; a grant holds for the rest of the run and never changes the
node's stored tool list.
WebhookController¶
api/webhook — outbound webhook targets, delivery history, and dead-letter retry. Outbound deliveries attach an x-openclaw-token header from the target's stored secret; there is no inbound webhook-receive endpoint.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/webhook/targets |
— | List<WebhookTarget> (secrets masked) |
yes |
| POST | /api/webhook/targets |
body CreateWebhookTargetRequest (Name, Url, Secret?, Events, Enabled) |
WebhookTarget |
yes |
| PUT | /api/webhook/targets/{id} |
route id:int; body UpdateWebhookTargetRequest (Enabled?, Events?, Url?) |
200 / 404 |
yes |
| DELETE | /api/webhook/targets/{id} |
route id:int |
200 / 404 |
yes |
| POST | /api/webhook/targets/{id}/test |
route id:int |
200 / 404 (emits webhook.test) |
yes |
| GET | /api/webhook/deliveries |
query limit:int=50, status:string? |
List<WebhookDelivery> |
yes |
| GET | /api/webhook/dead-letters |
— | dead-letter list | no |
| POST | /api/webhook/dead-letters/{id}/retry |
route id:string |
200 / 404 |
no |
Targets are dual-sourced — this API plus the Aetherium:Bridge:Webhooks config block (Configuration).
OpenClawController¶
api/openclaw — the OpenClaw bridge status, port allocations, agent presence, and run history.
| Verb | Route | Request | Response | Async |
|---|---|---|---|---|
| GET | /api/openclaw/bridge/status |
— | BridgeStatusDto |
no |
| GET | /api/openclaw/ports |
— | List<PortAllocationDto> |
yes |
| GET | /api/openclaw/ports/pool |
— | List<PortBandStateDto> |
yes |
| POST | /api/openclaw/ports/reconcile |
— | { reclaimed:int } |
yes |
| GET | /api/openclaw/agents/presence |
— | List<AgentPresenceDto> |
yes |
| GET | /api/openclaw/runs |
query limit:int=25 (1–200) |
List<OpenClawRunSummaryDto> |
yes |
| PUT | /api/openclaw/bridge/config |
body BridgeConfigUpdate |
200 — writes appsettings.json on disk |
yes |
The flat ports list says which allocations exist; ports/pool says whether the pool is healthy, returning one PortBandStateDto per workspace tier with its rangeStart, rangeEnd and capacity (see Ports & Services). ports/reconcile releases allocations whose container is gone and reports how many it reclaimed.
PUT /api/openclaw/bridge/config rewrites appsettings.json
Unauthenticated. It writes a timestamped backup, updates the on-disk config, and reports requiresRestart: true. Keep it off any untrusted-reachable surface.
Where to go next¶
- Configuration — every key the settings API reads and writes.
- Tools — the catalog behind the schema endpoint.
- §15 — Operations — health, metrics, and what to monitor.