Engines¶
| Engine | Reader (type) |
Found on its own | What Berth reads |
|---|---|---|---|
llama.cpp llama-server |
llamacpp |
:8080 | tokens, rates, queue, context window, KV slots |
| llama-swap | llama-swap |
:8080 | every running model's upstream, request log, per-model window |
| Ollama | ollama |
:11434 | loaded models and their VRAM |
| vLLM | vllm |
:8000 | models, window; tokens and queue from /metrics |
| SGLang | sglang |
:30000 | models, window; tokens and queue with --enable-metrics |
| ComfyUI | comfyui |
:8188 | jobs running and queued, version; job-boundary handoff |
| Kokoro-FastAPI (text to speech) | kokoro |
:8880 | health, models, voice count, default voice |
| LM Studio | lmstudio |
:1234 | loaded models, context length |
| KoboldCpp | koboldcpp |
:5001 | model, context length |
| TabbyAPI (ExLlamaV3) | tabbyapi |
:5000 | models, context length |
| LocalAI | localai |
:8080 | models, tokens |
| Unsloth Studio | unsloth |
:8888 | health; models with an API key |
| FreeToken | freetoken |
:1919 | tokens, queue |
| NInfer | ninfer |
as llama.cpp | as llama.cpp, plus draft acceptance |
| Strata | strata |
:8080 | model, window, tokens, queue |
| Any OpenAI-compatible server | openai |
:8080, :8000 | models |
| Anything else | custom |
never | your health, models, metrics and slots paths; kind = "tts", "stt", "image" or "other" tracks a voice or image server as one |
Every reader takes any URL, so any port and any host works. Engines that need a key name the
environment variable that holds it (api_key_env); the key never goes in the config.
On Unix, api_key_file instead names an absolute, owner-private regular file containing
one token. Symlinks and files accessible to other users are rejected. The two sources
are mutually exclusive. berth run --api-key-file uses the same source for drain requests;
detached gateway launches pass the source reference, never its contents. Other platforms
use api_key_env and explicitly reject file credentials. An engine's figures are its
own claims and are shown apart from what the driver measures.
ComfyUI as a suite¶
berth engine install|show|update|rollback|remove manages ComfyUI as a suite: the core,
ComfyUI-Manager and every git-based node pack are separate components, each pinned to a git
commit with one previous commit kept as the rollback snapshot. Several installs run side by
side on separate ports; each install's derived engine appears in berth engine check,
berth status and the gateway's wake page, and its start line is the exact berth run shape
an operator's unit already uses (the only alignment needed is --as <install-name>).
install NAME --adopt PATHrecords an existing checkout in place;--pathclones a fresh one (PyTorch is your step: the install is registered either way, and ComfyUI's own start error names the missing piece).update NAMEpins the core and each pack to a newer commit;rollback NAME --component Cswaps back. A dirty component, or an install with jobs running, is refused: the update happens at the job boundary.remove NAMEdrops the state entry;--delete --yesalso removes the directory.
The state lives in engines.toml beside berth.toml. Berth writes each install an
extra_model_paths.yaml naming the node's model roots from berth.toml, so an install's
models come from the same library the rest of the node uses; the daemon rewrites the file at
startup, so a change to the roots applies on the next restart.