AI Providers: Multi-LLM Model Routing and Local Models

Start with OctoHub

OctoHub is the default provider in the shipped configuration. The shortest setup is to run octomind in a terminal and choose the hub on the first-run screen, or to sign in directly:

octomind login
octomind

octomind login starts a device-authorization flow, displays a short code, and opens a browser approval page. After approval, Octomind stores OCTOHUB_API_KEY in <data-dir>/config/.env and the account session in <data-dir>/config/auth.json. You do not need separate credentials for models accessed through that gateway.

The default model is octohub:auto. To choose another model exposed by the gateway for one session, pass an explicit model override:

octomind run -m 'octohub:<model>'

Replace <model> with the gateway model identifier. The OctoHub client accepts any non-empty model name; the gateway decides whether that model is available to the credential.

For an existing alternative OctoHub deployment, set OCTOHUB_API_URL to its base URL and OCTOHUB_API_KEY to the credential issued by that deployment. For example, if your gateway listens locally on port 8080:

export OCTOHUB_API_URL="http://localhost:8080"
export OCTOHUB_API_KEY="your_key"
octomind run -m octohub:auto

Purpose routing with octohub:auto

Octomind attaches X-Model-Purpose to provider requests so a gateway can route octohub:auto by purpose. It uses exactly three values:

The shipped configuration uses octohub:auto for all three. These are edits to existing profiles, not a complete replacement config; preserve their other fields:

[model]
name = "octohub:auto"

[supervisor.model]
name = "octohub:auto"

[compression.model]
name = "octohub:auto"

All supervisor mechanics, including learning, share the supervisor purpose. The gateway controls the routing and model availability; Octomind does not implement its routing policy.

Bring Your Own Keys

You can skip the hub. The shortest path is the interactive setup:

octomind configure

Pick your provider, enter the key when asked (hidden input), and pick the main model. Setup then asks whether the supervisor and compression should use the same model; answer n to choose a cheaper one for each. The key is saved to the user-scope <data-dir>/config/.env (owner-only permissions on Unix), never to config.toml. The same screen appears on the first interactive octomind launch when no config exists yet.

To set it up by hand instead, export your credential. For example, for OpenAI:

export OPENAI_API_KEY="your_key"

Replace your_key with your credential. In the generated config, change name in all three existing profiles while keeping their other settings. A main-model override alone leaves the shipped internal profiles on OctoHub:

[model]
name = "openai:gpt-5.6-sol"

[supervisor.model]
name = "openai:gpt-5.6-sol"

[compression.model]
name = "openai:gpt-5.6-sol"

Then validate and start a session:

octomind config --validate
octomind run -m openai:gpt-5.6-sol

Use a model your provider account can access. An explicit role model can override the main default; check Roles if a different model appears.

Environment File Precedence

Provider clients read credentials from the environment. octomind config --api-key refuses to save a key; put credentials in the environment or a .env file, or let octomind configure write the user .env for you.

Octomind loads credentials from three sources, with later sources overriding earlier ones:

1. Process environment
2. User-scope <data-dir>/config/.env
3. Project-local ./.env

A project .env can select different credentials from the user file. For example, put this line in .env, using your real key, and keep the file out of version control:

OPENAI_API_KEY="your_key"

Remove stale assignments from later files if you want an exported value to take effect. Empty-value handling varies by provider; do not use a blank assignment as a credential fallback. OCTOMIND_CONFIG_PATH does not change the user credential directory.

Local Endpoint URLs

local: defaults to http://localhost:11434/v1/chat/completions. ollama: defaults to https://ollama.com/v1/chat/completions, so set an explicit endpoint to use a local Ollama server:

export OLLAMA_API_URL="http://localhost:11434/v1/chat/completions"
octomind run -m ollama:glm-5.3

This requires that model to be available on your running server. Configure supervisor and compression too if all model requests should stay local. Use the full endpoint path for LOCAL_API_URL and OLLAMA_API_URL; OCTOHUB_API_URL takes a base URL. The openai: adapter uses the Responses API; use local: for a local Chat Completions-compatible endpoint.

Local CLI-Backed Models

The special cli meta-provider executes a local agent CLI and skips provider credential validation. Its format is cli:<backend>/<model>:

[model]
name = "cli:codex/gpt-5.6-sol"

Choose a model accepted by your installed backend, and edit the existing [model] table. The backend must be installed and authenticated separately. Known adapters are codex, claude, cursor, and gemini; other backend names use the generic adapter. The executable defaults to the backend name, except Cursor defaults to cursor-agent.

The Codex backend also accepts these compatibility variables:

export CODEX_COMMAND="codex"
export CODEX_REASONING_EFFORT="medium"  # low | medium | high
export CODEX_SKIP_GIT_CHECK="false"
octomind run -m cli:codex/gpt-5.6-sol

CLI_CODEX_COMMAND and CLI_CODEX_REASONING_EFFORT take precedence over their CODEX_ aliases. The internal profiles still need their own working providers.

Switch Models

Override only the current invocation:

octomind run -m 'anthropic:claude-sonnet-4-6'

Change the active session:

/model openai:gpt-5.6-sol
/model anthropic:claude-sonnet-4-6
/model octohub:auto

Or edit [model].name for the persistent default. Role, supervisor, and compression profiles can override the main profile as described in Configuration.

Diagnose Provider Setup

Inspect the loaded settings and perform a small provider request:

octomind config --show
printf '%s\n' 'Reply with OK.' | octomind run --format plain

config --show reports Octomind sign-in separately from a manually exported gateway key. Its credential rows are incomplete and include legacy Google/AWS variable names; use the provider reference below for actual inputs. Config validation checks configuration and model names, not remote authentication or account access.

Common failures:

• Invalid model format: include both parts of provider:model.
• Unsupported provider: use a prefix from the provider reference below.
• Missing credentials: set the variables for the selected prefix or run octomind login for the default OctoHub path.
• OctoHub authentication rejected: force a new login to replace the machine's stored gateway credential:

octomind login --force

Provider Reference

The table reflects octolib 0.43.1, locked in Cargo.lock. Endpoint variables are optional overrides.

The historical google: prefix is not accepted by the current provider factory; use google-vertex: or google-studio:. The kimi: prefix is an alias for moonshot: and uses MOONSHOT_API_KEY.

For another prefix, follow the same export-and-profile procedure above with the corresponding variable and model. OpenRouter attribution defaults to Octomind and https://octomind.run only when the variables are absent:

export OPENROUTER_APP_TITLE="My terminal assistant"
export OPENROUTER_HTTP_REFERER="https://example.com"

See also

• Configuration — model profiles and precedence
• Environment Variables — broader runtime-variable reference
• Compression — compression profile behavior