Providers

Overview

Providers are resolved by packages/nikcli/src/provider/provider.ts. The service starts with models.dev, layers local and gateway providers, applies user config and saved auth, lets plugins modify providers, filters models, then caches SDK instances and model loaders.

Provider resolution

The provider service constructs the final catalog in this order:

1. Load models.dev and convert each provider/model into nikcli provider records.
2. Apply disabled_providers and enabled_providers filters.
3. Probe dynamic local providers such as ollama and nikcli-inference.
4. Merge provider overrides from config.provider.
5. Merge saved API auth records from Auth.Service.
6. Run plugin auth loaders and plugin model loaders.
7. Filter alpha, deprecated, unavailable, allow-list, and block-list models.
8. Create provider/model caches for SDK instances, language models, and image models.

Configure a provider

Provider config is stored under provider in nikcli.json. The schema is ModelsDev.Provider.partial plus nikcli options such as API key, base URL, timeout, header timeout, chunk timeout, model variants, allow-lists, and block-lists.

{
  "$schema": "https://nikcli-ai.dev/config.json",
  "model": "openrouter/openai/gpt-5",
  "small_model": "openai/gpt-5-mini",
  "provider": {
    "openrouter": {
      "options": {
        "apiKey": "{env:OPENROUTER_API_KEY}",
        "baseURL": "https://openrouter.ai/api/v1",
        "timeout": 300000,
        "headerTimeout": 10000,
        "chunkTimeout": 60000
      }
    },
    "ollama": {
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1",
        "apiKey": "ollama"
      }
    }
  }
}

Bundled SDK adapters

These provider packages are imported directly in provider/provider.ts, so they do not require a dynamic package install at runtime.

Dynamic providers

nikcli also creates providers that are not just copied from models.dev. The two important dynamic loaders are local Ollama and nikcli Inference Gateway.

Auth and credentials

Provider credentials can come from three places: environment variables read from model metadata and provider-specific loaders, saved auth records from Auth.Service, or explicit config options.

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export OPENROUTER_API_KEY=sk-or-...
export AWS_REGION=us-east-1
export AWS_PROFILE=default
export NIKCLI_INFERENCE_KEY=nik-...

Model selection

The active model is a provider/model string. It can come from CLI flags, session/TUI selection, config model, or defaults chosen by provider availability. small_model is used for supporting tasks such as title generation.

Runtime streaming

Runtime ownership starts in session/llm.ts. It builds headers, normalizes messages, converts tools, applies ProviderTransform, and streams output. A native runtime in session/llm/native-runtime.ts can use @nikcli-ai/llm for OpenAI, Anthropic, and OpenAI-compatible providers when requirements are met.

Images and speech

Image generation and speech use separate config sections so they can choose different providers and models from chat. The tool implementations also read provider-specific environment variables for OpenRouter and ElevenLabs.

{
  "image": {
    "provider": "openrouter",
    "model": "openai/gpt-5-image"
  },
  "speak": {
    "provider": "elevenlabs",
    "model": "voice-id",
    "modelId": "eleven_v3",
    "outputFormat": "mp3_44100_128"
  }
}

Refresh and cache

Provider service state is cached per instance. After writes that change auth or config.provider.*, route handlers call provider refresh so subsequent list and language-model reads see the new credentials and options.

Failure modes

Source anchors