# Providers

How nikcli builds its model catalog, resolves credentials, selects runtime adapters, and streams requests.

## Overview

Providers are resolved by `packages/nikcli/src/provider/provider.ts`. The service starts with `models.dev`, layers local and gateway providers, applies user config and saved auth, lets plugins modify providers, filters models, then caches SDK instances and model loaders.

Models

### Catalog

`ModelsDev.get` provides the default provider/model database and API package metadata.

Keys

### Auth

Credentials can come from environment variables, saved auth records, or `config.provider.*.options`.

Adapters

### SDK

Known AI SDK packages are imported directly; unknown provider packages can still be loaded dynamically.

Stream

### Runtime

Session execution enters `session/llm.ts`, then uses AI SDK or the native `@nikcli-ai/llm` runtime when supported.

## Provider resolution

The provider service constructs the final catalog in this order:

1.  Load `models.dev` and convert each provider/model into nikcli provider records.
2.  Apply `disabled_providers` and `enabled_providers` filters.
3.  Probe dynamic local providers such as `ollama` and `nikcli-inference`.
4.  Merge provider overrides from `config.provider`.
5.  Merge saved API auth records from `Auth.Service`.
6.  Run plugin auth loaders and plugin model loaders.
7.  Filter alpha, deprecated, unavailable, allow-list, and block-list models.
8.  Create provider/model caches for SDK instances, language models, and image models.

Filters are provider-level first

`enabled_providers` means only listed providers survive. `disabled_providers` removes providers from automatic loading. Model-level allow/block lists are applied later.

## Configure a provider

Provider config is stored under `provider` in `nikcli.json`. The schema is `ModelsDev.Provider.partial` plus nikcli options such as API key, base URL, timeout, header timeout, chunk timeout, model variants, allow-lists, and block-lists.

nikcli.json

json 

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

```
{
  "$schema": "https://nikcli-ai.dev/config.json",
  "model": "openrouter/openai/gpt-5",
  "small_model": "openai/gpt-5-mini",
  "provider": {
    "openrouter": {
      "options": {
        "apiKey": "{env:OPENROUTER_API_KEY}",
        "baseURL": "https://openrouter.ai/api/v1",
        "timeout": 300000,
        "headerTimeout": 10000,
        "chunkTimeout": 60000
      }
    },
    "ollama": {
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1",
        "apiKey": "ollama"
      }
    }
  }
}
```

| Option | Meaning |
| --- | --- |
| `options.apiKey` | Explicit key used when no saved auth or provider-specific env key wins first. |
| `options.baseURL` | OpenAI-compatible or gateway URL, normalized by provider-specific loaders when needed. |
| `options.timeout` | Total request timeout in milliseconds, or `false` to disable. |
| `options.headerTimeout` | Maximum wait for response headers. OpenAI defaults to a configured provider value. |
| `options.chunkTimeout` | Maximum gap between SSE chunks before aborting a stream. |
| `whitelist` | Only keep matching model ids for this provider. |
| `blacklist` | Remove matching model ids for this provider. |
| `models.*.variants` | Variant-level config, including disabled variants. |

## Bundled SDK adapters

These provider packages are imported directly in `provider/provider.ts`, so they do not require a dynamic package install at runtime.

| Package | Factory | Notes |
| --- | --- | --- |
| `@ai-sdk/openai` | `createOpenAI` | Uses Responses API through custom loader. |
| `@ai-sdk/anthropic` | `createAnthropic` | Adds Claude Code and fine-grained streaming beta headers. |
| `@ai-sdk/azure` | `createAzure` | Can choose chat or responses URL behavior. |
| `@ai-sdk/google` | `createGoogleGenerativeAI` | Google Generative AI provider. |
| `@ai-sdk/google-vertex` | `createVertex` | Vertex provider with project/location env handling. |
| `@ai-sdk/google-vertex/anthropic` | `createVertexAnthropic` | Anthropic on Vertex. |
| `@ai-sdk/openai-compatible` | `createOpenAICompatible` | Default fallback for compatible providers. |
| `@openrouter/ai-sdk-provider` | `createOpenRouter` | OpenRouter native adapter. |
| `@ai-sdk/xai` | `createXai` | xAI adapter. |
| `@ai-sdk/amazon-bedrock` | `createAmazonBedrock` | Credential chain, bearer token, region prefix logic. |
| `@ai-sdk/gateway` | `createGateway` | AI Gateway package support. |
| `@ai-sdk/vercel` | `createVercel` | Vercel AI Gateway package support. |
| `@gitlab/gitlab-ai-provider` | `createGitLab` | Uses OAuth/API/env token resolution. |
| `@ai-sdk/github-copilot` | Copilot compatibility factory | Switches between chat and responses for GPT-5 family models. |

## Dynamic providers

nikcli also creates providers that are not just copied from `models.dev`. The two important dynamic loaders are local Ollama and nikcli Inference Gateway.

| Provider | Detection | Result |
| --- | --- | --- |
| `ollama` | Configured `provider.ollama.options.baseURL`, `OLLAMA_BASE_URL`, or local `127.0.0.1:11434` probe. | Builds OpenAI-compatible models from `/models` with conservative limits. |
| `nikcli-inference` | `NIKCLI_INFERENCE_URL`, `NIKCLI_INFERENCE_KEY`, or config options. | Builds models from gateway `/models`, with pricing, context, thinking, and image input metadata. |
| Unknown configured provider | Entry exists in `config.provider` or has a key. | Creates a default provider record so custom compatible endpoints can appear. |
| Plugin provider | Plugin exposes auth or model hooks. | Merges options and model lists into the provider catalog. |

## Auth and credentials

Provider credentials can come from three places: environment variables read from model metadata and provider-specific loaders, saved auth records from `Auth.Service`, or explicit config options.

Terminal

bash 

```
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export OPENROUTER_API_KEY=sk-or-...
export AWS_REGION=us-east-1
export AWS_PROFILE=default
export NIKCLI_INFERENCE_KEY=nik-...
```

$ shell · paste into your terminal

| Provider family | Special handling |
| --- | --- |
| Amazon Bedrock | Config region wins over `AWS_REGION`, bearer token can override credential chain, endpoint/baseURL can override endpoint. |
| Azure | Can use resource-name derived base URL and switch completion URLs through options. |
| GitLab | Uses saved OAuth, saved API key, or `GITLAB_TOKEN`; instance URL defaults to `https://gitlab.com`. |
| Cloudflare | Uses account/gateway env plus token and transforms auth/header behavior for AI Gateway. |
| GitHub Copilot | Built-in auth plugin can register personal or enterprise Copilot providers. |

## Model selection

The active model is a `provider/model` string. It can come from CLI flags, session/TUI selection, config `model`, or defaults chosen by provider availability. `small_model` is used for supporting tasks such as title generation.

| Setting | Used for |
| --- | --- |
| `model` | Default primary model for agent turns. |
| `small_model` | Low-cost helper work such as title and summaries. |
| `NIKCLI_ENABLE_EXPERIMENTAL_MODELS` | Keeps alpha models in the catalog. |
| `NIKCLI_DISABLE_MODELS_FETCH` | Disables models.dev fetch path where supported. |
| `provider.*.whitelist` | Provider-local model allow-list. |
| `provider.*.blacklist` | Provider-local model block-list. |

Provider/model ids are exact

`Provider.ModelNotFoundError` includes provider id, model id, and suggestions. Check both sides of the slash before assuming the provider is unavailable.

## Runtime streaming

Runtime ownership starts in `session/llm.ts`. It builds headers, normalizes messages, converts tools, applies `ProviderTransform`, and streams output. A native runtime in `session/llm/native-runtime.ts` can use `@nikcli-ai/llm` for OpenAI, Anthropic, and OpenAI-compatible providers when requirements are met.

| Step | Source | Notes |
| --- | --- | --- |
| Message repair | `LLM.normalizeStreamMessages` | Repairs UI-shaped messages that would otherwise trigger invalid prompt errors. |
| Headers | `buildRequestHeaders` | Adds nikcli request headers for nikcli providers and user agent for most others. |
| Provider options | `ProviderTransform.providerOptions` | Transforms model/provider options before SDK/native call. |
| Native runtime | `native-runtime.ts` | Requires supported provider family and configured API key. |
| AI SDK fallback | `session/llm/ai-sdk.ts` | General runtime path for all supported SDK adapters. |

## Images and speech

Image generation and speech use separate config sections so they can choose different providers and models from chat. The tool implementations also read provider-specific environment variables for OpenRouter and ElevenLabs.

nikcli.json

json 

1

2

3

4

5

6

7

8

9

10

11

12

```
{
  "image": {
    "provider": "openrouter",
    "model": "openai/gpt-5-image"
  },
  "speak": {
    "provider": "elevenlabs",
    "model": "voice-id",
    "modelId": "eleven_v3",
    "outputFormat": "mp3_44100_128"
  }
}
```

| Env | Used by |
| --- | --- |
| `NIKCLI_OPENROUTER_API_KEY` / `OPENROUTER_API_KEY` | OpenRouter image, speech, and voice-related tools. |
| `NIKCLI_ELEVENLABS_API_KEY` / `ELEVENLABS_API_KEY` | ElevenLabs speech provider. |
| `NIKCLI_OPENROUTER_BASE_URL` | OpenRouter-compatible base URL override. |
| `NIKCLI_ELEVENLABS_BASE_URL` | ElevenLabs base URL override. |

## Refresh and cache

Provider service state is cached per instance. After writes that change auth or `config.provider.*`, route handlers call provider refresh so subsequent list and language-model reads see the new credentials and options.

| Mutation | Refresh path |
| --- | --- |
| Save provider API key | `server/httpapi/provider.ts` or `httpapi/provider.ts` writes auth and refreshes provider cache. |
| Remove provider auth | Auth removal clears the saved record and refreshes. |
| Config provider update | Config route update invalidates provider state through route helper. |
| Plugin auth loader change | Provider service rebuilds plugin-derived options during refresh. |

## Failure modes

| Symptom | Likely cause | Check |
| --- | --- | --- |
| Provider absent from model list | Filtered by `enabled_providers`/`disabled_providers`, missing auth, or failed dynamic probe. | Inspect config filters and provider log entries. |
| Model id missing | Alpha/deprecated filtering, provider allow/block list, or no API key for paid nikcli models. | Enable experimental models only if the model status requires it. |
| Bedrock authentication failure | Region/profile/bearer token mismatch. | Check config region, `AWS_REGION`, `AWS_PROFILE`, bearer token, and endpoint. |
| OpenAI-compatible usage missing | Provider options disabled usage inclusion or compatible endpoint omits usage. | Check `includeUsage` and provider response. |
| Invalid prompt error | Malformed message shape reached runtime. | Verify `normalizeStreamMessages` input and message history. |
| Fresh key not used | Provider cache was not refreshed after auth/config write. | Call provider refresh or use route handlers that already do it. |

## Source anchors

| Need | File |
| --- | --- |
| Provider catalog and SDK cache | `packages/nikcli/src/provider/provider.ts` |
| Provider transforms | `packages/nikcli/src/provider/transform.ts` |
| Provider auth routes | `packages/nikcli/src/server/httpapi/provider.ts`, `packages/nikcli/src/server/httpapi/provider.ts` |
| Session runtime hot path | `packages/nikcli/src/session/llm.ts` |
| Native runtime | `packages/nikcli/src/session/llm/native-runtime.ts` |
| LLM package providers | `packages/llm/src/providers` |
| Image and speech tools | `packages/nikcli/src/tool/generate_image.ts`, `packages/nikcli/src/tool/speak.ts` |

---

Source: https://nikcli-ai.dev/docs/providers