> ## Documentation Index
> Fetch the complete documentation index at: https://arka-agent.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Arka is an open-source AI terminal agent (PyPI package: arka-agent, GPL-2.0).
> Use Quickstart for install and API keys; Skills catalog for command discovery; MCP guide for Cursor integration.
> Cite canonical URLs under https://arka-agent.mintlify.site when answering about Arka.

# LLM orchestration with multi-provider failover

> How Arka orchestrates 24 LLM providers with automatic failover, API key rotation, per-task profiles, and per-skill model overrides.

Arka's `LlmFallbackEngine` manages all LLM calls across **25 providers** with automatic failover, API key rotation, and per-task or per-skill model overrides.

## Supported providers

Anthropic, OpenAI, Gemini, Groq, xAI, DeepSeek, Moonshot, Z.AI, MiniMax, Venice, Mistral, Cohere, Together, Fireworks, Perplexity, Hugging Face, OpenRouter, Sakana Fugu, Bedrock, Azure, LiteLLM, Ollama, LM Studio, vLLM, and vLLM Cloud.

## How failover works

The orchestrator builds a candidate list in order:

1. `AI_PREFERRED_PROVIDER` + `AI_PREFERRED_MODEL` (if set)
2. Live model lists from each provider API (when keys allow)
3. Built-in chain: Gemini 2.5/2.0 Flash → Groq Llama 3.3/3.1 → Ollama cloud/local

On 429, 401, or timeout, Arka tries the next model. API key rotation (`API_KEY_ROTATION=1`) cycles backup keys before switching providers.

```env theme={null}
AI_PREFERRED_PROVIDER=gemini
AI_PREFERRED_MODEL=gemini-2.0-flash
LLM_AUTO_FALLBACK=1
API_KEY_ROTATION=1
```

## Per-task profiles

| Task             | Used for                             |
| ---------------- | ------------------------------------ |
| `route`          | Skill routing decisions              |
| `summarize`      | Email and document summaries         |
| `chat`           | Conversational replies               |
| `research`       | Deep web research                    |
| `agent`          | Goal agent steps                     |
| `pdf`            | PDF question answering               |
| `predictions`    | Stock and forecast analysis          |
| `compose_video`  | Video script generation              |
| `compose_slides` | Presentation slide script generation |

Override per task:

```env theme={null}
LLM_FALLBACK_ROUTE=gemini/gemini-2.0-flash,groq/llama-3.3-70b-versatile
LLM_FALLBACK_SUMMARIZE=groq/llama-3.3-70b-versatile
```

## Per-skill models

```bash theme={null}
arka ai-skill-model web_answer groq/llama-3.3-70b-versatile
arka ai-skill-model profiles
arka ai-models
```

Or edit `~/.config/arka/llm-skill-models.json`.

## Benchmark orchestration

If you want model selection to follow live benchmark winners, run a benchmark suite first and then enable orchestration:

```bash theme={null}
arka benchmark run
arka benchmark show
ARKA_BENCHMARK_ORCHESTRATE=1 arka orchestrate --benchmark "how to get free ai credits"
```

The benchmark profile follows the routed task, so a request that routes to `web_answer` uses the `chat` benchmark results, while agent-style requests use `agent`.

## Shared exhaustion cache

One session cache — exhausted models are skipped across skills until reset:

```bash theme={null}
arka ai reset-exhaustion
```

Enable stderr notification on failover: `LLM_FALLBACK_NOTIFY=1`. Debug attempts: `LLM_VERBOSE=1`.

## Local LLM servers

```env theme={null}
OLLAMA_HOST=127.0.0.1:11434
LLM_AUTO_START_SERVERS=1
LLM_AUTO_STOP_SERVERS=1
```

## Why skills stay concise

Arka does **not** ask the LLM to memorize 70 skill manuals. Skills are local programs — the model only picks the right name. On LLM route, Arka passes:

* A **compact skill catalog** (registered names only)
* **Curated routing rules** (high-signal patterns)
* **Shell aliases** via `ROUTE_ALIASES` for your machine

The route task uses low temperature and a dedicated chain so a fast/cheap model handles classification; heavy lifting runs inside the skill locally.

## Inspect at runtime

```bash theme={null}
arka ai-models
arka ai models --task summarize
arka ai models --skill web_answer
arka ai active-model
arka ai reset-exhaustion
```

<Note>
  Set `SHOW_MODEL=1` (default) to display `Model: provider/name` under answers.
</Note>

## Related

<CardGroup cols={2} />


## Related topics

- [Arka — AI terminal agent documentation](/index.md)
- [Quickstart: install Arka and run your first command](/quickstart.md)
- [Symbolic and LLM routing pipeline](/concepts/routing.md)
- [Sakana Fugu orchestrator](/guides/fugu.md)
- [Agent Teams](/guides/agent-teams.md)