> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.joytokens.ai/llms.txt.

# Routing

> auto routing and tiers

`model` supports two ways of calling: send `model: "auto"` and JoyToken selects the final model and provider for the request (recommended), or send a concrete model ID from the catalog to pin the request to that model.

## Common Requests

| Goal                        | Request                                 |            |             |
| --------------------------- | --------------------------------------- | ---------- | ----------- |
| Smart routing (recommended) | `model: "auto"`                         |            |             |
| Select a tier               | `model: "auto"` + \`tier: "economy"     | "standard" | "premium"\` |
| Pin a specific model        | `model: "<modelName from the catalog>"` |            |             |

## Smart Routing

Send `model: "auto"` and JoyToken's smart routing engine takes over. It combines multi-dimensional request features, semantic rules, and LLM review to identify system settings, tool intent, context, and user needs, determine the task type and complexity, and apply the matching routing strategy.

For complex requests, the system automatically builds a task plan — breaking the request down, matching models and tiers to each task, and aggregating the results. Simple requests go straight to single-model selection.

Within policy constraints, the system schedules intelligently across a continuously expanding pool of online models and capabilities, weighing quality, cost, latency, and real-time success rate, and keeps service quality and continuity through automatic circuit breaking, traffic switching, and cross-tier fallback.

```json
{
  "model": "auto",
  "messages": [
    { "role": "user", "content": "Summarize this ticket in one sentence." }
  ]
}
```

```bash
curl -X POST "$JOY_TOKEN_API_BASE_URL/openai/v1/chat/completions" \
  -H "Authorization: Bearer $JOY_TOKEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{ "role": "user", "content": "Summarize this ticket in one sentence." }]
  }'
```

## Pin a Specific Model

Pass a `modelName` from `GET /api/v1/models` as the request `model`. JoyToken then skips scoring and routing and sends the request straight to that model.

```json
{
  "model": "deepseek-v4-flash",
  "messages": [
    { "role": "user", "content": "Summarize this ticket in one sentence." }
  ]
}
```

```bash
curl -X POST "$JOY_TOKEN_API_BASE_URL/openai/v1/chat/completions" \
  -H "Authorization: Bearer $JOY_TOKEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{ "role": "user", "content": "Summarize this ticket in one sentence." }]
  }'
```

## Tiers

`tier` narrows the candidate pool for an `auto` request. Omit it to let JoyToken decide.

| Tier       | Use                                        |
| ---------- | ------------------------------------------ |
| `economy`  | Cost-first, batch, low-risk tasks          |
| `standard` | Balanced default                           |
| `premium`  | Quality-first or critical evaluation tasks |

## Orchestration (Multi-task Planning)

For complex prompts, JoyToken may handle a single `model: "auto"` request in orchestration mode: it first builds a task plan, runs each task with a selected sub-model, then aggregates a final answer. You do not opt in explicitly; the gateway decides based on the request.

### How to detect it

| Signal                      | Meaning                                                                                                                            |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `plan` (top level)          | Ordered task list, present only in orchestration mode                                                                              |
| `choices[].message.content` | A JSON-encoded array of per-task outputs (`content` + `title`) instead of a single string                                          |
| `metadata` (array)          | One entry per task, including reserved `__planner__` and `__final__` tasks                                                         |
| streaming planning event    | A leading chunk whose `orchestration.phase` is `planning` carries the ordered `plan`, followed by the `__planner__` metadata event |
| streaming task chunk        | Each task's content chunk carries an `orchestration` field (`task_id`, `task_seq`, `task_status`, `title`)                         |
| streaming `metadata` event  | One standalone metadata event per task (a single object, not an array), emitted right after that task's content                    |

### Task lifecycle

| Field         | Meaning                                                                          |
| ------------- | -------------------------------------------------------------------------------- |
| `task_id`     | Task identifier; `__planner__` and `__final__` are reserved                      |
| `task_seq`    | Execution order; `0` is the planner                                              |
| `task_status` | Task status, e.g. `DONE`                                                         |
| `tag`         | Task role, e.g. `orchestration`, `search`, `reasoning`, `writing`, `aggregation` |
| `title`       | Human-readable task title                                                        |

> **Warning**
>
> In orchestration mode each task is billed separately (one `metadata`/`billing` entry per task) and may use a different sub-model. Parse `choices[].message.content` as JSON before rendering, and sum `billing.credits_used` across all task metadata entries.

## Confirm Routing

| Signal             | Meaning                  |
| ------------------ | ------------------------ |
| `metadata.model`   | Final JoyToken model     |
| `metadata.tier`    | Billing and routing tier |
| `metadata.billing` | Credits and token usage  |

## Troubleshooting

| Symptom                                | Check first                                                                   |
| -------------------------------------- | ----------------------------------------------------------------------------- |
| tier not allowed                       | API key policy allows the tier                                                |
| no eligible model                      | tier, blacklist, wallet, and key budget                                       |
| IP not allowed                         | server egress IP is allowlisted                                               |
| `auto` fails                           | policy, tier, and balance still leave candidate models                        |
| concrete model name rejected           | the name matches the catalog exactly and the API key policy allows that model |
| pinned model returns `model_exhausted` | the name matches the catalog exactly and the model has upstream capacity      |
| pinned model ignores `tier`            | expected: a pinned model always bills at its own tier                         |