Confidential AI API: Secure LLM Inference at Scale

Beta Router uses the OpenAI-compatible Chat Completions API. The capabilities on this page apply to the six models in the Models base catalog. Additional key-dependent models can have different capabilities. Use the quickstart to create the client used in these examples. These are API capabilities. Individual harnesses may not expose every control in their user interface. Use authenticated GET /v1/models with the same API key for the exact capability flags available to that key.

Router is not confidential. Do not send secrets, personal data, regulated data, or other sensitive data.

Compatibility at a glance

Capability	Current behavior
Tool calls	Supported by all six table models
Multiple tool calls	Supported by all six table models
`parallel_tool_calls: false`	Returns at most one tool call for all six table models
Strict JSON Schema	Supported by all six table models through `response_format`
Streamed usage	Supported by all six table models through `stream_options.include_usage`
Reasoning toggle	Supported by all six table models through `reasoning.enabled`
`max_completion_tokens`	Accepted for all six table models

Tool calling

Pass standard OpenAI function tools. Set parallel_tool_calls=False when your application must execute one action at a time.

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "What is the weather in Paris?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the weather for a city.",
                "strict": True,
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                    "additionalProperties": False,
                },
            },
        }
    ],
    tool_choice="auto",
    parallel_tool_calls=False,
)

for call in response.choices[0].message.tool_calls or []:
    print(call.id, call.function.name, call.function.arguments)

To continue the tool loop, execute the function, append the complete assistant message, then append a tool message whose tool_call_id matches the call ID. Send the updated message list in the next request.

Structured output

Use strict JSON Schema when your application needs a predictable response shape. Include the word JSON in the request message.

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Return a JSON summary."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "result",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {"summary": {"type": "string"}},
                "required": ["summary"],
                "additionalProperties": False,
            },
        },
    },
)

Reasoning and output limits

Use reasoning.enabled to turn model reasoning on or off. Both max_tokens and max_completion_tokens are accepted; do not send conflicting values for both fields.

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Summarize this."}],
    max_completion_tokens=4096,
    extra_body={"reasoning": {"enabled": False}},
)

Streamed usage

Set include_usage to receive token usage in the final stream chunk. That chunk can contain an empty choices array.

stream = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.usage is not None:
        print(chunk.usage.total_tokens)

Vision and log probabilities

Vision support currently covers base64-encoded images in image_url content parts.

Model ID	Base64 images	Logprobs
`kimi-k3`	Yes	No
`qwen-3.7-max`	No	Yes
`qwen-3.7-plus`	Yes	Yes
`qwen-3.6-plus`	Yes	Yes
`qwen-3.5-9b`	Yes	Key-dependent
`deepseek-v4-pro`	No	Key-dependent

Additional models are available only when returned by authenticated GET /v1/models for the same key:

Model ID	Base64 images	Logprobs
`deepseek-v4-flash`	No	Yes
`qwen-3.8-max`	Yes	Yes

Keep reasoning enabled when you use qwen-3.8-max. For a model that supports log probabilities, use the standard OpenAI fields:

response = client.chat.completions.create(
    model="qwen-3.5-9b",
    messages=[{"role": "user", "content": "Reply with one word."}],
    logprobs=True,
    top_logprobs=2,
)

See Custom Providers and Coding Tools for OpenCode, Pi, and other harnesses.

Get Started

Router

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

API Capabilities

Compatibility at a glance

Tool calling

Structured output

Reasoning and output limits

Streamed usage

Vision and log probabilities

​Compatibility at a glance

​Tool calling

​Structured output

​Reasoning and output limits

​Streamed usage

​Vision and log probabilities

Compatibility at a glance

Tool calling

Structured output

Reasoning and output limits

Streamed usage

Vision and log probabilities