Confidential AI API: Secure LLM Inference at Scale

Prem Router uses the OpenAI-compatible Chat Completions API. The capabilities on this page are verified against the current beta model catalog. Use the quickstart to create the client used in these examples. These are API capabilities. Individual harnesses may not expose every control in their user interface.

Prem Router is not confidential. Do not send secrets, personal data, regulated data, or other sensitive data.

Compatibility at a glance

Capability	Current behavior
Tool calls	Supported by all models
Multiple tool calls	Supported by all models
`parallel_tool_calls: false`	Returns at most one tool call
Strict JSON Schema	Supported by all models through `response_format`
Streamed usage	Supported by all models through `stream_options.include_usage`
Reasoning toggle	Supported by all models through `reasoning.enabled`
`max_completion_tokens`	Accepted for all models

Tool calling

Pass standard OpenAI function tools. Set parallel_tool_calls=False when your application must execute one action at a time.

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "What is the weather in Paris?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the weather for a city.",
                "strict": True,
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                    "additionalProperties": False,
                },
            },
        }
    ],
    tool_choice="auto",
    parallel_tool_calls=False,
)

for call in response.choices[0].message.tool_calls or []:
    print(call.id, call.function.name, call.function.arguments)

To continue the tool loop, execute the function, append the complete assistant message, then append a tool message whose tool_call_id matches the call ID. Send the updated message list in the next request.

Structured output

Use strict JSON Schema when your application needs a predictable response shape. Include the word JSON in the request message.

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Return a JSON summary."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "result",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {"summary": {"type": "string"}},
                "required": ["summary"],
                "additionalProperties": False,
            },
        },
    },
)

Reasoning and output limits

Use reasoning.enabled to turn model reasoning on or off. Both max_tokens and max_completion_tokens are accepted; do not send conflicting values for both fields.

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Summarize this."}],
    max_completion_tokens=4096,
    extra_body={"reasoning": {"enabled": False}},
)

Streamed usage

Set include_usage to receive token usage in the final stream chunk. That chunk can contain an empty choices array.

stream = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.usage is not None:
        print(chunk.usage.total_tokens)

Vision and log probabilities

Vision support currently covers base64-encoded images in image_url content parts.

Model ID	Base64 images	Logprobs
`kimi-k3`	Yes	No
`qwen-3.7-max`	No	Yes
`qwen-3.7-plus`	Yes	Yes
`qwen-3.6-plus`	Yes	Yes
`qwen-3.5-9b`	Yes	Yes
`deepseek-v4-pro`	No	Yes

For a model that supports log probabilities, use the standard OpenAI fields:

response = client.chat.completions.create(
    model="qwen-3.5-9b",
    messages=[{"role": "user", "content": "Reply with one word."}],
    logprobs=True,
    top_logprobs=2,
)

See Custom Providers and Coding Tools for OpenCode, Pi, and other harnesses.

Get Started

Router (Beta)

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

API Capabilities

Compatibility at a glance

Tool calling

Structured output

Reasoning and output limits

Streamed usage

Vision and log probabilities

​Compatibility at a glance

​Tool calling

​Structured output

​Reasoning and output limits

​Streamed usage

​Vision and log probabilities

Compatibility at a glance

Tool calling

Structured output

Reasoning and output limits

Streamed usage

Vision and log probabilities