Confidential AI API: Secure LLM Inference at Scale

Does your app already talk to the OpenAI API? Then you can point it at Prem instead. You change one setting: the base URL. Prem runs the model inside a secure enclave. Your prompts stay private on the way there. You connect through a small helper on your machine: the Confidential Proxy.

“OpenAI-compatible” describes the shape of the requests. It does not mean the requests go to OpenAI. It does not mean an OpenAI model runs behind them.

Confidential API
Router (Beta)

Use the local Confidential Proxy for sensitive model traffic. Go to the confidential setup.

Confidential API

What the proxy does

Accepts the client’s OpenAI-compatible request on your machine.
Encrypts request content before network egress.
Sends ciphertext and required metadata through the Prem API Gateway.
Decrypts the enclave response locally and returns the supported OpenAI-compatible response shape.

How the connection works

Your app and the Confidential Proxy stay on your machine. The Prem API Gateway sees only the sealed message. The Prem API Enclave opens it inside a Trusted Execution Environment (TEE).

What you can connect

You can connect any client that lets you set a custom base URL. For example:

OpenAI SDKs for Node.js, Python, and other languages.
Agent frameworks that support a custom OpenAI provider.
Internal apps that call /v1/chat/completions.
Audio apps that use the transcription or translation routes.

The proxy gives your app the routes below. It does not add every OpenAI feature.

Method	Route	What it does
API shape	Sends OpenAI-style JSON or multipart data	Accepts the supported OpenAI routes
Authentication	Sends `Authorization: Bearer <key>`	Uses the value as the Prem API key
Model	Sends a Prem model ID	Sends that model ID to Prem API
Request content	Plaintext inside the client process	Encrypts content before network egress
Response content	Receives OpenAI-style output	Decrypts and returns the response locally
Billing	Reads usage fields when available	Prem bills the organization associated with the API key

The client does not gain every OpenAI API feature. It gains the routes that the Confidential Proxy implements.

Supported routes

The inspected package implements these OpenAI-compatible routes:

Method	Route	Purpose
`GET`	`/v1/models`	List enabled Prem models
`POST`	`/v1/chat/completions`	Run chat completion requests, including streaming
`POST`	`/v1/audio/transcriptions`	Transcribe supported audio inputs
`POST`	`/v1/audio/translations`	Translation route; no enabled translation model is currently documented

The current Deepgram transcription response is not normalized to OpenAI’s { "text": "..." } shape. It returns Deepgram-style metadata and results fields.

Do not expect the Responses, Assistants, Realtime, Batches, Embeddings, or Files APIs. Check the Prem API reference before you use another route.

Before you start

Get these four things ready:

An app that lets you set a custom OpenAI base URL.
A Prem API key. This proves who you are. See API Keys.
A client Key Encryption Key (KEK). This is a master key that protects your other keys.
A model that your Prem account can use. See Models & Pricing.

The API key and the KEK are two different secrets. The API key handles sign-in, limits, and billing. The KEK protects your encryption keys. Keep both safe.

1. Set your secrets

Open a terminal. Set these values in the terminal that runs the proxy:

export PREM_API_KEY="your-prem-api-key"
export CLIENT_KEK="your-64-character-hex-kek"
export PROXY_URL="https://gateway.prem.io"
export ENCLAVE_URL="https://conf-engine.prem.io"

Do you not have a KEK yet? Make one, one time only:

openssl rand -hex 32

Store the KEK in a secret manager. Keep a backup. Use the same KEK every time. Do not make a new one each time you start the proxy.

PROXY_URL and ENCLAVE_URL show the current default endpoints. Check the Prem dashboard if the endpoints change.

2. Start the Confidential Proxy

Start the proxy in OpenAI mode. Keep this terminal open while you work:

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 127.0.0.1 \
  --port 8787 \
  --compat openai \
  --kek "$CLIENT_KEK"

The proxy now listens at http://127.0.0.1:8787/v1. Your app sends its requests there. Do you need one proxy for both OpenAI and Anthropic apps? Start it in both mode:

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 127.0.0.1 \
  --port 8787 \
  --compat both \
  --kek "$CLIENT_KEK"

In both mode, the OpenAI base URL is http://127.0.0.1:8787/openai/v1.

3. Point your app at the proxy

List the models that your Prem API key can use:

curl http://127.0.0.1:8787/v1/models \
  -H "Authorization: Bearer $PREM_API_KEY"

Use the returned id value in later requests. Do not substitute an OpenAI model name unless Prem lists that exact ID. The model list currently does not include a provider-backed or Reticle-verifiable assurance label. Do not infer one from the alias. See Platform Status for the current contract gap.

4. Test chat completions

Send a small non-streaming request:

curl http://127.0.0.1:8787/v1/chat/completions \
  -H "Authorization: Bearer $PREM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "messages": [{"role": "user", "content": "Reply with OK."}],
    "reasoning_effort": "none",
    "max_completion_tokens": 128,
    "stream": false
  }'

Replace glm-5.2 when your account uses another model. Inspect the response model field as well as the requested alias. A request for glm-5.2 currently returns the resolved ID zai-org/GLM-5.2. Do not continue if this request fails. See Errors and Rate limits.

5. Configure the application

Change only the API key and base URL when the application already uses Chat Completions.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.PREM_API_KEY,
  baseURL: "http://127.0.0.1:8787/v1",
});

const response = await client.chat.completions.create({
  model: "glm-5.2",
  messages: [{ role: "user", content: "Reply with OK." }],
  reasoning_effort: "none",
  max_completion_tokens: 128,
});

console.log(response.choices[0]?.message?.content);

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PREM_API_KEY"],
    base_url="http://127.0.0.1:8787/v1",
)

response = client.chat.completions.create(
    model="glm-5.2",
    messages=[{"role": "user", "content": "Reply with OK."}],
    reasoning_effort="none",
    max_completion_tokens=128,
)

print(response.choices[0].message.content)

Change glm-5.2 to a model your account can use. See your model IDs on the Prem dashboard. Other clients name the base URL setting in different ways. Look for baseURL, base_url, endpoint, host, or provider URL.

What this setup protects

Component	Reads your text	Notes
Your app	Yes	It creates the prompt and reads the reply on your machine.
Confidential Proxy	Yes, on your machine	It seals requests and opens replies.
Prem API Gateway	No	It handles the sealed message and metadata only.
Prem API Enclave	Yes, inside the TEE	It opens the request, runs the model, and seals the reply.
App tools and plugins	Depends on the tool	Prem does not put them inside the enclave.

This setup protects the model path only. It does not put your app, its tools, or its plugins inside the enclave.

Good to know

Use a Prem model ID. Compatibility does not give you OpenAI-hosted models. Use a model that /v1/models returns. One stream at a time. The encrypted endpoint allows one active stream for each API key. A second stream on the same key can return 429. Follow the Retry-After header when the response has one. See Agents & Automation. Check the finish reason. Read finish_reason on each response. Do not treat a cut-off reply as a complete one.

Troubleshooting

The app still calls api.openai.com

Check the base URL setting. Restart the app after you change it.

The API says model not found

List your models with /v1/models. Use an enabled Prem model ID from that list.

The app calls /v1/responses

Set the app to use Chat Completions. The proxy does not expose the Responses API.

Streaming stops early

Check finish_reason, timeout settings, proxy logs, and rate limits. Do not treat partial content as complete.

The API returns 429

Wait for the active request to finish. Follow Retry-After when the response has it.

Frequently asked questions

Does “OpenAI-compatible” mean OpenAI processes the request?

No. The term describes the HTTP interface only. The request goes through Prem and uses the Prem model you choose.

Can I use an OpenAI model name?

Only when /v1/models returns that exact ID. Compatibility does not give you OpenAI-hosted models.

Can I use the Responses API?

No. Set the client to use /v1/chat/completions.

Which key goes in the Authorization header?

The Prem API key. The KEK is a separate local secret. Never send the KEK.

Does changing the base URL protect my local tools?

No. It protects the model path after the proxy seals the request. Your local tools keep their own trust boundaries.

Router

Beta

Router is not confidential. Do not send secrets, personal data, regulated data, or other sensitive data through this path.

An OpenAI-compatible Chat Completions client can call Router directly:

Setting	Confidential API	Router
Base URL	`http://127.0.0.1:8787/v1`	`https://router.prem.io/v1`
API key	`PREM_API_KEY`	`PREM_ROUTER_API_KEY`
Client KEK	Required by the local proxy	Not used
Confidential	Yes, within the documented boundary	No
Model catalog	Confidential catalog	Broader current Router chat catalog

Confirm that kimi-k3 appears in GET /v1/models for this key. Otherwise, use an exact returned model ID. See Router models.

const routerClient = new OpenAI({
  apiKey: process.env.PREM_ROUTER_API_KEY,
  baseURL: "https://router.prem.io/v1",
});

const response = await routerClient.chat.completions.create({
  model: "kimi-k3",
  messages: [{ role: "user", content: "Reply with router ok." }],
});

Use a separate Router key and an exact ID returned for that key. See Router models.

Confidential Proxy

Review every mode, route, key, and daemon option.

OpenCode

Configure OpenCode as a specific OpenAI-compatible client.

OpenClaw

Configure a custom Chat Completions provider.

Hermes Agent

Use a named provider with the Chat Completions transport.

Goose

Point Goose’s built-in OpenAI provider at the proxy.

Anthropic-compatible clients

Use the Messages API shape through the same proxy.

Agents & Automation

Handle concurrency, retries, reasoning, and unattended operation.

Platform Status

Review current behavior and assurance gaps.

Get Started

Router

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

OpenAI-compatible clients

Confidential API

What the proxy does

How the connection works

What you can connect

Supported routes

Before you start

1. Set your secrets

2. Start the Confidential Proxy

3. Point your app at the proxy

4. Test chat completions

5. Configure the application

What this setup protects

Good to know

Troubleshooting

Frequently asked questions

Does “OpenAI-compatible” mean OpenAI processes the request?

Can I use an OpenAI model name?

Can I use the Responses API?

Which key goes in the Authorization header?

Does changing the base URL protect my local tools?

Router

Confidential Proxy

OpenCode

OpenClaw

Hermes Agent

Goose

Anthropic-compatible clients

Agents & Automation

Platform Status

​Confidential API

​What the proxy does

​How the connection works

​What you can connect

​Supported routes

​Before you start

​1. Set your secrets

​2. Start the Confidential Proxy

​3. Point your app at the proxy

​4. Test chat completions

​5. Configure the application

​What this setup protects

​Good to know

​Troubleshooting

​Frequently asked questions

​Does “OpenAI-compatible” mean OpenAI processes the request?

​Can I use an OpenAI model name?

​Can I use the Responses API?

​Which key goes in the Authorization header?

​Does changing the base URL protect my local tools?

​Router

​Related

Confidential Proxy

OpenCode

OpenClaw

Hermes Agent

Goose

Anthropic-compatible clients

Agents & Automation

Platform Status

Confidential API

What the proxy does

How the connection works

What you can connect

Supported routes

Before you start

1. Set your secrets

2. Start the Confidential Proxy

3. Point your app at the proxy

4. Test chat completions

5. Configure the application

What this setup protects

Good to know

Troubleshooting

Frequently asked questions

Does “OpenAI-compatible” mean OpenAI processes the request?

Can I use an OpenAI model name?

Can I use the Responses API?

Which key goes in the Authorization header?

Does changing the base URL protect my local tools?

Router

Related