Confidential AI API: Secure LLM Inference at Scale

Does your app already talk to the Anthropic Messages API? Then you can point it at Prem instead. You change one setting: the base URL. Prem runs the model inside a secure enclave. Your prompts stay private on the way there. You connect through a small helper on your machine: the Confidential Proxy.

“Anthropic-compatible” describes the shape of the requests. It does not mean Anthropic hosts the request. It does not mean a Claude model runs behind it.

What the proxy does

Accepts the client’s Anthropic Messages request on your machine.
Translates and encrypts the request before network egress.
Sends ciphertext and required metadata through the Prem API Gateway.
Decrypts the enclave response locally and returns Anthropic message objects or Server-Sent Events (SSE).

How the connection works

Your app and the Confidential Proxy stay on your machine. The proxy translates your Messages request into Prem’s format, then seals it. The Prem API Gateway sees only the sealed message. The Prem API Enclave opens it inside a Trusted Execution Environment (TEE). The proxy does two translations for you:

It turns your Messages request into Prem’s internal chat request.
It turns the reply back into Anthropic message objects or Server-Sent Events (SSE).

What you can connect

You can connect any client that lets you set a custom base URL. The proxy gives your app these routes:

Method	Route	What it does
`POST`	`/v1/messages`	Creates a message, with or without streaming.
`POST`	`/v1/messages/count_tokens`	Returns an estimated input token count.
`GET`	`/v1/models`	Lists the models your account can use.
`GET`	`/v1/models/:model_id`	Gets one model.

Send the Prem API key with x-api-key or with Authorization: Bearer. The proxy uses that value as your Prem API key.

Before you start

Get these four things ready:

An app that lets you set a custom Anthropic base URL.
A Prem API key. This proves who you are. See API Keys.
A client Key Encryption Key (KEK). This is a master key that protects your other keys.
A model that your Prem account can use. See Models & Pricing.

The API key and the KEK are two different secrets. The API key handles sign-in, limits, and billing. The KEK protects your encryption keys. Keep both safe.

1. Set your secrets

Open a terminal. Set these values in the terminal that runs the proxy:

export PREM_API_KEY="your-prem-api-key"
export CLIENT_KEK="your-64-character-hex-kek"
export PROXY_URL="https://gateway.prem.io"
export ENCLAVE_URL="https://conf-engine.prem.io"

Do you not have a KEK yet? Make one, one time only:

openssl rand -hex 32

Store the KEK in a secret manager. Keep a backup. Use the same KEK every time. Do not make a new one each time you start the proxy.

PROXY_URL and ENCLAVE_URL show the current default endpoints. Check the Prem dashboard if the endpoints change.

2. Start the Confidential Proxy

Start the proxy in Anthropic mode. Keep this terminal open while you work:

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 127.0.0.1 \
  --port 8787 \
  --compat anthropic \
  --kek "$CLIENT_KEK"

The proxy now listens at http://127.0.0.1:8787. Your app adds /v1/messages to that base URL. Do you need one proxy for both Anthropic and OpenAI apps? Start it in both mode:

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 127.0.0.1 \
  --port 8787 \
  --compat both \
  --kek "$CLIENT_KEK"

In both mode, the Anthropic base URL is http://127.0.0.1:8787/anthropic. Your app then adds /v1/messages.

3. Point your app at the proxy

List available models:

curl http://127.0.0.1:8787/v1/models \
  -H "x-api-key: $PREM_API_KEY" \
  -H "anthropic-version: 2023-06-01"

Use a returned model ID in the Messages request. The Anthropic adapter returns the requested alias in its model field. It does not currently expose the backend-resolved model ID or an assurance profile. See Platform Status.

4. Test a message

Send a non-streaming request:

curl http://127.0.0.1:8787/v1/messages \
  -H "x-api-key: $PREM_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "max_tokens": 128,
    "messages": [{"role": "user", "content": "Reply with OK."}]
  }'

Test streaming after the first request succeeds:

curl -N http://127.0.0.1:8787/v1/messages \
  -H "Authorization: Bearer $PREM_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "max_tokens": 128,
    "stream": true,
    "messages": [{"role": "user", "content": "Count to three."}]
  }'

The stream uses Anthropic event names. These include message_start, content block events, message_delta, and message_stop.

5. Configure a client

Set the client’s API key and base URL. This example uses the Anthropic Node.js SDK:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.PREM_API_KEY,
  baseURL: "http://127.0.0.1:8787",
});

const response = await client.messages.create({
  model: "glm-5.2",
  max_tokens: 128,
  messages: [{ role: "user", content: "Reply with OK." }],
});

console.log(response.content);

Change glm-5.2 to a model your account can use. See your model IDs on the Prem dashboard. Each SDK joins the base URL and the route in its own way. Some clients want a host only. Some clients want a path that ends in /v1. Check your SDK.

Send `claude` to a Prem model

Some apps send the literal model name claude. You can map that one name to a Prem model:

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 127.0.0.1 \
  --port 8787 \
  --compat anthropic \
  --default-model glm-5.2 \
  --kek "$CLIENT_KEK"

The proxy swaps in glm-5.2 only when the model equals claude. It does not change other Claude model names.

This swap changes routing only. It does not make the Prem model behave the same as a Claude model.

What this setup protects

Component	Reads your text	Notes
Your app	Yes	It creates the prompt and reads the reply on your machine.
Anthropic adapter	Yes, on your machine	It translates requests and replies.
Confidential Proxy	Yes, on your machine	It seals requests and opens replies.
Prem API Gateway	No	It handles the sealed message and metadata only.
Prem API Enclave	Yes, inside the TEE	It opens the request, runs the model, and seals the reply.
External tools	Depends on the tool	Prem does not put them inside the enclave.

This setup protects the model path only. It does not put your app or its tools inside the enclave.

Good to know

Tool calls run on your machine. The model can return a tool_use block. Your app decides whether to run the tool. The tool runs outside the enclave. The next tool result enters the sealed path only after the proxy receives it. Apply your own access, approval, and logging rules to each tool. Token counts are estimates. The count_tokens route estimates tokens from the text. Use Prem usage records for billing and quota. One stream at a time. The encrypted endpoint allows one active stream for each API key. A second stream on the same key can return 429. Follow the Retry-After header when the response has one. See Agents & Automation.

Troubleshooting

The app returns 404

Check that the app adds /v1/messages. In both mode, check the /anthropic prefix.

The API says model not found

List your models with /v1/models. Use an enabled Prem model ID, or map the claude name.

Authentication fails

Send the Prem API key with x-api-key or Authorization: Bearer. Never send the KEK.

Token counts do not match billing

The count route returns an estimate. Use Prem usage records for billing and quota.

An Anthropic feature does nothing

The adapter does not support every Anthropic feature. For example, it ignores top_k.

Frequently asked questions

Does this run a Claude model?

No. The request uses an Anthropic-compatible shape. The Prem model you choose runs the request.

Why does the proxy translate the request?

Prem’s enclave uses an OpenAI-style chat format. The proxy translates your Messages request before it seals and sends it.

Are token counts exact?

No. The count route estimates tokens from the text and tool definitions.

Are tool calls private?

The model request and reply use the sealed path. Your app runs the tool outside the enclave.

Can one proxy serve OpenAI and Anthropic apps?

Yes. Use --compat both with the /openai and /anthropic prefixes.

Confidential Proxy

Review every mode, route, key, and daemon option.

Claude for Microsoft 365

Apply the Anthropic surface to the Microsoft 365 add-ins.

OpenAI-compatible clients

Use the Chat Completions interface through the same proxy.

Agents & Automation

Review retries, concurrency, reasoning, and tool controls.

Claude Code

Use the verified launcher for Claude Code.

Platform Status

Review current behavior and assurance gaps.

Get Started

Router

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

Anthropic-compatible clients

What the proxy does

How the connection works

What you can connect

Before you start

1. Set your secrets

2. Start the Confidential Proxy

3. Point your app at the proxy

4. Test a message

5. Configure a client

Send `claude` to a Prem model

What this setup protects

Good to know

Troubleshooting

Frequently asked questions

Does this run a Claude model?

Why does the proxy translate the request?

Are token counts exact?

Are tool calls private?

Can one proxy serve OpenAI and Anthropic apps?

Confidential Proxy

Claude for Microsoft 365

OpenAI-compatible clients

Agents & Automation

Claude Code

Platform Status

​What the proxy does

​How the connection works

​What you can connect

​Before you start

​1. Set your secrets

​2. Start the Confidential Proxy

​3. Point your app at the proxy

​4. Test a message

​5. Configure a client

​Send claude to a Prem model

​What this setup protects

​Good to know

​Troubleshooting

​Frequently asked questions

​Does this run a Claude model?

​Why does the proxy translate the request?

​Are token counts exact?

​Are tool calls private?

​Can one proxy serve OpenAI and Anthropic apps?

​Related

Confidential Proxy

Claude for Microsoft 365

OpenAI-compatible clients

Agents & Automation

Claude Code

Platform Status

What the proxy does

How the connection works

What you can connect

Before you start

1. Set your secrets

2. Start the Confidential Proxy

3. Point your app at the proxy

4. Test a message

5. Configure a client

Send `claude` to a Prem model

What this setup protects

Good to know

Troubleshooting

Frequently asked questions

Does this run a Claude model?

Why does the proxy translate the request?

Are token counts exact?

Are tool calls private?

Can one proxy serve OpenAI and Anthropic apps?

Related