Confidential AI API: Secure LLM Inference at Scale

The Confidential Proxy is a local termination proxy that comes with @premai/api-sdk. It exposes OpenAI and Anthropic compatible HTTP routes on your machine. If you use an OpenAI SDK or an OpenAI-compatible client, you do not need to change your application logic. Set the base URL to http://127.0.0.1:8000/v1. The same base URL applies to Anthropic clients.

If you use the TypeScript SDK, you do not need the Confidential Proxy.The SDK encrypts in the same process. Use the Confidential Proxy for other languages and for existing OpenAI or Anthropic codebases.

How it works

The Confidential Proxy runs on your machine. It does the same client-side encryption as the SDK. The Confidential Proxy encrypts your plaintext before the data leaves your machine. The Prem API Gateway receives only ciphertext. The enclave decrypts the request inside its Trusted Execution Environment (TEE). See Encryption for the full cryptographic design: the XWing key exchange, the two-server model, and the threat model.

Run the server

Run the Confidential Proxy directly with bunx or npx. No installation is necessary. As an alternative, install it globally:

# Run without installing (bun or npm)
bunx -p @premai/api-sdk confidential-proxy
npx -p @premai/api-sdk confidential-proxy

# Or install globally, then run (ensure your global bin dir is on your PATH)
npm i -g @premai/api-sdk   # or: bun i -g @premai/api-sdk
confidential-proxy

A prebuilt Docker image is available at ghcr.io/premai-io/confidential-proxy:latest:

docker pull ghcr.io/premai-io/confidential-proxy:latest

# OpenAI-compatible server on port 8787
docker run -p 8787:8787 \
  -e PROXY_URL=... -e ENCLAVE_URL=... \
  ghcr.io/premai-io/confidential-proxy:latest

# Extra arguments go to the CLI
docker run -p 8787:8787 ghcr.io/premai-io/confidential-proxy:latest --log-level debug

The image sets HOST=0.0.0.0 and PORT=8787. You can override these values with environment variables or CLI flags. Use CLI flags for --compat and --tls. You cannot set these two flags with environment variables. By default, the server listens on http://127.0.0.1:8000.

Set PROXY_URL and ENCLAVE_URL to the values for your environment. Get the latest values from dashboard.prem.io/endpoints.json.

Configuration

Configure the Confidential Proxy with environment variables or CLI flags. Flags have precedence.

Environment variables

Variable	Required	Default	Description
`ENCLAVE_URL`	Yes	-	The enclave endpoint that decrypts the data and runs inference
`PROXY_URL`	Yes	-	The Prem API Gateway endpoint that routes encrypted payloads
`CLIENT_KEK`	Yes	-	Your Key Encryption Key (KEK). It wraps the DEKs (32 bytes, base64)
`JSON_BODY_LIMIT`	No	`32mb`	The maximum size of the request body
`HOST`	No	`127.0.0.1`	The interface to bind
`PORT`	No	`8000`	The port to listen on
`CONFIDENTIAL_PROXY_LOG_LEVEL`	No	`info`	`error`, `warn`, `info`, `http`, `verbose`, `debug`, or `silly`
`PREM_API_KEY`	No	-	A default API key. The proxy uses this key when a client does not send its own key

Set PREM_API_KEY to a default API key. The proxy applies this key when a client does not supply its own key. Each client can also send its own API key with each request. Use Authorization: Bearer <key> for OpenAI routes. Use x-api-key: <key> for Anthropic routes. The Confidential Proxy keeps one client in memory for each API key.CLIENT_KEK is a separate key. The Confidential Proxy uses it only to wrap encryption keys. It is not an API key.

CLI options

All commands accept the same server options:

# Bind host / port
confidential-proxy --host 127.0.0.1 --port 8000

# Override backend endpoints
confidential-proxy --proxy-url https://gateway.prem.io --enclave-url https://conf-engine.prem.io

# Pass the client KEK inline
confidential-proxy --kek your-kek

# Raise the JSON body size limit
confidential-proxy --json-body-limit 64mb

Compatibility modes

Use --compat to select the API surface:

Mode	Routes	Description
`openai`	`/v1/*`	The OpenAI-compatible API only
`anthropic`	`/v1/*`	The Anthropic-compatible Messages API only
`both`	`/openai/v1/` and `/anthropic/v1/`	The two APIs together, each with a separate prefix

# OpenAI only (default surface)
confidential-proxy --compat openai

# Anthropic only
confidential-proxy --compat anthropic

# Both, with custom prefixes
confidential-proxy --compat both --openai-prefix /openai --anthropic-prefix /anthropic

In both mode, the Confidential Proxy serves the two APIs under separate prefixes. This prevents route conflicts. Set the base URL to http://127.0.0.1:8000/openai/v1 for OpenAI clients. Set it to http://127.0.0.1:8000/anthropic/v1 for Anthropic clients. The Anthropic surface translates each Anthropic Messages request into the internal OpenAI-compatible enclave pipeline. It then returns the response as Anthropic SSE events.

Connect a client

OpenAI

Set the base URL to http://127.0.0.1:8000/v1. If you use --compat both, set it to http://127.0.0.1:8000/openai/v1. Send your API key as a bearer token:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-pro",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

As an alternative, use the OpenAI SDK in Node.js:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.PREM_API_KEY!,
  baseURL: "http://127.0.0.1:8000/v1",
});

const stream = await client.chat.completions.create({
  model: "deepseek-v4-pro",
  messages: [{ role: "user", content: "Count to 10" }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

The same pattern applies to all other languages. This is a Python example:

from openai import OpenAI

client = OpenAI(
    api_key="your-api-key",
    base_url="http://127.0.0.1:8000/v1",
)

response = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[{"role": "user", "content": "Hello, privately."}],
)

print(response.choices[0].message.content)

Anthropic

Set the base URL to http://127.0.0.1:8000/v1. If you use --compat both, set it to http://127.0.0.1:8000/anthropic/v1. Authenticate with x-api-key. Send the anthropic-version header:

curl http://127.0.0.1:8000/v1/messages \
  -H "x-api-key: your-api-key" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-pro",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Add "stream": true for incremental responses:

curl -N http://127.0.0.1:8000/v1/messages \
  -H "x-api-key: your-api-key" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-pro",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Count to 10"}],
    "stream": true
  }'

The Anthropic surface supports system prompts, tool use, image inputs, stop sequences, temperature, and top_p. Streaming responses follow the Anthropic SSE format (message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop).

Hermes Agent

You can use the Confidential Proxy as a custom provider in Hermes. Start the proxy in openai mode:

confidential-proxy start --compat openai

Add this configuration to ~/.hermes/config.yaml:

custom_providers:
  - name: Prem Confidential API
    base_url: http://127.0.0.1:8000/openai/v1
    api_key: your-api-key
    model: deepseek-v4-pro

Restart Hermes. Hermes sends requests through the encrypted proxy at localhost:8000.

Claude for Microsoft 365

The Confidential Proxy can be the gateway for the Claude for Microsoft 365 add-in. The add-in uses HTTPS and CORS. Start the proxy with these flags:

confidential-proxy \
  --host 0.0.0.0 --port 8787 \
  --compat anthropic \
  --default-model deepseek-v4-pro \
  --cors-origin https://pivot.claude.ai \
  --tls --tls-cert ./cert.pem --tls-key ./key.pem

The add-in cannot connect to 127.0.0.1 on macOS. macOS prevents cross-origin browser requests to loopback addresses. Run the proxy on a different machine on your network, or use a container with its own network interface. Point the add-in at that hostname.

Run the proxy as a daemon

By default, the Confidential Proxy runs in the foreground. The CLI can also manage it as a background daemon.

Command	Description
`confidential-proxy`	Run in the foreground, attached to the terminal
`confidential-proxy start`	Start the server as a background daemon
`confidential-proxy stop`	Stop the daemon with a graceful shutdown
`confidential-proxy status`	Show if the daemon runs and is reachable

start

The start command does these steps:

It checks for a daemon that is in operation. It does not start a second daemon.
It spawns itself as a child process. It sends the logs to the log file that you configure.
It polls the HTTP endpoint until the server is reachable.
It then exits and the daemon continues to run.

stop

The stop command stops the daemon. It waits a maximum of 5 seconds. If the daemon does not stop, the command stops it immediately.

status

The status command checks if the process is alive and if the HTTP endpoint is reachable.

These options apply to the daemon commands (start, stop, status):

Option	Default	Description
`--pid-file`	`<data-dir>/proxy.pid`	A custom path for the PID file
`--log-file`	stdout/stderr	The file for the daemon logs (with `start`)
`--log-level`	`info`	The log verbosity (`error` … `silly`)
`--shutdown-timeout`	`30000`	The maximum time (ms) to wait for requests in progress during a graceful shutdown

# Start in the background, then confirm it's up
confidential-proxy start --compat openai
confidential-proxy status

# Stop it when you're done
confidential-proxy stop

Next steps

Quickstart

The step-by-step guide to get your first request working.

Developer Experience

The two ways to integrate: the TypeScript SDK and the Confidential Proxy.

Chat completions

The chat API in detail, with streaming and vision payloads.

Encryption

The key exchange and the end-to-end encryption in detail.

The same proxy operates confidential-claude, an integration that comes with the SDK. This integration starts Claude Code connected to the encrypted gateway. All traffic uses the Confidential Proxy.

Get Started

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

Confidential Proxy

How it works

Run the server

Configuration

Environment variables

CLI options

Compatibility modes

Connect a client

OpenAI

Anthropic

Hermes Agent

Claude for Microsoft 365

Run the proxy as a daemon

Next steps

Quickstart

Developer Experience

Chat completions

Encryption

​How it works

​Run the server

​Configuration

​Environment variables

​CLI options

​Compatibility modes

​Connect a client

​OpenAI

​Anthropic

​Hermes Agent

​Claude for Microsoft 365

​Run the proxy as a daemon

​Next steps

Quickstart

Developer Experience

Chat completions

Encryption

How it works

Run the server

Configuration

Environment variables

CLI options

Compatibility modes

Connect a client

OpenAI

Anthropic

Hermes Agent

Claude for Microsoft 365

Run the proxy as a daemon

Next steps