Confidential AI API: Secure LLM Inference at Scale

This page is for anyone building a system that calls Prem API without a human watching each call: agent loops, batch jobs, evaluation harnesses, or any tool that fires many requests unattended. Unattended callers hit different failure modes than an interactive app. A truncated response, a burned rate-limit budget, or a duplicated retry may not be noticed until it has already happened many times.

If you want an AI coding agent (Claude Code, Cursor, Windsurf) to help you write this integration, see Use LLMs. It loads the Prem API docs into your coding agent’s context. This page is about the system you build calling Prem API, not the coding agent that helps you build it.

Pick your integration

	TypeScript SDK	Confidential Proxy	ZDR
Best for	Node.js agents, single-process automation	Polyglot agent frameworks (LangChain, Python/Go agent loops), anything already using an OpenAI or Anthropic SDK	OpenAI-compatible tools that need the ZDR model catalog
Encryption	In-process	A local HTTP process encrypts/decrypts for you	Not confidential
Code changes	Use `@premai/api-sdk` instead of `openai`	Point `baseURL` at the proxy	Point a Chat Completions client at `https://gateway.prem.io/openai`

The TypeScript SDK and Confidential Proxy are the confidential paths. ZDR does not provide enclave confidentiality. Use ZDR only when its model catalog is more important than confidential processing.

See Developer Experience for both, in detail. For an agent that runs continuously, run the Confidential Proxy as a background daemon rather than in the foreground:

confidential-proxy start --compat openai --kek "$CLIENT_KEK"
confidential-proxy status

See Confidential Proxy for the full daemon lifecycle (start / stop / status, PID and log files, graceful shutdown).

Required configuration

Both confidential integration paths need the same four values. Set them once in the environment where the agent runs, not for each call:

Variable	Used by	Description
`PREM_API_KEY`	Both	Your API key. See API Keys.
`CLIENT_KEK`	Both	A 32-byte key encoded as 64 hexadecimal characters. You generate and keep it. See Encryption.
`PROXY_URL`	Both	The Prem API Gateway endpoint.
`ENCLAVE_URL`	Both	The enclave endpoint.

Get the current PROXY_URL / ENCLAVE_URL values from dashboard.prem.io/endpoints.json at deploy time rather than hardcoding them. ZDR uses the same PREM_API_KEY at https://gateway.prem.io/openai. See ZDR overview for the ZDR setup.

Gotchas that only show up at automation scale

These are easy to miss in a one-off script and expensive to miss in a loop that runs thousands of times.

Reasoning is on by default and is billed

Reasoning models generate a chain-of-thought before every answer unless you turn it off. Those tokens are billed, count against your rate limit, and share the same inference’s token budget as the visible answer. Reasoning can consume a tight max_completion_tokens value and leave message.content: null.Default to the lowest reasoning_effort value that the model family accepts in automated pipelines unless you explicitly consume the reasoning trace. See Models & Pricing for the values that each family accepts.

Concurrent requests have a limit for each API key

Prem limits the number of inference requests that one API key can process at the same time. The limit applies to every inference endpoint. A call above the limit returns 429 with:

{"status": 429, "error": "Concurrent request limit reached. Your limit allows 5 concurrent request(s). Wait for ongoing requests to complete, or contact us for higher limits."}

The limit is most visible on the encrypted /rvenc/chat/completions endpoint, which the TypeScript SDK uses. That endpoint always responds over text/event-stream, even when you do not set stream: true. A call to it holds its slot until the stream closes. See Tiers. If your agent sends calls in parallel on one key, send them in sequence or supply a different API key to each worker.

429 responses don't all look the same

Some 429 responses have a structured body and a Retry-After header. Others, such as a rate limit upstream of Prem, return a generic body with no Retry-After header. Branch on the HTTP status code, not the body shape. Use your own exponential backoff when the header is not there. See Rate limits.

Retries need to be idempotent

An agent that retries blindly after a timeout or a 5xx can duplicate the underlying action (a file upload, a resource creation). Send an Idempotency-Key header so a retried request has the same effect as the original. See Idempotency.

Check finish_reason and content

A null or empty message.content is not necessarily a failed call. If finish_reason is "length", reasoning, a long answer, or both consumed the token budget. Treat it as truncation and retry with a larger max_completion_tokens or a lower reasoning_effort, not as an empty result to discard.

A request shaped for unattended use

import createRvencClient from "@premai/api-sdk";

const client = await createRvencClient({
  apiKey: process.env.PREM_API_KEY,
  clientKEK: process.env.CLIENT_KEK,
});

const response = await client.chat.completions.create({
  model: "qwen38-27b",
  messages: [{ role: "user", content: "Summarize this log file." }],
  reasoning_effort: "low",       // reduce reasoning unless you consume it
  max_completion_tokens: 4096,    // headroom for the answer
}, {
  headers: { "Idempotency-Key": crypto.randomUUID() }, // safe to retry
});

const choice = response.choices[0];
if (choice.finish_reason === "length") {
  // Truncated: raise max_completion_tokens or retry; do not treat as empty.
}
console.log(choice.message.content);

Before you deploy

reasoning_effort set deliberately (the lowest value the model family accepts, or a value you’ve budgeted max_completion_tokens for)
Retries use exponential backoff and don’t assume Retry-After is present
Idempotency keys on any retried write
Concurrent calls on the same API key are serialized, or spread across multiple keys
Every response checks finish_reason before treating content as final
support_id is logged on every error for support escalation

This checklist is agent-specific. For the full production checklist (key management, tiers, attestation), see Production Checklist.

Tiers

The request, token, and audio limits of each tier.

Rate limits

Retry code, the 429 shapes, and non-standard 429 bodies.

Idempotency

Make retries safe with the Idempotency-Key header.

Confidential Proxy

Daemon mode, config, and connecting any language.

OpenCode

Connect OpenCode to Prem API through the local proxy.

OpenClaw

Configure a Prem-backed OpenClaw provider and its tool boundary.

Hermes Agent

Use an explicit Chat Completions transport through Prem.

Goose

Connect Goose’s built-in OpenAI provider to Prem.

Cursor compatibility

Review why Cursor’s provider override is not a confidential local path.

Codex CLI compatibility

Review the current Responses API protocol mismatch.

​Pick your integration

​Required configuration

​Gotchas that only show up at automation scale

​A request shaped for unattended use

​Before you deploy

​Related

Tiers

Rate limits

Idempotency

Confidential Proxy

OpenCode

OpenClaw

Hermes Agent

Goose

Cursor compatibility

Codex CLI compatibility

Pick your integration

Required configuration

Gotchas that only show up at automation scale

A request shaped for unattended use

Before you deploy

Related