Confidential AI API: Secure LLM Inference at Scale

Attestation is hardware-signed evidence about a confidential-computing environment. A verifier combines that evidence with manufacturer collateral, freshness checks, and an explicit policy before deciding whether to trust a runtime.

Evidence verification and policy verification are different. A valid manufacturer signature proves that evidence came from the relevant hardware trust chain. It does not, by itself, prove that the measured software matches an approved Prem release.

Current implementation status

This table reflects @premai/reticle 0.5.1 and the reviewed Reticle source at commit 6024585.

Module	Implemented in the high-level client	Current limitation
AMD SEV-SNP	Generates a nonce, fetches the report and AMD certificate material, and verifies the report with the nonce	`attest_sev` still contains an explicit TODO for expected measurement comparison
Intel TDX	Generates a nonce, fetches the quote and Intel collateral, and runs quote verification with the nonce	`attest_tdx` still contains an explicit TODO for expected measurement comparison
NVIDIA GPU	Generates a nonce; verifies the overall and detached GPU JWTs; checks detached-token digests, nonce values, and the overall result; iterates the returned GPU claims	The public result does not expose per-GPU verdicts or prove that backend scheduling is bound to exactly that GPU set; the selected per-GPU check semantics also need code-owner review

The client discovers one CPU module (SEV-SNP or TDX) and an optional NVIDIA evidence bundle, then calls the corresponding high-level methods. The NVIDIA EAT token can contain multiple detached GPU claims even though module discovery reports only the NVIDIA vendor module.

Prem documentation and deployments target AMD SEV-SNP, Intel TDX, and NVIDIA confidential-computing GPUs. The exact hardware model, firmware, TCB, and policy acceptance decision must come from verified evidence and an explicit policy, not from a model alias.

What verified attestation policy means

A complete policy decision needs four distinct inputs:

Authenticity: the report or token verifies against manufacturer collateral.
Freshness: the evidence contains the verifier’s nonce.
Configuration: security-relevant claims meet required TCB, debug, and confidential-mode rules.
Identity: measured values match a release-specific approved reference set.

Reticle’s high-level methods perform evidence and freshness checks. The reviewed CPU paths do not yet complete step 4, so the client does not prove that the expected code is running or automatically match a published image fingerprint.

Where Reticle runs

Reticle is the verifier library, not a component that must run on the GPU cluster. The attestation endpoint runs with the confidential runtime to obtain hardware evidence. Reticle runs in the SDK or proxy process and verifies that evidence. The Rust implementation is also published as a WASM package for supported JavaScript environments.

How the SDK gates a request

With the default attest: true, @premai/api-sdk performs this sequence:

Builds a Reticle client against PROXY_URL and authenticates with the Prem API key.
Adds the requested model to the attestation query.
Calls the high-level client.attest() method.
Reads an x-session-id from the returned attestation headers.
Sends the encrypted inference request with that session ID.

The SDK caches a successful session ID for 30 seconds by API key and model and coalesces simultaneous attestation work. Transient attestation transport errors can be retried up to four attempts. A non-transient failure, or a missing session ID, throws before the protected inference request is sent.

attest: false and the proxy flag --no-attest bypass this gate. Do not use them for a workflow that claims attested confidential inference.

Routing and session pinning

Routing has two separate meanings:

Commercial/API routing: the gateway authenticates the API key, applies limits and billing metadata, and selects the service path for the requested alias while handling encrypted content.
Attestation-linked routing: the attestation response supplies a session ID that the SDK attaches to the encrypted inference request so the platform can associate it with the attested backend path.

The client source confirms model-scoped attestation and session-header forwarding. It does not prove every backend scheduling, load-balancing, or billing implementation detail. Billing is associated with the authenticated request; an attestation failure stops the inference request before completion, but final billing semantics must be confirmed from Prem usage records.

Model aliases are not assurance profiles

Model-list and completion types do not include assurance, attestation, tee, or provider-backed/Reticle-verifiable fields.

OpenAI-format and direct SDK completions can return a backend-resolved model ID in response.model.
The Anthropic adapter returns the requested alias in its model field.
Neither response is a substitute for an assurance profile.

Do not classify a model as provider-backed or Reticle-verifiable from its name. See Platform Status for the current contract gap.

Use the high-level Reticle API

Install the published package:

npm install @premai/reticle@0.5.1

Use attest() or a high-level component method:

import { ClientBuilder, QueryParams } from "@premai/reticle";

const apiKey = process.env.PREM_API_KEY;
const proxyUrl = process.env.PROXY_URL;
if (!apiKey || !proxyUrl) {
  throw new Error("Set PREM_API_KEY and PROXY_URL");
}

const client = await new ClientBuilder(proxyUrl)
  .with_authorization(apiKey)
  .build();

client.set_query(new QueryParams().with("model", "glm-5.2"));
const result = await client.attest();

console.log("Modules:", result.modules());
console.log("GPU session:", result.headers().gpu()?.get("x-session-id"));

The component methods are:

await client.attest_sev();
await client.attest_tdx();
await client.attest_nvidia();

Call only the method for hardware that the endpoint reports. client.attest() performs module discovery first.

Do not use request methods as verification

The published type definitions warn that these low-level methods request and parse evidence but do not perform cryptographic or measurement checks:

await client.request_sev(nonce);
await client.request_tdx(nonce);
await client.request_nvidia(nonce);

Use them only when you are implementing and testing your own complete verifier. A successful request_* call is not an attestation pass.

Multi-GPU attestation and load balancing

Multi-GPU inference can use several physical GPUs for one model backend. The inspected NVIDIA parser accepts multiple detached GPU JWTs, checks that overall-token submodule digests have matching detached tokens, verifies the returned JWT signatures, and applies nonce validation to each returned GPU claim. That is only part of a multi-GPU decision. The reviewed client does not expose the per-GPU claims or verdicts to the application, reject every possible extra detached token through an exact-set comparison, or demonstrate that the session and load balancer are bound to the appraised GPU set. The current CheckValidator implementation for selected per-GPU certificate fields also requires code-owner review because its boolean failure condition does not match the field names’ apparent meaning. End-to-end multi-GPU attestation is not established until the application can audit the exact GPU set and the serving path fails closed when topology or scheduling changes.

CPU technologies and other processors

AMD SEV-SNP and Intel TDX are confidential-VM technologies with remote-attestation formats and manufacturer collateral that Reticle implements. Other CPUs are not automatically insecure; they are unsupported by this verifier unless there is a corresponding confidential-computing architecture, evidence format, collateral path, and policy implementation. Support for one CPU family is significant because it lets a remote client authenticate hardware evidence and freshness for that family. It does not make all workloads, measurements, or firmware states acceptable without policy checks.

GPU technologies

NVIDIA confidential computing is hardware- and firmware-dependent. “Hopper” and “Blackwell” are architecture families, not sufficient policy inputs. A verifier must examine the actual signed token and claims for the deployed SKU and software stack. GPUs without a supported confidential-computing mode and attestation format cannot provide the same remote evidence through Reticle. They may still run inference, but the client cannot transfer NVIDIA confidential-computing claims to them.

Audit records

An audit record should contain non-sensitive decision metadata, such as:

request/support ID and time;
requested alias and resolved model ID when exposed;
verifier and policy version;
evidence type, hardware identity fields, and TCB result;
nonce/freshness result;
reference-set identifier and measurement result when implemented;
allow/deny decision and reason.

Do not record prompts, responses, API keys, KEKs, raw decrypted payloads, or secrets in an attestation audit log.

Known gaps

Expected CPU measurement comparison remains TODO in the reviewed client source.
The API does not expose a model assurance profile.
NVIDIA tokens can carry multiple GPU claims, but exact-set appraisal, exposed per-GPU verdicts, and scheduler binding are not established end to end.
Reproducible release images and a public release-to-measurement mapping are separate requirements.
Package and runtime version reporting currently disagree for the Confidential Proxy.

These findings apply to the reviewed package and source versions. Changes to Reticle, the SDK, endpoints, or policy require new source and live validation.

Get Started

Router

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

Attestation

Current implementation status

What verified attestation policy means

Where Reticle runs

How the SDK gates a request

Routing and session pinning

Model aliases are not assurance profiles

Use the high-level Reticle API

Do not use request methods as verification

Multi-GPU attestation and load balancing

CPU technologies and other processors

GPU technologies

Audit records

Known gaps

​Current implementation status

​What verified attestation policy means

​Where Reticle runs

​How the SDK gates a request

​Routing and session pinning

​Model aliases are not assurance profiles

​Use the high-level Reticle API

​Do not use request methods as verification

​Multi-GPU attestation and load balancing

​CPU technologies and other processors

​GPU technologies

​Audit records

​Known gaps

Current implementation status

What verified attestation policy means

Where Reticle runs

How the SDK gates a request

Routing and session pinning

Model aliases are not assurance profiles

Use the high-level Reticle API

Do not use request methods as verification

Multi-GPU attestation and load balancing

CPU technologies and other processors

GPU technologies

Audit records

Known gaps