Confidential AI API: Secure LLM Inference at Scale

Hermes Agent supports named custom providers that use the OpenAI Chat Completions protocol. Point one of those providers at Prem’s local Confidential Proxy.

Do not configure Prem under Hermes’ openai-api provider. That provider can select the OpenAI Responses API, which the Confidential Proxy does not expose. Use a named custom provider with transport: chat_completions.

Confidential API
Router (Beta)

Use the local Confidential Proxy for sensitive model traffic. Go to the confidential setup.

Confidential API

How the connection works

Before you start

You need:

Hermes Agent installed. See the Hermes Agent repository.
A Prem API key. See API Keys.
A 32-byte client KEK encoded as 64 hexadecimal characters.
The current Prem gateway and enclave endpoints.
A chat model returned by the Prem model-list endpoint.

Check the installed version:

hermes --version

The configuration below has been exercised with Hermes Agent 0.20.0 using a one-shot response and a local file-reading tool call.

1. Set your secrets

export PREM_API_KEY="your-prem-api-key"
export CLIENT_KEK="your-64-character-hex-kek"
export PROXY_URL="https://gateway.prem.io"
export ENCLAVE_URL="https://conf-engine.prem.io"

Generate the KEK once if you do not have one:

openssl rand -hex 32

Keep PREM_API_KEY out of config.yaml. Hermes resolves it from the environment variable named by key_env.

2. Start the Confidential Proxy

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 127.0.0.1 \
  --port 8787 \
  --compat openai \
  --kek "$CLIENT_KEK"

Do not use --no-attest. Keep attestation enabled for confidential inference.

3. Select an enabled model

curl http://127.0.0.1:8787/v1/models \
  -H "Authorization: Bearer $PREM_API_KEY"

Use a returned id. This example uses qwen36-27b:

curl http://127.0.0.1:8787/v1/chat/completions \
  -H "Authorization: Bearer $PREM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen36-27b",
    "messages": [{"role": "user", "content": "Reply with OK."}],
    "stream": false
  }'

Do not continue until this request succeeds.

4. Configure Hermes

Add this provider to ~/.hermes/config.yaml:

providers:
  prem-confidential:
    api: http://127.0.0.1:8787/v1
    key_env: PREM_API_KEY
    transport: chat_completions
    default_model: qwen36-27b
    models:
      qwen36-27b:
        context_length: 131072

model:
  default: qwen36-27b
  provider: custom:prem-confidential
  context_length: 131072

Replace the model ID and context length when you choose another enabled model. Keep transport: chat_completions explicit. The keyed providers format is Hermes’ current custom-provider schema. See the Hermes custom-provider documentation. Check the configuration:

hermes config check

5. Run Hermes

Start with a bounded one-shot request:

hermes --oneshot "Reply with exactly OK. Do not use tools."

Then start an interactive session:

hermes chat

Hermes can stream responses and perform tool calls through this provider. The model request and tool-call response use the encrypted model channel. Hermes runs the selected tool on your machine, then sends any resulting context in a later encrypted model request.

Tool, memory, and gateway boundary

Component	Plaintext access	Notes
Hermes Agent	Yes	Reads prompts, memory, rules, workspace data, and tool results
Confidential Proxy	Yes, locally	Encrypts model requests and decrypts responses
Prem API Gateway	No content access	Receives ciphertext and operational metadata
Prem confidential runtime	Yes, inside the protected runtime	Runs inference and encrypts the result
Shell, skills, messaging gateways, and MCP servers	Depends on the integration	Remain outside the Prem confidential boundary

Hermes can connect to messaging channels and run scheduled work. Review sender allowlists, approval rules, toolsets, hooks, and outbound network access before enabling those surfaces.

Agent limits

The encrypted chat endpoint permits one active stream for each API key. Parallel Hermes sessions, delegation, fallback attempts, or messaging workers that share one key can receive 429 responses. Begin with one session and no delegation. Serialize turns or use a separate Prem API key per concurrent worker. Do not configure a non-Prem fallback if every model turn must use Prem’s encrypted path.

Troubleshooting

Hermes calls /v1/responses

The session is using the wrong provider. Set model.provider to custom:prem-confidential and keep transport: chat_completions in the named provider.

Hermes reports a missing API key

Export PREM_API_KEY in the process that starts Hermes. Confirm that key_env has the exact same name.

The API returns model not found

Call /v1/models. Use a returned ID in default_model, providers.prem-confidential.models, and model.default.

A tool can read more than expected

Limit Hermes toolsets, workspace access, approval settings, hooks, and MCP servers. Model-path confidentiality does not restrict local tool permissions.

The API returns 429

Wait for the active stream to finish. Reduce delegation and parallel workers or use distinct API keys.

Frequently asked questions

Is Hermes itself running inside the enclave?

No. Hermes runs locally. The Prem confidential boundary starts after the local proxy encrypts a supported model request.

Why not use `OPENAI_BASE_URL` with the built-in OpenAI provider?

That route can use /v1/responses. The named custom provider fixes the protocol to /v1/chat/completions, which the Confidential Proxy supports.

Can Hermes tools use the encrypted connection?

The model can request a tool through the encrypted model channel. Hermes executes the tool outside the enclave. A tool’s own network traffic does not pass through Prem unless you configure that tool separately.

Can Hermes use fallback providers?

Yes, but a non-Prem fallback leaves the Prem inference path. Omit those fallbacks when the encrypted route is a requirement.

Router

Beta

Router is not confidential. Use this path only for non-sensitive prompts, memory, workspace context, and tool results.

Hermes uses Router through a second named provider.

export PREM_ROUTER_API_KEY="your-router-api-key"

Confirm that kimi-k3 appears in GET /v1/models for this key. Otherwise, use an exact returned model ID. See Router models. Use the prem-router configuration in the Router Hermes configuration:

Field	Value
`api`	`https://router.prem.io/v1`
`key_env`	`PREM_ROUTER_API_KEY`
`transport`	`chat_completions`
`default_model`	`kimi-k3`
`model.provider`	`custom:prem-router`

Run hermes config check after switching. Do not reuse PREM_API_KEY or the client KEK for Router.

Confidential Proxy

Review proxy modes, routes, keys, and daemon controls.

OpenAI-compatible clients

Understand the protocol used by Hermes.

Agents & Automation

Review concurrency, retries, and unattended operation.

Security Model

Review the protected and unprotected parts of the flow.

Get Started

Router

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

Hermes Agent

Confidential API

How the connection works

Before you start

1. Set your secrets

2. Start the Confidential Proxy

3. Select an enabled model

4. Configure Hermes

5. Run Hermes

Tool, memory, and gateway boundary

Agent limits

Troubleshooting

Frequently asked questions

Is Hermes itself running inside the enclave?

Why not use `OPENAI_BASE_URL` with the built-in OpenAI provider?

Can Hermes tools use the encrypted connection?

Can Hermes use fallback providers?

Router

Confidential Proxy

OpenAI-compatible clients

Agents & Automation

Security Model

​Confidential API

​How the connection works

​Before you start

​1. Set your secrets

​2. Start the Confidential Proxy

​3. Select an enabled model

​4. Configure Hermes

​5. Run Hermes

​Tool, memory, and gateway boundary

​Agent limits

​Troubleshooting

​Frequently asked questions

​Is Hermes itself running inside the enclave?

​Why not use OPENAI_BASE_URL with the built-in OpenAI provider?

​Can Hermes tools use the encrypted connection?

​Can Hermes use fallback providers?

​Router

​Related

Confidential Proxy

OpenAI-compatible clients

Agents & Automation

Security Model

Confidential API

How the connection works

Before you start

1. Set your secrets

2. Start the Confidential Proxy

3. Select an enabled model

4. Configure Hermes

5. Run Hermes

Tool, memory, and gateway boundary

Agent limits

Troubleshooting

Frequently asked questions

Is Hermes itself running inside the enclave?

Why not use `OPENAI_BASE_URL` with the built-in OpenAI provider?

Can Hermes tools use the encrypted connection?

Can Hermes use fallback providers?

Router

Related