Confidential AI API: Secure LLM Inference at Scale

The Claude add-ins for Microsoft 365 can send inference to Prem. Prem runs the model inside a confidential enclave. The add-ins keep their normal look and workflow. Your organization controls the connection. This guide helps you set up that connection. It uses the LLM gateway path from Anthropic. Prem provides the gateway software: the Confidential Proxy. A Microsoft 365 tenant has not been tested end to end with this setup. Complete the acceptance tests before production rollout.

Anthropic documents four connection paths for the Office add-ins: LLM gateway, Bedrock direct, Vertex AI direct, and Foundry direct. This guide covers the LLM gateway path only. Read the Anthropic third-party platforms guide first. It is the source of truth for tenant setup, network allowlists, and feature support.

Who this guide is for

Two roles complete this setup:

An IT administrator deploys the add-in and runs the gateway. Sections 1 to 8 are for this role.
An end user connects the add-in in their Office application. The End-user connection section is for this role.

You can do both roles yourself for a small pilot.

What you set up

The add-in talks to a gateway that you run. The gateway is the Prem Confidential Proxy. The proxy encrypts each request before it leaves your network. The Prem enclave decrypts the request, runs the model, and encrypts the response. The add-in and the proxy host read plaintext. The Prem API Gateway sees only ciphertext. The Prem Enclave reads plaintext inside its Trusted Execution Environment (TEE).

What this integration does

Users open the Claude task pane in these applications:

Excel
PowerPoint
Word
Outlook

The add-in sends its model requests to your Prem gateway. Prem runs the model with confidential inference.

What this integration does not do

This setup does not do these things:

It does not set up Microsoft Copilot.
It does not protect the full Microsoft 365 workflow. It protects the model request only.
It does not make a Prem model behave the same as a Claude model.
It does not protect Microsoft Graph calls, connectors, or local add-in storage.
It does not enable work across applications. Anthropic lists this feature as unavailable through third-party platforms.

Check the current feature support table before each rollout. The list changes.

Before you start

Get these items ready:

Microsoft 365 with Microsoft Entra ID for admin consent.
Claude for Excel, PowerPoint, Word, or Outlook, installed from Microsoft AppSource or by admin deployment.
A gateway hostname that every approved device can reach.
A TLS certificate that those devices trust. For a local pilot on macOS, you create one with mkcert.
Docker Desktop for a local pilot on macOS, or Node.js for a server.
A dedicated Prem API key.
A 32-byte KEK encoded as 64 hexadecimal characters.
The current Prem gateway and enclave endpoints.
A Prem model that supports your workflow.

For Outlook, a Global Administrator grants Microsoft Graph consent one time. The Graph token stays in the user’s Outlook client. It never reaches your gateway or Prem. See the Anthropic Outlook guide for the consent step.

1. Choose the gateway hostname

The add-in cannot use localhost or 127.0.0.1 on macOS. So the gateway needs a real hostname.

Choose a hostname, for example prem-office-gateway.example.com.
Point it at the gateway host’s network IP address, not 127.0.0.1.
Add the hostname to your local DNS, or to the /etc/hosts file on each approved device.
Confirm that every approved device can resolve the hostname.

Use the same hostname everywhere. It must match the certificate and the URL that users enter.

2. Create a trusted certificate

The task pane runs in a browser. It requires an HTTPS gateway with a trusted certificate. For a local pilot on macOS, create the certificate with mkcert:

brew install mkcert
mkcert -install
mkcert prem-office-gateway.example.com

mkcert -install adds a local certificate authority to the Mac’s trust store. mkcert then writes two files in the current folder:

prem-office-gateway.example.com.pem       # the certificate
prem-office-gateway.example.com-key.pem   # the private key

To trust the certificate on other devices, install the mkcert root certificate on each one. Find it with mkcert -CAROOT. For a wider rollout, use an organization-issued or public certificate instead. Keep the private key safe. Do not commit it to a repository.

3. Set the gateway secrets

Set these values on the gateway host:

export PREM_API_KEY="your-dedicated-prem-api-key"
export CLIENT_KEK="your-64-character-hex-kek"
export PROXY_URL="https://gateway.prem.io"
export ENCLAVE_URL="https://conf-engine.prem.io"

Store the API key, KEK, certificate key, and endpoints in a secret manager. Do not commit them to a repository.

4. Start the gateway

The gateway software is the confidential-proxy command in the @premai/api-sdk npm package. The package is an end-to-end encrypted client that speaks both the OpenAI and the Anthropic API formats. Local pilot on macOS (Docker): run the proxy in a container. Run this command from the folder that holds your certificate files:

docker run --rm -p 8787:8787 \
  -e PREM_API_KEY -e CLIENT_KEK -e PROXY_URL -e ENCLAVE_URL \
  -v "$(pwd):/tls:ro" \
  node:22-alpine \
  npx -p @premai/api-sdk@1.0.59 confidential-proxy \
    --host 0.0.0.0 --port 8787 \
    --compat anthropic \
    --kek "$CLIENT_KEK" \
    --default-model glm-5.2 \
    --cors-origin https://pivot.claude.ai \
    --tls \
    --tls-cert /tls/prem-office-gateway.example.com.pem \
    --tls-key /tls/prem-office-gateway.example.com-key.pem

Server (Node.js): run the proxy directly with npx:

npx -p @premai/api-sdk@1.0.59 confidential-proxy \
  --host 0.0.0.0 --port 8787 \
  --compat anthropic \
  --kek "$CLIENT_KEK" \
  --default-model glm-5.2 \
  --cors-origin https://pivot.claude.ai \
  --tls \
  --tls-cert /path/to/cert.pem \
  --tls-key /path/to/key.pem

The proxy now serves the add-in at https://prem-office-gateway.example.com:8787. The options mean:

--compat anthropic makes the proxy speak the Anthropic Messages API. The add-in uses this format.
--default-model is built for the Claude for Microsoft 365 integration. It replaces the literal claude request value with a Prem model. Change glm-5.2 to the Prem model you want.
--cors-origin lets the task pane read the responses. The task pane loads from https://pivot.claude.ai.

Do not use --cors-origin *. CORS is not authentication. An exact origin limits unwanted browser access.

5. Restrict network access

Do not expose an unauthenticated proxy to the public internet. Allow inbound traffic to port 8787 only from approved networks or your reverse proxy. Choose a topology that fits the rollout:

Topology	Use it for	Required controls
Dedicated LAN host	A small pilot on one managed network	Internal DNS, trusted TLS, firewall, dedicated key
Private VM or container	A managed organization deployment	Private routing, health checks, key storage, updates
Internet-reachable gateway	Users without private network access	Reverse proxy, strict authentication, firewall, monitoring

For an internet-reachable deployment, put an authenticated gateway in front of the proxy. Apply request limits, authentication, and alerts there.

6. Deploy the add-in

Anthropic provides a setup wizard for tenant deployment. The wizard generates the add-in manifest and handles admin consent. Install the plugin in your shell:

claude plugin marketplace add anthropics/financial-services
claude plugin install claude-for-msft-365-install@claude-for-financial-services

Run the wizard inside Claude:

/claude-for-msft-365-install:setup

Choose the LLM gateway path when the wizard asks. Give it these values:

Gateway URL: the base URL of your proxy, for example https://prem-office-gateway.example.com:8787.
Gateway token: the dedicated Prem API key.

The wizard writes a manifest.xml file. For Outlook, it also writes manifest-outlook.xml. Upload each manifest in the Microsoft 365 Admin Center. Open Settings → Integrated apps → Upload custom apps. Assign the add-in to a pilot group first. Widen the assignment after the pilot passes.

By default, the add-in sends the token in the x-api-key header. To use Authorization: Bearer instead, set gateway_auth_header: authorization in the manifest. See the Anthropic gateway requirements for all manifest options.

End-user connection

Give these steps to each user. They connect the add-in one time.

Open Excel, PowerPoint, Word, or Outlook.
Launch the Claude add-in.
On the sign-in screen, select Cloud provider or gateway.
Select Gateway.
Enter the gateway URL, for example https://prem-office-gateway.example.com:8787.
Enter the Prem API key as the token.
Select Connect.

Enter the base URL only. Do not add /v1/messages to the URL. The add-in stores the token in the browser’s local storage, inside its sandboxed frame. To change the token later, open Settings in the add-in sidebar, enter the new value, and test the connection.

Validate the rollout

Test each approved application on its own. Use non-sensitive test files first.

Open a small range, slide, paragraph, or email.
Ask for a short model response.
Apply one reversible edit through the add-in.
Confirm that streaming finishes.
Confirm that the Prem usage record appears.
Confirm that an invalid token returns 401.
Confirm that an unapproved user or network cannot reach the gateway.
Confirm that a certificate error blocks the connection.

Do not widen the pilot until every test passes.

Responsibility boundaries

Layer	Owner	Responsibility
Add-in deployment	Microsoft 365 administrator	Assign add-ins, users, groups, and permissions
Identity and consent	Microsoft Entra ID administrator	Approve sign-in and Microsoft Graph permissions
Office content access	Add-in and Microsoft 365	Read or change selected content
Gateway host	Your organization	Run TLS, CORS, proxy, firewall, keys, and monitoring
Confidential Proxy	Your organization and Prem package	Translate, encrypt, decrypt, and return traffic
Prem API Gateway	Prem	Authenticate, limit, bill, and route encrypted payloads
Prem API Enclave	Prem	Decrypt, run inference, and encrypt the response

Security boundary

Component	Reads plaintext	Note
Microsoft 365 application	Yes	Holds the source document or message
Claude task pane	Yes	Reads selected content and stores the token locally
Microsoft Entra ID and Graph	Depends on tenant	Outside the Prem inference boundary
Confidential Proxy host	Yes	Translates, encrypts, decrypts, and holds local secrets
Prem API Gateway	No	Processes ciphertext, authentication, and metadata
Prem API Enclave	Yes, inside the TEE	Runs the selected Prem model
Connectors and external tools	Depends on the feature	Need a separate data-flow review

Anthropic states that prompts and responses go only to your gateway. The add-in loads its interface and telemetry from Anthropic and Microsoft domains. That traffic does not carry prompts or responses.

The add-in stores its token in browser localStorage, inside the sandboxed task pane. The current Prem proxy uses that token as the Prem API key. Treat each device and browser profile as a key-bearing asset. For short-lived user tokens, run a separate authenticated gateway in front of the proxy.

Operational controls

Apply these controls for a managed rollout:

Pin the @premai/api-sdk package version.
Run the proxy as a supervised service.
Monitor certificate expiry and endpoint health.
Rotate the Prem API key and KEK under a written procedure.
Turn off debug logs in production unless support needs them.
Keep document content out of support bundles.
Record add-in assignments, gateway changes, and key rotations.
Test rollback before each package or manifest update.

Troubleshooting

The add-in reports connection refused

Check DNS, firewall rules, gateway health, and TLS trust. Confirm the base URL. Do not include /v1/messages.

The browser reports a CORS error

Confirm the response includes the allowed origin on OPTIONS, GET, POST, and error responses.

The add-in reports no models

Test GET /v1/models with the same token. Confirm the Prem API key can list an enabled model.

The request uses an unknown Claude model

Send claude as the model, or send a supported Prem model ID. The default substitution matches only claude.

The token expired or was rotated

Update the token in the add-in Settings. Test the connection again.

Streaming hangs

Confirm that firewalls and reverse proxies do not buffer Server-Sent Events. Test the direct gateway path.

Frequently asked questions

Is this Microsoft Copilot?

No. This connects Anthropic’s Claude add-ins for Microsoft 365 to the Prem gateway.

Does Prem run a Claude model in this setup?

Not always. The add-in uses the Anthropic API format. The proxy maps the claude placeholder to a Prem model.

Why can the add-in not use localhost on macOS?

The add-in runs in a sandboxed browser frame. It cannot use the loopback path. Use a reachable gateway host with trusted TLS.

Does CORS protect the API key?

No. CORS controls which browser origin can read responses. Authentication, TLS, device controls, and firewall rules protect access.

Where does the add-in store the token?

The add-in stores the token in browser localStorage, inside its sandboxed task pane. Treat the device and browser profile as key-bearing assets.

Does this protect the full document lifecycle?

No. It protects the model request after the proxy encrypts it. Microsoft 365 and local add-in processing stay separate.

Does work across Microsoft 365 applications work through this gateway?

No. Anthropic lists this feature as unavailable through third-party platforms. Check the limitation before each rollout.

Anthropic third-party platforms

Read Anthropic’s source guide for tenant setup and feature support.

Anthropic-compatible clients

Understand the Messages adapter and its compatibility limits.

Confidential Proxy

Review every proxy option and daemon control.

Security Model

Review the protected and unprotected parts of the data path.

Production Checklist

Complete key, attestation, reliability, and support checks.

Platform Status

Review current behavior and the untested tenant boundary.

Get Started

Router

Core Concepts

Handling Errors

Guides

API Reference

Billing

Support

Claude for Microsoft 365

Who this guide is for

What you set up

What this integration does

What this integration does not do

Before you start

1. Choose the gateway hostname

2. Create a trusted certificate

3. Set the gateway secrets

4. Start the gateway

5. Restrict network access

6. Deploy the add-in

End-user connection

Validate the rollout

Responsibility boundaries

Security boundary

Operational controls

Troubleshooting

Frequently asked questions

Is this Microsoft Copilot?

Does Prem run a Claude model in this setup?

Why can the add-in not use localhost on macOS?

Does CORS protect the API key?

Where does the add-in store the token?

Does this protect the full document lifecycle?

Does work across Microsoft 365 applications work through this gateway?

Anthropic third-party platforms

Anthropic-compatible clients

Confidential Proxy

Security Model

Production Checklist

Platform Status

​Who this guide is for

​What you set up

​What this integration does

​What this integration does not do

​Before you start

​1. Choose the gateway hostname

​2. Create a trusted certificate

​3. Set the gateway secrets

​4. Start the gateway

​5. Restrict network access

​6. Deploy the add-in

​End-user connection

​Validate the rollout

​Responsibility boundaries

​Security boundary

​Operational controls

​Troubleshooting

​Frequently asked questions

​Is this Microsoft Copilot?

​Does Prem run a Claude model in this setup?

​Why can the add-in not use localhost on macOS?

​Does CORS protect the API key?

​Where does the add-in store the token?

​Does this protect the full document lifecycle?

​Does work across Microsoft 365 applications work through this gateway?

​Related

Anthropic third-party platforms

Anthropic-compatible clients

Confidential Proxy

Security Model

Production Checklist

Platform Status

Who this guide is for

What you set up

What this integration does

What this integration does not do

Before you start

1. Choose the gateway hostname

2. Create a trusted certificate

3. Set the gateway secrets

4. Start the gateway

5. Restrict network access

6. Deploy the add-in

End-user connection

Validate the rollout

Responsibility boundaries

Security boundary

Operational controls

Troubleshooting

Frequently asked questions

Is this Microsoft Copilot?

Does Prem run a Claude model in this setup?

Why can the add-in not use localhost on macOS?

Does CORS protect the API key?

Where does the add-in store the token?

Does this protect the full document lifecycle?

Does work across Microsoft 365 applications work through this gateway?

Related