Confidential AI API: Secure LLM Inference at Scale

Decisions answers structured questions about a piece of context. You send a state and a set of typed questions. You get one probability distribution per question. The current catalogue exposes one decision model: dgemma. Decisions runs in Zero Data Retention mode. Your content is plaintext at the Prem API Gateway and at the partner. There is no client-side encryption and no attestation. See ZDR overview and ZDR security boundary.

Basic decision

Ask a routing question about an incident:

curl https://gateway.prem.io/typesafe/v1/systemone \
  -H "Authorization: Bearer $PREM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dgemma",
    "state": "All users see HTTP 503 and cannot sign in.",
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Payments and invoices",
          "technical": "Software failures",
          "sales": "New purchases"
        }
      }
    }
  }'

The response carries one answer per question id:

{
  "model": "dgemma",
  "answers": {
    "team": {
      "type": "choice",
      "choice": "technical",
      "probabilities": {
        "billing": 0.019,
        "technical": 0.981,
        "sales": 0.000
      },
      "confidence": 0.981
    }
  },
  "usage": {
    "input_tokens": 109,
    "output_tokens": 8
  }
}

choice is the most likely label. probabilities holds one value per criterion key. confidence rescales the top probability against a uniform distribution. It is not a calibrated correctness estimate.

Question types

Each question declares a type and a criteria shape that matches the type.

`choice`

Pick one label from a named list. criteria maps each label to a description. The list holds 1 to 26 entries.

{
  "topic": {
    "type": "choice",
    "instructions": "Which topic does this cover?",
    "criteria": {
      "billing": "Payments and invoices",
      "technical": "Software failures",
      "sales": "New purchases"
    }
  }
}

`noul`

Answer yes or no. The answer is a single probability. criteria is optional and can name the two outcomes.

{
  "delivered": {
    "type": "noul",
    "instructions": "Was the package delivered?"
  }
}

The answer shape is a scalar in noul:

{
  "delivered": { "type": "noul", "noul": 0.02 }
}

Do not read probabilities for a noul question. That field is not part of this answer.

`score`

Rate on an ordered scale. criteria lists the levels in ascending order, from 2 to 10 levels.

{
  "severity": {
    "type": "score",
    "instructions": "Rate incident severity.",
    "criteria": ["low", "medium", "high"]
  }
}

The answer carries score, legend, probabilities, and confidence. The score is the probability-weighted mean over the zero-indexed levels.

Several questions in one request

Each question is independent. One request can carry up to 16 questions. The questions share the state, and each returns its own distribution.

{
  "model": "dgemma",
  "state": "Refunds are allowed within 30 days unless the item is final sale. Bought 10 days ago, final sale.",
  "questions": {
    "refund": {
      "type": "noul",
      "instructions": "Is a refund allowed?"
    },
    "severity": {
      "type": "score",
      "instructions": "Rate how clear the policy is.",
      "criteria": ["unclear", "clear"]
    }
  }
}

Questions cannot read one another. There is no shared prefix across questions in a request, and one question cannot use the answer to another. Ask a follow-up request if a later question depends on an earlier answer.

Image input

Send one image as a base64 data URL in images. The image is placed ahead of the state.

{
  "model": "dgemma",
  "state": "Inspect the attached image.",
  "questions": {
    "color": {
      "type": "choice",
      "instructions": "What color is the square?",
      "criteria": { "red": "Red", "blue": "Blue", "green": "Green" }
    }
  },
  "images": ["data:image/png;base64,iVBORw0KGgo..."]
}

Video input

Send one MP4 as a base64 data URL in videos. This is a dgemma extension and is not part of the base request shape.

{
  "model": "dgemma",
  "state": "Inspect the attached clip.",
  "questions": {
    "hazard": {
      "type": "choice",
      "instructions": "Is a hazard visible?",
      "criteria": { "yes": "A hazard is visible", "no": "No hazard" }
    }
  },
  "videos": ["data:video/mp4;base64,AAAAIGZ0eXBpc29t..."]
}

The clip must be at most 10 MiB, 30 seconds, and 1920 x 1080 pixels. Four frames are sampled. Audio is not processed. Use images or videos, not both in one request. Use a raw JSON request if your HTTP client does not expose the field.

Response shape

The response contains:

model, the resolved model name;
answers, one entry per question id, with type, the answer, and the distribution;
usage, with input_tokens and output_tokens;
diagnostics, engine detail for the call.

Read answers keyed by your own question ids. diagnostics is informational and can change without notice.

The values in probabilities are normalized scores over the supplied labels. They are not calibrated correctness estimates, and label order can change the prediction. See Current limits.

Request with the TypeSafe SDK

Decisions uses a TypeSafe-compatible API, so the official TypeSafe client works against this route. Install it and point the base URL at https://gateway.prem.io/typesafe.

npm install @typesafe-ai/sdk

import { choice, noul, score, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  apiKey: process.env.PREM_API_KEY,
  baseURL: "https://gateway.prem.io/typesafe",
  defaultModel: "dgemma",
});

const res = await client.systemOne({
  state: "I was charged twice for my monthly subscription.",
  questions: {
    team: choice("Which team should handle this?", {
      billing: "Payments and invoices",
      technical: "Software failures",
      sales: "New purchases",
    }),
    urgent: noul("Is this request urgent?"),
    severity: score("How severe is this issue?", ["low", "medium", "high"] as const),
  },
});

console.log(res.answers.team.choice, res.answers.team.confidence);
console.log(res.answers.urgent.noul);
console.log(res.answers.severity.score, res.answers.severity.legend);

The choice, noul, and score helpers build the typed questions and validate them before the request is sent. state accepts any JSON value, so an object such as { document: "..." } works as well as a string. The client throws a TypeSafeError on an invalid key and a BadRequestError when the server rejects the request, for example when a request carries more than 16 questions.

Request with the standard library

Decisions is not the OpenAI Chat Completions shape, so an OpenAI client does not work against this route. Any HTTP client can send the JSON directly:

const response = await fetch("https://gateway.prem.io/typesafe/v1/systemone", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.PREM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "dgemma",
    state: "I was charged twice for my monthly subscription.",
    questions: {
      team: {
        type: "choice",
        instructions: "Which team should handle this?",
        criteria: {
          billing: "Payments and invoices",
          technical: "Software failures",
          sales: "New purchases",
        },
      },
    },
  }),
});

const data = await response.json();
console.log(data.answers.team.choice);

Current limits

The route is non-streaming. stream is not supported.
Up to 16 questions per request. Up to 4 concurrent requests per key before a 429.
choice holds 1 to 26 criteria. score holds 2 to 10 levels.
At most one image or one video per request, not both.
Questions are independent and cannot read one another.
Probabilities are not calibrated. On a public benchmark slice, 8 of 113 answers above a 0.99 top-probability threshold were wrong, and reversing the order of the criteria changed 17 of 139 predictions. Do not use confidence as a correctness guarantee.
The route runs in ZDR mode. There is no client-side encryption and no attestation. See ZDR vs Confidential.
Verify a production input format with your own representative fixture before rollout.

Next steps

Decisions API reference

The full request and response reference.

ZDR overview

What ZDR guarantees and what it does not.

Models & Pricing

The models, the prices, and the regions.

ZDR vs Confidential

Which mode fits your workload.

​Basic decision

​Question types

​choice

​noul

​score

​Several questions in one request

​Image input

​Video input

​Response shape

​Request with the TypeSafe SDK

​Request with the standard library

​Current limits

​Next steps

Decisions API reference

ZDR overview

Models & Pricing

ZDR vs Confidential

Basic decision

Question types

`choice`

`noul`

`score`

Several questions in one request

Image input

Video input

Response shape

Request with the TypeSafe SDK

Request with the standard library

Current limits

Next steps