> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prem.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate limits

> Understand how Prem enforces limits and how to handle a 429.

Several controls apply to each request: the request rate, concurrency, token or audio quotas, the budget, and model capacity. The API refuses requests that cannot pass these controls. For tier qualifications, see [Tiers](https://docs.prem.io/tiers.md).

## The purpose of rate limits

We apply rate limits for three reasons. They protect the API against abuse, because a malicious actor can send too many requests and make the service unavailable. They make access fair for all users, because one organization cannot make the API slow for the other users. They hold the total load on our infrastructure at a level that gives a constant response time.

## The operation of rate limits

Each API tier has separate policies for **Confidential** and **ZDR** inference. Each policy has a concurrency limit, a request burst capacity, a refill rate, a token allowance, and an audio allowance. Use in one family does not consume the other family's allowance.

> [!NOTE]
>
> See your account settings for each family's current limits.

| Control | Scope |
| - | - |
| Inference request rate | The organization and the inference family, across all API keys, users, and endpoints in that family. |
| Concurrent inference requests | The organization and the inference family, across all API keys and users. |
| Token and audio quotas | The organization, the inference family, the quota type, and the period. |
| Spending budget and balance | The organization, across all inference requests. |
| Model capacity | The shared capacity for inference. Different model names can use one capacity pool. |

A different API key does not increase the family's allowance. Keys from different members share that allowance. A different endpoint within the family does not increase it either. Requests with and without a stream share the family's policy. `/rvenc/chat/completions` and `/rvenc/audio/*` use the Confidential policy. Direct calls to `/openai/chat/completions` and `/typesafe/v1/systemone` use the ZDR policy. OpenAI-compatible clients that use the local Confidential Proxy use Confidential allowances. A model in both families uses the allowance for the family of the request.

API requests use the organization's tier. Anonymous requests use IP-based controls. The family split does not create separate balances, budgets, file storage, or model-capacity pools.

## Request limits

A request-rate bucket has a capacity and a refill rate. The API deducts one request credit when a request passes the rate limit. Credits refill continuously. This lets you send a short burst, then send requests at the refill rate. Request credits are separate from model tokens.

### Burst capacity and RPM

Burst capacity is the maximum number of request credits that the bucket holds. **RPM** is the refill amount over 60 seconds. A bucket with a capacity of 10 and a refill of 60 RPM lets you send 10 requests at once. It then adds one request credit each second.

A family can have several active request windows, such as per-minute and daily allowances. Each active window with a finite limit must have a request credit. The API deducts one credit from these windows together. If one window has no credit, the API deducts no credits from the other windows. The API does not return these credits if it refuses the request later or inference does not finish.

If Prem disables a request window, the API keeps its stored balance. An unlimited window keeps its stored balance too. The API deducts no credits from these windows. If an allowance had no refill, its consumed credits stay consumed when Prem restores that allowance without refill.

Your account settings show the burst capacity, refill rate, and period for each family's active windows. One tier can have different values for Confidential and ZDR. These values are maximum request rates. Concurrency, usage quotas, and model capacity can further limit the rate of successful requests.

Non-inference endpoints and anonymous inference use their separate request-rate controls.

## Token and audio quotas

Each usage quota has a capacity, a refill amount, and a period in seconds. A per-minute bucket refills continuously over 60 seconds. A daily bucket refills over 86,400 seconds. These buckets do not reset at each clock minute or at midnight. Each active period for the quota must have sufficient allowance for a new request.

Before a chat request begins, the API makes sure that the token quota has sufficient allowance. For audio requests, the API makes sure that both the token and audio quotas have sufficient allowance. The API deducts reported usage after inference finishes. Audio usage keeps fractional seconds.

**Usage quotas do not reserve allowance for requests that have not finished.** Several requests can begin before the API receives their total usage. This can put a bucket below zero. A depleted bucket refuses new requests until it refills to 20% of its capacity. The API rounds this threshold up to a whole unit. A bucket below zero must first refill to zero, then to this threshold.

### Token limits (TPM / Tokens DQ)

Each family's token policy has a capacity and a refill rate for each active period. TPM uses a 60-second period. A daily token allowance uses an 86,400-second period. Your account settings show your tier's current values for Confidential and ZDR.

The API deducts the total token usage that the model reports. If the model reports no positive total, the API adds the prompt and completion tokens. Quota usage is separate from the price. A discount for cached tokens does not change the token count.

### Audio processing limits (APM / Audio DQ)

Audio policies measure the duration in seconds, with fractional seconds. Each inference family has its own capacity, refill rate, and periods. The available audio endpoints use the Confidential family.

### Quota configuration values

* A limit of `-1` means no limit for that quota.
* A limit of `0` stops requests that need that quota.
* A positive limit defines the capacity. A refill of `0` adds no allowance over time.
* These values apply independently within each inference family. Other controls still apply when a quota is unlimited.

## Concurrent request limits

Tier concurrency counts active requests across the organization within one inference family. Consider an organization with a Confidential limit of 5 and a ZDR limit of 20. Five active Confidential requests use all its Confidential slots. The organization still has its separate ZDR allowance.

Requests that wait for model capacity occupy a family concurrency slot. Each request holds its slot until it finishes or stops. A client disconnect does not always stop the model immediately.

At the concurrency limit, the API returns `429` for new requests until a slot becomes available. A concurrency limit of `-1` means unlimited. A limit of `0` stops new requests. A positive limit is the maximum number of active requests. Model capacity and the other limits still apply when concurrency is unlimited.

## Usage tiers

Your account settings show the organization's current API tier.

### Spending budgets

Before inference begins, the API makes sure that the organization has sufficient balance and budget. The budget applies across the organization. It does not divide by API key or inference family.

The optional monthly budget has no tier-based ceiling. If you do not set a budget, the API applies no budget limit. A budget of `0` has this effect too. A positive budget stops new requests when the period's cost reaches that amount. The balance requirement and other limits still apply.

The API does not reserve funds for requests that have not finished. Their total cost can be more than the budget left when they began. The budget stops new requests, but it does not cap the cost of requests that already began.

### Rate limits in headers

A rejection can have a `Retry-After` header with a delay in seconds. Some concurrency and model-service errors omit this header. A window with zero capacity or no credits and no refill omits it too. Use your account limits and the error details to understand the rejection.

## Error mitigation

### Steps to decrease rate limit errors

Be careful when you give users programmatic access, bulk processing features, and automated posting. Enable these features only for trusted customers.

Set a usage limit for each user in a specified period (daily, weekly, or monthly). This protects against automated and high-volume misuse. Add a hard cap or a manual review process for users who exceed the limit.

### Retry with exponential backoff

Retry temporary rate-limit rejections with bounded exponential backoff and random jitter. The client waits before each retry. It increases the delay after each rejection, up to a maximum number of retries. This method has three benefits:

* Automatic retries let you recover from rate limit errors without crashes or lost data
* With exponential backoff, the first retries have short delays, and later retries have longer delays
* With random jitter, clients retry at different times

> [!TIP]
>
> The API deducts a request credit when the request passes the rate limit. It does not return this credit if it later refuses the request. Wait before each retry.

Do not automatically retry `Inference request allowance exhausted. Contact us for higher limits.` This `429` means a request window has zero capacity, or no credits and no refill. It has no `Retry-After` header because a delay cannot restore a credit. Prem must change the limit before another request can pass.

The example that follows uses exponential backoff.

#### Example:

This example reads the SDK error's `status`, `message` or `error`, `support_id`, and optional `headers["retry-after"]` fields. It stops if the error states that the request allowance has no credits and no refill, or zero capacity. For other `429` responses, it waits before each retry. Use it before the first output arrives. Do not replay a request after you receive output from its stream.

```typescript theme={"system"}
import createRvencClient from "@premai/api-sdk";

async function retryWithExponentialBackoff<T>(
  fn: () => Promise<T>,
  maxRetries: number = 6,
  baseDelay: number = 1000
): Promise<T> {
  let retries = 0;

  while (true) {
    try {
      return await fn();
    } catch (error: any) {
      const isRateLimitError = error?.status === 429;
      const message = error?.message ?? error?.error;
      const allowanceExhausted =
        message === "Inference request allowance exhausted. Contact us for higher limits.";

      if (!isRateLimitError || allowanceExhausted || retries >= maxRetries) {
        throw error;
      }

      const parsedRetryAfter = Number(error?.headers?.["retry-after"]);
      const retryDelay = Number.isFinite(parsedRetryAfter) && parsedRetryAfter > 0
        ? parsedRetryAfter * 1000
        : baseDelay * 2 ** retries;
      const delay = retryDelay + Math.random() * baseDelay;

      retries++;
      console.log(
        `Rate limited (support_id ${error?.support_id || "unknown"}). ` +
        `Retrying in ${Math.round(delay)}ms... (attempt ${retries}/${maxRetries})`
      );

      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });

  try {
    const response = await retryWithExponentialBackoff(
      () =>
        client.chat.completions.create({
          model: "deepseek-v4.1-flash-1017",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
        }),
      6,
      1000
    );

    console.log(response.choices[0].message.content);
  } catch (error) {
    console.error("Request failed after retries:", error);
  }
}

main().catch(console.error);
```

### Handle concurrent request limits

Make sure that your application does not send more concurrent requests than the family's limit. Use a shared queue for the organization's requests in each family. The queue must limit requests across all your workers.

```typescript theme={"system"}
import createRvencClient from "@premai/api-sdk";

class RequestQueue {
  private maxConcurrent: number;
  private running: number = 0;
  private queue: Array<() => void> = [];

  constructor(maxConcurrent: number = 3) {
    this.maxConcurrent = maxConcurrent;
  }

  async execute<T>(requestFn: () => Promise<T>): Promise<T> {
    while (this.running >= this.maxConcurrent) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }

    this.running++;
    try {
      return await requestFn();
    } finally {
      this.running--;
      const resolve = this.queue.shift();
      if (resolve) resolve();
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  const queue = new RequestQueue(3); // max 3 concurrent requests

  const prompts = [
    "Explain photosynthesis",
    "What is machine learning?",
    "Describe the water cycle",
    "Explain gravity",
    "What is DNA?",
  ];

  try {
    const results = await Promise.all(
      prompts.map((prompt) =>
        queue.execute(() =>
          client.chat.completions.create({
            model: "deepseek-v4.1-flash-1017",
            messages: [
              { role: "system", content: "You are a helpful assistant." },
              { role: "user", content: prompt },
            ],
          })
        )
      )
    );

    results.forEach((response, index) => {
      console.log(`Response ${index + 1}:`, response.choices[0].message.content);
    });
  } catch (error) {
    console.error("One or more requests failed:", error);
  }
}

main().catch(console.error);
```

## Tips for developers

* **Group related operations** to send fewer requests.
* **Cache frequently accessed data** so that you do not request the same data again and again.
* **Watch for `TOKEN_LIMIT_EXCEEDED` and `AUDIO_DURATION_LIMIT_EXCEEDED`** on `400` responses, not only for `429`.
* **Log the `support_id`** for each unsuccessful request.
* **Use `Retry-After` when present** for temporary rejections. Stop automatic retries if the request allowance has zero capacity, or no credits and no refill.
* **Read both HTTP and stream errors**: Read `429` responses before the stream begins. Read error events after an SSE connection opens. Do not automatically replay a request after you receive output.

## Example error response

If a request bucket has no credits, but has positive capacity and refill, the API returns `429 Too Many Requests`. Its `Retry-After` header states the delay in seconds.

A window with zero capacity, or no credits and no refill, returns `429`. The error is `Inference request allowance exhausted. Contact us for higher limits.` The response has no `Retry-After` header. Do not automatically retry this response.

Token and audio quota errors return `400` with `TOKEN_LIMIT_EXCEEDED` or `AUDIO_DURATION_LIMIT_EXCEEDED`. Budget and balance errors return `400` with `BUDGET_LIMIT_EXCEEDED` or `INSUFFICIENT_FUNDS`. The API returns `503` if it cannot get the limits for your request.

```json theme={"system"}
{
  "status": 429,
  "data": null,
  "error": "Rate limit exceeded, try again in 1 seconds",
  "message": null,
  "env": "production",
  "log": null,
  "validator": null,
  "support_id": "req_support_019a3f7c8d2e7b41a0c6"
}
```

Record the `support_id`. Send it to support with a timestamp if you need us to trace a request.

## Non-standard 429 responses

The API's rate limiter returns the response format in the example. Other services do not always use this format for `429` responses. A model service can return a `429` status with a generic error and no `Retry-After` header:

```json theme={"system"}
{
  "error": {
    "message": "Internal server error",
    "type": "server_error"
  }
}
```

The message suggests a `500` error, but the status code is `429`. Your client must use the status code and error details:

* **Read the HTTP status and error details.** A `429` refers to the request rate, concurrency, or shared model capacity. Use bounded backoff for temporary rejections. Quota and billing errors can use `400`. Read these errors too.
* **Never assume that `Retry-After` is present.** For temporary concurrency or model-service rejections, use bounded exponential backoff with jitter. Stop automatic retries if the request allowance has zero capacity, or no credits and no refill.

After an SSE connection sends HTTP `200`, the API sends later errors in the stream. These errors do not change the HTTP status. Read the SDK's stream exception or SSE error event, and the initial HTTP errors. HTTP `200` alone does not mean that inference finished successfully.

For repeated rejections without a `Retry-After` header or with an unexpected body, [contact us](https://docs.prem.io/contact-us.md). Send the `support_id`, if present, and a timestamp. We use these values to trace the request.

## Higher rate limits

Your limits increase when you move to a higher tier. See [Tiers](https://docs.prem.io/tiers.md#how-to-move-to-a-higher-tier).

If you need higher limits, [contact us](https://docs.prem.io/contact-us.md) to discuss custom rate limits.

## Related

> [card] [Tiers](https://docs.prem.io/tiers.md)
>
> The request, token, and audio limits of each tier.

> [card] [Errors](https://docs.prem.io/errors.md)
>
> Every status code that the API returns, and what to do about it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.