> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prem.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate limits

> Understand API rate limits and restrictions.

Rate limits restrict the number of requests that a user or client can send to the API in a given period.

## The purpose of rate limits

Rate limits are common for APIs. We apply them for these reasons:

* **Rate limits help protect the API against abuse or misuse.** For example, a malicious actor could flood the API with requests to overload it or to cause a disruption in service. Rate limits prevent this activity.
* **Rate limits help give all users fair access to the API.** If one person or organization sends an excessive number of requests, the API can become slow for all other users. We limit the number of requests from a single user. So the largest possible number of people can use the API without slowdowns.
* **Rate limits help us manage the total load on our infrastructure.** If requests to the API increase quickly, the servers can have performance problems. Rate limits help us keep a smooth and constant experience for all users.

Read this full document to understand how our rate limit system operates. We include code examples and possible solutions for common problems.

## The operation of rate limits

We measure rate limits in six ways:

* **RPS** (requests per second)
* **TPM** (tokens per minute)
* **Tokens DQ** (tokens daily quota)
* **concurrent requests**
* **APM** (audio minutes per minute)
* **Audio DQ** (audio minutes daily quota)

Not all limit types are active at the same time. You can reach a rate limit on any active option, whichever occurs first. For example, assume that your concurrent request limit is 5 and your TPM limit is 150k. If you send 5 requests with only 100 tokens to the chat completions endpoint, you fill your concurrent request limit. This occurs although the 5 requests do not use 150k tokens.

Also note these points:

* **We define rate limits at the organization level.** All users and API keys in an organization share the same rate limit pool.
* **Rate limits vary by request type.** General API requests and inference requests have different rate limits.
* **We also set limits on token usage and audio processing time.** These limits refill continuously at the refill rate of your tier. We can enforce limits per minute and/or per day. Not all periods are active at the same time.

## Identifier scope

The rate limiter uses the most specific and reliable identifier that is available to track usage:

* **Organization ID**: The primary rate limit scope (tied to your API key)
* **API Key**: The rate limiter uses this if the organization context is not clearly available
* **User ID**: The rate limiter uses this if the organization context is not available
* **IP Address**: The rate limiter uses this only when no other identifier is available (e.g., unauthenticated or anonymous requests)

So all requests that use the API keys of the same organization share the same rate limit bucket.

### Rate limits by tier

Rate limits vary with the **tier of your organization**. Each tier defines different capacities and refill rates for each request type.

### Available request types

* `DEFAULT`: General-purpose API requests (e.g., models, projects, settings)
* `INFERENCE`: Paid model inference requests (e.g., chat completions, embeddings)
* `INFERENCE_FREE`: Free model inference requests
* `AUTH`: Authentication-related requests (e.g., login, token exchanges)

### Request limits

Request limits control the number of API requests that you can send per second. Each request type has a different limit that matches its sensitivity and resource usage.

| Tier     | Type            | Capacity | Refill Rate (tokens/sec) |
| -------- | --------------- | -------- | ------------------------ |
| `BASE`   | DEFAULT         | 50       | 5                        |
| `BASE`   | INFERENCE       | 5        | 1                        |
| `BASE`   | INFERENCE\_FREE | 1        | 1                        |
| `BASE`   | AUTH            | 5        | 1                        |
| `TIER_1` | DEFAULT         | 150      | 15                       |
| `TIER_1` | INFERENCE       | 5        | 1                        |
| `TIER_1` | INFERENCE\_FREE | 1        | 1                        |
| `TIER_1` | AUTH            | 5        | 1                        |
| `TIER_2` | DEFAULT         | 450      | 45                       |
| `TIER_2` | INFERENCE       | 5        | 1                        |
| `TIER_2` | INFERENCE\_FREE | 1        | 1                        |
| `TIER_2` | AUTH            | 5        | 1                        |
| `TIER_3` | DEFAULT         | 1000     | 100                      |
| `TIER_3` | INFERENCE       | 5        | 1                        |
| `TIER_3` | INFERENCE\_FREE | 1        | 1                        |
| `TIER_3` | AUTH            | 5        | 1                        |

We use a **token bucket algorithm**. Each request consumes 1 token from your bucket. The bucket refills automatically at the specified rate per second. This permits short bursts of requests but prevents continuous overuse.

### Usage tiers

You can see the rate and usage limits for your organization in the limits section of your account settings. When your usage of our API increases, we automatically move you to the next usage tier. This usually increases the rate limits for most endpoints.

| Tier   | Qualification                                            | Usage limits     |
| ------ | -------------------------------------------------------- | ---------------- |
| Base   | Default tier                                             | \$100 / month    |
| Tier 1 | \$5 paid                                                 | \$500 / month    |
| Tier 2 | \$100 paid and 7+ days since first successful payment    | \$5,000 / month  |
| Tier 3 | \$1,000 paid and 30+ days since first successful payment | \$20,000 / month |

### Token limits (TPM / Tokens DQ)

Token limits apply to the inference endpoints. These limits restrict the total number of tokens that you can process in a given period. We can enforce token limits per minute (TPM) and/or per day (Tokens DQ). Not all periods are active at the same time.

| Tier     | Token Limit | Refill Rate | Period     |
| -------- | ----------- | ----------- | ---------- |
| `BASE`   | 38,000      | 38,000      | per minute |
| `TIER_1` | 540,000     | 540,000     | per minute |
| `TIER_2` | 1,000,000   | 1,000,000   | per minute |
| `TIER_3` | 2,500,000   | 2,500,000   | per minute |

Token limits operate in the same way as request limits:

* Each tier has a **maximum capacity** and a **refill rate**
* When you process a request, the request consumes the **total tokens used** from your bucket
* Your bucket **refills continuously** at the specified rate per period

### Audio processing limits (APM / Audio DQ)

An **audio duration limit** also applies to the audio transcription and translation endpoints. This limit restricts the total duration of audio that you can process in a given period. We can enforce audio limits per minute (APM) and/or per day (Audio DQ). Not all periods are active at the same time.

| Tier     | Audio Limit (seconds) | Refill Rate (sec) | Period     |
| -------- | --------------------- | ----------------- | ---------- |
| `BASE`   | 10                    | 10                | per minute |
| `TIER_1` | 60                    | 60                | per minute |
| `TIER_2` | 300                   | 300               | per minute |
| `TIER_3` | 1,200                 | 1,200             | per minute |

Audio limits operate in the same way as request limits:

* Each tier has a **maximum capacity** and a **refill rate**.
* When you process audio, the audio consumes its **duration in seconds** from your bucket
* Your bucket **refills continuously** at the specified rate per period

### Concurrent request limits

Concurrent request limits control the number of inference requests that your organization can **process at the same time**. This limit is separate from the request rate limit (RPS).

| Tier     | Concurrent Requests |
| -------- | ------------------- |
| `BASE`   | 5                   |
| `TIER_1` | 50                  |
| `TIER_2` | 500                 |
| `TIER_3` | 5,000               |

When you reach your concurrent request limit, more requests receive a `429` error until an active request completes.

### Rate limits in headers

You can see your rate limit in your account settings. You can also see important rate limit information in the headers of the HTTP response.

The response can include these header fields:

| Field       | Sample Value | Description                                          |
| ----------- | ------------ | ---------------------------------------------------- |
| Retry-After | 1            | The time in seconds until you can retry the request. |

## Error mitigation

### Steps to decrease rate limit errors

Be careful when you give users programmatic access, bulk processing features, and automated posting. Enable these features only for trusted customers.

Set a usage limit for each user in a specified period (daily, weekly, or monthly). This protects against automated and high-volume misuse. Add a hard cap or a manual review process for users who exceed the limit.

### Retry with exponential backoff

One simple way to prevent rate limit errors is to retry requests automatically with a random exponential backoff. With this method, the client sleeps for a short time when a rate limit error occurs. Then the client retries the unsuccessful request. If the request is again unsuccessful, the client increases the sleep time and retries again. This continues until the request is successful or until the client reaches a maximum number of retries. This approach has many benefits:

* Automatic retries let you recover from rate limit errors without crashes or lost data
* With exponential backoff, the first retries occur quickly, and later retries have longer delays
* Random jitter in the delay prevents all retries from occurring at the same time

<Tip>Unsuccessful requests count against your rate limit. Do not send the same request again and again without a delay.</Tip>

The example below uses exponential backoff.

#### Example:

This advanced implementation uses the `Retry-After` header from the API response for more efficient retries:

```typescript theme={"system"}
import createRvencClient from "@premai/api-sdk";

async function retryWithExponentialBackoff<T>(
  fn: () => Promise<T>,
  maxRetries: number = 6,
  baseDelay: number = 1000
): Promise<T> {
  let retries = 0;
  
  while (true) {
    try {
      return await fn();
    } catch (error: any) {
      const isRateLimitError = error?.status === 429;
      
      if (!isRateLimitError || retries >= maxRetries) {
        throw error;
      }
      
      let delay = parseInt(error.headers["retry-after"], 10) * 1000;
      
      retries++;
      console.log(
        `Rate limited (${error?.log?.rate_limit?.tier || 'unknown'} tier). ` +
        `Retrying in ${Math.round(delay)}ms... (attempt ${retries}/${maxRetries})`
      );
      
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  
  try {
    const response = await retryWithExponentialBackoff(
      () =>
        client.chat.completions.create({
          model: "deepseek-v4-pro",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
        }),
      6,
      1000
    );
    
    console.log(response.choices[0].message.content);
  } catch (error) {
    console.error("Request failed after retries:", error);
  }
}

main().catch(console.error);
```

### Handle concurrent request limits

If your application sends many requests at the same time, monitor the concurrent request limits. Use a request queue that processes requests in sequence or in controlled batches. This prevents errors from the concurrent request limit.

```typescript theme={"system"}
import createRvencClient from "@premai/api-sdk";

class RequestQueue {
  private maxConcurrent: number;
  private running: number = 0;
  private queue: Array<() => void> = [];

  constructor(maxConcurrent: number = 3) {
    this.maxConcurrent = maxConcurrent;
  }

  async execute<T>(requestFn: () => Promise<T>): Promise<T> {
    while (this.running >= this.maxConcurrent) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }

    this.running++;
    try {
      return await requestFn();
    } finally {
      this.running--;
      const resolve = this.queue.shift();
      if (resolve) resolve();
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  const queue = new RequestQueue(3); // max 3 concurrent requests

  const prompts = [
    "Explain photosynthesis",
    "What is machine learning?",
    "Describe the water cycle",
    "Explain gravity",
    "What is DNA?",
  ];

  try {
    const results = await Promise.all(
      prompts.map((prompt) =>
        queue.execute(() =>
          client.chat.completions.create({
            model: "deepseek-v4-pro",
            messages: [
              { role: "system", content: "You are a helpful assistant." },
              { role: "user", content: prompt },
            ],
          })
        )
      )
    );

    results.forEach((response, index) => {
      console.log(`Response ${index + 1}:`, response.choices[0].message.content);
    });
  } catch (error) {
    console.error("One or more requests failed:", error);
  }
}

main().catch(console.error);
```

## Tips for developers

* **Group related operations** to decrease the number of requests.
* **Cache frequently accessed data** so that you do not request the same data again and again.
* **Monitor response headers** for usage patterns. Add alerts that operate when you are near the limits.
* **Use the `Retry-After` header** from the error response to set the correct delay before your retry.
* **Implement proper error handling**: Always check for 429 status codes and handle them safely.

## Example error response

When your bucket has no tokens, your request returns an error with a `429 Too Many Requests` status. The response includes a `Retry-After` header (in seconds). This header tells you the time to wait before a retry.

```json theme={"system"}
{
  "status": 429,
  "error": "Rate limit exceeded, try again in 1 seconds",
  "log": {
    "support": "Reach out to support@premai.io to request a higher tier, or upgrade your plan.",
    "rate_limit": {
      "resource": "/rvenc/chat/completions",
      "tier": "BASE",
      "type": "INFERENCE"
    }
  }
}
```

## Higher rate limits

If you need higher rate limits for your use case, you can:

* **Contact our support team** at [support@premai.io](mailto:support@premai.io) for enterprise plans with custom rate limits