Rate limits

Documentation Index

Fetch the complete documentation index at: https://docs.prem.io/llms.txt

Use this file to discover all available pages before exploring further.

​

Why do we have rate limits?

• They help protect against abuse or misuse of the API. For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, we can prevent this kind of activity.
• Rate limits help ensure that everyone has fair access to the API. If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, we ensure that the most number of people have an opportunity to use the API without experiencing slowdowns.
• Rate limits can help manage the aggregate load on our infrastructure. If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, we can help maintain a smooth and consistent experience for all users.

​

How do these rate limits work?

• Rate limits are defined at the organization level. All users and API keys within an organization share the same rate limit pool.
• Rate limits vary by the request type being used. General API requests and inference requests each have different rate limits.
• Limits are also placed on the token usage and audio processing time. These limits refill continuously based on your tier’s refill rate. Limits can be enforced per minute and/or per day — not all periods are necessarily enabled simultaneously.

​

Identifier scope

• Organization ID – Primary rate limit scope (tied to your API key)
• API Key – Used if org-level context is not clearly available
• User ID – Used if org-level context is not available
• IP Address – Only used as a last resort (e.g., unauthenticated or anonymous requests)

​

Rate limits by tier

​

Available request types

• DEFAULT: General-purpose API requests (e.g., models, projects, settings)
• INFERENCE: Paid model inference requests (e.g., chat completions, embeddings)
• INFERENCE_FREE: Free model inference requests
• AUTH: Authentication-related requests (e.g., login, token exchanges)

​

Request limits

​

Usage tiers

​

Token limits (TPM / Tokens DQ)

• Each tier has a maximum capacity and a refill rate
• When you process a request, the total tokens used are consumed from your bucket
• Your bucket refills continuously at the specified rate per period

​

Audio processing limits (APM / Audio DQ)

• Each tier has a maximum capacity and a refill rate.
• When you process audio, the duration in seconds is consumed from your bucket
• Your bucket refills continuously at the specified rate per period

​

Concurrent request limits

​

Rate limits in headers

​

Error mitigation

​

What are some steps I can take to mitigate this?

​

Retrying with exponential backoff

• Automatic retries means you can recover from rate limit errors without crashes or missing data
• Exponential backoff means that your first retries can be tried quickly, while still benefiting from longer delays if your first few retries fail
• Adding random jitter to the delay helps retries from all hitting at the same time

​

Example:

import createRvencClient from "@premai/api-sdk";

async function retryWithExponentialBackoff<T>(
  fn: () => Promise<T>,
  maxRetries: number = 6,
  baseDelay: number = 1000
): Promise<T> {
  let retries = 0;
  
  while (true) {
    try {
      return await fn();
    } catch (error: any) {
      const isRateLimitError = error?.status === 429;
      
      if (!isRateLimitError || retries >= maxRetries) {
        throw error;
      }
      
      let delay = parseInt(error.headers["retry-after"], 10) * 1000;
      
      retries++;
      console.log(
        `Rate limited (${error?.log?.rate_limit?.tier || 'unknown'} tier). ` +
        `Retrying in ${Math.round(delay)}ms... (attempt ${retries}/${maxRetries})`
      );
      
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  
  try {
    const response = await retryWithExponentialBackoff(
      () =>
        client.chat.completions.create({
          model: "openai/gpt-oss-120b",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
        }),
      6,
      1000
    );
    
    console.log(response.choices[0].message.content);
  } catch (error) {
    console.error("Request failed after retries:", error);
  }
}

main().catch(console.error);

​

Handle concurrent request limits

import createRvencClient from "@premai/api-sdk";

class RequestQueue {
  private maxConcurrent: number;
  private running: number = 0;
  private queue: Array<() => void> = [];

  constructor(maxConcurrent: number = 3) {
    this.maxConcurrent = maxConcurrent;
  }

  async execute<T>(requestFn: () => Promise<T>): Promise<T> {
    while (this.running >= this.maxConcurrent) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }

    this.running++;
    try {
      return await requestFn();
    } finally {
      this.running--;
      const resolve = this.queue.shift();
      if (resolve) resolve();
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  const queue = new RequestQueue(3); // max 3 concurrent requests

  const prompts = [
    "Explain photosynthesis",
    "What is machine learning?",
    "Describe the water cycle",
    "Explain gravity",
    "What is DNA?",
  ];

  try {
    const results = await Promise.all(
      prompts.map((prompt) =>
        queue.execute(() =>
          client.chat.completions.create({
            model: "openai/gpt-oss-120b",
            messages: [
              { role: "system", content: "You are a helpful assistant." },
              { role: "user", content: prompt },
            ],
          })
        )
      )
    );

    results.forEach((response, index) => {
      console.log(`Response ${index + 1}:`, response.choices[0].message.content);
    });
  } catch (error) {
    console.error("One or more requests failed:", error);
  }
}

main().catch(console.error);

​

Tips for developers

• Group related operations to reduce the number of requests.
• Cache frequently accessed data instead of refetching it constantly.
• Monitor response headers for usage patterns and implement alerts when nearing limits.
• Use the Retry-After header from the error response to delay your retry appropriately.
• Implement proper error handling: Always check for 429 status codes and handle them gracefully.

​

Example error response

{
  "status": 429,
  "error": "Rate limit exceeded, try again in 1 seconds",
  "log": {
    "support": "Reach out to support@premai.io to request a higher tier, or upgrade your plan.",
    "rate_limit": {
      "resource": "/rvenc/chat/completions",
      "tier": "BASE",
      "type": "INFERENCE"
    }
  }
}

​

Need higher limits?

• Contact our support team at support@premai.io for enterprise plans with custom rate limits