Rate limits

​

The purpose of rate limits

• Rate limits help protect the API against abuse or misuse. For example, a malicious actor could flood the API with requests to overload it or to cause a disruption in service. Rate limits prevent this activity.
• Rate limits help give all users fair access to the API. If one person or organization sends an excessive number of requests, the API can become slow for all other users. We limit the number of requests from a single user. So the largest possible number of people can use the API without slowdowns.
• Rate limits help us manage the total load on our infrastructure. If requests to the API increase quickly, the servers can have performance problems. Rate limits help us keep a smooth and constant experience for all users.

​

The operation of rate limits

• RPS (requests per second)
• TPM (tokens per minute)
• Tokens DQ (tokens daily quota)
• concurrent requests
• APM (audio minutes per minute)
• Audio DQ (audio minutes daily quota)

• We define rate limits at the organization level. All users and API keys in an organization share the same rate limit pool.
• Rate limits vary by request type. General API requests and inference requests have different rate limits.
• We also set limits on token usage and audio processing time. These limits refill continuously at the refill rate of your tier. We can enforce limits per minute and/or per day. Not all periods are active at the same time.

​

Identifier scope

• Organization ID: The primary rate limit scope (tied to your API key)
• API Key: The rate limiter uses this if the organization context is not clearly available
• User ID: The rate limiter uses this if the organization context is not available
• IP Address: The rate limiter uses this only when no other identifier is available (e.g., unauthenticated or anonymous requests)

​

Rate limits by tier

​

Available request types

• DEFAULT: General-purpose API requests (e.g., models, projects, settings)
• INFERENCE: Paid model inference requests (e.g., chat completions, embeddings)
• INFERENCE_FREE: Free model inference requests
• AUTH: Authentication-related requests (e.g., login, token exchanges)

​

Request limits

​

Usage tiers

​

Token limits (TPM / Tokens DQ)

• Each tier has a maximum capacity and a refill rate
• When you process a request, the request consumes the total tokens used from your bucket
• Your bucket refills continuously at the specified rate per period

​

Audio processing limits (APM / Audio DQ)

• Each tier has a maximum capacity and a refill rate.
• When you process audio, the audio consumes its duration in seconds from your bucket
• Your bucket refills continuously at the specified rate per period

​

Concurrent request limits

​

Rate limits in headers

​

Error mitigation

​

Steps to decrease rate limit errors

​

Retry with exponential backoff

• Automatic retries let you recover from rate limit errors without crashes or lost data
• With exponential backoff, the first retries occur quickly, and later retries have longer delays
• Random jitter in the delay prevents all retries from occurring at the same time

​

Example:

import createRvencClient from "@premai/api-sdk";

async function retryWithExponentialBackoff<T>(
  fn: () => Promise<T>,
  maxRetries: number = 6,
  baseDelay: number = 1000
): Promise<T> {
  let retries = 0;
  
  while (true) {
    try {
      return await fn();
    } catch (error: any) {
      const isRateLimitError = error?.status === 429;
      
      if (!isRateLimitError || retries >= maxRetries) {
        throw error;
      }
      
      let delay = parseInt(error.headers["retry-after"], 10) * 1000;
      
      retries++;
      console.log(
        `Rate limited (${error?.log?.rate_limit?.tier || 'unknown'} tier). ` +
        `Retrying in ${Math.round(delay)}ms... (attempt ${retries}/${maxRetries})`
      );
      
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  
  try {
    const response = await retryWithExponentialBackoff(
      () =>
        client.chat.completions.create({
          model: "deepseek-v4-pro",
          messages: [
            { role: "system", content: "You are a helpful assistant." },
            { role: "user", content: "What is the capital of France?" },
          ],
        }),
      6,
      1000
    );
    
    console.log(response.choices[0].message.content);
  } catch (error) {
    console.error("Request failed after retries:", error);
  }
}

main().catch(console.error);

​

Handle concurrent request limits

import createRvencClient from "@premai/api-sdk";

class RequestQueue {
  private maxConcurrent: number;
  private running: number = 0;
  private queue: Array<() => void> = [];

  constructor(maxConcurrent: number = 3) {
    this.maxConcurrent = maxConcurrent;
  }

  async execute<T>(requestFn: () => Promise<T>): Promise<T> {
    while (this.running >= this.maxConcurrent) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }

    this.running++;
    try {
      return await requestFn();
    } finally {
      this.running--;
      const resolve = this.queue.shift();
      if (resolve) resolve();
    }
  }
}

// Usage
async function main() {
  const client = await createRvencClient({
    apiKey: process.env.PREM_API_KEY,
    clientKEK: process.env.CLIENT_KEK
  });
  const queue = new RequestQueue(3); // max 3 concurrent requests

  const prompts = [
    "Explain photosynthesis",
    "What is machine learning?",
    "Describe the water cycle",
    "Explain gravity",
    "What is DNA?",
  ];

  try {
    const results = await Promise.all(
      prompts.map((prompt) =>
        queue.execute(() =>
          client.chat.completions.create({
            model: "deepseek-v4-pro",
            messages: [
              { role: "system", content: "You are a helpful assistant." },
              { role: "user", content: prompt },
            ],
          })
        )
      )
    );

    results.forEach((response, index) => {
      console.log(`Response ${index + 1}:`, response.choices[0].message.content);
    });
  } catch (error) {
    console.error("One or more requests failed:", error);
  }
}

main().catch(console.error);

​

Tips for developers

• Group related operations to decrease the number of requests.
• Cache frequently accessed data so that you do not request the same data again and again.
• Monitor response headers for usage patterns. Add alerts that operate when you are near the limits.
• Use the Retry-After header from the error response to set the correct delay before your retry.
• Implement proper error handling: Always check for 429 status codes and handle them safely.

​

Example error response

{
  "status": 429,
  "error": "Rate limit exceeded, try again in 1 seconds",
  "log": {
    "support": "Reach out to support@premai.io to request a higher tier, or upgrade your plan.",
    "rate_limit": {
      "resource": "/rvenc/chat/completions",
      "tier": "BASE",
      "type": "INFERENCE"
    }
  }
}

​

Higher rate limits

• Contact our support team at support@premai.io for enterprise plans with custom rate limits