Rate Limiting

API rate limits and best practices for staying within quotas.

Overview

AIDP API implements rate limiting to ensure fair usage and system stability. During our free beta, all users have generous rate limits.

---

Current Rate Limits (Free Beta)

All users currently have the same generous rate limits:

• Search API: 10,000 requests/hour

• Analytics API: 5,000 requests/hour

• Business API: 10,000 requests/hour

• Burst: 100 requests/second

These limits are designed to support production workloads while preventing abuse.

---

Rate Limit Headers

Every API response includes rate limit information:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 847
X-RateLimit-Reset: 1702389600
X-RateLimit-Retry-After: 3600

Headers:

• X-RateLimit-Limit: Total requests allowed in window

• X-RateLimit-Remaining: Requests remaining in current window

• X-RateLimit-Reset: Unix timestamp when limit resets

• X-RateLimit-Retry-After: Seconds until you can retry (when rate limited)

---

Handling Rate Limits

Check Headers

Always check rate limit headers before making requests:

const response = await fetch('https://api.aidp.dev/v1/api/v1/consumer/search', {
  headers: { Authorization: `Bearer ${API_KEY}` },
});

const remaining = response.headers.get('X-RateLimit-Remaining');
const reset = response.headers.get('X-RateLimit-Reset');

if (remaining < 10) {
  console.warn(`Only ${remaining} requests remaining until ${new Date(reset * 1000)}`);
}

Handle 429 Responses

When rate limited, you'll receive a 429 status:

{
  "error": "Rate limit exceeded",
  "retryAfter": 3600,
  "limit": 1000,
  "resetAt": "2024-12-12T15:00:00Z"
}

Retry Logic:

async function makeRequestWithRetry(url, options, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    const response = await fetch(url, options);

    if (response.status === 429) {
      const retryAfter = response.headers.get('X-RateLimit-Retry-After');
      await sleep(retryAfter * 1000);
      continue;
    }

    return response;
  }

  throw new Error('Max retries exceeded');
}

---

Best Practices

1. Implement Exponential Backoff

async function exponentialBackoff(fn, maxRetries = 5) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      if (error.status !== 429) throw error;

      const delay = Math.min(1000 * Math.pow(2, i), 32000);
      await sleep(delay);
    }
  }
}

2. Use Request Queuing

class RateLimitedQueue {
  constructor(requestsPerSecond) {
    this.queue = [];
    this.interval = 1000 / requestsPerSecond;
    this.processing = false;
  }

  async add(fn) {
    return new Promise((resolve, reject) => {
      this.queue.push({ fn, resolve, reject });
      this.process();
    });
  }

  async process() {
    if (this.processing || this.queue.length === 0) return;

    this.processing = true;
    const { fn, resolve, reject } = this.queue.shift();

    try {
      const result = await fn();
      resolve(result);
    } catch (error) {
      reject(error);
    }

    setTimeout(() => {
      this.processing = false;
      this.process();
    }, this.interval);
  }
}

3. Cache Responses

Reduce API calls by caching responses:

const cache = new Map();

async function cachedSearch(query, ttl = 3600000) {
  const key = JSON.stringify(query);
  const cached = cache.get(key);

  if (cached && Date.now() - cached.timestamp < ttl) {
    return cached.data;
  }

  const data = await searchAPI(query);
  cache.set(key, { data, timestamp: Date.now() });
  return data;
}

---

Increasing Limits

Need higher limits?

1. Upgrade Tier: Move to Professional or Enterprise

2. Contact Sales: Custom limits for high-volume use cases

3. Optimize Usage: Implement caching and batching

---

Monitoring Usage

Track your API usage in the Developer Dashboard:

1. Go to Developer Dashboard

2. View "API Usage" section

3. Monitor requests per endpoint

4. Set up usage alerts

---

Need Help?

• 📧 Email: [e-mail]

• 📚 Documentation: docs.aidp.dev/rate-limiting