Skip to Content

Architecture, Resiliency & Scale

Production integration architecture covering queue-based decoupling, rate limiting, exponential backoff with jitter, and circuit breaker patterns.

View Markdown (.md)

Building a mission-critical SMS integration requires architecting for upstream network latency, rate limits, and carrier outages. Direct, blocking synchronous calls from your web request handlers to an external API introduce fragility and performance bottlenecks.


Never call the SMS API from the user-facing request. Put the send on a queue in your application and let a worker call SMS Pro.

Your app, your queue, then the SMS Pro API

  • The user request returns as soon as your queue accepts the job.
  • If SMS Pro or the carrier is unavailable, the job stays in your queue.
  • Pace the worker so it stays under the key's rate limit.

2. Rate Limits & Concurrency Management

SMS Pro protects carrier signaling channels by enforcing rate limits per API key:

Rate Limit HeaderDescription
X-RateLimit-LimitMaximum number of requests permitted in the current 60-second window.
X-RateLimit-RemainingRemaining request allowance in the current window.
X-RateLimit-ResetUnix epoch timestamp (seconds) when the quota window resets.
Retry-AfterIncluded on 429 Too Many Requests; number of seconds to wait.

Client-Side Token Bucket Pacing

Configure your worker queues (e.g. BullMQ, Celery, or Sidekiq) to rate-limit outbound requests to stay comfortably beneath your tier threshold (e.g. rate: { max: 100, duration: 1000 } for 100 TPS).


3. Exponential Backoff with Full Jitter

When encountering transient network timeouts (ETIMEDOUT, ECONNRESET), HTTP 429 Too Many Requests, or 503 Service Unavailable, implement an exponential backoff algorithm with full jitter:

wait_time = min(max_backoff, base_backoff * (2 ^ attempt)) ± jitter

Why Jitter Is Essential

Without jitter, hundreds of worker processes that encounter a transient rate limit at the exact same instant will back off for the exact same duration and retry simultaneously. This creates a thundering herd that repeatedly overwhelms the gateway. Adding random jitter desynchronizes the retries, allowing queues to drain smoothly.


4. Idempotency Key Preservation

Whenever a worker retries an API request, it must attach the identical Idempotency-Key:

  1. Worker picks up job job_order_84920.
  2. Worker generates a stable UUID c7a8b9e0-1234-5678-9abc-def012345678 tied to this job.
  3. Worker sends request. Network drops before the HTTP response headers arrive.
  4. Worker catches error and retries 2 seconds later using the same UUID.
  5. The SMS Pro gateway recognizes the cached UUID and returns the original 202 Accepted response without billing your wallet or sending a duplicate message.

5. Circuit Breaker Pattern

If downstream carrier networks experience a major disruption, repeatedly battering the endpoint will exhaust your worker connections. Implement a circuit breaker:

  • Closed (Normal): All traffic flows to SMS Pro normally.
  • Open (Failing): If the failure rate exceeds 25% over a 30-second window, the circuit trips Open. Outbound calls immediately fail fast or divert to local persistent storage without making network calls.
  • Half-Open (Testing): After a sleep window (e.g. 60 seconds), a single canary request is allowed through. If successful, the circuit closes; if it fails, the open state resets.

On this page