# Architecture, Resiliency & Scale
URL: /docs/guides/architecture-and-resilience

> Production integration architecture covering queue-based decoupling, rate limiting, exponential backoff with jitter, and circuit breaker patterns.

Building a mission-critical SMS integration requires architecting for upstream network latency, rate limits, and carrier outages. Direct, blocking synchronous calls from your web request handlers to an external API introduce fragility and performance bottlenecks.

---

## 1. Recommended Production Architecture [#1-recommended-production-architecture]

Never call the SMS API from the user-facing request. Put the send on a queue in your application and let a worker call SMS Pro.

![Your app, your queue, then the SMS Pro API](/images/resilient-queue-architecture.svg)

- The user request returns as soon as your queue accepts the job.
- If SMS Pro or the carrier is unavailable, the job stays in your queue.
- Pace the worker so it stays under the key's rate limit.

---

## 2. Rate Limits & Concurrency Management [#2-rate-limits--concurrency-management]

SMS Pro protects carrier signaling channels by enforcing rate limits per API key:

| Rate Limit Header | Description |
| :--- | :--- |
| `X-RateLimit-Limit` | Maximum number of requests permitted in the current 60-second window. |
| `X-RateLimit-Remaining` | Remaining request allowance in the current window. |
| `X-RateLimit-Reset` | Unix epoch timestamp (seconds) when the quota window resets. |
| `Retry-After` | Included on `429 Too Many Requests`; number of seconds to wait. |

### Client-Side Token Bucket Pacing [#client-side-token-bucket-pacing]
Configure your worker queues (e.g. BullMQ, Celery, or Sidekiq) to rate-limit outbound requests to stay comfortably beneath your tier threshold (e.g. `rate: { max: 100, duration: 1000 }` for 100 TPS).

---

## 3. Exponential Backoff with Full Jitter [#3-exponential-backoff-with-full-jitter]

When encountering transient network timeouts (`ETIMEDOUT`, `ECONNRESET`), HTTP `429 Too Many Requests`, or `503 Service Unavailable`, implement an **exponential backoff algorithm with full jitter**:

```text
wait_time = min(max_backoff, base_backoff * (2 ^ attempt)) ± jitter
```

### Why Jitter Is Essential [#why-jitter-is-essential]
Without jitter, hundreds of worker processes that encounter a transient rate limit at the exact same instant will back off for the exact same duration and retry simultaneously. This creates a **thundering herd** that repeatedly overwhelms the gateway. Adding random jitter desynchronizes the retries, allowing queues to drain smoothly.

---

## 4. Idempotency Key Preservation [#4-idempotency-key-preservation]

Whenever a worker retries an API request, it **must attach the identical `Idempotency-Key`**:

1. Worker picks up job `job_order_84920`.
2. Worker generates a stable UUID `c7a8b9e0-1234-5678-9abc-def012345678` tied to this job.
3. Worker sends request. Network drops before the HTTP response headers arrive.
4. Worker catches error and retries 2 seconds later using the **same** UUID.
5. The SMS Pro gateway recognizes the cached UUID and returns the original `202 Accepted` response without billing your wallet or sending a duplicate message.

---

## 5. Circuit Breaker Pattern [#5-circuit-breaker-pattern]

If downstream carrier networks experience a major disruption, repeatedly battering the endpoint will exhaust your worker connections. Implement a circuit breaker:

- **Closed (Normal)**: All traffic flows to SMS Pro normally.
- **Open (Failing)**: If the failure rate exceeds 25% over a 30-second window, the circuit trips **Open**. Outbound calls immediately fail fast or divert to local persistent storage without making network calls.
- **Half-Open (Testing)**: After a sleep window (e.g. 60 seconds), a single canary request is allowed through. If successful, the circuit closes; if it fails, the open state resets.

---

## Related Documentation [#related-documentation]
- 📖 [Message Lifecycle & Routing](/docs/guides/message-lifecycle-and-routing)
- 📖 [Error Handling & Diagnostics](/docs/guides/error-handling-and-troubleshooting)
- ⚡ [Recipe: Idempotent SMS Dispatch](/docs/recipes/idempotent-sms-dispatch)