Architecture pattern
Multi-region gateway with failover
Run the gateway and its services in more than one region, route clients to the nearest healthy one, and fail over when a region goes down.
- Caller
- Gateway
- Component you run
- Request
- Event or background
How it flows
- Clients to Global traffic layerClients resolve one global hostname.
- Global traffic layer to Gateway, region ARoutes to the closest healthy region, based on health checks.
- Global traffic layer to Gateway, region BShifts traffic to region B when region A fails its health checks.
- Gateway, region A to Services, region A; Gateway, region B to Services, region BEach gateway prefers services in its own region.
- Gateway, region A to Services, region BOptional cross-region upstream failover, at the cost of extra latency.
- Services, region A and Services, region B to Replicated data (asynchronous)State replicates between regions, usually asynchronously.
The problem
A single-region deployment is one incident away from a full outage. Users far from that region also pay a latency cost on every call. Adding a second region sounds simple, but the hard parts are routing, health detection and, above all, data.
When to use it
- Availability targets that one region cannot meet, or a regulator expects regional resilience.
- Users spread across continents, where round-trip time matters.
- Services that are stateless, or whose state can replicate between regions with acceptable lag.
- Reducing dependency on a single cloud provider is a stated goal.
When not to
- State that must be strongly consistent everywhere, and cannot tolerate replication lag or conflicts. Solve the data design first.
- Small teams without the operational maturity to test failover regularly. An untested second region gives false confidence.
- When a well-architected single region across several availability zones already meets the target at much lower cost.
Key components
- Global traffic layer. DNS-based or anycast routing that steers clients by health and proximity.
- Regional gateways. Identical configuration in every region, ideally managed from one control plane so they cannot drift.
- Health checks. Active probes at both the global layer and the gateway, checking real dependencies rather than only that a port is open.
- Upstream failover. Optional gateway-level retry to another region's service when a local one fails.
- Data replication. A clear strategy per data store: active-active, active-passive, or region-pinned.
Trade-offs
- Cost. A second active region roughly doubles the infrastructure for the services involved. A warm standby is cheaper but slower to take full load.
- Consistency. Asynchronous replication means a failover can lose or reorder recent writes. Know your recovery point objective.
- DNS caching. DNS-based failover is limited by client caching and time-to-live values; anycast reacts faster but needs suitable infrastructure.
- Cross-region calls. Upstream failover keeps requests succeeding, but adds latency and egress cost, and can overload the healthy region.
Explore further
Lab: Same prompt. Three clocks.→Send one curated prompt to several LLM providers at once and compare time to first token, total time, tokens per second and cost.
Lab: A demo is the beginning.→Explore what changes when an AI application meets the real world.