Architecture pattern
Service mesh with an edge gateway
Use a gateway for north-south traffic entering your platform and a service mesh for east-west traffic between services, each doing the job it is built for.
- Caller
- Gateway
- Component you run
- Control, identity or policy
- Request
- Configuration or credentials
How it flows
- Public clients and Partners to Edge gatewayExternal API calls arrive at the edge (north-south traffic).
- Edge gateway to Service AThe gateway authenticates, applies limits, and forwards into the mesh over mTLS.
- Service A to Service BService-to-service calls go through the sidecars (east-west traffic).
- Service B to Service CThe mesh adds retries, timeouts, mTLS and traffic splitting without code changes.
- Mesh control plane to Service A, Service B and Service C (control)Pushes certificates and routing policy to every proxy.
The problem
Teams often ask whether they need an API gateway or a service mesh. They solve different problems. External consumers need products, keys, rate limits and documentation. Internal services need encrypted, observable, resilient calls to each other. Forcing one tool to do both leads to either a gateway in every service hop or a mesh exposed to the internet.
When to use it
- Many services on Kubernetes or across clusters, owned by different teams.
- You need mutual TLS and consistent retries and timeouts between services without changing application code.
- External consumers need a curated API surface with authentication, quotas and a developer portal.
- You want traffic splitting for canary releases inside the platform.
When not to
- A handful of services. A mesh adds proxies, control plane components and concepts that small estates do not need; a gateway plus good client libraries may be enough.
- Teams new to Kubernetes. Get the basics stable before adding a mesh.
- When the mesh is adopted only for encryption, and the platform already offers transparent encryption between nodes.
Key components
- Edge gateway. Terminates external TLS, authenticates consumers, applies rate limits and exposes versioned API products.
- Sidecar or node proxies. Intercept service-to-service calls to add mTLS, retries, timeouts and metrics.
- Mesh control plane. Issues workload certificates and distributes routing and authorisation policy to every proxy.
- Shared observability. Traces that follow a request from the edge through every internal hop.
Trade-offs
- Two systems to run. Separate upgrade cycles, configuration models and failure modes. Some vendors offer both under one control plane to reduce this.
- Resource overhead. Per-pod sidecars use memory and CPU; sidecar-less or node-level proxy designs reduce this but change the security model.
- Overlapping features. Both layers can retry and time out. Decide which layer owns each policy, or retries multiply during an incident.
- Learning curve. Engineers must understand where a request was rejected: at the edge, in a sidecar, or in the service.
Explore further
Lab: One request, eight plugins.→Follow a single AI request through an ordered chain of gateway plugins, switch any of them off and see what changes.
Lab: Same prompt. Three clocks.→Send one curated prompt to several LLM providers at once and compare time to first token, total time, tokens per second and cost.