Internal traffic and the layer cake
When every internal call crosses four network layers, each justified on its own. How the stack builds up, what it costs, and how to reduce it.
A call between two internal services rarely crosses a single layer. In the platforms we come across, it crosses three or four: a load balancer, a cluster ingress, an API gateway, sometimes a mesh proxy. Each was added for a good reason, none of them was ever removed, and nobody owns the stack.
None of these layers is absurd taken on its own, and that is exactly the problem. The load balancer predates the platform, the ingress arrived with Kubernetes, the gateway with API management, the mesh with encryption between services. Each was justified before a different board, at a different time, by a different team. The question "how many hops for an internal call" was never put to anyone, because it belongs to none of those projects.
What it costs
Latency, and above all variance. Every hop adds its own processing and a network crossing: count two to five milliseconds per correctly sized layer, so ten to twenty milliseconds for a stack of four. On an internal call that should take twenty, the plumbing doubles the response time.
But the real cost is not that average, it is the variance. Four layers are four chances to land on a full queue, a runtime pause or a connection renegotiation. Each layer has its own 99th percentile, and they do not happen at the same moment: a call is therefore four times as likely to hit one. That is why the stack shows on the high percentiles, the ones that produce the tickets, and not on the dashboard displaying a reassuring average.
And infrastructure. A layer that applies no useful policy still consumes instances, certificates, monitoring and a place in every on-call rotation. It is the easiest figure to pull out for a simplification case, and the one nobody remembers to count.
Diagnosis becomes impossible. When latency rises, the question becomes "which of the four" and nobody can answer it without correlating logs from four different tools. This is the situation observability is there to avoid, and the stack mechanically makes it harder.
Duplicated policies, or contradictory ones. Two layers apply rate limiting with different thresholds, two layers retry, and a retry of a retry multiplies the load on a backend already in trouble. That case is covered in resilience: retries do not compose, they multiply.
Workarounds. The highest cost is not technical. An internal team subjected to four hops, to quotas designed for strangers and to a publication delay ends up calling the service directly. The platform loses its visibility, and that is the subject of why your teams bypass the platform.
The principle that sorts the stack out
One simple rule settles most cases: one layer per boundary crossed, not one layer per function wanted.
An internal call between two services of the same trust domain crosses no boundary. It therefore needs no API gateway. What it does need, service identity, encryption, telemetry, retries, belongs to the network or to the mesh. On one condition, that the mesh is not itself one more hop: that is what a mesh without sidecars changes, or telemetry taken at kernel level. The precise split is in gateway and service mesh.
A call that crosses a trust boundary, between domains, to a partner, from outside, goes through the gateway, and once only.
What it looks like in practice
| Type of call | What it should cross |
|---|---|
| Service to service, same domain | The network, with identity and encryption |
| Service to service, different domains | An internal gateway, one only |
| External consumer | TLS termination, then the exposure gateway |
| Platform administration | A separate path, never the traffic one |
Applying this table often removes an entire layer, and it is almost always the API gateway on internal traffic within the same domain. It had been put there for visibility. Visibility is better obtained from network telemetry, without sitting on the path.
Where to start
The prerequisite is an inventory, and it fits on one page: for three representative internal calls, the list of hops, with the time spent in each. That document is built in a day with distributed traces and it almost always produces at least one surprise: a hop nobody knew was there, or a policy applied twice.
Removal then proceeds layer by layer, starting with the one that applies no policy and produces no telemetry the others do not produce already. There is almost always one.
One precaution: removing a layer changes the source addresses seen by downstream services. The network filtering rules that rely on them must be revised beforehand, and that is the kind of detail that turns a simplification into an incident.
Published in March 2026.
On the same subject