Operate

An API platform is chosen on functions and judged on operations. The gateway is the only component every call goes through: what it cannot do on a Sunday at three in the morning decides the value of everything else.

This page deals with the gateway itself. How the chain survives when a backend degrades is another subject, that of resilience.

Its availability is an architectural property

A single-instance gateway makes the platform a single point of failure, and that is the starting configuration of almost every project. Moving to a tenable one takes three properties, in this order.

Several instances, spread across several zones. Two instances in the same zone protect against a process failure, not against an infrastructure failure. Sizing targets the loss of a whole zone with no loss of capacity at peak traffic. That amounts to carrying a permanent reserve you pay for without using it.

Survival when the control plane is lost. A gateway must keep serving with its last known configuration while the control plane is down. It is the property you test by switching it off for a day, described in the control plane, and it is missing more often than people think.

The fate of the dependencies on the path. The identity provider and the counter store are called during processing. Decide explicitly what happens when they do not answer: serve degraded on the last known state, or refuse. Both answers can be defended, the absence of an answer is paid for in incidents.

What the instances share, and the quota trap

This is the most widespread configuration flaw, and it is invisible in acceptance testing.

A gateway replicated over five instances, each of them counting calls in its own memory, applies five times the announced quota. The consumer limited to a hundred calls a minute makes five hundred, and nobody notices while the backend holds. The day it stops holding, the quota is blamed for not playing its part, when it played the one it was given.

Three pieces of state are therefore settled explicitly.

Sharing has a cost: a round trip to a shared store on the path of the call, and one more failure mode. The usual answer is the compromise: a local counter for protection by the second, a shared counter for the contractual quota by the day. The choice is made consumer by consumer, never globally.

Sizing, and the measurement that lies

A gateway is sized by measurement, and the measurement almost always goes wrong the same way: an empty route is tested, with no policy, with tiny payloads. The figure obtained is the proxy's, not the platform's.

A useful measurement meets four conditions. The real policy chain is switched on, since that is where the cost is. The payloads are the size of the real ones. The load ramp lasts long enough to trigger the runtime pauses and the connection renewals, which a two-minute run never does. And the indicator to keep is the 99th percentile of the gateway's own latency, not the maximum throughput: a saturated gateway still answers, more and more slowly.

The reflex that saves months of suspicion is to measure the cost policy by policy, in a test environment, before stacking them in production. Half a day is enough, and the result then serves every discussion about latency.

Upgrading without downtime

This is the operation that will come back several times a year, and the question to put to the vendor before buying. Four points decide whether it goes well.

The gradual withdrawal of instances. An instance being stopped must cease to receive new requests, then finish the ones it is handling, before shutting down. Without that delay, every deployment produces errors at the consumers, in small numbers, which makes them easy to ignore and impossible to explain.

Long-lived connections. An SSE stream, a WebSocket or a gRPC stream does not end on its own: it lasts for hours. Gradual withdrawal does not drain them, it waits. Decide whether you cut them, and in that case the client has to know how to reconnect, which is checked before the first upgrade, not during it.

The order between control plane and data plane. The two are not upgraded together, and compatibility between neighbouring versions is a property of the product. The exact question to ask: which data plane versions a given control plane can drive, and for how long.

Rolling back. It must be possible without restoring a backup, so without an irreversible migration of the configuration format. An upgrade that transforms the configuration store with no way back is not an upgrade, it is a migration.

Running an incident

Diagnosing an API platform always comes back to the same question: where the time goes, or who is refusing. Four suspects, in order of frequency.

1. The backend. The majority case, and the least examined, because the gateway offers a more convenient culprit.
2. The outbound connections. The gateway waits for a free connection before it even sends the request. The backend shows normal times, the client measures double.
3. The identity provider. Expired key cache, synchronous introspection, quota reached. Typical symptom: latency in steps.
4. A single consumer. The average rises because one caller has started sending pathological requests. The breakdown by consumer singles it out in one query.

One artefact alone makes that sorting possible, and it is prepared before the incident: a call's time split into measured segments, inbound network, gateway processing, connection wait, backend, return. If the platform can give those five figures for a given call, the incident is steered in minutes. Otherwise it is settled by teams blaming each other, and that is the sign that observability stopped at the volume dashboards.

What makes operations bearable

Three things, none of them a product.

An on-call rota of four people at least, failing which it rests on one or two and collapses at the first departure. A configuration versioned and published by a pipeline, since console changes with no history start most avoidable incidents, the whole subject of industrialise. And a transit time published by the platform team itself, in percentiles, per API: that is what turns the first minute of an incident into a diagnosis, not a trial.