Observability

Observability is what lets you monitor the API estate. All the traffic goes through the gateway, so that is where, and nowhere else, you see in one place an attack under way, a service degrading, a partner that started calling ten times more since this morning.

Hence the test that judges this layer, and it is a harsh one: when a service goes wrong, where does the alert come from? If the platform raises it, the layer is doing its job. If the service team discovers the outage through its own monitoring, and tells the platform team about it, the platform's observability has failed. It saw the same errors and the same latencies go past, and did nothing with them.

During an incident, then, only one question counts: is the degradation coming from the gateway, the backend, the identity provider, the network, or one particular consumer? A platform is observable when that question finds its answer in a few minutes, with the data already in place. Volume dashboards show that a problem exists, they do not locate it.

Three signals, one recurring gap

Metrics, aggregated and cheap, serve trends and alerts. Logs, detailed, serve the analysis of a specific case, provided they are structured. Traces follow a request across the components with the time spent in each: they are the ones that locate, and they are the ones most often missing.

Without distributed tracing, locating means correlating timestamps by hand across several tools, under full incident pressure. Every recent gateway can produce these traces, and the absence of tracing is a choice by omission, no longer a technical obstacle.

The format, on the other hand, has to be checked product by product. OpenTelemetry has established itself among open source and cloud native gateways, while the three large public cloud offerings stay on their proprietary telemetry: Application Insights at Azure, X-Ray at AWS, Cloud Trace at Google. The question to put to the vendor is therefore not "can you trace" but "to which collector, in which format, and at what egress cost".

Four dimensions at the gateway

Every call has to be recorded with at least:

• the consumer, in the sense of the application identity, OAuth2 client or API key, never the IP address alone: without it, attributing a degradation is impossible.
• the templated route, /clients/{id} and not the concrete URL, otherwise the number of metric series explodes.
• the status code, separating the families: a rise in 401 is an authentication problem, a rise in 429 a consumer past its quota, a rise in 502 a backend in trouble.
• latency broken down: total time, backend time, time spent in the gateway itself. The breakdown lets you attribute the latency, the total only lets you record it.

The correlation identifier

Every incoming request carries a unique identifier, propagated to all the components and present in all the logs. The gateway generates it when it is missing and keeps it when it is supplied, which lets a partner find their call again with their own identifier. It is returned to the consumer in a response header: a support ticket that contains it is handled in a few seconds of searching.

The point to watch is the crossing into asynchronous processing: message queues, deferred jobs. That is where the identifier gets lost, and that is often where the missing minutes of a latency are hiding.

Logging without leaking

Filtering has to exist before the first incident, because the reflex during an incident is to turn up verbosity, precisely the moment a leak happens. Never record:

• the access token: record its subject, its issuer and its jti identifier when it is present, failing that a fingerprint of the token, never its value, which is a usable credential.
• authorisation headers, including in debug traces.
• bodies containing personal data, unless there is explicit filtering, short retention and restricted access. Truncation does not automatically put the data out of scope: what counts is whether re-identification is possible from the other data available, not the nature of the field.

The indicators worth having

Average latency is the most misleading indicator in the field: an average of 120 ms can contain one per cent of calls at eight seconds, and those are the ones that reach support.

The probe that sees what the inside cannot

The instrumentation described above is internal. It knows nothing of what the consumer actually experiences: DNS resolution, TLS handshake, intermediate network. Synthetic monitoring, a real call run from outside at regular intervals with a real token, catches the failures that are invisible from inside: expired certificate, wrong DNS entry, firewall rule. It fails at the same time as the clients, which makes it the first useful alert.