Migrating gateway without cutting traffic

Every API gateway migration starts with the same sentence in committee: "we cut over during the Easter maintenance window". And it almost always ends the same way: the window is too short, you call it off after three hours, you roll back, and the project slips by a year.

The problem is not the window. The problem is believing that a gateway migration is an event, when it is a transitional regime you have to be able to sustain for months.

Why the big bang does not work here

A gateway is not a database. It holds no state to transfer: it holds rules, and those rules were written over several years by people who have left, in reaction to incidents nobody documented.

That is what makes this migration unusual:

• You cannot test what you do not know exists. The configuration holds special cases whose reason nobody remembers any more, a header transformation for a single consumer, a quota exception, a route to a fallback backend.
• Not every consumer is reachable. On a platform that has been around a while, a fraction of the callers are applications nobody owns any more.
• Rolling back is easy technically and impossible politically. After three retreats, the migration is dead, whatever the merits of the plan.

The method that works therefore runs both gateways side by side, moves the traffic in slices, and keeps the ability to roll back per consumer, not globally.

The method: six phases

1. Inventory what actually goes through

Before anything else: the list of routes actually called over a rolling 90 days, with their volumes, their error rates, and the identity of the caller.

This is not the list of declared APIs. It is almost always a shorter list, and it holds three or four entries nobody expected.

Anything not called in 90 days does not get migrated: you announce its deprecation and leave it on the old gateway until it dies out. It is the only moment in the project when you can cut scope without negotiating, and in our experience that is a quarter to a third of the routes.

2. Rebuild the rules, do not translate them

The temptation is to write an automatic converter from the old configuration to the new one. That is almost always a mistake: the policy models do not map one to one, and a converter faithfully reproduces the historical mistakes.

The right approach is to rebuild from intent: for each route, what is the business rule? The result is shorter, understandable, and it excludes the special cases that no longer have any purpose.

In practice: one sheet per route, three columns, expected authentication, limits and quotas, transformations. If nobody can fill a cell, that is a question to ask, not a cell to copy.

3. Run both gateways in parallel, with no real traffic

The new gateway is deployed, configured, and receives a copy of the traffic, either by mirroring (shadow traffic) or by replaying the access logs.

You do not compare functional responses at this stage: you compare status codes and latencies. A gap in status codes reveals a missing rule. A gap in latency reveals a sizing or DNS resolution problem better discovered now.

4. Cut over by consumer, not by route

This is the point that separates a migration that lands from a migration that bogs down.

Intuition pushes you to migrate route by route: "we start with the least critical APIs". But an incident on a route then hits every one of its consumers, including the ones you had not warned.

The correct order is the reverse: you cut over one consumer at a time, across all its routes. You start with an internal consumer, cooperative, reachable, with meaningful but not critical volume. If something goes wrong, a single team is affected, that team knows about it, and the rollback concerns it alone.

The cutover itself happens at DNS or load balancer level, with a weighted split: 5%, then 25%, then 50%, then 100%.

The promotion criterion is written before the first step, otherwise it gets negotiated under pressure. Three conditions are enough: the 5xx error rate does not exceed the old gateway's over the same period, latency at the 99th percentile stays within an announced margin, no status code gap has appeared.

Observation time counts as much as the thresholds. A step cleared on a Tuesday at three in the afternoon has seen nothing of the month-end batch: allow at least one full business cycle per step, more for the move to 100%.

A rollback is prepared, it is not improvised. A DNS switch is not reversible in a minute. Resolvers, and above all clients that hold a resolution in memory, keep sending traffic for the lifetime of the record. Bring the record's TTL down to one minute several days before the first step, long enough for the old value to expire everywhere.

And when recovery has to be immediate, do not go through DNS: the weighting at the load balancer changes in a few seconds, without depending on anyone. Announce the real delay to the consumers concerned. That is what separates a rollback from an incident.

5. Sustain the transitional regime

For several months, two gateways are in production. That requires:

• A single dashboard aggregating both, otherwise nobody knows where to look during an incident.
• An explicit rule on where a change is made during the transition. The default answer has to be "on the new one only", with a documented exception.
• An announced end date, failing which the transition becomes permanent. That is the most frequent outcome: the last tenth of the traffic stays on the old platform for years, with two platforms to maintain for it alone.

6. Shut it down, for real

The old gateway does not simply stop: you first put it into explicit rejection (410 Gone with a message naming the new URL and a contact), for four to six weeks, before unplugging it.

This step reveals the ghost consumers, the ones that appeared in no inventory and show up only when things break.

The five traps

TLS and client certificates. This is the first source of incidents. Consumers on mTLS have pinned something somewhere: an authority, a certificate, sometimes an IP address. Treat them as a separate workstream, upfront, and never in the same cutover step as the rest.

Outbound IP addresses. Backends and partners often filter by IP. The new gateway goes out through different addresses. That list has to be communicated several weeks ahead, and verified, not announced.

Differences in default behaviour. Two gateways do not handle duplicate headers, casing, encoded characters in paths, requests with no Host, or maximum body size the same way. These gaps do not show up in acceptance testing and break in production on one exotic caller.

The identity provider. If token validation changes mechanism, key cache, clock tolerance, accepted audiences, you will discover the problem under load, not in testing.

The organisation. Two gateways mean two on-call rotas, or one rota that has to know both. This point never appears in the migration plan and it is the one that wears the team down.

What it really costs

The advertised cost of a migration is the cost of rebuilding the rules. That is only a quarter of it. The split we observe, as a share of the total effort:

The practical consequence sits in the first row: the critical path is not your calendar, it is your consumers'. A migration is therefore planned backwards, from the acceptance windows of the teams that will have to move. The number of consumers you can cut over per quarter is the only figure that predicts the end date.

If you are preparing a migration and want an outside view of the plan before committing to it, that is exactly what the flash audit covers.