Multipath: complete server-side NAT rebinding migration - #524
Open
aswanthk777 wants to merge 4 commits into
Open
Multipath: complete server-side NAT rebinding migration#524aswanthk777 wants to merge 4 commits into
aswanthk777 wants to merge 4 commits into
Conversation
added 4 commits
July 21, 2026 20:04
The server-side anti-amplification helpers (inc/dec/cmp_anti_ampl_limit) all stop applying the limit once verified_peer_address is set: the limit value is frozen from that point on. need_send_validation_frames and need_expand_padding_frames, however, kept consulting the frozen value. For a path opened by a small packet — exactly what a NAT rebinding looks like to a server — the sequence deadlocks: 1. The first packet on the new 4-tuple is small (e.g. a bare PING), so the path starts with anti_ampl_limit of only a few hundred bytes. 2. The server sends an unpadded PATH_CHALLENGE (padding is refused because anti_ampl_limit <= max_datagram_size), and the peer answers. 3. on_path_resp_received sets verified_peer_address, which freezes anti_ampl_limit; the path is promoted to ValidatingMTU and probes again, needing a challenge padded to MIN_CLIENT_INITIAL_LEN. 4. need_expand_padding_frames still compares the frozen limit against max_datagram_size and refuses the padding — forever. The path never reaches Validated, never becomes active, and the server keeps sending user data to the stale address. Fix: skip the anti-amplification checks in both helpers once the peer address is verified, matching the semantics of the other three helpers. Address-validated paths are exempt from the 3x limit per RFC 9000 Section 8; the MTU-validating padded challenge is then allowed out and validation completes. Found while adding server-side NAT rebind handling; covered by the conn_multipath_nat_rebind test added in a follow-up commit (the test deadlocks at ValidatingMTU without this fix).
RFC 9000 Section 5.1.1: "An endpoint SHOULD supply a new connection ID when the peer retires a connection ID." try_schedule_control_frames issues active_conn_id_limit - 1 source CIDs once, at handshake completion, and nothing ever tops the pool back up: every RETIRE_CONNECTION_ID permanently shrinks it. Any workload that retires CIDs over a connection lifetime — most notably a server retiring the superseded path CID after each NAT rebinding — eventually drains the pool, after which new paths cannot be created at all. Emit ScidToAdvertise(1) after processing a retirement so the endpoint generates a replacement, keeping the advertised pool at the size the peer asked for.
get_or_create_path already recognizes a NAT rebinding (a known CID arriving from a new 4-tuple), creates a new path sharing the packet number space, and queues a PATH_CHALLENGE. But the migration never completes, in two ways: 1. The new path is created with dcid_seq=None and nothing ever assigns one, and select_send_path skips dcid-less paths for probing — so the PATH_CHALLENGE is never actually sent. The path never validates, never becomes active, and the server keeps striping user data to the stale (dead) address until the connection idle-times-out. 2. Nothing retires the superseded path. Each rebinding permanently leaks one path slot and one DCID; insert_path only evicts unused() paths (inactive AND dcid-less), which a formerly-active path never becomes. After max_paths rebindings, path creation fails outright. Complete the migration: - Assign an unused DCID to the server-created path at creation time, so its PATH_CHALLENGE can actually be sent. - Track the superseded path in Path::migrated_from. When the rebind path completes validation (PATH_RESPONSE handler), deactivate the old path, drop its 4-tuple mapping (PathMap::retire_path) so a NAT flapping back can re-create it, and free its DCID via RETIRE_CONNECTION_ID — which, with the previous commit, also makes the peer issue a replacement, so chained rebindings never drain the CID pool. - Re-bind the packet SCID to the freshest path on every rebinding, so a later rebinding of the same CID never dereferences an evicted path id (and tolerate an already-evicted binding by treating the packet as opening a new path). The new conn_multipath_nat_rebind test drives two successive rebinds through an emulated NAT and asserts the server converges on the new tuple each time, retires the stale one, keeps exactly one active path, and still delivers stream data end-to-end. Note the test only passes together with the earlier anti-amplification fix: rebind paths open with a small packet, and without that fix validation deadlocks in ValidatingMTU. Verified live on a multipath IP tunnel (server on a VPS, client behind an emulated rebinding NAT): the server migrated within seconds of each rebinding with zero packet loss across a continuous 5/s ping stream, where previously all tunnel traffic was stranded until a 15s application-level liveness redial.
abandon_path() only set path.is_abandon = true — a flag that nothing reads anywhere in the codebase. The path stayed active, the schedulers kept sending on it, its DCID stayed bound, and its slot could never be evicted: for the caller the API was a silent no-op. Make it effective with the machinery the NAT rebinding migration introduced: deactivate the path and drop its 4-tuple mapping (PathMap::retire_path), and free its DCID via RETIRE_CONNECTION_ID so the peer issues a replacement. Also refuse to abandon the last active path — a connection with no active path cannot send at all. is_abandon is kept for a future PATH_ABANDON frame implementation (the frame codec exists but the frame is currently neither sent nor acted upon when received).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #523 (its commit is included as the base of this branch; the test below deadlocks without it).
get_or_create_pathalready recognizes a NAT rebinding (a known CID arriving from a new 4-tuple), creates a new path sharing the packet number space, and queues a PATH_CHALLENGE. But the migration never completes, in two ways:dcid_seq=Noneand nothing ever assigns one, andselect_send_pathskips dcid-less paths for probing — so the PATH_CHALLENGE is never actually sent. The path never validates, never becomes active, and the server keeps striping user data to the stale (dead) address until the connection idle-times-out.insert_pathonly evictsunused()paths (inactive AND dcid-less), which a formerly-active path never becomes. Aftermax_pathsrebindings, path creation fails outright.This series completes the migration (see individual commit messages):
Path::migrated_fromand retire it on validation (deactivate, drop its 4-tuple mapping, free its DCID via RETIRE_CONNECTION_ID); re-bind the packet SCID to the freshest path so chained rebindings never dereference an evicted path id. Adds aconn_multipath_nat_rebindtest driving two successive rebindings through an emulated NAT.is_abandonwas a write-only flag; the API was a silent no-op. Reuses the same retire machinery, and refuses to abandon the last active path.Verified live on a multipath IP tunnel (server on a VPS, client behind an emulated rebinding NAT): the server migrated within seconds of each rebinding with zero packet loss across a continuous ping stream, where previously all tunnel traffic was stranded until an application-level liveness redial. Full suite: 559/0.
🤖 Generated with Claude Code