Repository navigation
policy: implement NPRDS with NPDS fallback - #2154
Draft
jrajahalme wants to merge 11 commits into
Draft
jrajahalme wants to merge 11 commits into
jrajahalme wants to merge 11 commits into
Conversation
tools/bazel_adaptive.py runs Bazel with a concrete integer --jobs value, monitors progress output and memory pressure, stops and resumes bazel build processes as needed under memory pressure, and if that does not relieve memory pressure, restarts with lower or higher parallelism as memory allows. Integer --jobs values are used directly; Bazel-style HOST_CPUS and HOST_RAM values with optional multipliers, such as HOST_CPUS*.5 and HOST_RAM*.0002, are resolved once as the maximum adaptive jobs cap. The wrapper rewrites bazel output to indicate the number of paused jobs (if any). "(13 actions, 12 running)" is shown as "(13 actions, 2 paused, 10 running)" when 2 of the running actions have been paused by the wrapper. This wrapper is only modestly faster than builds with carefully tuned --jobs count and ample available swap space. The main benefit of this wrapper is that it removes the tuning work and automatically restarts bazel when if fails for out-of-memory or job termination. To allow Bazel to get through memory bottlenecks during the build without unnecessary restarts the build environment should have swap available. This wrapper was tested on a 24 GB VM with 16 GB swap. Running the wrapper with BAZEL_ADAPTIVE_DISABLE_PTY=1 defined allows more of the diagnostic lines to remain visible even when running in an interactive terminal. Each diagnostic line starts with "[bazel-adaptive/<XXX>s]", where "<XXX>" is the number of seconds since the wrapper started. Written by Codex through an iterative session where codex has run the included Python tests and has observed the behavior of live runs to observe the runtime behavior. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
This commit changes the Makefile to use the wrapper by default. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Refactor host map to new policy_host_map.h/cc that contains the xDS implementation while host_map.h/cc retains the core data structures. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Keep shared, immutable policies behind Policy and PolicyImpl, and make PolicyInstance a read-only borrowed value. Return const instances by value from policy resolvers and default-policy helpers, preserving the existing worker grace-period lifetime contract. Simplify policy reuse and lookup helpers, remove redundant code, and adapt ownership tests to retain policies explicitly. Pass resource type URLs to decoder factories and move Delta detection into the subscription. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Support policies referencing independently updated selector resources by name. Use stable, versioned selector handles so selector-only updates reach existing policies without rebuilding the policy lookup map. Automatically negotiate policy discovery for SotW and Delta over ADS and dedicated services. Try NPRDS first, falling back to NPDS if its first response is empty or its dedicated RPC returns UNIMPLEMENTED before any response. Supported NPRDS servers must send a nonempty initial response; later empty responses are ordinary updates. Retain published state during negotiation until a replacement update is accepted. Retry NPRDS after transport closure, while preserving healthy Delta NPRDS subscriptions across listener reconfiguration. Reject failing NPRDS resources individually, including validation and SDS initialization failures. Publish accepted changes atomically and NACK the response with each failed resource named using the LDS/CDS error-detail format. Legacy NPDS retains whole-update rejection. Replace the resource namespace on the first accepted update of a new stream. Accepting only selectors therefore removes all old policies, even if other resources cause the response to be NACKed. Rebind explicitly refreshed policies after selector removal and recreation. Keep enforcement lock-free and defer reclamation until worker threads have quiesced. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Bind Endpoint IPs and IDs to shared policies. Retain each Endpoint's immutable named ports for pod ingress enforcement, allowing endpoints sharing a policy to have independent mappings. Empty Endpoint mappings remain authoritative even when peer mappings exist. Maintain PolicyPeer address, security identity and named-port mappings using the shared HostMap implementation. Treat each peer resource as the complete state for its identity; removing it removes all associated addresses and named ports. Publish accepted Endpoint and peer changes atomically with policies and selectors, following NPRDS per-resource rejection semantics. Resolve peers at enforcement time so their updates remain independent of policy bindings. Skip peer-map rebuilding on same-stream updates that leave peer resources untouched. Store the original destination's PolicyInfo in filter state and carry identity and named ports together through network, HTTP and TLS enforcement. Resolve the selected destination again where routing may change it. Retained lookup results remain valid across later updates. Choose peer or legacy identity lookup from the published snapshot, preserving retained state during negotiation. Keep NPHDS and ipcache available for NPDS fallback. Separate ingress and egress port lookup. Evaluate numeric and matching named policies together, preserving verdict precedence and applying default deny only when no lookup produces a verdict. Reject nonempty port_name fields in legacy SotW and Delta NPDS, where the agent resolves names to numeric ports. Cover independent endpoint mappings, policy rebinding, peer lifecycle and prefix lookups, retained results, verdict precedence, backend selection and legacy NPDS rejection. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
jrajahalme
force-pushed
the
add-nprds-support
branch
from
November 2, 2026 10:34
7ed0e93 to
7a60e7c
Compare
NPRDS relies on Delta framing for passing the resource names. Rather than add another copy of the resource name for SotW make NPRDS Delta-only and keep using NPDS for SotW subscriptions, Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
NPRDS relies on Delta xDS, but Cilium agent's ADS currently does not implement Delta ADS, so switch to delta-split mode for now. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
jrajahalme
force-pushed
the
add-nprds-support
branch
from
November 2, 2026 14:17
24efc14 to
bd3a408
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add support for NetworkPolicyResourceDiscoveryService (NPRDS), where policy information is split into four sub-resources:
Agent support for NPRDS is automatically detected on new streams, falling back to legacy NPDS when not supported.