Repository navigation
Conversation
Enabling the kernel integration ([zebra.config] enabled = true) on a topology with directly-connected eBGP peers made the daemon withdraw routes it had just learned and forwarded. Sequence observed with two on-link peers (sender -> rustybgpd -> receiver): 1. Address monitoring injects each connected prefix into the BGP RIB as a kernel-sourced path (Handle::Address -> inject_kernel_route). 2. distribute_update unconditionally syncs the rank-1 best into the FIB, so the connected route is re-installed as 'via <own address> ... proto bgp' with .replace(), clobbering the kernel's connected 'dev ethX proto kernel scope link' entry. 3. Installing a BGP-learned route via that nexthop now fails with 'Network unreachable' (gateway resolution goes through the poisoned entry), and NHT marks the nexthop unreachable: lookup_route() rejects fib-match routes with protocol == Bgp, and the poisoned entry is exactly that. The daemon then withdraws the routes peers announced through it (receiver observes announce followed by withdraw). Fix: skip the FIB write when the new best path is kernel- or local- sourced. The kernel already owns these routes (connected entries and gRPC/CLI-injected static routes were never ours to install), so the skip is a no-op there, while export to peers and NHT registration are unchanged. A best path that vanished (new_best() == None) still syncs, so stale BGP-installed routes are withdrawn. Additionally, treat ESRCH from RTM_DELROUTE as success: a withdraw for a never-installed or already-removed prefix is routine and should not surface as 'kernel route update failed: No such process'. With the fix and [zebra.config] enabled, the same topology shows: connected entries intact, the learned route installed in the FIB (proto bgp), no netlink errors, and a stable announce on the downstream peer. Assisted-by: GLM (ZCode CLI) Signed-off-by: qfr23 <qfr23@mails.tsinghua.edu.cn>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Enabling the kernel integration (
[zebra.config] enabled = true) on atopology with directly-connected eBGP peers made the daemon withdraw
routes it had just learned and forwarded. Reproduced with a minimal
three-node lab (sender → rustybgpd → receiver, all on-link, peers inside
the connected subnets of the daemon):
kernel-sourced path (
KernelEvent::Address→inject_kernel_route).TableShard::distribute_updateunconditionally syncs the rank-1 bestpath into the kernel FIB — including kernel-sourced paths. The
connected route is re-installed as
10.0.0.0/24 via 10.0.0.1 dev eth1 proto bgp(via its own interface address) with
.replace(), clobbering thekernel's connected entry
(
10.0.0.0/24 dev eth1 proto kernel scope link src 10.0.0.1).fails: the gateway is resolved through the poisoned entry, and netlink
returns
Network unreachable:ERROR rustybgp_kernel] kernel route update failed: netlink: Received a netlink error message Network unreachable (os error 101)Handle::lookup_routerejects fib-match routes withprotocol == Bgp, and the poisoned entry is exactly that(
RouteProtocol::Bgp). The daemon therefore withdraws the routes thepeers announced through it — the downstream peer observes
announce followed by withdraw for a route that never changed.
RTM_DELROUTEhits anon-existent entry and logs
kernel route update failed: netlink: ... No such process (os error 3).The same topology without the kernel integration (or with a v0.1-style
setup) is stable: the learned route stays installed and advertised.
Root cause
distribute_updatetreats every best-path change as BGP-owned work. Forkernel-sourced (connected) and local-sourced (gRPC/CLI injected) paths
the FIB write is at best redundant — the kernel already owns the route —
and in the connected case actively harmful, because the path's nexthop is
the local interface address and reinstalling it via
.replace()destroys the connected route that makes on-link nexthops resolvable.
Fix
fib_sync_required()and call it fromdistribute_update: skipthe FIB sync when the new best path is kernel- or local-sourced. The
skip is a no-op for the kernel (it already owns those routes) and does
not affect BGP-side behavior: redistribution to peers, NHT registration
(
nht_registeralready skips these sources), and Loc-RIB visibilityare unchanged. A prefix whose best path vanished (
new_best() == None)still syncs, so a stale BGP-installed route is withdrawn when the
connected route wins back the prefix.
ESRCHfromRTM_DELROUTEas success inrustybgp_kernel::Handle::withdraw: a withdraw for a never-installedor already-removed prefix is routine (e.g. the kernel removed its
connected route first) and should not surface as an error log.
Verification
cargo test -p rustybgpd: 581 passed (7 new:fib_sync_required_*decision table for BGP / kernel / local / emptied / BGP-take-over /
kernel-restore cases, plus a Loc-RIB integration test).
cargo clippy -p rustybgp-kernel -p rustybgpdandcargo fmt --check: clean.End-to-end on a containerlab
sender → rustybgpd → receivereBGP labwith
[zebra.config] enabled = true, before vs after:via <own addr> proto bgpproto kernel scope link)Network unreachable)<learned prefix> via <peer addr> dev ethX proto bgp)gobgp global rib