From 08c4854cbde13a76d66e3de3dd5d2496cb2278f5 Mon Sep 17 00:00:00 2001 From: Jason Patterson Date: Mon, 5 Oct 2026 21:23:08 +0000 Subject: [PATCH 1/2] SONiC containerlab: name the endpoints ethN so docker-sonic-vs maps its ports docker-sonic-vs builds lanemap.ini/port_config.ini from the ethN netdevs present at boot, and syncd then creates a TAP named Ethernet<4n> for each port it maps a veth to -- eth1 to Ethernet0, eth2 to Ethernet4, and so on, the contract containerlab documents for the sonic-vs kind. The veth keeps its name; both netdevs exist. The device declared no clab.interface.name, so containerlab injected the veth as Ethernet0 itself: the lane map came out empty, syncd failed to create every TAP (TUNSETIFF: name in use), and SONiC port state and admin control were detached from the link. "show interfaces status" reported oper/admin down on a port carrying a Full OSPF adjacency, and "config interface shutdown" changed nothing. Declare clab.interface.name eth{ifindex + 1} -- ifindex starts at 0 here and eth0 is management, the same offset crpd, csrx, vmx, vptx, vsrx and the two vjunos devices already use. With the port a syncd TAP rather than the raw veth, CONFIG_DB reaches the netdev again: the MTU goes back through "config interface mtu" (measured on a 9000-byte link -- netdev, CONFIG_DB and APPL_DB all agree, an 8972-byte DF ping crosses) and the loop that mirrored CONFIG_DB INTERFACE rows onto the netdev with "ip addr add" is gone, because intfmgrd programs routed-port addresses. portmgrd still leaves every front-panel port admin-down at boot, so "config interface startup" stays, now issued for the ports the lab uses and under the script's set -e. The vxlan template loses "routed-port IPs" from the list of gaps it works around for the same reason, and its live-proof note is qualified: that datapath enslaves what is now a TAP, and it was measured before this change. The vlan, lag and vxlan datapaths are otherwise out of scope: those features are off on this device (vlan: False, lag: False, vxlan: false) and were not re-measured. Nothing here touches the wire veth. Left alone it answers ARP for the port's address with its own MAC, which is containerlab's to fix because containerlab creates it, and srl-labs/containerlab#3390 now does what SONiC's own virtual switch harness does and marks it ARP-off and IPv6-off. The transformation case pins the container name on the node interface and, by covering the whole transformed topology, also pins that the node keeps its own config_templates rather than inheriting FRR's bind mounts. Co-Authored-By: Claude Fable 5.1 Co-Authored-By: Claude Opus 5 --- .../ansible/templates/initial/sonic-clab.j2 | 27 ++-- netsim/ansible/templates/vxlan/sonic-clab.j2 | 6 +- netsim/devices/sonic.yml | 5 + .../topology/expected/sonic-clab-ifnames.yml | 127 ++++++++++++++++++ tests/topology/input/sonic-clab-ifnames.yml | 11 ++ 5 files changed, 155 insertions(+), 21 deletions(-) create mode 100644 tests/topology/expected/sonic-clab-ifnames.yml create mode 100644 tests/topology/input/sonic-clab-ifnames.yml diff --git a/netsim/ansible/templates/initial/sonic-clab.j2 b/netsim/ansible/templates/initial/sonic-clab.j2 index ffcb3b1a27..927a2c78da 100644 --- a/netsim/ansible/templates/initial/sonic-clab.j2 +++ b/netsim/ansible/templates/initial/sonic-clab.j2 @@ -109,33 +109,22 @@ else fi {% endif %} {% endif %} -{# Set the MTU on the kernel netdev, not through "config interface mtu": on the interfaces - containerlab injects for netlab links that command writes CONFIG_DB and never reaches the - netdev, exactly like the routed-port IPs below -- measured, the interface stayed at 9500 - while CONFIG_DB claimed otherwise. The value is netlab MTU with no adjustment: - docker-sonic-vs programs it literally (1548 in -> 1548 on the netdev), so the +48 this - used to add left every interface 48 bytes above its peers and stalled OSPF in ExStart. #} +{# The port is the TAP syncd manages, so portmgrd programs the CONFIG_DB MTU onto the netdev. #} {# L2 switchports (LAG members and VLAN access/trunk ports) reject a direct MTU set once they are members ("'interface_name' is in portchannel!" / "is in vlan"); the PortChannel/port MTU propagates, so skip them #} {% if l.mtu is defined and l.mtu >= 1500 and not (l.virtual_interface|default(False)) and l.lag._parentindex is not defined and l.vlan.access_id is not defined and l.vlan.trunk_id is not defined %} -ip link set {{ l.ifname }} mtu {{ l.mtu }} +config interface mtu {{ l.ifname }} {{ l.mtu }} {% endif %} ! {% endfor %} # -# docker-sonic-vs's intfmgrd/orchagent do not program routed-port IPs from CONFIG_DB onto the -# kernel netdev (loopbacks work; routed ports do not), so BGP/OSPF over links can't come up -# without this. Bring the data ports admin-up and mirror the CONFIG_DB INTERFACE IPs onto the -# kernel netdev. Idempotent. -# -for p in $(redis-cli -n 4 keys "PORT|Ethernet*" 2>/dev/null | sed 's/PORT|//'); do - config interface startup "$p" >/dev/null 2>&1 || true -done -redis-cli -n 4 keys "INTERFACE|Ethernet*|*" 2>/dev/null | while IFS= read -r k; do - ifc=$(echo "$k" | cut -d'|' -f2); addr=$(echo "$k" | cut -d'|' -f3) - [ -n "$addr" ] && ip addr add "$addr" dev "$ifc" 2>/dev/null || true -done +# portmgrd leaves every front-panel port admin-down at boot, so start the ones this lab +# uses. clab.name is set on exactly those: the interfaces containerlab creates. +# +{% for l in interfaces if l.clab.name is defined %} +config interface startup {{ l.ifname }} +{% endfor %} # # Enable the FRR daemons needed by the configured modules (docker-sonic-vs ships most of them # disabled) and restart FRR once, only if something actually changed. diff --git a/netsim/ansible/templates/vxlan/sonic-clab.j2 b/netsim/ansible/templates/vxlan/sonic-clab.j2 index 69738dd449..ac36224d54 100644 --- a/netsim/ansible/templates/vxlan/sonic-clab.j2 +++ b/netsim/ansible/templates/vxlan/sonic-clab.j2 @@ -8,12 +8,14 @@ # SONiC EVPN-VXLAN L2VNI — FRR/kernel data-plane. # # docker-sonic-vs's orchagent/vxlanmgrd do NOT program the VXLAN dataplane (same class -# of gap as routed-port IPs and VLAN members), so the config_db VXLAN_TUNNEL model gives +# of gap as VLAN members), so the config_db VXLAN_TUNNEL model gives # no datapath on the VS. Instead we build the L2VNI exactly the way FRR/Cumulus does on # plain Linux — a traditional per-VNI bridge holding the access port(s) + a kernel vxlan # netdev — and let FRR zebra drive EVPN off that kernel state (advertise-all-vni). This # is the same path proven live between two VS nodes (VNI up, remote VTEP learned, iBGP -# l2vpn evpn Established, host-to-host ping across the tunnel). +# l2vpn evpn Established, host-to-host ping across the tunnel) -- measured when the port +# was the containerlab veth. It enslaves the port into a Linux bridge, which is now the +# syncd TAP, and has not been re-measured; the vxlan feature is off on this device. # # config_db is intentionally bypassed for VNI-backed VLANs: initial/sonic-clab.j2 skips # `config vlan add` for them and vlan/sonic-clab.j2 skips their member-add, so nothing competes diff --git a/netsim/devices/sonic.yml b/netsim/devices/sonic.yml index 76b6c85916..6d3f27b504 100644 --- a/netsim/devices/sonic.yml +++ b/netsim/devices/sonic.yml @@ -89,6 +89,11 @@ clab: # key-value passthrough loop, so this device needs no provider template of its own. image: docker-sonic-vs:latest mtu: 1500 + # docker-sonic-vs maps its front-panel ports from the ethN netdevs present at boot + # (eth1 -> Ethernet0): the containerlab endpoint is ethN, Ethernet<4n> is the SONiC-side + # name netlab configures. eth0 is management and ifindex starts at 0, hence the +1. + interface: + name: eth{ifindex+1} node: kind: sonic-vs # Do NOT inherit FRR's bind-mounted /etc/frr/daemons and /etc/hosts config templates: diff --git a/tests/topology/expected/sonic-clab-ifnames.yml b/tests/topology/expected/sonic-clab-ifnames.yml new file mode 100644 index 0000000000..1e485c9496 --- /dev/null +++ b/tests/topology/expected/sonic-clab-ifnames.yml @@ -0,0 +1,127 @@ +groups: + netlab_no_reload: + members: + - s1 + - s2 +input: +- topology/input/sonic-clab-ifnames.yml +- package:topology-defaults.yml +links: +- _linkname: links[1] + interfaces: + - ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.1/30 + node: s1 + - ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.2/30 + node: s2 + linkindex: 1 + node_count: 2 + prefix: + ipv4: 10.1.0.0/30 + type: p2p +name: input +nodes: + s1: + _node_config: + initial: /etc/config/01-initial.sh:sh + af: + ipv4: true + box: docker-sonic-vs:latest + clab: + binds: + - source: node_files/s1/initial + target: /etc/config/01-initial.sh + config_templates: + - mode: sh + source: initial + target: /etc/config/01-initial.sh + exec: + - sleep 1 + kind: sonic-vs + device: sonic + hostname: clab-input-s1 + id: 1 + interfaces: + - clab: + name: eth1 + ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.1/30 + linkindex: 1 + mtu: 1500 + name: s1 -> s2 + neighbors: + - ifname: Ethernet0 + ipv4: 10.1.0.2/30 + node: s2 + type: p2p + loopback: + ifindex: 0 + ifname: Loopback0 + ipv4: 10.0.0.1/32 + neighbors: [] + type: loopback + virtual_interface: true + mgmt: + ifname: eth0 + ipv4: 192.168.121.101 + mac: ca:fe:00:01:00:00 + mtu: 1500 + name: s1 + netlab_ansible_skip_module: + - initial + role: router + s2: + _node_config: + initial: /etc/config/01-initial.sh:sh + af: + ipv4: true + box: docker-sonic-vs:latest + clab: + binds: + - source: node_files/s2/initial + target: /etc/config/01-initial.sh + config_templates: + - mode: sh + source: initial + target: /etc/config/01-initial.sh + exec: + - sleep 1 + kind: sonic-vs + device: sonic + hostname: clab-input-s2 + id: 2 + interfaces: + - clab: + name: eth1 + ifindex: 0 + ifname: Ethernet0 + ipv4: 10.1.0.2/30 + linkindex: 1 + mtu: 1500 + name: s2 -> s1 + neighbors: + - ifname: Ethernet0 + ipv4: 10.1.0.1/30 + node: s1 + type: p2p + loopback: + ifindex: 0 + ifname: Loopback0 + ipv4: 10.0.0.2/32 + neighbors: [] + type: loopback + virtual_interface: true + mgmt: + ifname: eth0 + ipv4: 192.168.121.102 + mac: ca:fe:00:02:00:00 + mtu: 1500 + name: s2 + netlab_ansible_skip_module: + - initial + role: router +provider: clab diff --git a/tests/topology/input/sonic-clab-ifnames.yml b/tests/topology/input/sonic-clab-ifnames.yml new file mode 100644 index 0000000000..771f864100 --- /dev/null +++ b/tests/topology/input/sonic-clab-ifnames.yml @@ -0,0 +1,11 @@ +# +# docker-sonic-vs maps its front-panel ports from the ethN netdevs present at boot +# (eth1 -> Ethernet0), so the containerlab endpoint name must be ethN. +# +provider: clab +nodes: + s1: + device: sonic + s2: + device: sonic +links: [ s1-s2 ] From 96bac67c1aa30915252f4b491b3d49e721ffad02 Mon Sep 17 00:00:00 2001 From: Jason Patterson Date: Mon, 5 Oct 2026 21:23:08 +0000 Subject: [PATCH 2/2] SONiC containerlab: document the two-interface port layout A containerlab SONiC node has two kernel interfaces per link: the wire veth that containerlab creates, which keeps its ethN name, and the port that syncd creates for it, four apart in the numbering. Only the port is configured and addressed. The second bullet is the part a user can act on: an older containerlab release leaves the veth answering ARP for the port's address, so say to use a recent one, and point at srl-labs/containerlab#3390, which marks the veth ARP-off and IPv6-off so only the port answers on the link. Co-Authored-By: Claude Fable 5.1 Co-Authored-By: Claude Opus 5 --- docs/caveats.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/caveats.md b/docs/caveats.md index f628ad5506..34e2e291b4 100644 --- a/docs/caveats.md +++ b/docs/caveats.md @@ -733,6 +733,8 @@ See also [](caveats-sros) caveats for further details. The `sonic` device also runs under *containerlab* with the community `docker-sonic-vs` image; see [](build-sonic-container) for how to obtain it and how the two deployments differ. +* Every port is two kernel interfaces: the wire veth that *containerlab* creates, which keeps its `ethN` name, and the port that SONiC derives from it, which is named four apart -- `eth1` is the wire behind `Ethernet0`, `eth2` behind `Ethernet4`. Configure and address only the port; the veth just carries its frames. +* Use a recent *containerlab* release with this device. Older releases leave that veth answering ARP for the port's address, so a neighbor can cache the veth's MAC instead of the port's. [Current *containerlab* code](https://github.com/srl-labs/containerlab/pull/3390) marks the veth ARP-off and IPv6-off, leaving only the port answering on the link. * Configuration is deployed with **docker exec** commands, not over an SSH session. * `docker-sonic-vs` ships `sshd` but does not start it. * `srv6` is control-plane and kernel-plane only: the locator and End/End.X SIDs are advertised in the IS-IS LSDB and installed as kernel `seg6local` routes, but the end-to-end SRv6 datapath does not resolve -- the same open item as FRR/IS-IS SRv6 on other platforms.