Script C: Switchdev Mode with VXLAN/GRETAP/GENEVE/GTP Linux Bridge Configuration
The following commands are used to create and bring up a VF in switchdev mode, and to configure TC-Flower filters on VXLAN/GRETAP/GENEVE/GTP tunnels.
The DDP comms package is required by the parser to distinguish the GTP traffic. Refer to the Intel® Ethernet Controller E810 Dynamic Device Personalization (DDP) Technology Guide for loading the DDP comms package. ===========================================================================================
#!/bin/bash set -x
#set -e
DEVLINK=devlink
TC=tc
BR=br0
PF1=ens9f1 #PF whose eSwitch will be configured in switchdev mode. Change accordingly.
PF1_PCI="pci/0000:4b:00.1"
VF1=ens9f1v0
VF1_MAC=52:54:00:00:16:01
VF1_PR=eth0
TNL_IP=172.31.123.11
PEER_TNL_IP=172.31.123.12
INNER_IP=172.31.100.11
PEER_IP=172.31.100.12
MASK=24
TNL_KEY_OR_ID=100 # GRETAP KEY or VXLAN/GENEVE ID
TNL_NAME=tnl100
GTP_TEID=1234
GTP_OPTS=00:2b/00:ff # <pdu type>:<qfi>/<pdu mask>:<qfi mask>
#1. Make sure that there are no VFs
echo 0 > /sys/class/net/$PF1/device/sriov_numvfs
#2. Create a bridge
ip link add $BR type bridge 2> /dev/null
# To allow PF to be added to bridge as uplink
# PF needs to be added to bridge prior to entering switchdev and creating VFs
#3. Add PF as UpLink port to the bridge
ip link set $PF1 master $BR
#4. Change eSwitch mode to switchdev
$DEVLINK dev eswitch set $PF1_PCI mode switchdev
# Check the current eSwitch mode
$DEVLINK dev eswitch show $PF1_PCI
#5. Create 1 SR-IOV VF
echo 1 > /sys/class/net/$PF1/device/sriov_numvfs
#6. Configure VF MAC Address
ip link set $PF1 vf 0 mac $VF1_MAC
#7. Add VF Port Representor to the bridge and bring it up
ip link set $VF1_PR master $BR
ip link set $VF1_PR up
ip link set $PF1 up
ip link set $BR up
#8. Create 1 network namespace: ns1
ip netns add ns1 2> /dev/null
sleep 10
#9. Move VF1 to ns
ip link set $VF1 netns ns1
#10. Create a tunnel (VXLAN/GRETAP/GENEVE) on PF and VF
1. VXLAN Tunnel:
ip link add name $TNL_NAME type vxlan id $TNL_KEY_OR_ID dstport 4789 dev $PF1
ip netns exec ns1 ip link add $TNL_NAME type vxlan id $TNL_KEY_OR_ID remote $PEER_TNL_IP
dstport 4789 dev $VF1
2. GRETAP Tunnel:
ip link add name $TNL_NAME type gretap local $TNL_IP remote $PEER_TNL_IP key $TNL_KEY_OR_ID
dev $PF1
ip netns exec ns1 ip link add name $TNL_NAME type gretap local $TNL_IP remote $PEER_TNL_IP
key $TNL_KEY_OR_ID dev $VF1
3. GENEVE Tunnel:
ip link add name $TNL_NAME type geneve id $TNL_KEY_OR_ID remote $PEER_TNL_IP dstport 6081
ip netns exec ns1 ip link add $TNL_NAME type geneve id $TNL_KEY_OR_ID remote $PEER_TNL_IP
dstport 6081
4. GTP Tunnel:
# Only GTP role SGSN is supported
ip link add name $TNL_NAME type gtp role sgsn
ip netns exec ns1 ip link add name $TNL_NAME type gtp role sgsn
#11. Add IP Addresses and bring up the VF and tunnel interface created on VF
ip netns exec ns1 ip link set $VF1 up
ip netns exec ns1 ip addr add $TNL_IP/$MASK dev $VF1
ip netns exec ns1 ip link set $TNL_NAME up
ip netns exec ns1 ip addr add $INNER_IP/$MASK dev $TNL_NAME
#12. Bring up the tunnel interface created on PF
ip link set $TNL_NAME up
# Enable hw-tc-offload on PF (Uplink port) and VF Port Representors
#13. To offload tc filters to the hardware hw-tc-offload must be enabled on the VFs Port Representor (VF_PR)
ethtool -K $PF1 hw-tc-offload on
ethtool -K $VF1_PR hw-tc-offload on
# Verify settings:
ethtool -k $PF1 | grep "hw-tc"
ethtool -k $VF1_PR | grep "hw-tc"
#14. Enable ingress qdisc on Tunnel port (Uplink port) and VF Port Representors
$TC qdisc add dev $TNL_NAME ingress
$TC qdisc add dev $VF1_PR ingress
#15. Add filter to offload to hardware
# skip_sw flag is not applicable for tunnel filters.
1. VXLAN/GRETAP/GENEVE Tunnel TC filter configuration:
# Add tc filter for ingress traffic
$TC filter add dev $TNL_NAME protocol ip parent ffff: flower enc_key_id
$TNL_KEY_OR_ID src_ip $INNER_IP dst_ip $PEER_IP action mirred egress redirect dev $VF1_PR
# Add tc filter for egress traffic
$TC filter add dev $TNL_NAME protocol ip parent ffff: flower enc_key_id
$TNL_KEY_OR_ID src_ip $PEER_IP dst_ip $INNER_IP action mirred egress redirect dev $VF1_PR
2. GTP Tunnel TC filter configuration:
$TC filter add dev $TNL_NAME ingress priority 1 flower enc_key_id $GTP_TEID gtp_opts
$GTP_OPTS action mirred egress redirect dev $VF1_PR
# Verify filter programming
$TC filter show dev $TNL_NAME ingress
#16. Do a ping from VF1 to PEER_IP
ip netns exec ns1 ping $PEER_IP
===========================================================================================