Script B: Switchdev Mode with OVS Configuration
The following example shows step-by-step commands to create and bring up two VFs when the PF is in switchdev mode, and to configure data path through OVS. How to check OVS flows, offloaded fields, and filters is also included. Namespaces on the host allow for easy testing of the switchdev feature without VM creation, but a similar exercise could be done with VMs instead of namespaces.
===========================================================================================
#!/bin/bash
set -x
#set -e
DEVLINK=devlink
TC=tc
$BR=br1
PF1=ens4f0
PF1_PCI="pci/0000:af:00.0"
PF1_IP=192.168.66.16
VF1=ens4f0v0
VF2=ens4f0v1
VF1_PCI=0000:af:01.0
VF2_PCI=0000:af:01.1
VF1_MAC=52:54:00:00:16:01
VF2_MAC=52:54:00:00:16:02
VF1_IP=192.168.66.161
VF2_IP=192.168.66.162
VF1_PR=eth0
VF2_PR=eth1
PEER_IP=192.168.66.10
MASK=24
PEER_MAC=68:05:ca:a3:7b:10
#1. Load the ICE driver
rmmod ice
modprobe ice
sleep 2
#2. Install the Open vSwitch package & start the service
#2.1. Install the OVS package
zypper install openvswitch
#2.2. Start the open vSwitch service
systemctl status openvswitch
#2.3. Check the vSwitch status
systemctl status openvswitch
#3. Create an OVS bridge
ovs-vsctl add-br $BR
#4. Add PF as an Uplink Port to the bridge:
ovs-vsctl add-port $BR $PF1
ovs-vsctl show
#5. Check and change the Mode to switchdev
#eSwitch mode could be changed only if there are no VFs created and the PF has been added to the OVS bridge
#PF1_PCI should look like that: pci/0000:03:00.1
#To Find it lspci -D | grep Eth
#5.1. Show current eSwitch mode - should be legacy
devlink dev eswitch show $PF1_PCI
#5.2. Change eSwitch mode to switchdev
devlink dev eswitch set $PF1_PCI mode switchdev
#5.3. Show current eSwitch mode - should be switchdev
devlink dev eswitch show $PF1_PCI
sleep 2
#6. Enable SRIOV and create 2 VFs
echo 2 > /sys/class/net/$PF1/device/sriov_numvfs
sleep 2
#7. Enable hw-tc-offload on PF (Uplink port) and VF Port Representors
ethtool -K $PF1 hw-tc-offload on
ethtool -K $VF1_PR hw-tc-offload on
ethtool -K $VF2_PR hw-tc-offload on
#8. Configure OVS (Enable hardware offload, which is disabled by default)
ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
# tc flow placement. one of: none, skip_sw, skip_hw
ovs-vsctl set Open_vSwitch . other_config:tc-policy=skip_sw
#9. Restart Open Switch service
systemctl restart openvswitch
#10. Add VF Port Representors to OVS bridge
ovs-vsctl add-port $BR $VF1_PR
ovs-vsctl add-port $BR $VF2_PR
#Set them to UP State
ip link set $PF1 up
ip link set $VF1_PR up
ip link set $VF2_PR up
#11. Configure VFs
#create 1 network namespace for each VF
ip netns add ns1 2> /dev/null
ip netns add ns2 2> /dev/null
sleep 1
#12. Move VFs to namespaces
ip link set $VF1 netns ns1
ip link set $VF2 netns ns2
#13) Set VFs to up state and give them IP Addresses
ip netns exec ns1 ip link set $VF1 up
ip netns exec ns2 ip link set $VF2 up
ip netns exec ns1 ip a $VF1_IP/$MASK dev $VF1
ip netns exec ns2 ip a $VF2_IP/$MASK dev $VF2
#14. Enable bridge
ip link set $BR up
#15. Check connections and Watch rules being added via tc tool
#ping 2nd VF from 1st VF
ip netns exec ns1 ping -c3 $VF2_IP
#16. Watch rules being added via tc tool
tc -s monitor
# If Flag shows as in_hw that means flows are offloaded to hardware
#17. Check connections and Watch rules offloaded via OVS
ip netns exec ns1 ping -c3 $VF2_IP
ovs-appctl dpctl/dump-flows -m
# In output, check for offload and datapath field as offloaded: yes, dp:tc, it means flows are offloaded to HW.
===========================================================================================