​IP Forwarding

• ​​Update the kernel.

  Some recent in-distro kernels have degraded routing performance due to kernel changes in the routing code starting with the removal of the routing cache due to security. Recent out-of-distro kernels should have patches that alleviate the performance impact of these changes and might provide improved performance.

• ​​Disable hyper-threading (logical cores).
• ​​Edit kernel boot parameters (useful for older kernels):
  • Force iommu off (intel_​iommu=off or iommu=off) from the kernel boot line unless required for virtualization.
  • Turn off power management: processor.max_cstates=1 idle=poll pcie_aspm=off
• ​​Limit number of queues to be equal to the number of cores on the local socket (32 in this example). ethtool -L ethX combined 32
• ​​Pin interrupts to local socket only (script packaged with ice driver source). set_irq_affinity -X local ethX
• ​​Enable symmetric queuing to align Tx/Rx queues (script packaged with ice driver source). set_xps_rxqs ethX
• ​​Change the Tx and Rx ring sizes as needed, matching Rx/Tx values typically improves forwarding performance. ethtool -G ethX rx 512 tx 512
• ​​Disable adaptive interrupt moderation and set a static value. ethtool -C ethX adaptive-rx off adaptive-tx off ethtool -C ethX rx-usecs 64 tx-usecs 64
• ​​Disable the firewall. sudo systemctl disable firewalld sudo systemctl stop firewalld
• ​​Enable IP forwarding. sysctl -w net.ipv4.ip_forward=1