​IP Forwarding

• ​​Update the kernel.

  Some recent in-distro kernels have degraded routing performance due to kernel changes in the routing code starting with the removal of the routing cache due to security. Recent out-of-distro kernels should have patches that alleviate the performance impact of these changes and might provide improved performance.

• ​​Disable hyper-threading (logical cores).
• ​​Edit kernel boot parameters (useful for older kernels):
  • Force iommu off (intel_​iommu=off or iommu=off) from the kernel boot line unless required for virtualization.
  • Turn off power management: processor.max_cstates=1 idle=poll pcie_aspm=off
• ​​Limit number of queues to be equal to the number of cores on the local socket (32 in this example). ethtool -L <ethX> combined 32
• ​​Pin interrupts to local socket only (script packaged with ice driver source). set_irq_affinity -X local <ethX>

  OR

  set_irq_affinity -X local <ethX> Note:"-X or -x" can be used depending on the workload.

• ​​Enable symmetric queuing to align Tx/Rx queues (script packaged with ice driver source). set_xps_rxqs <ethX>
• ​​Change the Tx and Rx ring sizes as needed, matching Rx/Tx values typically improves forwarding performance. ethtool -G <ethX> rx 512 tx 512
• ​​Disable adaptive interrupt moderation and set a static value. ethtool -C <ethX> adaptive-rx off adaptive-tx off ethtool -C <ethX> rx-usecs 64 tx-usecs 64 Note:Depending on the type of processor and workload, the coalescing parameters for Rx and Tx can be adjusted for improved performance (or less frame loss).
• ​​Disable the firewall. sudo systemctl disable firewalld sudo systemctl stop firewalld
• ​​Enable IP forwarding. sysctl -w net.ipv4.ip_forward=1
• ​​Configure max values for the receive and send socket buffer sizes. sysctl -w net.core.rmem_max=16777216 sysctl -w net.core.wmem_max=16777216 Note:Depending on the workload or requirement, these values can be changed from the default.