Intel® Ethernet 800 Series
Linux Performance Tuning Guide
6th Gen Intel Xeon Scalable Processors
The 6thGen Intel Xeon platform introduces with E-cores (Efficiency cores) and P-cores(Performance cores), all built on the Birch Stream Intel Xeon processor platform.
Because Intel Xeon 6 processors feature extremely high core counts across multi-die/chiplet layouts, traversing the physical interconnect mesh to access distant cache slices and memory controllers introduces significant latency.
Sub-NUMA Clustering (SNC) addresses this by partitioning a single physical socket into localized, independent NUMA domains pairing local CPU cores directly with adjacent Last-Level Cache (LLC) slices and memory channels to keep memory access local and eliminate cross-mesh bottlenecks.
SNC divides a single CPU socket into 2 (SNC2) or 3 (SNC3) logical NUMA nodes, pairing CPU cores directly with adjacent Last-Level Cache (LLC) slices and memory controllers.
Tuning Requirement: Network card interrupts (IRQs) must be pinned to the specific local SNC node (NUMA domain) to avoid cross-node latency penalties
Recommended BIOS settings:
- Set system profile to "Performance".
- Set SNC (Sub-NUMA Cluster) to either SN2 or 3 (it depends on the CPU model).
- Set CPU Power Management to "Max Performance".
- Disable deep CPU idle states (C-states) to keep cores continuously active.
- Enable/Disable Hyper threading depends on the use case.
- Enable Turbo boost.
Hyper-Threading and Turbo Boost options are not supported on E-core platform.
Recommended OS-level Settings:
- Set CPU frequency scaling governor to “performance”.
cpupower frequency-set -g performance - Disable deep idle C-States (C6/C6p).
cpupower idle-set -D3 cpupower idle-set -D4 - Configure NAPI budget and socket buffers.
sysctl -w net.core.netdev_budget=900 sysctl -w net.core.netdev_budget_usecs=8000 sysctl -w net.core.rmem_max=134217728 sysctl -w net.core.wmem_max=134217728 Note: E-cores require a lower net.core.netdev_budget (e.g., 200-300) due to lower IPC, while P-cores can handle a larger budget (e.g., 900)
- Disable Automatic NUMA Balancing:
(To avoid unexpected latency spikes and cross-node memory traffic)
sysctl -w kernel.numa_balancing=0 Note: On E-core platforms, it is mandatory to disable xps_cpu and enable xps_rxqs to prevent the performance degradation and ensure balanced core utilization. - On E-core platforms, increasing DDIO ways can degrade performance by stealing L2 cache from NAPI. On P-Core platforms, DDIO has negligible impact.
- Disable busy poll (this can cause an estimated 13% performance drop).
Sysctl –w net.core.busy_poll =0 Sysctl –w net.core.busy_read: 0
Recommended ICE Driver level Settings:
Along with other tunings updated in Section 5.0 Tuning ice Driver Settings, always use tx_scheduling_layers=5. It provides 4x fewer retransmits and 2x better consistency with zero throughput cost. Ensure the DDP package is updated to support this. Refer the Section 5.4 Transmit Scheduling Layer for the configuration.