​C-State Control

Limiting C-state entry to C0 or C1 improves performance and increases power utilization.

Disabling CPU Package C6 state entry can improve network performance. However, this increases power usage.

The following options are available:

• Dynamically control the C-state entry:

  Open /dev/cpu_dma_latency and write the maximum allowable latency to it.

  Note:There is a small program called cpudmalatency.c that can be downloaded from the open source community, compiled, and run from the command line to do exactly this.

  The following example allows five μs of wake time, and thus allows C1 entry:

  cpudmalatency 5 &
• Limit the maximum C-state in the kernel boot settings:

  For Intel CPUs:

  intel_idle.max_cstates=1

  For non-Intel CPUs:

  processor.max_cstates=1
• Use the cpupower command to check and disable CPU C6 state:

  Check:

  cpupower monitor or cpupower idle-info

  Disable C6:

  cpupower idle-set -d3 or

  Disable C-States:

  cpupower idle-set -D0

  Notes:

  1. Disable C-states on the CPU if the server has Intel® 4th Gen Intel® Xeon® Scalable Processor(s). When Hyper Threading is enabled or disabled, disabling idle states (-D0) prevents cores from entering low-power states during idle periods and reduces the latency for the CPU to transition between idle and active states.

  2. The power management of the Intel® 4th Gen Intel® Xeon® Scalable Processor is extremely aggressive. To avoid cores from entering low-power states, try reducing the number of cores in use to keep them awake for longer (ethtool -L <ethX> combined <numcores>). Also, bind interrupts to specific cores using set irq affinity (most often with -X local or list of CPU cores), and ensure the workload runs on those same cores with taskset or numactl. This improves performance by keeping cores active and optimizing interrupt handling.

  Enable C6:

  cpupower idle-set -d3

  Enable C-States:

  cpupower idle-set -E
• Another method is to utilize the tuned tool (included with many Linux distributions) to set a performance profile. These profiles modify several OS settings that can affect performance across many applications. It has been found that the network-throughput profile provides improvement to most workloads.

  Check: tuned-adm active

  Set: tuned-adm profile network-throughput

  Note:Tuned service must be running for above commands. To check/restart, tuned: systemctl status tuned systemctl restart tuned

  You can also disallow any C-state entry by adding the following to the kernel boot line:

  idle=poll
• Limit the C-state through the system's BIOS power management settings, which might have a performance profile available.

  Tools such as turbostat or x86_​energy_​perf_​policy can be used to check or set power management settings.