​CPU Utilization

Check CPU utilization per core while the workload is running. Note that utilization per core is more relevant to performance than overall CPU utilization since it provides an idea of the CPU utilization per network queue. If you have only a few threads running network traffic, then you might only have a few cores being used. However, if those cores are at 100%, then your network throughput is likely limited by CPU utilization and it is time to perform the following:

1. ​Tune IRQ moderation/ring size as detailed in ​Interrupt Moderation.
2. ​Increase the number of application threads to spread out the CPU load over more cores. If all cores are running at 100% then your application might be CPU bound rather than network bound.

Commonly available tools:

• top
  • Press 1 to expand list of CPUs and check which ones are being used.
  • Notice the level of utilization.
  • Notice which processes are listed as most active (top of list).
• mpstat

  The following example command line was tested on Red Hat Enterprise Linux 7.x. It displays CPU utilization per core (by finding the total percent idle and subtracting from 100) and highlights the values above 80% in red.

  mpstat -P ALL 1 1 | grep -v Average | tail -n +5 | head -n -1 | awk '{ print (100-$13)}' | egrep -color=always '[^\.][8-9][0-9][\.]?.*|^[8-9][0-9][\.]?.*|100|' | column
• perf top

  Look for where cycles are being spent.