Intel® Ethernet Controller E810 Application Device Queues (ADQ)

Configuration Guide

ID 609008
Date 04/03/2023
Version 2.8
Document Table of Contents

netperf Client

Note:This section uses application dependent polling (ADQ1.0) for application and testing and does not use an independent poller (ADQ2.0).

The following variables are used in the examples in this section:

$iface The interface in use (PF).
$ipaddrserver The IP Address of the netperf server interface under test.
$ipaddrclient The IP of the netperf client interface under test.
$app_​port

The starting port of the netperf application.

Note:This should be available on both sever and client. Usually a high number (>2000) is best, but check that it is available (for example: sudo netstat -tulpn | grep LISTEN).
$app_​threads The number of netperf client instances to be run (must be less than or equal to $num_​queues_​tc1).
$testtime The length of time for each test.
ADQ Setup Script variables
$bp Global busy_​poll value to be set by ADQ Setup Script (system-wide setting, not to be used with independent poller).
$br Global busy_​read value to be set by ADQ Setup Script (system-wide setting, not to be used with independent poller).
$file_​name Application name to be set and used by ADQ Setup Script (any descriptive string, i.e., netperf).
$portrange TCP ports to be configured by ADQ Setup Script.
$addr Destination IP address of traffic to be configured by ADQ Setup Script.

Using ADQ Setup Script

The ADQ Setup script allows you to quickly and easily configure required ADQ parameters such as traffic classes, priority, filters, and ethtool parameters etc.

  1. To configure ADQ, run the following command: adqsetup --dev=$iface –-priority=skbedit --busypoll=$bp --busyread=$br create $file_name mode shared \ queues $num_queues_tc1 ports $portrange addrs $addr

    See Notes below for customizing the ADQ configuration. Once ADQ is configured by adqsetup, start the netperf client.

    Notes:
    1. The example command above creates both ingress (Rx) and egress (Tx) filters, and Linux cgroups are not needed to be created and can be skipped (cgroup is only needed if --priority=skbedit was NOT specified in adqsetup command).
    2. ADQ setup script handles symmetric queues and affinity.
    3. Set the transmit and receive interrupt coalescing values to --rxadapt=off --rxusecs=0 --txadapt=off --txusecs=500 for improved ADQ performance.
    4. To configure independent pollers, add the -pollers=$pollers parameter in the adqsetup command (and optionally --pollers_​timeout), and remove the flags to set global --busypoll=$bp --busyread=$br.
    5. Use the cpu parameter in the command to bind the independent pollers to specific CPU cores. Refer to ADQ Setup Using ADQ Setup Script for more information on pinning pollers to specific CPU cores.
    6. The --debug parameter is optional, but it is useful for obtaining complete stack trace.
    7. For more information on how to use the script, refer to ADQ Setup Using ADQ Setup Script.
  2. Run traffic.

    Start simultaneous netperf client instances.

    Pinning the application to a CPU core on the local NUMA node provides more consistent results for ADQ. The following example uses the netperf -T option to select which core to use for each instance of the application.

    numa=0 cpus_allowed=(`lscpu | grep node${numa} | cut -d: -f2 | sed -e 's/ //g' | sed -e 's/,/ /g'`) start_core=${cpus_allowed[$app_threads]} for ((i = 0; i < app_threads; i++)); do core=$(( $i * 2 + $start_core )) netperf -j -H $ipaddrserver -t \ TCP_RR -T $core -l ${testtime} -- -k \ THROUGHPUT,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,\ P99_LATENCY,STDDEV_LATENCY,MEAN_LATENCY -P $((app_port + i)),$((app_port + i)) & done Note:Use netperf -h for parameter details. Reference here:

    https://hewlettpackard.github.io/netperf/doc/netperf.html#TCP_005fRR

  3. Verify ADQ traffic is on the correct queues.

    While ADQ application traffic is running, watch ethtool statistics to check that only the ADQ queues are being used (have significant traffic) with busy poll (pkt_​busy_​poll) for ADQ traffic. If non busy poll (pkt_​not_​busy_​poll) have significant counts and/or if traffic is not confined to ADQ queues, recheck the configuration steps carefully.

    Example:

    watch -d -n 0.5 "ethtool -S $iface | grep busy | column"

    See Verify ADQ Application Traffic and Independent Pollers (If Applicable) for example watch output.

Note: The ADQ Setup script clears the existing configuration before proceeding with the new ADQ configuration. To clear manually, follow the steps inClear the ADQ Configuration.