Intel® Ethernet Controller E810 Application Device Queues (ADQ)

Configuration Guide

ID 609008
Date 04/03/2023
Version 2.8
Document Table of Contents

Redis Clients (non-ADQ) Configuration

The following variables are used in the examples in this section:

$iface The interface in use.
$ipaddr The IP Address of the Redis server interface under test.
$app_​port The starting port of the application (for example, 7001). Note:Port number greater than 5000 is best, but check that it is available (for example: sudo netstat -tulpn | grep LISTEN)
$app_​threads The number of Redis server instances to be run (must be less than or equal to $num_​queues_​tc1).
$pathtoicepackage The path to the ice driver package.
$redispath The path to Redis install (for example, /opt/redis-4.0.11/bin).
$clients The array of hostnames or IPs of the physical Redis client systems (for example, declare -a clients=("server1" "server2")).
$redisbench_​keysize The Redis benchmark key size in bytes.
$redisbench_​conn The number of connections per Redis benchmark thread.
  1. Perform Client configuration.
    1. Enable throughput-performance tuned profile. tuned-adm profile throughput-performance Note:The tuned-adm daemon is not installed by default in RHEL 9.0 systems. Install it with the command yum install tuned.

      Check settings are applied correctly:

      cat /etc/tuned/active_profile

      Output: throughput-performance

      cat /etc/tuned/profile_mode

      Output: manual

    2. Stop the irqbalance service. systemctl stop irqbalance
    3. Run the set_​irq_​affinity script for all interfaces (included in the scripts folder of the ice package). ${pathtoicepackage}/scripts/set_irq_affinity -X all $iface
    4. Set interrupt moderation rate to a static value for Tx and turn off interrupt moderation for Rx. ethtool --coalesce $iface adaptive-rx off rx-usecs 0 ethtool --coalesce $iface adaptive-tx off tx-usecs 25
  2. Perform Redis build.
    1. Download the Redis release. wget http://download.redis.io/releases/redis-4.0.11.tar.gz
    2. Untar the package. tar -xvzf redis-4.0.11.tar.gz
    3. Apply redis-benchmark.patch for more detailed benchmark statistics and a duration (-D) command flag. Note:If you are viewing this document in HTML, click the Download PDF button to download the zip package containing the redis-benchmark.patch patch. cd redis-4.0.11/src patch -p1 < redis-benchmark.patch
    4. Install Redis. cd redis-4.0.11 make PREFIX=/opt/redis-4.0.11 install
    5. After this, the redis-server and redis-benchmark are found under /opt/redis-4.0.11/bin.
  3. Perform Redis benchmarking.

    A typical test run would be one redis-benchmark instance per SUT queue/Redis instance, distributing the benchmark instances evenly across all the client systems. The request size (-d) can be scaled for testing as well as the connection rate per benchmark instances (-c).

    For example, in the case of 90 queues and 10 client systems, there would be 90 instances of redis-server on the SUT listening on 90 different TCP ports, and 9 instances of redis-benchmark on each client, each connecting to a different port on the SUT. If a total of 900 connections/users were desired, then each of the 10 client benchmark threads would have a connection rate of -c 10.

    To run the automated benchmarking script below, the server that the script is run on must have passwordless ssh access to all clients.

    To set this up, run the following from the server that the script is going to be run on:

    ssh-keygen for client in ${clients[@]}; do ssh-copy-id $client; done Note:Refer to https://redis.io/topics/benchmarks for more detail on Redis benchmarking.

    Example:

    trafficgen="${redispath}/redis-benchmark" num_clients=${#clients[@]} for ((i = 0; i < app_threads; i++)); do (ssh ${clients[$((i % num_clients))]} "$trafficgen -t GET -n 100000 -r 1000 \ -d $redisbench_keysize -c $redisbench_conn -P 1 -h $ipaddr -p $((app_port + \ i))") & done Note:Pinning the application processes to CPU cores using numactl or similar might provide more consistent results.

    Parameter Definitions:

    '-t {tests}', '-c {threads}', '-n {requests}', '-r {keys}', '-P {pipeline}', '-d {size}', '-h {server}', '-p {port}'
  4. Verify ADQ traffic is on the correct queues.

    While ADQ application traffic is running, watch ethtool statistics to check that only the ADQ queues are being used (have significant traffic) with busy poll (pkt_​busy_​poll) for ADQ traffic. If non busy poll (pkt_​not_​busy_​poll) have significant counts and/or if traffic is not confined to ADQ queues, recheck the configuration steps carefully.

    watch -d -n 0.5 "ethtool -S $iface | grep busy | column"

    See Verify ADQ Application Traffic and Independent Pollers (If Applicable) for example watch output.

Note:The ADQ Setup script clears the existing configuration before proceeding with the new ADQ configuration. To clear manually, follow the steps in Clear the ADQ Configuration.