Intel® Ethernet Controller E810 Application Device Queues (ADQ)
Configuration Guide
Redis Clients (non-ADQ) Configuration
The following variables are used in the examples in this section:
- Perform Client configuration.
- Enable throughput-performance tuned profile.
tuned-adm profile throughput-performance Note:The tuned-adm daemon is not installed by default in RHEL 9.0 systems. Install it with the command yum install tuned .Check settings are applied correctly:
cat /etc/tuned/active_profile Output:
throughput-performance cat /etc/tuned/profile_mode Output:
manual - Stop the irqbalance service.
systemctl stop irqbalance - Run the set_irq_affinity script for all interfaces (included in the scripts folder of the ice package).
${pathtoicepackage}/scripts/set_irq_affinity -X all $iface - Set interrupt moderation rate to a static value for Tx and turn off interrupt moderation for Rx.
ethtool --coalesce $iface adaptive-rx off rx-usecs 0 ethtool --coalesce $iface adaptive-tx off tx-usecs 25
- Enable throughput-performance tuned profile.
- Perform Redis build.
- Download the Redis release.
wget http://download.redis.io/releases/redis-4.0.11.tar.gz - Untar the package.
tar -xvzf redis-4.0.11.tar.gz - Apply redis-benchmark.patch for more detailed benchmark statistics and a duration (-D) command flag.
Note:If you are viewing this document in HTML, click the Download PDF button to download the zip package containing the redis-benchmark.patch patch. cd redis-4.0.11/src patch -p1 < redis-benchmark.patch - Install Redis.
cd redis-4.0.11 make PREFIX=/opt/redis-4.0.11 install - After this, the redis-server and redis-benchmark are found under /opt/redis-4.0.11/bin.
- Download the Redis release.
- Perform Redis benchmarking.
A typical test run would be one redis-benchmark instance per SUT queue/Redis instance, distributing the benchmark instances evenly across all the client systems. The request size (-d) can be scaled for testing as well as the connection rate per benchmark instances (-c).
For example, in the case of 90 queues and 10 client systems, there would be 90 instances of redis-server on the SUT listening on 90 different TCP ports, and 9 instances of redis-benchmark on each client, each connecting to a different port on the SUT. If a total of 900 connections/users were desired, then each of the 10 client benchmark threads would have a connection rate of -c 10.
To run the automated benchmarking script below, the server that the script is run on must have passwordless ssh access to all clients.
To set this up, run the following from the server that the script is going to be run on:
ssh-keygen for client in ${clients[@]}; do ssh-copy-id $client; done Note:Refer to https://redis.io/topics/benchmarks for more detail on Redis benchmarking. Example:
trafficgen="${redispath}/redis-benchmark" num_clients=${#clients[@]} for ((i = 0; i < app_threads; i++)); do (ssh ${clients[$((i % num_clients))]} "$trafficgen -t GET -n 100000 -r 1000 \ -d $redisbench_keysize -c $redisbench_conn -P 1 -h $ipaddr -p $((app_port + \ i))") & done Note:Pinning the application processes to CPU cores using numactl or similar might provide more consistent results. Parameter Definitions:
'-t {tests}', '-c {threads}', '-n {requests}', '-r {keys}', '-P {pipeline}', '-d {size}', '-h {server}', '-p {port}' - Verify ADQ traffic is on the correct queues.
While ADQ application traffic is running, watch ethtool statistics to check that only the ADQ queues are being used (have significant traffic) with busy poll (pkt_busy_poll) for ADQ traffic. If non busy poll (pkt_not_busy_poll) have significant counts and/or if traffic is not confined to ADQ queues, recheck the configuration steps carefully.
watch -d -n 0.5 "ethtool -S $iface | grep busy | column" See Verify ADQ Application Traffic and Independent Pollers (If Applicable) for example watch output.