Intel® Ethernet Controller E810 Application Device Queues (ADQ)
Configuration Guide
ADQ Setup Script Usage
The following variables are used in the examples in this section:
| $iface | The interface in use (PF). |
| $pathtoicepackage | The path to the ice driver package. |
| $port | Destination port of traffic |
| $portrange | Destination port range (ex: 11211-11218) |
| $addr | Destination IP address of traffic |
| $remote_addr | Source IP address of traffic |
| $remote_port | Source port of traffic |
| $num_queues_tc0 | The number of queues for default traffic class (default queues are 2) |
| $num_queues_tc1 | The number of queues for application traffic class 1 |
| $protocol | IP Protocol of the traffic ( tcp/udp) |
| $conf_file | Config file name to apply the settings to create the ADQ setup( ex: Memcached.cong, redis.conf and nginx.conf) |
| $driverpath | Device driver path to .ko file (ex: src/ice.ko) |
| $file_name | Creates a quick-config consisting of one application traffic class (it is a user defined name ex: netperf, redis, nginx etc) |
| $poller_number | The number of queue pairs per independent poller for a given TC (max value is #queues in the TC). |
| $poller_timeout | The timeout value for the independent pollers for a given TC (nonzero integer value in jiffies, default value 10000). |
| $mask | The bit mask used to select which queues to run the subcommand on (ex: for ADQ queues 2-9 : 0x3fc (0011 1111 1100)). |
| $cpus | Assigns pollers to specific CPU cores (ex: 4, 5, 6, 7) |
Using Command Line Parameters
- Create ADQ filters in exclusive mode for multithreaded applications (uses inline flow director to distribute the connections to queues).
adqsetup --dev=$iface --priority=skbedit create [$file name] queues \ $num_queues_tc1 ports $port - Create ADQ filters in shared mode for single threaded applications (creates perfect filters using Ntuple rules to distribute connections to queues).
adqsetup --dev=$iface --priority=skbedit create [$file name] mode shared \ queues $num_queues_tc1 protocol $protocol addrs $addr ports $portrange - Create ADQ setup with interrupt moderation rate values for Tx/RX.
adqsetup --dev=$iface --priority=skbedit --rxusecs=0 \ --txadapt=off --txusecs=500 create [$file name] mode shared queues \ $num_queues_tc1 ports $portrange - Create an ADQ setup by using various device drivers in the syntax.
adqsetup --dev=$iface --priority=skbedit create [$file name] queues \ $num_queues_tc1 protocol $protocol ports $port --driver $driverpath \ --reload - Create ADQ setup using independent pollers configuration with default timeout.
adqsetup --dev=$iface --priority=skbedit create [$file name] queues \ $num_queues_tc1 ports $port addrs $addr pollers $poller_number - Create ADQ setup using independent pollers configuration with pollers timeout in jiffies.
adqsetup --dev=$iface --priority=skbedit create [$file name] queues \ $num_queues_tc1 ports $port addrs $addr pollers \ $poller_number \ pollers_timeout $poller_timeout - Create ADQ setup with independent pollers by pinning pollers to specific CPU cores.
adqsetup --dev=$iface --priority=skbedit create [$file_name] mode shared queues $num_queues ports $portrange cpus $cpus pollers $poller_number
- Adaptive transmit and receive interrupt coalescing is enabled by default. Coalesce settings must be turned off (
-- txadapt=off -rxadapt=off ) to set the Tx interrupt moderation rate to a static value and turn off Rx interrupt moderation (ex.-- txusec=500 --rxusec=0 ). - In ADQ setup script, the independent pollers argument (
pollers ) is not directly related to qps_per_poller arguments passed to devlink. The script allows the user to specify how many pollers a particular TC should have, rather than specifying qps_per_poller (queue pairs per poller). - In the above examples, the driver is not reloaded, but can be reloaded by using the
--reload parameter while creating the ADQ setup. - To load/use a different device driver while creating the setup, the
--driver parameter may be used. Device driver path is the full path to the .ko file (ex:ice-1.9.x/src/ice.ko ). Interface must be set to come up automatically with an ip address (via NetworkManager or other). adqsetup will wait up to three seconds for this to occur before erroring out. Conversely, you can load the driver and setup the interface manually before running the adqsetup.Note:To load/use the various device drivers in adqsetup V1.x, use the --reload parameter in conjunction with the--drivers parameter. - If the
--priority=skbedit parameter is not specified in the command syntax, a Cgroup must be created to align ingress and egress traffic. The performance of the two methods (cgroup priority or TC filters with skbedit priority) may differ slightly based on the application or workload. - The
--debug parameter is optional, but it is useful for obtaining complete configuration details. - adqsetup 1.x required updated versions of the
tc ,ethtool , anddevlink commands to be installed on the system. With adqsetup 2. x and onward, this requirement has been removed. - Pinning pollers to specific CPU cores, as well as 256 queue pair support, were added in adqsetup V2.x and later.
Using Configuration File
- Apply a configuration file to create ADQ setup.
adqsetup --dev=$iface apply $conf_file The adqsetup script can automatically generate an
examples subdirectory containing a set of example config files using the following command that can be customized as needed.adqsetup examples - To apply a configuration file for persistent ADQ settings across system reboots, use the following commands
adqsetup --dev=$iface persist $conf_file systemctl enable --now adqsetup@$iface
Limitations
The ADQ setup script does not support the following configurations currently: