Configure Intel® Ethernet Flow Director Settings

Intel® Ethernet Controller E810 Application Device Queues (ADQ)

Configuration Guide

Download as ZIP

ID 609008

Date 04/03/2023

Version 2.8

Linux traffic control filtering allows an application to be mapped to a set of specified queues on an interface, using dedicated queue sets (also called traffic classes) for application traffic with ADQ. Intel® Ethernet Flow Director allows for incoming connections to be distributed and load balanced across these application queues, within the queue set, in a more reliably defined way than the default RSS hash. This is important to the inner workings of ADQ because it is recommended that no two application threads are busy polling on the same queue.

There are two options for Intel® Ethernet Flow Director configuration in the ice driver:

Option 1: channel inline or per-TC inline

Option 2: ntuple

The following variables are used in the examples in this section:

$iface	The interface in use (PF).
$iface_bdf	The network interface BDF notation (Bus:Device.Function) used by devlink (PF).
$port	The starting port of the application.
$queues	The number of queues in the TC.
$tc1offset	The offset for the application TC (for example, if TC0 has 2 queues, TC1 offset would be 2).

Note:Commands in this section must be applied to the PF interface (not available on VF interfaces).

Option 1: channel inline or per-TC inline

Intel® Ethernet Flow Director distributes connections to queues in a round-robin fashion. This is recommended for ADQ-enabled, single-instance, multi-threaded applications that manage their own worker thread load distribution (such as Memcached).

ice version 1.7.x (and earlier):

channel inline: Intel® Ethernet Flow Director is configured using a global private flag and gets applied to all TCs on the interface:

ethtool --set-priv-flags $iface channel-inline-flow-director on

ice version 1.8.x (and later):

per-TC inline: Intel® Ethernet Flow Director is configured using a per-TC configuration and gets applied to only the queues within the TC specified for the interface. A different devlink command is required for each TC. Example for tc1:

iface_bdf=$(ethtool -i ${iface} | grep bus-info | awk '{print $2}') devlink dev param set pci/${iface_bdf} name tc1_inline_fd value true cmode runtime Note:Kernel with devlink param support is required for ice-1.8.x and later. See Install OS and Update Kernel (If Needed) to determine OS and kernel requirements. Note:Valid devlink param flags include tc1_inline_fd through tc15_inline_fd, to configure inline flow director on up to 16 TCs (max #TCs). Note:XPS using receive queues map must be configured for per-TC inline flow director to work correctly. See Configure Symmetric Queues to configure XPS.

Option 2: ntuple

Intel® Ethernet Flow Director assigns perfect filters to direct each application TCP port to a specific queue within the application TC range. This is useful for applications (such as Redis) where each instance of the application runs a single worker thread, and load balancing is done either manually or through a separate load balancer application.

ethtool --features $iface ntuple on for ((i=0; i < queues; i++)) do ethtool --config-ntuple $iface flow-type tcp4 dst-port $((port + i)) action \ $(((i %queues) + tc1offset)) sleep 0.5 done #verify set correctly ethtool --show-ntuple $iface Note:If rmgr: Cannot insert Rx class rule: Invalid argument error message appears, verify that no previous assignment is in place: ethtool --show-ntuple $iface

Assignments as a whole can be deleted:

ethtool --features $iface ntuple off Note: Linux forward to queue using class ID is an alternative to ntuple filter rules using the action forward to queue to forward packets to a specific device queue. Ntuple can set the specific packet flow types (i.e., tcp4, udp4) and depends on the ethtool version, RHEL 8.x, or upstream kernel to configure Ntuple flags, whereas forward to queues can be set per-filter using the class ID parameter. See TC Filtering, Example of Forwarding to a Queue Using classid as an alternative to ntuple filter rules in this section.