Set cgroup Priority for Application Network Traffic (If Needed)

Linux Control Groups (cgroups) are used to steer outgoing (Tx) traffic to a set of dedicated hardware queues on the NIC. If egress tc filter(s) were configured in TC Filtering, Linux cgroups are not needed and this step can be skipped. If only ingress tc filter(s) were created, follow the steps in this section to create a cgroup and set the priority of network traffic to align TX packets to the Rx filter queues.

The following variables are used in the examples this section:

1. Configure cgroup.

   The Network Priority (net_​prio) subsystem of the Control Groups (cgroups) Linux kernel feature provides a way to set the priority of network traffic per each network interface for applications within various cgroups. Typically, an application sets the priority of its traffic via the SO_​PRIORITY socket option. However, if applications are not coded to set the priority value, the administrator can assign a process to a specific cgroup which defines the priority of outgoing traffic on a given network interface.

   Note:Network Priority (net_​prio) using cgroup configuration in this section, or the SO_​PRIORITY socket option in the application, must be configured for symmetric queuing (Configure Symmetric Queues) to work correctly with ADQ. Failure to configure either 1) Network Priority (net_​prio) using cgroups, or 2) SO_​PRIORITY in the application, will result in Tx traffic going to the default TC queues.

   There are two options for cgroup configuration: manually, or through the cgconfig service tools.

   Option 1: Manual cgroup Configuration

   Note:RHEL9.0 does not include the net_​prio module by default. A net_​prio directory must be created under /sys/fs/cgroup/ before mounting it. For example: mkdir /sys/fs/cgroup/net_prio

   This step is not required in RHEL8.X or earlier versions.

   1. Mount the cgroup filesystem. mount -t cgroup -o net_prio none /sys/fs/cgroup/net_prio
   2. Create a new cgroup. mkdir /sys/fs/cgroup/net_prio/${cgroup_name}
   3. Assign the application process to cgroup. echo $app_pid > /sys/fs/cgroup/net_prio/${cgroup_name}/tasks Note:Before running this command, start the server application (for example, redis server, netperf, etc.) and include the specific application PID as $app_​pid.
   4. Set priority to $prio for traffic originating from processes belonging to the net_​prio cgroup. echo "$iface $prio" > /sys/fs/cgroup/net_prio/${cgroup_name}/ \ net_prio.ifpriomap

      Example:

      echo "eth0 1" > /sys/fs/cgroup/net_prio/foo/net_prio.ifpriomap

   Option 2 (Recommended): cgroup Configuration Using CGConfig Service Tools

   1. Make a net_​prio directory and mount it in the /sys/fs/cgroup directory. mkdir /sys/fs/cgroup/net_prio mount -t cgroup -o net_prio none /sys/fs/cgroup/net_prio Note:This step is only applicable for RHEL9.0 and later operating systems because the net_​prio module is not available in the OS repository.
   2. Create a new cgroup. cgcreate -g net_prio:${cgroup_name}
   3. Set the priority for processes belonging to the cgroup.

      The $prio value should map to the position of the targeted TC defined in Create TCs.

      cgset -r net_prio.ifpriomap="$iface $prio" ${cgroup_name}

      Example:

      For example, in the following TC command, map 0 0 1 2 sets priority 0-1 to use TC0, prio 2 to TC1, and prio 3 to TC2.

      ${pathtotc}/tc qdisc add dev $iface root mqprio num_tc 3 map \ 0 0 1 2 queues 2@0 40@2 54@42 hw 1 mode channel cgcreate -g net_prio:app_tc1 cgset -r net_prio.ifpriomap="$iface 2" app_tc1 cgcreate -g net_prio:app_tc2 cgset -r net_prio.ifpriomap="$iface 3" app_tc2
2. Run the task in the given cgroup (for example, ${cgroup_​name}) cgexec -g net_prio:${cgroup_name} --sticky $command

   Example:

   cgexec -g net_prio:${cgroup_name} --sticky netserver
3. Verify cgroup configuration:
   1. Verify that the interface is mapped to the right interface: cat /sys/fs/cgroup/net_prio/${cgroup_name}/net_prio.ifpriomap
   2. Show the Process IDs being run in this cgroup and match them to the application Process IDs: cat /sys/fs/cgroup/net_prio/${cgroup_name}/tasks

Example:

For convenience, the commands in this section are condensed below with example values. Variables must be changed to match each environment.