Intel® Ethernet 800 Series Linux Flow Control

Configuration Guide for RDMA Use Cases

ID 635330
Date 07/13/2023
Version 1.3
Document Table of Contents

Congestion Control Parameter Settings

Congestion control settings are accessed through configfs. Additional DCQCN tunings are available via module parameters can be tuned when using out of tree irdma driver.

Note: Always consult the README_​irdma.txt for the latest information on congestion control support in CVL.

To access congestion control settings:

  1. After driver load, change to the irdma configfs directory: cd /sys/kernel/config/irdma
  2. Create a new directory for each RDMA device you want to configure. Note: Use ibv_​devices for a list of RDMA devices.

    For example, to create configfs entries for the rdmap<interface> device:

    mkdir rdmap<interface>
  3. List the new directory to get its dynamic congestion control knobs and values: cd rdmap<interface> for f in *; do echo -n "$f: "; cat "$f"; done;

    If the interface is in iWARP mode, the files have an "iw_​" prefix:

    • iw_​dctcp_​enable
    • iw_​ecn_​enable
    • iw_​timely_​enable

    If the interface is in RoCEv2 mode, the files have a "roce_​" prefix:

    • roce_​dcqcn_​enable
    • roce_​dctcp_​enable
    • roce_​timely_​enable
  4. Enable or disable the desired algorithms.
    1. To enable an algorithm: echo 1 > <attribute>

      For example, to add ECN marker processing to the default TCP New Reno iWARP congestion control algorithm:

      echo 1 > /sys/kernel/config/irdma/rdmap<interface>/iw_ecn_enable
    2. To disable an algorithm: echo 0 > <attribute>

      For example: echo 0 > /sys/kernel/config/irdma/rdmap<interface>/iw_ecn_enable

    3. To read the current status: cat <attribute>

      Default values:

      • iwarp_​dctcp_​en: off
      • iwarp_​timely_​en: off
      • iwarp_​ecn_​en: ON
      • roce_​timely_​en: off
      • roce_​dctcp_​en: off
      • roce_​dcqcn_​en: off
  5. Remove the configfs directory created above: rmdir /sys/kernel/config/irdma/rdmap<interface> Note:The driver will not unload until you remove the configfs directory.

Advanced Congestion Control Knobs

Module parameters on Intel Ethernet 800 Series for RoCEv2 DCQCN tuning that can be accessed when using an out-of-tree RDMA driver:

  • dcqcn_​enable

    Enables the DCQCN algorithm for RoCEv2.

    Note: roce_​ena must also be set to True.
  • dcqcn_​cc_​cfg_​valid

    Indicates that all DCQCN parameters are valid and should be updated in the registers or QP context.

  • dcqcn_​min_​dec_​factor

    The minimum factor by which the current transmit rate can be changed when processing a CNP, given as a percentage (1 to100).

  • dcqcn_​min_​rate

    The minimum value for the rate to limit, in Megabits per second (Mbps).

  • dcqcn_​F

    The number of times to stay in each stage of the bandwidth recovery.

  • dcqcn_​T

    The number of microseconds that should elapse before increasing the CWND in DCQCN mode.

  • dcqcn_​B

    The number of bytes to transmit before updating the CWND in DCQCN mode.

  • dcqcn_​rai_​factor

    The number of MSS to add to the congestion window in additive increase mode.

  • dcqcn_​hai_​factor

    The number of MSS to add to the congestion window in hyperactive increase mode.

  • dcqcn_​rreduce_​mperiod

    The minimum time between 2 consecutive rate reductions for a single flow. Rate reduction will occur only if a CNP is received during the relevant time interval.