Intel® Ethernet 800 Series Linux Flow Control

Configuration Guide for RDMA Use Cases

ID Date Version Classification
635330 11/13/2025 Public
Document Table of Contents
Document Table of Contents
Notices and Disclaimers Revision History Introduction Background Link-Level Flow Control Priority Flow Control - Fundamentals Congestion Management – Tuning Parameters Priority Flow Control - Planning and Guidelines Priority Flow Control - Verification Troubleshooting
Notices and Disclaimers
Revision History
Introduction
Introduction QoS/Flow Control Limitations on the 800 Series
QoS/Flow Control Limitations on the 800 Series
Background
Ethernet Flow Control Flow Control in RDMA Networks Types of Flow Control: LFC vs. PFC
Ethernet Flow Control
Flow Control in RDMA Networks
Types of Flow Control: LFC vs. PFC
Link-Level Flow Control
LFC Setup Instructions Symmetric vs. Asymmetric LFC
LFC Setup Instructions
Symmetric vs. Asymmetric LFC
Priority Flow Control - Fundamentals
Priority Flow Control - Fundamentals IEEE Standards for DCB Determining PFC Priority Mode: PCP vs. DSCP Assigning an Application to a Traffic Class
IEEE Standards for DCB
IEEE Standards for DCB DCB Willing vs. Non-willing Modes
DCB Willing vs. Non-willing Modes
Determining PFC Priority Mode: PCP vs. DSCP
Assigning an Application to a Traffic Class
Assigning an Application to a Traffic Class Mapping Steps Details
Mapping Steps Details
Step 1: Determine Flow Control Design Needed Step 2: Kernel Priority (sk_prio) or DSCP to UP Mapping Step 3: UP to TC Step 4: Set ToS in the Application (or for All RoCEv2 Traffic)
Step 1: Determine Flow Control Design Needed
Step 2: Kernel Priority (sk_prio) or DSCP to UP Mapping
Step 3: UP to TC
Step 4: Set ToS in the Application (or for All RoCEv2 Traffic)
Congestion Management – Tuning Parameters
Overview DCQCN Congestion Control Parameter Settings
Overview
DCQCN
DCQCN DCQCN Specification
DCQCN Specification
Congestion Control Parameter Settings
Priority Flow Control - Planning and Guidelines
Priority Flow Control - Planning and Guidelines Steps Example Configurations
Steps
Steps Network Host and Switch Setup Willing vs. Non-willing DCB Mode Firmware vs. Software DCB Software DCB Willing Mode Separating and Prioritizing Traffic Streams Configuring ETS: Map Priorities to TCs/Allocate Bandwidth Configuring PFC Run Applications with the Right Priority
Network Host and Switch Setup
Willing vs. Non-willing DCB Mode
Firmware vs. Software DCB
Software DCB Willing Mode
Separating and Prioritizing Traffic Streams
Configuring ETS: Map Priorities to TCs/Allocate Bandwidth
Configuring PFC
Configuring PFC Option A: PCP PFC - Set Priorities for Drop or No-drop Option B: DSCP PFC
Option A: PCP PFC - Set Priorities for Drop or No-drop
Option B: DSCP PFC
Run Applications with the Right Priority
Example Configurations
Example 1 - 800 Series-800 Series Back-to-Back – PCP PFC with Single TC Example 2 - Adapters Connected Through a Switch – Willing Mode on Adapters, DCB Configured on Switch Example 3 - DSCP PFC with Non-Default TCs - Non-Willing Mode on Adapters, ECN Configured Example 4 - PCP PFC with Multiple TCs (1 for RDMA, 1 for LAN) – No VLANs Example 5 - PCP PFC with Multiple TCs (1 for RDMA, 1 for LAN) – with VLANs
Example 1 - 800 Series-800 Series Back-to-Back – PCP PFC with Single TC
Example 2 - Adapters Connected Through a Switch – Willing Mode on Adapters, DCB Configured on Switch
Example 3 - DSCP PFC with Non-Default TCs - Non-Willing Mode on Adapters, ECN Configured
Example 4 - PCP PFC with Multiple TCs (1 for RDMA, 1 for LAN) – No VLANs
Example 5 - PCP PFC with Multiple TCs (1 for RDMA, 1 for LAN) – with VLANs
Priority Flow Control - Verification
Priority Counters Discard Counters tcpdump dcb Tool
Priority Counters
Discard Counters
Discard Counters LAN Packet Drops RDMA Discards
LAN Packet Drops
RDMA Discards
tcpdump
dcb Tool
Troubleshooting

Option B: DSCP PFC

When configuring Priority Flow Control (PFC) using DSCP (Differentiated Services Code Point), traffic classes are assigned based on DSCP values in the IP header. Unlike PCP-based configuration, DSCP mode does not involve DCBx negotiation, and therefore the concept of willing/non-willing does not apply. Use the dcb utility to configure DSCP-to-traffic class mappings directly on the interface. This method is preferred over lldptool , which is designed for DCBx-based negotiation and is not applicable in DSCP mode.

Run Applications with the Right Priority