Intel® Ethernet Controller E810 Data Plane Development Kit (DPDK) 22.11/23.03

Configuration Guide

Download as PDF
ID 764257
Date 06/16/2023
Version 22.11
Document Table of Contents
Document Table of Contents
Notices and Disclaimers Revision History Introduction DPDK Requirements DPDK Installation and Configuration Virtual Function (VF) Setup with DPDK Advanced Features and Debug Test Applications (pktgen and testpmd)
Notices and Disclaimers
Revision History
Introduction
Introduction DPDK Overview Known Issues
DPDK Overview
Known Issues
DPDK Requirements
System Requirements Software/Firmware Requirements
System Requirements
Software/Firmware Requirements
Software/Firmware Requirements Updating the NVM with a DPDK Driver
Updating the NVM with a DPDK Driver
DPDK Installation and Configuration
DPDK Installation and Configuration System Configuration BIOS Settings Hugepages Setup IOMMU CPU Isolation RCU Callbacks Tickless Kernel vt.handoff NUMA Balancing and MCE Active-State Power Management High Performance of Small Packets on 100G NIC- Use 16 Bytes Rx Descriptor Size Downloading and Installing the ice Driver Getting the Latest DPDK Code Prerequisite Library Installing DPDK Linux Drivers
System Configuration
BIOS Settings
Hugepages Setup
IOMMU
CPU Isolation
RCU Callbacks
Tickless Kernel
vt.handoff
NUMA Balancing and MCE
Active-State Power Management
High Performance of Small Packets on 100G NIC- Use 16 Bytes Rx Descriptor Size
Downloading and Installing the ice Driver
Getting the Latest DPDK Code
Prerequisite Library
Installing DPDK
Installing DPDK Using the Meson Build System (Recommended) Setting 16 Bytes Rx Descriptor Size
Installing DPDK Using the Meson Build System (Recommended)
Setting 16 Bytes Rx Descriptor Size
Linux Drivers
Linux Drivers Loading the vfio-pci Module Binding and Unbinding Network Ports
Loading the vfio-pci Module
Binding and Unbinding Network Ports
Virtual Function (VF) Setup with DPDK
Advanced Features and Debug
Q-in-Q Support Malicious Driver Detection Receive Side Scaling Configuration
Q-in-Q Support
Malicious Driver Detection
Receive Side Scaling Configuration
Test Applications (pktgen and testpmd)
Test Applications (pktgen and testpmd) Setting Up pktgen Server Setting Up Testpmd Application Example pktgen Configuration Example Testpmd Configuration Running a Sample Application L3fwd Server DPDK Test Plans
Setting Up pktgen Server
Setting Up Testpmd Application
Setting Up Testpmd Application Running Testpmd Testpmd Runtime Functions Debugging in Testpmd
Running Testpmd
Testpmd Runtime Functions
Debugging in Testpmd
Example pktgen Configuration
Example Testpmd Configuration
Running a Sample Application L3fwd Server
DPDK Test Plans
Notices and Disclaimers
Revision History
Introduction
DPDK Requirements
DPDK Installation and Configuration
Virtual Function (VF) Setup with DPDK
Advanced Features and Debug
Test Applications (pktgen and testpmd)

Hugepages Setup

Hugepage support is required for the large memory pool allocation used for packet buffers. By using hugepage allocations, performance is increased since fewer pages are needed, and therefore less Translation Lookaside Buffers (TLBs, high speed translation caches), which reduce the time it takes to translate a virtual page address to a physical page address. Without hugepages, high TLB miss rates would occur with the standard 4K page size, slowing performance.

For 1 GB pages

It is not possible to reserve the hugepage memory after the system has booted. The size must be specified explicitly and can also be optionally set as the default hugepage size for the system.

The 1 GB hugepage option can be added in Grub along with IOMMU in kernel command line, as shown in High Performance of Small Packets on 100G NIC- Use 16 Bytes Rx Descriptor Size.

To reserve 4 GB of hugepage memory in the form of four 1 GB pages, the following options should be passed to the kernel:

default_hugepagesz=1G hugepagesz=1G hugepages=4

Once the hugepage memory is reserved, to make the memory available for DPDK use, execute the following:

mkdir /mnt/huge mount -t hugetlbfs nodev /mnt/huge

For 1 GB pages, the mount point can be made permanent across reboots, by adding the following line to the /etc/fstab file:

nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 Note:There are multiple ways to create hugepages under RHEL 7.x, following is one example that shows the steps to create four pages of 1 GB:
  1. Create a mounting point for huge pages with auto-mount. cd /mnt mkdir huge
  2. Modify /etc/fstab to include: nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
  3. Modify and update grub to set up hugepages. # /etc/default/grub GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=4"

    For RHEL:

    grub2-mkconfig -o /boot/grub/grub.cfg

    For Ubuntu:

    Update-grub
  4. Reboot the system and check Huge Page allocation. # cat /proc/meminfo | grep HugePages_Total HugePages_Total: 4 # cat /proc/meminfo | grep Hugepagesize Hugepagesize: 1048576 kB

For 2 MB pages

Hugepages can be allocated after the system has booted. This is done by echoing the number of hugepages required to a nr_​hugepages file in the /sys/devices/ directory.

For a single-node system, the command to use is as follows (assuming that 1024 pages are required):

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages mount -t hugetlbfs nodev /mnt/huge

On a NUMA machine, pages should be allocated explicitly on separate nodes:

mkdir -p /mnt/huge echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages mount -t hugetlbfs nodev /mnt/huge
IOMMU