Example: Running DPDK-pktgen Traffic
This section provides an example for how to configure a standard server with a DPDK-pktgen software packet generator to generate network traffic to exercise the AF_XDP server configured in AF_XDP Server Configuration. Note that the server generating the traffic does not have to be configured with ADQ.
The following are the attributes used to generate the traffic:
| $pci_id | PCIe address of the interface in use. |
| $cores | List of cores used to generate the traffic. |
| $mem_channels | Number of memory channels. |
| $socket_mem | DPDK allocated memory from each CPU. |
| $log_level | Set the specific log level (example: 1-8). |
| $protocol | Traffic protocol (AF_XDP must use UDP). |
| $packet_size | Size of the packet (example: 64-1518). |
| $src_mac | Client interface MAC Address. |
| $dst_mac | AF_XDP server interface MAC Address. |
| $src_ip | IP Address configured on the client interface using pktgen set command. |
| $dst_ip | IP Address of the AF_XDP server interface. |
| $src_port | Port number configured on client interface using pktgen set command. |
| $dst_port | Port number configured in the traffic class filters on the AF_XDP server. |
| $min_value | Minimum range value configured using pktgen range command. |
| $max_value | Maximum range value configured using pktgen range command. |
| $increment | Increment value configured using pktgen range command. |
| $RTE_SDK | Points to the DPDK installation directory. |
| $pathtopktgen | Path to pktgen application. |
| $file_prefix | Hugepage filename prefix. |
| $port_num | Interface port number (example: port 0,1). |
| $rx_cores | Cores to handle specific port's Rx. |
| $tx_cores | Cores to handle specific port's Tx. |
| |
To use all queues, the number of traffic streams (ports) should be equal to or a multiple of the number of ADQ queues configured on the AF_XDP server ($num_queues_tc1 in AF_XDP Server Configuration).
- pktgen configuration command.
cd $RTE_SDK ./$pathtopktgen/pktgen -l $cores -n $mem_channels --proc-type auto --log-level $log_level --socket-mem=$socket_mem --file-prefix $file_prefix -a $pci_id -- -N -P -T -m [$rx_cores:$tx_cores].$port_num Following are the mandatory and optional command-line options:
-l:
-n:
-m:
-N:
-P:
--file-prefix:
List of cores on which to run.
Number of memory channels (range is 2-4).
Matrix for mapping ports to logical cores.
Enable NUMA support.
Enable PROMISCUOUS mode on all ports.
Prefix for huge page filenames.
-T:
--proc-type:
--log-level:
--socket-mem:
Enable the color output.
Type of the process (primary| secondary |auto).
Set specific log level.
Memory to allocate on specific sockets (comma separated values).
Example commands:
The following examples configure pktgen to use CPU cores specified by -l $cores, where the first core is used for the pktgen command line, for timers and for displaying the runtime metrics text on the terminal, and the remaining cores are used for the generated UDP packet streams. $cores are CPU cores on the same (local) NUMA node as the E810 network adapter. $rx_cores and $tx_cores are used to map Rx and Tx traffic from Port 0 of the network adapter to the CPU cores specified by -m.
Note:The PCIe device 12:00.0 that is used in pktgen is on NUMA node 0, so -l <CPUs> are from NUMA node 0 inventory. The following command can be used to list inventory of available NUMA nodes on the system, and the respective CPU cores:
numactl -hardware (or) lscpu | grep -i numa - Pktgen commands to generate the traffic.
Pktgen:/> set $port_num proto $protocol Pktgen:/> set $port_num size $packet_size Pktgen:/> set $port_num src mac $src_mac Pktgen:/> set $port_num dst mac $dst_mac Pktgen:/> set $port_num src ip $src_ip/mask Pktgen:/> set $port_num dst ip $dst_ip pktgen:/> set $port_num dport $dst_port Pktgen:/> set $port_num sport $src_port Pktgen:/> enable $port_num range Pktgen:/> range $port_num proto $protocol Pktgen:/> range $port_num size $packet_size $min_value $max_value $increment Pktgen:/> range $port_num src mac $src_mac $min_value $max_value $increment Pktgen:/> range $port_num dst mac $dst_mac $min_value $max_value $increment Pktgen:/> range $port_num dst ip $dst_ip $min_value $max_value $increment Pktgen:/> range $port_num src ip $src_ip $min_value $max_value $increment Pktgen:/> range $port_num dst port $dst_port $min_value $max_value $increment Pktgen:/> range $port_num src port $src_port $min_value $max_value $increment Pktgen:/> start $port_num #To start the traffic Pktgen:/> stop $port_num #To stop the traffic Pktgen:/> quit #To quit the application Example commands:
The following commands configure 64-byte UDP packets from the pktgen host (src) to the AF_XDP server (dst), generating 8 traffic streams across 8 application ports. The destination port numbers (dport and dst port) must match the port numbers used in the ADQ configuration steps on the AF_XDP server in AF_XDP Server Configuration.
Must send UDP traffic to an AF_XDP socket.
Pktgen:/> set 0 proto udp Pktgen:/> set 0 size 64 Pktgen:/> set 0 src mac b4:96:91:9c:f9:90 Pktgen:/> set 0 dst mac 40:a6:b7:19:08:d0 Tested with /16 network mask, other range of network masks could also be used (Default is /32 for IPv4). Subnet mask not required on the destination IP address.
Pktgen:/> set 0 src ip 61.1.1.50/16 Pktgen:/> set 0 dst ip 61.1.1.100 Destination port number must match the starting $app_port number configured on the ADQ AF_XDP server (AF_XDP Server Configuration).
pktgen:/> set 0 dport 5002 Pktgen:/> set 0 sport 3002 Pktgen:/> enable 0 range Pktgen:/> range 0 proto udp Pktgen:/> range 0 size 64 64 64 1 Pktgen:/> range 0 src mac b4:96:91:9c:f9:90 b4:96:91:9c:f9:90 b4:96:91:9c:f9:90 00:00:00:00:00:01 Pktgen:/> range 0 dst mac 40:a6:b7:19:08:d0 40:a6:b7:19:08:d0 40:a6:b7:19:08:d0 00:00:00:00:00:01 Pktgen:/> range 0 dst ip 61.1.1.100 61.1.1.100 61.1.1.100 0.0.0.1 Pktgen:/> range 0 src ip 61.1.1.50 61.1.1.50 61.1.1.50 0.0.0.1 Number of traffic streams (ports) should be equal to or a multiple of the number of ADQ queues configured on the AF_XDP server ($num_queues_tc1 in AF_XDP Server Configuration). $dst_port numbers must match the port numbers configured on the ADQ server.
Pktgen:/> range 0 dst port 5002 5002 5009 1 #Ports used to add TC filters in ADQ server Pktgen:/> range 0 src port 3002 3002 3009 1 #8 source ports Pktgen:/> start 0 Pktgen:/> stop 0 - Verify ADQ traffic is on the correct queues.
After traffic is started in the client machine, on the AF_XDP server start the XDPSOCK application and watch ethtool statistics to check that only the ADQ queues are being used. The pkt_busy_poll counter should have significantly higher traffic. If instead, pkt_not_busy_poll show significantly higher traffic, or if traffic is not confined to ADQ queues, re-check the configuration steps carefully and/or make sure XDPSOCK is running on the AF_XDP server.
watch -d -n 0.5 "ethtool -S $iface | grep busy | column" See Verify ADQ Application Traffic and Independent Pollers (If Applicable) for example ethtool watch output.