Change Necessary for ADQ Enablement

To enable ADQ in the NGINX application, the kernel's accept code path must be modified to establish an association between network adapter receive queues and application threads during the kernel's thread selection process.

The application thread selection can be done using a cBPF receive queue mapping program for each incoming connection. The cBPF program needs to be attached to each listening socket. Based on results from that cBPF program, one of the application threads is selected to handle the incoming connection instead of JHASH-based waiting-thread selection.

The following sections provide an overview of the necessary changes. For more information, including a full example NGINX patch, refer to Appendix, SO_​REUSEPORT Patch - NGINX.

Tthe following steps provide the pseudo code for enabling ADQ.

Step 1: Define the cBPF program to perform receive queue mapping, where n = #listening sockets.

The cBPF program accesses skb fields to determine receive queue information, using packet data at a fixed, pre-defined offset of SKF_​AD_​QUEUE. For more information, refer to https://netdev.vger.kernel.narkive.com/Hew6XXb3/patch-net-next-2-6-filter-add-skf-ad-queue-instruction.

Step 2: Attach the cBPF program to the listening socket using SO_​ATTACH_​REUSEPORT_​CBPF.

For each of n listening sockets, represented by fd in the following code snippet, attach the cBPF function p using SO_​ATTACH_​REUSEPORT_​CBPF to determine the selected listening thread via receive queue mapping.