Integrating AF_XDP into DPDK

1 . x Integrating AF_XDP into DPDK XIAO LONG YE INTEL 

2 .Agenda • AF_XDP introduction • AF_XDP PMD in DPDK • Performance • Future work 2 

3 .AF_XDP • Overview • XDP’s user space interface • Use XDP program to trigger Rx path for selected queue • Zero Copy from DMA buffers to user space with driver support • Copy mode for non-modified drivers • Benefits • Performance boost • Support all Linux network devices 3 

4 .AF_XDP sfd = socket(PF_XDP, SOCK_RAW, 0); buffs = calloc(num_buffs, FRAME_SIZE); setsockopt(sfd, SOL_XDP, XDP_MEM_REG, buffs); setsockopt(sfd, SOL_XDP, XDP_{RX|TX|FILL|COMPLETION}_RING, ring_size); mmap(..., sfd, ......); /* map kernel rings */ bind(sfd, ”/dev/eth0”, queue_id,....); for (;;) { read_process_send_messages(sfd); }; Completion Q User Space Filling Q RXQ TXQ XDP Socket Kernel Space UMEM 4 

5 .AF_XDP integrated into DPDK • Motivation • Support DPDK in container usecase • Support non DPDK NICs • Reuse DPDK libraries for applications • Use of hugepages for performance • Linux kernel driver handles hardware • Better security and isolation • Utilize existing Linux network tools • Goal • DPDK apps can run unmodified using AF_XDP interface • Performance is on par with kernel xdpsock sample 5 

6 .AF_XDP integrated into DPDK Current Model DPDK with AF_XDP user space user space DPDK apps DPDK apps DPDK AF_XDP DPDK PMD DPDK PMD PMD DPDK PMD Kernel space Kernel space Igb_uio NIC driver 6 

7 . AF_XDP PMD User Space • Implemented by vdev DPDK q0 DPDK q1 DPDK q2 DPDK q0 DPDK q1 • Create af_xdp socket, umem, XSK A XSK B XSK C XSK D XSK E fill/completion/Rx/Tx rings • Features: VDEV 0 VDEV 1 • Multi-queues NETDEV q0 NETDEV q1 NETDEV q2 NETDEV q3 NETDEV q4 • Zero copy between mbuf and umem NETDEV eth0 ID=56 KEY XSK 0 XSK A eBPF redirect 1 XSK B 2 XSK C ./build/app/testpmd -l 5,6,7 -n 4 --log- 3 XSK D level=pmd.net.af_xdp:info --no-pci --vdev Kernel Space 4 XSK E net_af_xdp0,iface=ens786f1,start_queue=0,queue_count= 3 -- -i --rxq=3 --txq=3 7 

8 .ZERO-COPY (through external mbuf) • Use external buf to hold the payload 2K aligned address MP HDROBJ size (64 bytes) xsk_umem rte_mempool *buf_addr Mbuf_0 struct rte_mbuf (128 2K Chunk bytes) Mbuf_1 2K Chunk Mbuf_2 umem data 2K Chunk mbuf headroom (128 Mbuf_3 bytes) 2K Chunk Mbuf_4 2K Chunk … 2K Chunk 8 

9 .Performance • DPDK 19.05 + ZC patch l2fwd • Broadwell E5 2660 @ 2.0 GHz • IXIA load generator blasting at full 40 Gbit/s Throughput • Intel XL710 card (40G, i40e driver) Tx Rx App Core 1 Core 2 64 bytes 256 bytes 1024 bytes xdpsock i40e pmd AF_XDP pmd AF_XDP pmd zero copy 9 

10 .Future work • Flexible umem chunk size and alignment • Optimize AF_XDP pmd in container scenario 10 

11 .Thanks