Brainstorm notes

See areas/arm64/README.html

Ref slides: http://people.netfilter.org/hawk/presentations/NetConf2017_Seoul/XDP_devel_update_NetConf2017_Seoul.pdf

Idea slide 31: On XDP-redirect packets can be dropped, after the XDP prog have run, and today we have a 'drop-tracepoint' to handle/debug this. It would be more interesting to call a new XDP-type program that get the about-to-be-dropped packet, and allow it to take a new decision. Wondering what this can be used for, click the youtube link in the slide, and blow-your-mind! (https://youtu.be/BO0QhaxBRr0)

Want to track when BPF progs gets loaded/unloaded

• Both for system audit and troubleshooting purposes

BPF code do have LSM/SElinux hooks

• LSM maintainers request, use Audit subsystem instead
• Jiri Olsa currently working on this

E.g. AF_XDP via xdpsock uses redirect, and there is not feedback when running prog on driver without redirect-support. Need to look in dmesg to see big splast.

E.g. Loading XDP/ebpf prog with HW offload on Netronome NIC, but wrong firmware, just says EINVAL. No help or hints that firmware needs to be upgraded.

How can we improve the usability / user-experience here?

Same EINVAL when loading on a NIC driver other than Netronome. Can we instead return ENOTSUP ("Operation not supported"), to differentiate the cases?

Issue arose when driver ixgbe and i40e choose to violate one-page-per-packet. After this point, the frame-tail size became variable. This cause issues when creating an SKB from an xdp_frame, see the workarounds in cpumap and veth, where the truesize is wrong and skb_shared_info placement is variable.

This can be solved by introducing a new xdp_buff member named data_hard_end, which specify the maximum address data_end can be extended to.

This will allow:

1. BPF helper (bpf_xdp_adjust_tail) to extend "tail" (not just shrink as today). This allows for use-cases (1) do fast DNS replies via XDP_TX, (2) NDIV (haproxy) also have some use-case that use tail-extend.
2. Create better SKBs based on an xdp_frame. (1) Allow placing skb_shared_info section consistently. (2) Use correct truesize. (3) Allow for TCP coalescing these kind of SKBs.

Keeping DMA mapping in XDP return API.

Missing an ifindex to match on in tracepoint

(Jakub question this)

(Plus other info)

Proprietary commercial companies like NapaTech, which also maps packets into userspace, deliver dynamic descriptor info in-front of the packet, like our data_meta area.

As far as I can see, via their public docs[4], they have 4 different dynamic descriptor formats. With BTF and metadata we should have more flexibility than them :-)

It is a bit interesting to look at what they expose. I recommend looking/clicking at the header-file[5][6][7][8] as it shows they use a lot of C-bit-fields to compress the size. Do BTF support C-bit fields?

[4] https://docs.napatech.com/search/all?query=Dynamic+Packet+Descriptor [5] pktdescr_dyn1.h https://docs.napatech.com/reader/Gtwjm73bddn7nrHz1NxHZw/leAUnFb_t2il~h4y1tNPpw [6] pktdescr_dyn2.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/7cYsE5yb3DLpomSTEeL_bQ [7] pktdescr_dyn3.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/hcQobdatqtY2j2nmrZ577A [8] pktdescr_dyn4.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/2LaoD2p2mvxpkNOVvBPBqg

I find it interesting to see that in (dyn1+2), default decode offsets into the packet of L3 and L4 (and L4 payload), but allows them to be programmable. Also notice the "color" members, which are programmable, and sometimes are use as a 64-bit unique correlation key[8] (e.g. identifying flows).

Maybe I should have looked at their standard format before the dynamic(?) Their pktdescr_std0.h [9] is placed in front of all packets being received by the adapter when the adapter is operating in STANDARD or EXTENDED mode. The Extended descriptors are placed after pktdescr_std0.h [9], which contains a lot of info on the types in different layers, see[10][11][12].

[9] pktdescr_std0.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/mRPP74KNQXIJtSyWBBHMKA [10] pktdescr_ext7.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/GXhjyPfAPJ6Rr7k8KR4KsQ [11] pktdescr_ext8.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/NWaAIAROOdyXvy~OV4pl6A [12] pktdescr_ext9.h https://docs.napatech.com/reader/GHSQQPQbWLPdJUmxIkO91Q/LVIq8m_b0_44QIIAqcdLew