Project management for XDP-tutorial

Table of Contents

This document contains org-mode tasks and TODOs for XDP-tutorial. It is recommended to use emacs when viewing and editing these .org files, as the github rendering view removes the TODO and DONE marking on the tasks.

The XDP-tutorial is more than a tutorial. We want to evolve this to become a resource for XDP/BPF developer to learn good programming practices. This also means aspiring to high code quality, and show good practise for error handling, and show how code are placed into common/ directory files/objects.

Top-level headings indicate category, as explained in ../xdp-project.html.

Important medium-term tasks   @medium

TODO XDP-tutorial ideas for assignments/examples

DONE xdp-tutorial: rebase libbpf to get AF_XDP headers

Updated to the latest github libbpf commit 910c475f09e5 ("sync: latest libbpf changes from kernel") details see:

TODO xdp-tutorial: Split packet parsing between two XDP programs

This exercise/lesson is also meant to show how XDP hardware offloading can interact with regular XDP programs (software mode running driver level).

The assignment is to create two XDP programs. The first XDP-prog does packet header parsing, and stores parse information in metadata area (located just before payload) and BPF tail-calls into the second XDP-prog. The second XDP-prog reads info from metadata area, and acts on parse-info, e.g. shows it can read part of the packet, without doing parsing. The tricky part of the second XDP-prog is how to ensure the verifier is able to perform its static analysis and prove the program is safe(?).

As part of the assignment, it would be relevant to use BTF for giving the metadata area structure.

TODO xdp-tutorial: Extend to HW-offload: Split packet parsing between two XDP programs

When above assignment is complete. Then extend to HW-offload the first XDP-prog that does the parsing and storing in metadata area. It doesn't BPF tail-call the second XDP-prog, instead it loads the second XDP-prog in regular XDP driver hook.

This will demonstrate how to HW-offload packet header parse information via metadata area.

BTF considerations, how can we transfer BTF info from HW-offloaded XDP-prog?

TODO xdp-tutorial for tracepoints project

Tutorial sub-project for teaching people about using tracepoints.

The XDP angle is that infra depend on some tracepoints for stats, debugging and troubleshooting XDP. E.g. see the kernel/samples/bpf program xdp_monitor (xdp_monitor_kern.c and xdp_monitor_user.c). But the issue is that xdp_monitor (and other trace samples) use another ELF loader than libbpf (the samples local bpf_load.c), which we want to move people way from.

DONE xdp-tutorial: add tracepoint examples for reading XDP tracepoints

(This was merged as part of PR#42)

Work in progress by Jiri Olsa:

Tutorial already covers load BPF programs, but attaching to tracepoints need to be covered by tutorial. Attaching to tracepoints is a multi-step process, that involves reading an id from debugfs, using sys_perf_event_open, and calling some ioctl() events.

Ultimately we should consider adding to libbpf, tracepoint attach helper functions, to hide these multi-step attach steps. For the purpose of the tutorial we can demonstrate the steps, and have several tracing0x assignments that abstract and generalise this. Notice functions that we consider upstreaming to libbpf are placed in common/common_libbpf.c.

Another consideration: We should promote using raw_tracepoints for BPF as it have much less overhead for getting info that BPF need.

DONE xdp-tutorial: relate tracing to other assignments

(This was merged as part of PR#42)

When building up tracing assignments relate this to other parts of tutorial.

We can already relate this to other assignments. The assignment basic02 show how to use perf record + script with tracepoint event xdp:xdp_exception. And basic04 program have a "xdp_abort" section program, that can trigger this tracepoint.

DONE xdp-tutorial: fix bugs for raw_tracepoints

The PR#42 contained some bugs for tracing02-xdp-monitor. The bug is tracked by issue#48.

Deadline: This must be fixed before talk at Bornhack.

TODO xdp-tutorial: selftests project

We would like to have a system that (integrated with Makefile) that can run a selftest per assignment directory.

Longer-term preliminary plans   @long

Date: 2020-04-08 Wed 13:20