alistair23-linux/tools/lib/bpf
David S. Miller 150f29f5e6 Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Daniel Borkmann says:

====================
pull-request: bpf-next 2020-09-01

The following pull-request contains BPF updates for your *net-next* tree.

There are two small conflicts when pulling, resolve as follows:

1) Merge conflict in tools/lib/bpf/libbpf.c between 88a8212028 ("libbpf: Factor
   out common ELF operations and improve logging") in bpf-next and 1e891e513e
   ("libbpf: Fix map index used in error message") in net-next. Resolve by taking
   the hunk in bpf-next:

        [...]
        scn = elf_sec_by_idx(obj, obj->efile.btf_maps_shndx);
        data = elf_sec_data(obj, scn);
        if (!scn || !data) {
                pr_warn("elf: failed to get %s map definitions for %s\n",
                        MAPS_ELF_SEC, obj->path);
                return -EINVAL;
        }
        [...]

2) Merge conflict in drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c between
   9647c57b11 ("xsk: i40e: ice: ixgbe: mlx5: Test for dma_need_sync earlier for
   better performance") in bpf-next and e20f0dbf20 ("net/mlx5e: RX, Add a prefetch
   command for small L1_CACHE_BYTES") in net-next. Resolve the two locations by retaining
   net_prefetch() and taking xsk_buff_dma_sync_for_cpu() from bpf-next. Should look like:

        [...]
        xdp_set_data_meta_invalid(xdp);
        xsk_buff_dma_sync_for_cpu(xdp, rq->xsk_pool);
        net_prefetch(xdp->data);
        [...]

We've added 133 non-merge commits during the last 14 day(s) which contain
a total of 246 files changed, 13832 insertions(+), 3105 deletions(-).

The main changes are:

1) Initial support for sleepable BPF programs along with bpf_copy_from_user() helper
   for tracing to reliably access user memory, from Alexei Starovoitov.

2) Add BPF infra for writing and parsing TCP header options, from Martin KaFai Lau.

3) bpf_d_path() helper for returning full path for given 'struct path', from Jiri Olsa.

4) AF_XDP support for shared umems between devices and queues, from Magnus Karlsson.

5) Initial prep work for full BPF-to-BPF call support in libbpf, from Andrii Nakryiko.

6) Generalize bpf_sk_storage map & add local storage for inodes, from KP Singh.

7) Implement sockmap/hash updates from BPF context, from Lorenz Bauer.

8) BPF xor verification for scalar types & add BPF link iterator, from Yonghong Song.

9) Use target's prog type for BPF_PROG_TYPE_EXT prog verification, from Udip Pant.

10) Rework BPF tracing samples to use libbpf loader, from Daniel T. Lee.

11) Fix xdpsock sample to really cycle through all buffers, from Weqaar Janjua.

12) Improve type safety for tun/veth XDP frame handling, from Maciej Żenczykowski.

13) Various smaller cleanups and improvements all over the place.
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
2020-09-01 13:22:59 -07:00
..
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
bpf.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
bpf.h tools/bpf: Support new uapi for map element bpf iterator 2020-08-06 16:39:14 -07:00
bpf_core_read.h libbpf: Implement enum value-based CO-RE relocations 2020-08-19 14:19:39 -07:00
bpf_endian.h libbpf: Make bpf_endian co-exist with vmlinux.h 2020-07-01 09:06:12 +02:00
bpf_helpers.h libbpf: Add __noinline macro to bpf_helpers.h 2020-08-21 15:40:22 -07:00
bpf_prog_linfo.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
bpf_tracing.h libbpf: Switch tracing and CO-RE helper macros to bpf_probe_read_kernel() 2020-08-18 17:16:15 -07:00
btf.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
btf.h libbpf: Clean up and improve CO-RE reloc logging 2020-08-18 18:12:45 -07:00
btf_dump.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2020-09-01 13:22:59 -07:00
Build libbpf: Add BPF ring buffer support 2020-06-01 14:38:22 -07:00
hashmap.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
hashmap.h libbpf: Fix libbpf hashmap on (I)LP32 architectures 2020-07-09 19:38:55 -07:00
libbpf.c libbpf: Support sleepable progs 2020-08-28 21:20:33 +02:00
libbpf.h libbpf: Add perf_buffer APIs for better integration with outside epoll loop 2020-08-21 14:26:55 -07:00
libbpf.map libbpf: Support shared umems between queues and devices 2020-08-31 21:15:05 +02:00
libbpf.pc.template libbpf: Add zlib as a dependency in pkg-config template 2019-12-16 14:55:29 -08:00
libbpf_common.h libbpf: Fix libbpf_common.h when installing libbpf through 'make install' 2019-12-18 00:19:31 +01:00
libbpf_errno.c libbpf: Poison kernel-only integer types 2020-01-10 10:38:00 -08:00
libbpf_internal.h libbpf: Fix libbpf build on compilers missing __builtin_mul_overflow 2020-08-20 16:45:09 +02:00
libbpf_probes.c bpf: Implement bpf_local_storage for inodes 2020-08-25 15:00:04 -07:00
libbpf_util.h libbpf: move logging helpers into libbpf_internal.h 2019-05-16 12:47:47 -07:00
Makefile tools, bpf/build: Cleanup feature files on make clean 2020-08-28 14:04:27 +02:00
netlink.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
nlattr.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
nlattr.h libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
README.rst libbpf: add perf_buffer_ prefix to README 2019-07-08 15:35:43 +02:00
ringbuf.c libbpf: Centralize poisoning and poison reallocarray() 2020-08-18 18:38:25 -07:00
str_error.c libbpf: Poison kernel-only integer types 2020-01-10 10:38:00 -08:00
str_error.h libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
xsk.c libbpf: Support shared umems between queues and devices 2020-08-31 21:15:05 +02:00
xsk.h libbpf: Support shared umems between queues and devices 2020-08-31 21:15:05 +02:00

.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)

libbpf API naming convention
============================

libbpf API provides access to a few logically separated groups of
functions and types. Every group has its own naming convention
described here. It's recommended to follow these conventions whenever a
new function or type is added to keep libbpf API clean and consistent.

All types and functions provided by libbpf API should have one of the
following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``,
``perf_buffer_``.

System call wrappers
--------------------

System call wrappers are simple wrappers for commands supported by
sys_bpf system call. These wrappers should go to ``bpf.h`` header file
and map one-on-one to corresponding commands.

For example ``bpf_map_lookup_elem`` wraps ``BPF_MAP_LOOKUP_ELEM``
command of sys_bpf, ``bpf_prog_attach`` wraps ``BPF_PROG_ATTACH``, etc.

Objects
-------

Another class of types and functions provided by libbpf API is "objects"
and functions to work with them. Objects are high-level abstractions
such as BPF program or BPF map. They're represented by corresponding
structures such as ``struct bpf_object``, ``struct bpf_program``,
``struct bpf_map``, etc.

Structures are forward declared and access to their fields should be
provided via corresponding getters and setters rather than directly.

These objects are associated with corresponding parts of ELF object that
contains compiled BPF programs.

For example ``struct bpf_object`` represents ELF object itself created
from an ELF file or from a buffer, ``struct bpf_program`` represents a
program in ELF object and ``struct bpf_map`` is a map.

Functions that work with an object have names built from object name,
double underscore and part that describes function purpose.

For example ``bpf_object__open`` consists of the name of corresponding
object, ``bpf_object``, double underscore and ``open`` that defines the
purpose of the function to open ELF file and create ``bpf_object`` from
it.

Another example: ``bpf_program__load`` is named for corresponding
object, ``bpf_program``, that is separated from other part of the name
by double underscore.

All objects and corresponding functions other than BTF related should go
to ``libbpf.h``. BTF types and functions should go to ``btf.h``.

Auxiliary functions
-------------------

Auxiliary functions and types that don't fit well in any of categories
described above should have ``libbpf_`` prefix, e.g.
``libbpf_get_error`` or ``libbpf_prog_type_by_name``.

AF_XDP functions
-------------------

AF_XDP functions should have an ``xsk_`` prefix, e.g.
``xsk_umem__get_data`` or ``xsk_umem__create``. The interface consists
of both low-level ring access functions and high-level configuration
functions. These can be mixed and matched. Note that these functions
are not reentrant for performance reasons.

Please take a look at Documentation/networking/af_xdp.rst in the Linux
kernel source tree on how to use XDP sockets and for some common
mistakes in case you do not get any traffic up to user space.

libbpf ABI
==========

libbpf can be both linked statically or used as DSO. To avoid possible
conflicts with other libraries an application is linked with, all
non-static libbpf symbols should have one of the prefixes mentioned in
API documentation above. See API naming convention to choose the right
name for a new symbol.

Symbol visibility
-----------------

libbpf follow the model when all global symbols have visibility "hidden"
by default and to make a symbol visible it has to be explicitly
attributed with ``LIBBPF_API`` macro. For example:

.. code-block:: c

        LIBBPF_API int bpf_prog_get_fd_by_id(__u32 id);

This prevents from accidentally exporting a symbol, that is not supposed
to be a part of ABI what, in turn, improves both libbpf developer- and
user-experiences.

ABI versionning
---------------

To make future ABI extensions possible libbpf ABI is versioned.
Versioning is implemented by ``libbpf.map`` version script that is
passed to linker.

Version name is ``LIBBPF_`` prefix + three-component numeric version,
starting from ``0.0.1``.

Every time ABI is being changed, e.g. because a new symbol is added or
semantic of existing symbol is changed, ABI version should be bumped.
This bump in ABI version is at most once per kernel development cycle.

For example, if current state of ``libbpf.map`` is:

.. code-block::
        LIBBPF_0.0.1 {
        	global:
                        bpf_func_a;
                        bpf_func_b;
        	local:
        		\*;
        };

, and a new symbol ``bpf_func_c`` is being introduced, then
``libbpf.map`` should be changed like this:

.. code-block::
        LIBBPF_0.0.1 {
        	global:
                        bpf_func_a;
                        bpf_func_b;
        	local:
        		\*;
        };
        LIBBPF_0.0.2 {
                global:
                        bpf_func_c;
        } LIBBPF_0.0.1;

, where new version ``LIBBPF_0.0.2`` depends on the previous
``LIBBPF_0.0.1``.

Format of version script and ways to handle ABI changes, including
incompatible ones, described in details in [1].

Stand-alone build
=================

Under https://github.com/libbpf/libbpf there is a (semi-)automated
mirror of the mainline's version of libbpf for a stand-alone build.

However, all changes to libbpf's code base must be upstreamed through
the mainline kernel tree.

License
=======

libbpf is dual-licensed under LGPL 2.1 and BSD 2-Clause.

Links
=====

[1] https://www.akkadia.org/drepper/dsohowto.pdf
    (Chapter 3. Maintaining APIs and ABIs).