1. 19 Jun, 2019 11 commits
    • Daniel Borkmann's avatar
      Merge branch 'bpf-bounded-loops' · 94079b64
      Daniel Borkmann authored
      Alexei Starovoitov says:
      
      ====================
      v2->v3: fixed issues in backtracking pointed out by Andrii.
      The next step is to add a lot more tests for backtracking.
      
      v1->v2: addressed Andrii's feedback.
      
      this patch set introduces verifier support for bounded loops and
      adds several other improvements.
      Ideally they would be introduced one at a time,
      but to support bounded loop the verifier needs to 'step back'
      in the patch 1. That patch introduces tracking of spill/fill
      of constants through the stack. Though it's a useful feature
      it hurts cilium tests.
      Patch 3 introduces another feature by extending is_branch_taken
      logic to 'if rX op rY' conditions. This feature is also
      necessary to support bounded loops.
      Then patch 4 adds support for the loops while adding
      key heuristics with jmp_processed.
      Introduction of parentage chain of verifier states in patch 4
      allows patch 9 to add backtracking of precise scalar registers
      which finally resolves degradation from patch 1.
      
      The end result is much faster verifier for existing programs
      and new support for loops.
      See patch 8 for many kinds of loops that are now validated.
      Patch 9 is the most tricky one and could be rewritten with
      a different algorithm in the future.
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      94079b64
    • Alexei Starovoitov's avatar
      bpf: precise scalar_value tracking · b5dc0163
      Alexei Starovoitov authored
      Introduce precision tracking logic that
      helps cilium programs the most:
                        old clang  old clang    new clang  new clang
                                with all patches         with all patches
      bpf_lb-DLB_L3.o      1838     2283         1923       1863
      bpf_lb-DLB_L4.o      3218     2657         3077       2468
      bpf_lb-DUNKNOWN.o    1064     545          1062       544
      bpf_lxc-DDROP_ALL.o  26935    23045        166729     22629
      bpf_lxc-DUNKNOWN.o   34439    35240        174607     28805
      bpf_netdev.o         9721     8753         8407       6801
      bpf_overlay.o        6184     7901         5420       4754
      bpf_lxc_jit.o        39389    50925        39389      50925
      
      Consider code:
      654: (85) call bpf_get_hash_recalc#34
      655: (bf) r7 = r0
      656: (15) if r8 == 0x0 goto pc+29
      657: (bf) r2 = r10
      658: (07) r2 += -48
      659: (18) r1 = 0xffff8881e41e1b00
      661: (85) call bpf_map_lookup_elem#1
      662: (15) if r0 == 0x0 goto pc+23
      663: (69) r1 = *(u16 *)(r0 +0)
      664: (15) if r1 == 0x0 goto pc+21
      665: (bf) r8 = r7
      666: (57) r8 &= 65535
      667: (bf) r2 = r8
      668: (3f) r2 /= r1
      669: (2f) r2 *= r1
      670: (bf) r1 = r8
      671: (1f) r1 -= r2
      672: (57) r1 &= 255
      673: (25) if r1 > 0x1e goto pc+12
       R0=map_value(id=0,off=0,ks=20,vs=64,imm=0) R1_w=inv(id=0,umax_value=30,var_off=(0x0; 0x1f))
      674: (67) r1 <<= 1
      675: (0f) r0 += r1
      
      At this point the verifier will notice that scalar R1 is used in map pointer adjustment.
      R1 has to be precise for later operations on R0 to be validated properly.
      
      The verifier will backtrack the above code in the following way:
      last_idx 675 first_idx 664
      regs=2 stack=0 before 675: (0f) r0 += r1         // started backtracking R1 regs=2 is a bitmask
      regs=2 stack=0 before 674: (67) r1 <<= 1
      regs=2 stack=0 before 673: (25) if r1 > 0x1e goto pc+12
      regs=2 stack=0 before 672: (57) r1 &= 255
      regs=2 stack=0 before 671: (1f) r1 -= r2         // now both R1 and R2 has to be precise -> regs=6 mask
      regs=6 stack=0 before 670: (bf) r1 = r8          // after this insn R8 and R2 has to be precise
      regs=104 stack=0 before 669: (2f) r2 *= r1       // after this one R8, R2, and R1
      regs=106 stack=0 before 668: (3f) r2 /= r1
      regs=106 stack=0 before 667: (bf) r2 = r8
      regs=102 stack=0 before 666: (57) r8 &= 65535
      regs=102 stack=0 before 665: (bf) r8 = r7
      regs=82 stack=0 before 664: (15) if r1 == 0x0 goto pc+21
       // this is the end of verifier state. The following regs will be marked precised:
       R1_rw=invP(id=0,umax_value=65535,var_off=(0x0; 0xffff)) R7_rw=invP(id=0)
      parent didn't have regs=82 stack=0 marks         // so backtracking continues into parent state
      last_idx 663 first_idx 655
      regs=82 stack=0 before 663: (69) r1 = *(u16 *)(r0 +0)   // R1 was assigned no need to track it further
      regs=80 stack=0 before 662: (15) if r0 == 0x0 goto pc+23    // keep tracking R7
      regs=80 stack=0 before 661: (85) call bpf_map_lookup_elem#1  // keep tracking R7
      regs=80 stack=0 before 659: (18) r1 = 0xffff8881e41e1b00
      regs=80 stack=0 before 658: (07) r2 += -48
      regs=80 stack=0 before 657: (bf) r2 = r10
      regs=80 stack=0 before 656: (15) if r8 == 0x0 goto pc+29
      regs=80 stack=0 before 655: (bf) r7 = r0                // here the assignment into R7
       // mark R0 to be precise:
       R0_rw=invP(id=0)
      parent didn't have regs=1 stack=0 marks                 // regs=1 -> tracking R0
      last_idx 654 first_idx 644
      regs=1 stack=0 before 654: (85) call bpf_get_hash_recalc#34 // and in the parent frame it was a return value
        // nothing further to backtrack
      
      Two scalar registers not marked precise are equivalent from state pruning point of view.
      More details in the patch comments.
      
      It doesn't support bpf2bpf calls yet and enabled for root only.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      b5dc0163
    • Alexei Starovoitov's avatar
      selftests/bpf: add realistic loop tests · b061017f
      Alexei Starovoitov authored
      Add a bunch of loop tests. Most of them are created by replacing
      '#pragma unroll' with '#pragma clang loop unroll(disable)'
      
      Several tests are artificially large:
        /* partial unroll. llvm will unroll loop ~150 times.
         * C loop count -> 600.
         * Asm loop count -> 4.
         * 16k insns in loop body.
         * Total of 5 such loops. Total program size ~82k insns.
         */
        "./pyperf600.o",
      
        /* no unroll at all.
         * C loop count -> 600.
         * ASM loop count -> 600.
         * ~110 insns in loop body.
         * Total of 5 such loops. Total program size ~1500 insns.
         */
        "./pyperf600_nounroll.o",
      
        /* partial unroll. 19k insn in a loop.
         * Total program size 20.8k insn.
         * ~350k processed_insns
         */
        "./strobemeta.o",
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      b061017f
    • Alexei Starovoitov's avatar
      selftests/bpf: add basic verifier tests for loops · 0d3679e9
      Alexei Starovoitov authored
      This set of tests is a rewrite of Edward's earlier tests:
      https://patchwork.ozlabs.org/patch/877221/Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      0d3679e9
    • Alexei Starovoitov's avatar
      selftests/bpf: fix tests · aeee380c
      Alexei Starovoitov authored
      Fix tests that assumed no loops.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      aeee380c
    • Alexei Starovoitov's avatar
      bpf: fix callees pruning callers · eea1c227
      Alexei Starovoitov authored
      The commit 7640ead9 partially resolved the issue of callees
      incorrectly pruning the callers.
      With introduction of bounded loops and jmps_processed heuristic
      single verifier state may contain multiple branches and calls.
      It's possible that new verifier state (for future pruning) will be
      allocated inside callee. Then callee will exit (still within the same
      verifier state). It will go back to the caller and there R6-R9 registers
      will be read and will trigger mark_reg_read. But the reg->live for all frames
      but the top frame is not set to LIVE_NONE. Hence mark_reg_read will fail
      to propagate liveness into parent and future walking will incorrectly
      conclude that the states are equivalent because LIVE_READ is not set.
      In other words the rule for parent/live should be:
      whenever register parentage chain is set the reg->live should be set to LIVE_NONE.
      is_state_visited logic already follows this rule for spilled registers.
      
      Fixes: 7640ead9 ("bpf: verifier: make sure callees don't prune with caller differences")
      Fixes: f4d7e40a ("bpf: introduce function calls (verification)")
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      eea1c227
    • Alexei Starovoitov's avatar
      bpf: introduce bounded loops · 2589726d
      Alexei Starovoitov authored
      Allow the verifier to validate the loops by simulating their execution.
      Exisiting programs have used '#pragma unroll' to unroll the loops
      by the compiler. Instead let the verifier simulate all iterations
      of the loop.
      In order to do that introduce parentage chain of bpf_verifier_state and
      'branches' counter for the number of branches left to explore.
      See more detailed algorithm description in bpf_verifier.h
      
      This algorithm borrows the key idea from Edward Cree approach:
      https://patchwork.ozlabs.org/patch/877222/
      Additional state pruning heuristics make such brute force loop walk
      practical even for large loops.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      2589726d
    • Alexei Starovoitov's avatar
      bpf: extend is_branch_taken to registers · fb8d251e
      Alexei Starovoitov authored
      This patch extends is_branch_taken() logic from JMP+K instructions
      to JMP+X instructions.
      Conditional branches are often done when src and dst registers
      contain known scalars. In such case the verifier can follow
      the branch that is going to be taken when program executes.
      That speeds up the verification and is essential feature to support
      bounded loops.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fb8d251e
    • Alexei Starovoitov's avatar
      selftests/bpf: fix tests due to const spill/fill · fc559a70
      Alexei Starovoitov authored
      fix tests that incorrectly assumed that the verifier
      cannot track constants through stack.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fc559a70
    • Alexei Starovoitov's avatar
      bpf: track spill/fill of constants · f7cf25b2
      Alexei Starovoitov authored
      Compilers often spill induction variables into the stack,
      hence it is necessary for the verifier to track scalar values
      of the registers through stack slots.
      
      Also few bpf programs were incorrectly rejected in the past,
      since the verifier was not able to track such constants while
      they were used to compute offsets into packet headers.
      
      Tracking constants through the stack significantly decreases
      the chances of state pruning, since two different constants
      are considered to be different by state equivalency.
      End result that cilium tests suffer serious degradation in the number
      of states processed and corresponding verification time increase.
      
                           before  after
      bpf_lb-DLB_L3.o      1838    6441
      bpf_lb-DLB_L4.o      3218    5908
      bpf_lb-DUNKNOWN.o    1064    1064
      bpf_lxc-DDROP_ALL.o  26935   93790
      bpf_lxc-DUNKNOWN.o   34439   123886
      bpf_netdev.o         9721    31413
      bpf_overlay.o        6184    18561
      bpf_lxc_jit.o        39389   359445
      
      After further debugging turned out that cillium progs are
      getting hurt by clang due to the same constant tracking issue.
      Newer clang generates better code by spilling less to the stack.
      Instead it keeps more constants in the registers which
      hurts state pruning since the verifier already tracks constants
      in the registers:
                        old clang  new clang
                               (no spill/fill tracking introduced by this patch)
      bpf_lb-DLB_L3.o      1838    1923
      bpf_lb-DLB_L4.o      3218    3077
      bpf_lb-DUNKNOWN.o    1064    1062
      bpf_lxc-DDROP_ALL.o  26935   166729
      bpf_lxc-DUNKNOWN.o   34439   174607
      bpf_netdev.o         9721    8407
      bpf_overlay.o        6184    5420
      bpf_lcx_jit.o        39389   39389
      
      The final table is depressing:
                        old clang  old clang    new clang  new clang
                                 const spill/fill        const spill/fill
      bpf_lb-DLB_L3.o      1838    6441          1923      8128
      bpf_lb-DLB_L4.o      3218    5908          3077      6707
      bpf_lb-DUNKNOWN.o    1064    1064          1062      1062
      bpf_lxc-DDROP_ALL.o  26935   93790         166729    380712
      bpf_lxc-DUNKNOWN.o   34439   123886        174607    440652
      bpf_netdev.o         9721    31413         8407      31904
      bpf_overlay.o        6184    18561         5420      23569
      bpf_lxc_jit.o        39389   359445        39389     359445
      
      Tracking constants in the registers hurts state pruning already.
      Adding tracking of constants through stack hurts pruning even more.
      The later patch address this general constant tracking issue
      with coarse/precise logic.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      f7cf25b2
    • Andrii Nakryiko's avatar
      libbpf: constify getter APIs · a324aae3
      Andrii Nakryiko authored
      Add const qualifiers to bpf_object/bpf_program/bpf_map arguments for
      getter APIs. There is no need for them to not be const pointers.
      
      Verified that
      
      make -C tools/lib/bpf
      make -C tools/testing/selftests/bpf
      make -C tools/perf
      
      all build without warnings.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      a324aae3
  2. 17 Jun, 2019 13 commits
    • Daniel T. Lee's avatar
      samples: bpf: refactor header include path · 4d18f6de
      Daniel T. Lee authored
      Currently, header inclusion in each file is inconsistent.
      For example, "libbpf.h" header is included as multiple ways.
      
          #include "bpf/libbpf.h"
          #include "libbpf.h"
      
      Due to commit b552d33c ("samples/bpf: fix include path
      in Makefile"), $(srctree)/tools/lib/bpf/ path had been included
      during build, path "bpf/" in header isn't necessary anymore.
      
      This commit removes path "bpf/" in header inclusion.
      Signed-off-by: default avatarDaniel T. Lee <danieltimlee@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      4d18f6de
    • Daniel T. Lee's avatar
      samples: bpf: remove unnecessary include options in Makefile · fa206dcc
      Daniel T. Lee authored
      Due to recent change of include path at commit b552d33c
      ("samples/bpf: fix include path in Makefile"), some of the
      previous include options became unnecessary.
      
      This commit removes duplicated include options in Makefile.
      Signed-off-by: default avatarDaniel T. Lee <danieltimlee@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fa206dcc
    • Daniel Borkmann's avatar
      Merge branch 'bpf-libbpf-btf-defined-maps' · 32b88d37
      Daniel Borkmann authored
      Andrii Nakryiko says:
      
      ====================
      This patch set implements initial version (as discussed at LSF/MM2019
      conference) of a new way to specify BPF maps, relying on BTF type information,
      which allows for easy extensibility, preserving forward and backward
      compatibility. See details and examples in description for patch #6.
      
      [0] contains an outline of follow up extensions to be added after this basic
      set of features lands. They are useful by itself, but also allows to bring
      libbpf to feature-parity with iproute2 BPF loader. That should open a path
      forward for BPF loaders unification.
      
      Patch #1 centralizes commonly used min/max macro in libbpf_internal.h.
      Patch #2 extracts .BTF and .BTF.ext loading loging from elf_collect().
      Patch #3 simplifies elf_collect() error-handling logic.
      Patch #4 refactors map initialization logic into user-provided maps and global
      data maps, in preparation to adding another way (BTF-defined maps).
      Patch #5 adds support for map definitions in multiple ELF sections and
      deprecates bpf_object__find_map_by_offset() API which doesn't appear to be
      used anymore and makes assumption that all map definitions reside in single
      ELF section.
      Patch #6 splits BTF intialization from sanitization/loading into kernel to
      preserve original BTF at the time of map initialization.
      Patch #7 adds support for BTF-defined maps.
      Patch #8 adds new test for BTF-defined map definition.
      Patches #9-11 convert test BPF map definitions to use BTF way.
      
      [0] https://lore.kernel.org/bpf/CAEf4BzbfdG2ub7gCi0OYqBrUoChVHWsmOntWAkJt47=FE+km+A@mail.gmail.com/
      
      v1->v2:
      - more BTF-sanity checks in parsing map definitions (Song);
      - removed confusing usage of "attribute", switched to "field;
      - split off elf_collect() refactor from btf loading refactor (Song);
      - split selftests conversion into 3 patches (Stanislav):
        1. test already relying on BTF;
        2. tests w/ custom types as key/value (so benefiting from BTF);
        3. all the rest tests (integers as key/value, special maps w/o BTF support).
      - smaller code improvements (Song);
      
      rfc->v1:
      - error out on unknown field by default (Stanislav, Jakub, Lorenz);
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      32b88d37
    • Andrii Nakryiko's avatar
      selftests/bpf: convert tests w/ custom values to BTF-defined maps · df0b7792
      Andrii Nakryiko authored
      Convert a bulk of selftests that have maps with custom (not integer) key
      and/or value.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      df0b7792
    • Andrii Nakryiko's avatar
      selftests/bpf: switch BPF_ANNOTATE_KV_PAIR tests to BTF-defined maps · f6544074
      Andrii Nakryiko authored
      Switch tests that already rely on BTF to BTF-defined map definitions.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      f6544074
    • Andrii Nakryiko's avatar
      selftests/bpf: add test for BTF-defined maps · 9e3d709c
      Andrii Nakryiko authored
      Add file test for BTF-defined map definition.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      9e3d709c
    • Andrii Nakryiko's avatar
      libbpf: allow specifying map definitions using BTF · abd29c93
      Andrii Nakryiko authored
      This patch adds support for a new way to define BPF maps. It relies on
      BTF to describe mandatory and optional attributes of a map, as well as
      captures type information of key and value naturally. This eliminates
      the need for BPF_ANNOTATE_KV_PAIR hack and ensures key/value sizes are
      always in sync with the key/value type.
      
      Relying on BTF, this approach allows for both forward and backward
      compatibility w.r.t. extending supported map definition features. By
      default, any unrecognized attributes are treated as an error, but it's
      possible relax this using MAPS_RELAX_COMPAT flag. New attributes, added
      in the future will need to be optional.
      
      The outline of the new map definition (short, BTF-defined maps) is as follows:
      1. All the maps should be defined in .maps ELF section. It's possible to
         have both "legacy" map definitions in `maps` sections and BTF-defined
         maps in .maps sections. Everything will still work transparently.
      2. The map declaration and initialization is done through
         a global/static variable of a struct type with few mandatory and
         extra optional fields:
         - type field is mandatory and specified type of BPF map;
         - key/value fields are mandatory and capture key/value type/size information;
         - max_entries attribute is optional; if max_entries is not specified or
           initialized, it has to be provided in runtime through libbpf API
           before loading bpf_object;
         - map_flags is optional and if not defined, will be assumed to be 0.
      3. Key/value fields should be **a pointer** to a type describing
         key/value. The pointee type is assumed (and will be recorded as such
         and used for size determination) to be a type describing key/value of
         the map. This is done to save excessive amounts of space allocated in
         corresponding ELF sections for key/value of big size.
      4. As some maps disallow having BTF type ID associated with key/value,
         it's possible to specify key/value size explicitly without
         associating BTF type ID with it. Use key_size and value_size fields
         to do that (see example below).
      
      Here's an example of simple ARRAY map defintion:
      
      struct my_value { int x, y, z; };
      
      struct {
      	int type;
      	int max_entries;
      	int *key;
      	struct my_value *value;
      } btf_map SEC(".maps") = {
      	.type = BPF_MAP_TYPE_ARRAY,
      	.max_entries = 16,
      };
      
      This will define BPF ARRAY map 'btf_map' with 16 elements. The key will
      be of type int and thus key size will be 4 bytes. The value is struct
      my_value of size 12 bytes. This map can be used from C code exactly the
      same as with existing maps defined through struct bpf_map_def.
      
      Here's an example of STACKMAP definition (which currently disallows BTF type
      IDs for key/value):
      
      struct {
      	__u32 type;
      	__u32 max_entries;
      	__u32 map_flags;
      	__u32 key_size;
      	__u32 value_size;
      } stackmap SEC(".maps") = {
      	.type = BPF_MAP_TYPE_STACK_TRACE,
      	.max_entries = 128,
      	.map_flags = BPF_F_STACK_BUILD_ID,
      	.key_size = sizeof(__u32),
      	.value_size = PERF_MAX_STACK_DEPTH * sizeof(struct bpf_stack_build_id),
      };
      
      This approach is naturally extended to support map-in-map, by making a value
      field to be another struct that describes inner map. This feature is not
      implemented yet. It's also possible to incrementally add features like pinning
      with full backwards and forward compatibility. Support for static
      initialization of BPF_MAP_TYPE_PROG_ARRAY using pointers to BPF programs
      is also on the roadmap.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      abd29c93
    • Andrii Nakryiko's avatar
      libbpf: split initialization and loading of BTF · 063183bf
      Andrii Nakryiko authored
      Libbpf does sanitization of BTF before loading it into kernel, if kernel
      doesn't support some of newer BTF features. This removes some of the
      important information from BTF (e.g., DATASEC and VAR description),
      which will be used for map construction. This patch splits BTF
      processing into initialization step, in which BTF is initialized from
      ELF and all the original data is still preserved; and
      sanitization/loading step, which ensures that BTF is safe to load into
      kernel. This allows to use full BTF information to construct maps, while
      still loading valid BTF into older kernels.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      063183bf
    • Andrii Nakryiko's avatar
      libbpf: identify maps by section index in addition to offset · db48814b
      Andrii Nakryiko authored
      To support maps to be defined in multiple sections, it's important to
      identify map not just by offset within its section, but section index as
      well. This patch adds tracking of section index.
      
      For global data, we record section index of corresponding
      .data/.bss/.rodata ELF section for uniformity, and thus don't need
      a special value of offset for those maps.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      db48814b
    • Andrii Nakryiko's avatar
      libbpf: refactor map initialization · bf829271
      Andrii Nakryiko authored
      User and global data maps initialization has gotten pretty complicated
      and unnecessarily convoluted. This patch splits out the logic for global
      data map and user-defined map initialization. It also removes the
      restriction of pre-calculating how many maps will be initialized,
      instead allowing to keep adding new maps as they are discovered, which
      will be used later for BTF-defined map definitions.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      bf829271
    • Andrii Nakryiko's avatar
      libbpf: streamline ELF parsing error-handling · 01b29d1d
      Andrii Nakryiko authored
      Simplify ELF parsing logic by exiting early, as there is no common clean
      up path to execute. That makes it unnecessary to track when err was set
      and when it was cleared. It also reduces nesting in some places.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      01b29d1d
    • Andrii Nakryiko's avatar
      libbpf: extract BTF loading logic · 9c6660d0
      Andrii Nakryiko authored
      As a preparation for adding BTF-based BPF map loading, extract .BTF and
      .BTF.ext loading logic.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      9c6660d0
    • Andrii Nakryiko's avatar
      libbpf: add common min/max macro to libbpf_internal.h · d7fe74f9
      Andrii Nakryiko authored
      Multiple files in libbpf redefine their own definitions for min/max.
      Let's define them in libbpf_internal.h and use those everywhere.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      d7fe74f9
  3. 14 Jun, 2019 11 commits
  4. 12 Jun, 2019 1 commit
    • Valdis Klētnieks's avatar
      bpf: silence warning messages in core · aee450cb
      Valdis Klētnieks authored
      Compiling kernel/bpf/core.c with W=1 causes a flood of warnings:
      
      kernel/bpf/core.c:1198:65: warning: initialized field overwritten [-Woverride-init]
       1198 | #define BPF_INSN_3_TBL(x, y, z) [BPF_##x | BPF_##y | BPF_##z] = true
            |                                                                 ^~~~
      kernel/bpf/core.c:1087:2: note: in expansion of macro 'BPF_INSN_3_TBL'
       1087 |  INSN_3(ALU, ADD,  X),   \
            |  ^~~~~~
      kernel/bpf/core.c:1202:3: note: in expansion of macro 'BPF_INSN_MAP'
       1202 |   BPF_INSN_MAP(BPF_INSN_2_TBL, BPF_INSN_3_TBL),
            |   ^~~~~~~~~~~~
      kernel/bpf/core.c:1198:65: note: (near initialization for 'public_insntable[12]')
       1198 | #define BPF_INSN_3_TBL(x, y, z) [BPF_##x | BPF_##y | BPF_##z] = true
            |                                                                 ^~~~
      kernel/bpf/core.c:1087:2: note: in expansion of macro 'BPF_INSN_3_TBL'
       1087 |  INSN_3(ALU, ADD,  X),   \
            |  ^~~~~~
      kernel/bpf/core.c:1202:3: note: in expansion of macro 'BPF_INSN_MAP'
       1202 |   BPF_INSN_MAP(BPF_INSN_2_TBL, BPF_INSN_3_TBL),
            |   ^~~~~~~~~~~~
      
      98 copies of the above.
      
      The attached patch silences the warnings, because we *know* we're overwriting
      the default initializer. That leaves bpf/core.c with only 6 other warnings,
      which become more visible in comparison.
      Signed-off-by: default avatarValdis Kletnieks <valdis.kletnieks@vt.edu>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      aee450cb
  5. 11 Jun, 2019 4 commits
    • Daniel Borkmann's avatar
      Merge branch 'bpf-libbpf-num-cpus' · 5e2ac390
      Daniel Borkmann authored
      Hechao Li says:
      
      ====================
      Getting number of possible CPUs is commonly used for per-CPU BPF maps
      and perf_event_maps. Add a new API libbpf_num_possible_cpus() that
      helps user with per-CPU related operations and remove duplicate
      implementations in bpftool and selftests.
      
      v2: Save errno before calling pr_warning in case it is changed.
      v3: Make sure libbpf_num_possible_cpus never returns 0 so that user only
          has to check if ret value < 0.
      v4: Fix error code when reading 0 bytes from possible CPU file.
      v5: Fix selftests compliation issue.
      v6: Split commit to reuse libbpf_num_possible_cpus() into two commits:
          One commit to remove bpf_util.h from test BPF C programs.
          One commit to reuse libbpf_num_possible_cpus() in bpftools
          and bpf_util.h.
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      5e2ac390
    • Hechao Li's avatar
      bpf: use libbpf_num_possible_cpus internally · 4c587c19
      Hechao Li authored
      Use the newly added bpf_num_possible_cpus() in bpftool and selftests
      and remove duplicate implementations.
      Signed-off-by: default avatarHechao Li <hechaol@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      4c587c19
    • Hechao Li's avatar
      selftests/bpf: remove bpf_util.h from BPF C progs · ebb88607
      Hechao Li authored
      Though currently there is no problem including bpf_util.h in kernel
      space BPF C programs, in next patch in this stack, I will reuse
      libbpf_num_possible_cpus() in bpf_util.h thus include libbpf.h in it,
      which will cause BPF C programs compile error. Therefore I will first
      remove bpf_util.h from all test BPF programs.
      
      This can also make it clear that bpf_util.h is a user-space utility
      while bpf_helpers.h is a kernel space utility.
      Signed-off-by: default avatarHechao Li <hechaol@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      ebb88607
    • Hechao Li's avatar
      bpf: add a new API libbpf_num_possible_cpus() · 6446b315
      Hechao Li authored
      Adding a new API libbpf_num_possible_cpus() that helps user with
      per-CPU map operations.
      Signed-off-by: default avatarHechao Li <hechaol@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      6446b315