1. 26 Apr, 2018 12 commits
    • Quentin Monnet's avatar
      bpf: add documentation for eBPF helpers (33-41) · fa15601a
      Quentin Monnet authored
      Add documentation for eBPF helper functions to bpf.h user header file.
      This documentation can be parsed with the Python script provided in
      another commit of the patch series, in order to provide a RST document
      that can later be converted into a man page.
      
      The objective is to make the documentation easily understandable and
      accessible to all eBPF developers, including beginners.
      
      This patch contains descriptions for the following helper functions, all
      written by Daniel:
      
      - bpf_get_hash_recalc()
      - bpf_skb_change_tail()
      - bpf_skb_pull_data()
      - bpf_csum_update()
      - bpf_set_hash_invalid()
      - bpf_get_numa_node_id()
      - bpf_set_hash()
      - bpf_skb_adjust_room()
      - bpf_xdp_adjust_meta()
      
      v4:
      - bpf_skb_change_tail(): Clarify comment about invalidated verifier
        checks.
      - bpf_skb_pull_data(): Clarify the motivation for using this helper or
        bpf_skb_load_bytes(), on non-linear buffers. Fix RST formatting for
        *skb*. Clarify comment about invalidated verifier checks.
      - bpf_csum_update(): Fix description of checksum (entire packet, not IP
        checksum). Fix a typo: "header" instead of "helper".
      - bpf_set_hash_invalid(): Mention bpf_get_hash_recalc().
      - bpf_get_numa_node_id(): State that the helper is not restricted to
        programs attached to sockets.
      - bpf_skb_adjust_room(): Clarify comment about invalidated verifier
        checks.
      - bpf_xdp_adjust_meta(): Clarify comment about invalidated verifier
        checks.
      
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fa15601a
    • Quentin Monnet's avatar
      bpf: add documentation for eBPF helpers (23-32) · 1fdd08be
      Quentin Monnet authored
      Add documentation for eBPF helper functions to bpf.h user header file.
      This documentation can be parsed with the Python script provided in
      another commit of the patch series, in order to provide a RST document
      that can later be converted into a man page.
      
      The objective is to make the documentation easily understandable and
      accessible to all eBPF developers, including beginners.
      
      This patch contains descriptions for the following helper functions, all
      written by Daniel:
      
      - bpf_get_prandom_u32()
      - bpf_get_smp_processor_id()
      - bpf_get_cgroup_classid()
      - bpf_get_route_realm()
      - bpf_skb_load_bytes()
      - bpf_csum_diff()
      - bpf_skb_get_tunnel_opt()
      - bpf_skb_set_tunnel_opt()
      - bpf_skb_change_proto()
      - bpf_skb_change_type()
      
      v4:
      - bpf_get_prandom_u32(): Warn that the prng is not cryptographically
        secure.
      - bpf_get_smp_processor_id(): Fix a typo (case).
      - bpf_get_cgroup_classid(): Clarify description. Add notes on the helper
        being limited to cgroup v1, and to egress path.
      - bpf_get_route_realm(): Add comparison with bpf_get_cgroup_classid().
        Add a note about usage with TC and advantage of clsact. Fix a typo in
        return value ("sdb" instead of "skb").
      - bpf_skb_load_bytes(): Make explicit loading large data loads it to the
        eBPF stack.
      - bpf_csum_diff(): Add a note on seed that can be cascaded. Link to
        bpf_l3|l4_csum_replace().
      - bpf_skb_get_tunnel_opt(): Add a note about usage with "collect
        metadata" mode, and example of this with Geneve.
      - bpf_skb_set_tunnel_opt(): Add a link to bpf_skb_get_tunnel_opt()
        description.
      - bpf_skb_change_proto(): Mention that the main use case is NAT64.
        Clarify comment about invalidated verifier checks.
      
      v3:
      - bpf_get_prandom_u32(): Fix helper name :(. Add description, including
        a note on the internal random state.
      - bpf_get_smp_processor_id(): Add description, including a note on the
        processor id remaining stable during program run.
      - bpf_get_cgroup_classid(): State that CONFIG_CGROUP_NET_CLASSID is
        required to use the helper. Add a reference to related documentation.
        State that placing a task in net_cls controller disables cgroup-bpf.
      - bpf_get_route_realm(): State that CONFIG_CGROUP_NET_CLASSID is
        required to use this helper.
      - bpf_skb_load_bytes(): Fix comment on current use cases for the helper.
      
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      1fdd08be
    • Quentin Monnet's avatar
      bpf: add documentation for eBPF helpers (12-22) · c456dec4
      Quentin Monnet authored
      Add documentation for eBPF helper functions to bpf.h user header file.
      This documentation can be parsed with the Python script provided in
      another commit of the patch series, in order to provide a RST document
      that can later be converted into a man page.
      
      The objective is to make the documentation easily understandable and
      accessible to all eBPF developers, including beginners.
      
      This patch contains descriptions for the following helper functions, all
      written by Alexei:
      
      - bpf_get_current_pid_tgid()
      - bpf_get_current_uid_gid()
      - bpf_get_current_comm()
      - bpf_skb_vlan_push()
      - bpf_skb_vlan_pop()
      - bpf_skb_get_tunnel_key()
      - bpf_skb_set_tunnel_key()
      - bpf_redirect()
      - bpf_perf_event_output()
      - bpf_get_stackid()
      - bpf_get_current_task()
      
      v4:
      - bpf_redirect(): Fix typo: "XDP_ABORT" changed to "XDP_ABORTED". Add
        note on bpf_redirect_map() providing better performance. Replace "Save
        for" with "Except for".
      - bpf_skb_vlan_push(): Clarify comment about invalidated verifier
        checks.
      - bpf_skb_vlan_pop(): Clarify comment about invalidated verifier
        checks.
      - bpf_skb_get_tunnel_key(): Add notes on tunnel_id, "collect metadata"
        mode, and example tunneling protocols with which it can be used.
      - bpf_skb_set_tunnel_key(): Add a reference to the description of
        bpf_skb_get_tunnel_key().
      - bpf_perf_event_output(): Specify that, and for what purpose, the
        helper can be used with programs attached to TC and XDP.
      
      v3:
      - bpf_skb_get_tunnel_key(): Change and improve description and example.
      - bpf_redirect(): Improve description of BPF_F_INGRESS flag.
      - bpf_perf_event_output(): Fix first sentence of description. Delete
        wrong statement on context being evaluated as a struct pt_reg. Remove
        the long yet incomplete example.
      - bpf_get_stackid(): Add a note about PERF_MAX_STACK_DEPTH being
        configurable.
      
      Cc: Alexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      c456dec4
    • Quentin Monnet's avatar
      bpf: add documentation for eBPF helpers (01-11) · ad4a5223
      Quentin Monnet authored
      Add documentation for eBPF helper functions to bpf.h user header file.
      This documentation can be parsed with the Python script provided in
      another commit of the patch series, in order to provide a RST document
      that can later be converted into a man page.
      
      The objective is to make the documentation easily understandable and
      accessible to all eBPF developers, including beginners.
      
      This patch contains descriptions for the following helper functions, all
      written by Alexei:
      
      - bpf_map_lookup_elem()
      - bpf_map_update_elem()
      - bpf_map_delete_elem()
      - bpf_probe_read()
      - bpf_ktime_get_ns()
      - bpf_trace_printk()
      - bpf_skb_store_bytes()
      - bpf_l3_csum_replace()
      - bpf_l4_csum_replace()
      - bpf_tail_call()
      - bpf_clone_redirect()
      
      v4:
      - bpf_map_lookup_elem(): Add "const" qualifier for key.
      - bpf_map_update_elem(): Add "const" qualifier for key and value.
      - bpf_map_lookup_elem(): Add "const" qualifier for key.
      - bpf_skb_store_bytes(): Clarify comment about invalidated verifier
        checks.
      - bpf_l3_csum_replace(): Mention L3 instead of just IP, and add a note
        about bpf_csum_diff().
      - bpf_l4_csum_replace(): Mention L4 instead of just TCP/UDP, and add a
        note about bpf_csum_diff().
      - bpf_tail_call(): Bring minor edits to description.
      - bpf_clone_redirect(): Add a note about the relation with
        bpf_redirect(). Also clarify comment about invalidated verifier
        checks.
      
      v3:
      - bpf_map_lookup_elem(): Fix description of restrictions for flags
        related to the existence of the entry.
      - bpf_trace_printk(): State that trace_pipe can be configured. Fix
        return value in case an unknown format specifier is met. Add a note on
        kernel log notice when the helper is used. Edit example.
      - bpf_tail_call(): Improve comment on stack inheritance.
      - bpf_clone_redirect(): Improve description of BPF_F_INGRESS flag.
      
      Cc: Alexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      ad4a5223
    • Quentin Monnet's avatar
      bpf: add script and prepare bpf.h for new helpers documentation · 56a092c8
      Quentin Monnet authored
      Remove previous "overview" of eBPF helpers from user bpf.h header.
      Replace it by a comment explaining how to process the new documentation
      (to come in following patches) with a Python script to produce RST, then
      man page documentation.
      
      Also add the aforementioned Python script under scripts/. It is used to
      process include/uapi/linux/bpf.h and to extract helper descriptions, to
      turn it into a RST document that can further be processed with rst2man
      to produce a man page. The script takes one "--filename <path/to/file>"
      option. If the script is launched from scripts/ in the kernel root
      directory, it should be able to find the location of the header to
      parse, and "--filename <path/to/file>" is then optional. If it cannot
      find the file, then the option becomes mandatory. RST-formatted
      documentation is printed to standard output.
      
      Typical workflow for producing the final man page would be:
      
          $ ./scripts/bpf_helpers_doc.py \
                  --filename include/uapi/linux/bpf.h > /tmp/bpf-helpers.rst
          $ rst2man /tmp/bpf-helpers.rst > /tmp/bpf-helpers.7
          $ man /tmp/bpf-helpers.7
      
      Note that the tool kernel-doc cannot be used to document eBPF helpers,
      whose signatures are not available directly in the header files
      (pre-processor directives are used to produce them at the beginning of
      the compilation process).
      
      v4:
      - Also remove overviews for newly added bpf_xdp_adjust_tail() and
        bpf_skb_get_xfrm_state().
      - Remove vague statement about what helpers are restricted to GPL
        programs in "LICENSE" section for man page footer.
      - Replace license boilerplate with SPDX tag for Python script.
      
      v3:
      - Change license for man page.
      - Remove "for safety reasons" from man page header text.
      - Change "packets metadata" to "packets" in man page header text.
      - Move and fix comment on helpers introducing no overhead.
      - Remove "NOTES" section from man page footer.
      - Add "LICENSE" section to man page footer.
      - Edit description of file include/uapi/linux/bpf.h in man page footer.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      56a092c8
    • Daniel Borkmann's avatar
      Merge branch 'bpf-tunnel-metadata-selftests' · 3f13de6d
      Daniel Borkmann authored
      William Tu says:
      
      ====================
      The patch series provide end-to-end eBPF tunnel testsute.  A common topology
      is created below for all types of tunnels:
      
      Topology:
      ---------
           root namespace   |     at_ns0 namespace
                            |
            -----------     |     -----------
            | tnl dev |     |     | tnl dev |  (overlay network)
            -----------     |     -----------
            metadata-mode   |     native-mode
             with bpf       |
                            |
            ----------      |     ----------
            |  veth1  | --------- |  veth0  |  (underlay network)
            ----------    peer    ----------
      
      Device Configuration
      --------------------
       Root namespace with metadata-mode tunnel + BPF
       Device names and addresses:
             veth1 IP: 172.16.1.200, IPv6: 00::22 (underlay)
             tunnel dev <type>11, ex: gre11, IPv4: 10.1.1.200 (overlay)
      
       Namespace at_ns0 with native tunnel
       Device names and addresses:
             veth0 IPv4: 172.16.1.100, IPv6: 00::11 (underlay)
             tunnel dev <type>00, ex: gre00, IPv4: 10.1.1.100 (overlay)
      
      End-to-end ping packet flow
      ---------------------------
       Most of the tests start by namespace creation, device configuration,
       then ping the underlay and overlay network.  When doing 'ping 10.1.1.100'
       from root namespace, the following operations happen:
       1) Route lookup shows 10.1.1.100/24 belongs to tnl dev, fwd to tnl dev.
       2) Tnl device's egress BPF program is triggered and set the tunnel metadata,
          with remote_ip=172.16.1.200 and others.
       3) Outer tunnel header is prepended and route the packet to veth1's egress
       4) veth0's ingress queue receive the tunneled packet at namespace at_ns0
       5) Tunnel protocol handler, ex: vxlan_rcv, decap the packet
       6) Forward the packet to the overlay tnl dev
      
      Test Cases
      -----------------------------
       Tunnel Type |  BPF Programs
      -----------------------------
       GRE:          gre_set_tunnel, gre_get_tunnel
       IP6GRE:       ip6gretap_set_tunnel, ip6gretap_get_tunnel
       ERSPAN:       erspan_set_tunnel, erspan_get_tunnel
       IP6ERSPAN:    ip4ip6erspan_set_tunnel, ip4ip6erspan_get_tunnel
       VXLAN:        vxlan_set_tunnel, vxlan_get_tunnel
       IP6VXLAN:     ip6vxlan_set_tunnel, ip6vxlan_get_tunnel
       GENEVE:       geneve_set_tunnel, geneve_get_tunnel
       IP6GENEVE:    ip6geneve_set_tunnel, ip6geneve_get_tunnel
       IPIP:         ipip_set_tunnel, ipip_get_tunnel
       IP6IP:        ipip6_set_tunnel, ipip6_get_tunnel,
                     ip6ip6_set_tunnel, ip6ip6_get_tunnel
       XFRM:         xfrm_get_state
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      3f13de6d
    • William Tu's avatar
      samples/bpf: remove the bpf tunnel testsuite. · b05cd740
      William Tu authored
      Move the testsuite to
      selftests/bpf/{test_tunnel_kern.c, test_tunnel.sh}
      Signed-off-by: default avatarWilliam Tu <u9012063@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      b05cd740
    • William Tu's avatar
      selftests/bpf: bpf tunnel test. · 933a741e
      William Tu authored
      The patch migrates the original tests at samples/bpf/tcbpf2_kern.c
      and samples/bpf/test_tunnel_bpf.sh to selftests.  There are a couple
      changes from the original:
          1) add ipv6 vxlan, ipv6 geneve, ipv6 ipip tests
          2) simplify the original ipip tests (remove iperf tests)
          3) improve documentation
          4) use bpf_ntoh* and bpf_hton* api
      
      In summary, 'test_tunnel_kern.o' contains the following bpf program:
        GRE: gre_set_tunnel, gre_get_tunnel
        IP6GRE: ip6gretap_set_tunnel, ip6gretap_get_tunnel
        ERSPAN: erspan_set_tunnel, erspan_get_tunnel
        IP6ERSPAN: ip4ip6erspan_set_tunnel, ip4ip6erspan_get_tunnel
        VXLAN: vxlan_set_tunnel, vxlan_get_tunnel
        IP6VXLAN: ip6vxlan_set_tunnel, ip6vxlan_get_tunnel
        GENEVE: geneve_set_tunnel, geneve_get_tunnel
        IP6GENEVE: ip6geneve_set_tunnel, ip6geneve_get_tunnel
        IPIP: ipip_set_tunnel, ipip_get_tunnel
        IP6IP: ipip6_set_tunnel, ipip6_get_tunnel,
               ip6ip6_set_tunnel, ip6ip6_get_tunnel
        XFRM: xfrm_get_state
      Signed-off-by: default avatarWilliam Tu <u9012063@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      933a741e
    • Nikita V. Shirokov's avatar
      bpf: fix xdp_generic for bpf_adjust_tail usecase · f7613120
      Nikita V. Shirokov authored
      When bpf_adjust_tail was introduced for generic xdp, it changed skb's tail
      pointer, so it was pointing to the new "end of the packet". However skb's
      len field wasn't properly modified, so on the wire ethernet frame had
      original (or even bigger, if adjust_head was used) size. This diff is
      fixing this.
      
      Fixes: 198d83bb (" bpf: make generic xdp compatible w/ bpf_xdp_adjust_tail")
      Signed-off-by: default avatarNikita V. Shirokov <tehnerd@tehnerd.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      f7613120
    • Jiri Olsa's avatar
      tools, bpftool: Display license GPL compatible in prog show/list · 9b984a20
      Jiri Olsa authored
      Display the license "gpl" string in bpftool prog command, like:
      
        # bpftool prog list
        5: tracepoint  name func  tag 57cd311f2e27366b  gpl
                loaded_at Apr 26/09:37  uid 0
                xlated 16B  not jited  memlock 4096B
      
        # bpftool --json --pretty prog show
        [{
                "id": 5,
                "type": "tracepoint",
                "name": "func",
                "tag": "57cd311f2e27366b",
                "gpl_compatible": true,
                "loaded_at": "Apr 26/09:37",
                "uid": 0,
                "bytes_xlated": 16,
                "jited": false,
                "bytes_memlock": 4096
            }
        ]
      Signed-off-by: default avatarJiri Olsa <jolsa@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      9b984a20
    • Jiri Olsa's avatar
      tools, bpf: Sync bpf.h uapi header · fb6ef42b
      Jiri Olsa authored
      Syncing the bpf.h uapi header with tools.
      Signed-off-by: default avatarJiri Olsa <jolsa@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fb6ef42b
    • Jiri Olsa's avatar
      bpf: Add gpl_compatible flag to struct bpf_prog_info · b85fab0e
      Jiri Olsa authored
      Adding gpl_compatible flag to struct bpf_prog_info
      so it can be dumped via bpf_prog_get_info_by_fd and
      displayed via bpftool progs dump.
      
      Alexei noticed 4-byte hole in struct bpf_prog_info,
      so we put the u32 flags field in there, and we can
      keep adding bit fields in there without breaking
      user space.
      Signed-off-by: default avatarJiri Olsa <jolsa@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      b85fab0e
  2. 25 Apr, 2018 6 commits
  3. 24 Apr, 2018 13 commits
  4. 23 Apr, 2018 3 commits
  5. 21 Apr, 2018 6 commits