• Andrii Nakryiko's avatar
    selftests/bpf: support stats ordering in comparison mode in veristat · fa9bb590
    Andrii Nakryiko authored
    Introduce the concept of "stat variant", by which it's possible to
    specify whether to use the value from A (baseline) side, B (comparison
    or control) side, the absolute difference value or relative (percentage)
    difference value.
    
    To support specifying this, veristat recognizes `_a`, `_b`, `_diff`,
    `_pct` suffixes, which can be appended to stat name(s). In
    non-comparison mode variants are ignored (there is only `_a` variant
    effectively), if no variant suffix is provided, `_b` is assumed, as
    control group is of primary interest in comparison mode.
    
    These stat variants can be flexibly combined with asc/desc orders.
    
    Here's an example of ordering results first by verdict match/mismatch (or n/a
    if one of the sides is missing; n/a is always considered to be the lowest
    value), and within each match/mismatch/n/a group further sort by number of
    instructions in B side. In this case we don't have MISMATCH cases, but N/A are
    split from MATCH, demonstrating this custom ordering.
    
      $ ./veristat -e file,prog,verdict,insns -s verdict_diff,insns_b_ -C ~/base.csv ~/comp.csv
      File                Program                         Verdict (A)  Verdict (B)  Verdict (DIFF)  Insns (A)  Insns (B)  Insns   (DIFF)
      ------------------  ------------------------------  -----------  -----------  --------------  ---------  ---------  --------------
      bpf_xdp.o           tail_lb_ipv6                    N/A          success      N/A                   N/A     151895             N/A
      bpf_xdp.o           tail_nodeport_nat_egress_ipv4   N/A          success      N/A                   N/A      15619             N/A
      bpf_xdp.o           tail_nodeport_ipv6_dsr          N/A          success      N/A                   N/A       1206             N/A
      bpf_xdp.o           tail_nodeport_ipv4_dsr          N/A          success      N/A                   N/A       1162             N/A
      bpf_alignchecker.o  tail_icmp6_send_echo_reply      N/A          failure      N/A                   N/A         74             N/A
      bpf_alignchecker.o  __send_drop_notify              success      N/A          N/A                    53        N/A             N/A
      bpf_host.o          __send_drop_notify              success      N/A          N/A                    53        N/A             N/A
      bpf_host.o          cil_from_host                   success      N/A          N/A                   762        N/A             N/A
      bpf_xdp.o           tail_lb_ipv4                    success      success      MATCH               71736      73430  +1694 (+2.36%)
      bpf_xdp.o           tail_handle_nat_fwd_ipv4        success      success      MATCH               21547      20920   -627 (-2.91%)
      bpf_xdp.o           tail_rev_nodeport_lb6           success      success      MATCH               17954      17905    -49 (-0.27%)
      bpf_xdp.o           tail_handle_nat_fwd_ipv6        success      success      MATCH               16974      17039    +65 (+0.38%)
      bpf_xdp.o           tail_nodeport_nat_ingress_ipv4  success      success      MATCH                7658       7713    +55 (+0.72%)
      bpf_xdp.o           tail_rev_nodeport_lb4           success      success      MATCH                7126       6934   -192 (-2.69%)
      bpf_xdp.o           tail_nodeport_nat_ingress_ipv6  success      success      MATCH                6405       6397     -8 (-0.12%)
      bpf_xdp.o           tail_nodeport_nat_ipv6_egress   failure      failure      MATCH                 752        752     +0 (+0.00%)
      bpf_xdp.o           cil_xdp_entry                   success      success      MATCH                 423        423     +0 (+0.00%)
      bpf_xdp.o           __send_drop_notify              success      success      MATCH                 151        151     +0 (+0.00%)
      bpf_alignchecker.o  tail_icmp6_handle_ns            failure      failure      MATCH                  33         33     +0 (+0.00%)
      ------------------  ------------------------------  -----------  -----------  --------------  ---------  ---------  --------------
    Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
    Link: https://lore.kernel.org/r/20221103055304.2904589-10-andrii@kernel.orgSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    fa9bb590
veristat.c 45.3 KB