• Dave Marchevsky's avatar
    bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref · 1b121715
    Dave Marchevsky authored
    This patch enables the following pattern:
    
      /* mapval contains a __kptr pointing to refcounted local kptr */
      mapval = bpf_map_lookup_elem(&map, &idx);
      if (!mapval || !mapval->some_kptr) { /* omitted */ }
    
      p = bpf_refcount_acquire(&mapval->some_kptr);
    
    Currently this doesn't work because bpf_refcount_acquire expects an
    owning or non-owning ref. The verifier defines non-owning ref as a type:
    
      PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF
    
    while mapval->some_kptr is PTR_TO_BTF_ID | PTR_UNTRUSTED. It's possible
    to do the refcount_acquire by first bpf_kptr_xchg'ing mapval->some_kptr
    into a temp kptr, refcount_acquiring that, and xchg'ing back into
    mapval, but this is unwieldy and shouldn't be necessary.
    
    This patch modifies btf_ld_kptr_type such that user-allocated types are
    marked MEM_ALLOC and if those types have a bpf_{rb,list}_node they're
    marked NON_OWN_REF as well. Additionally, due to changes to
    bpf_obj_drop_impl earlier in this series, rcu_protected_object now
    returns true for all user-allocated types, resulting in
    mapval->some_kptr being marked MEM_RCU.
    
    After this patch's changes, mapval->some_kptr is now:
    
      PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF | MEM_RCU
    
    which results in it passing the non-owning ref test, and the motivating
    example passing verification.
    
    Future work will likely get rid of special non-owning ref lifetime logic
    in the verifier, at which point we'll be able to delete the NON_OWN_REF
    flag entirely.
    Signed-off-by: default avatarDave Marchevsky <davemarchevsky@fb.com>
    Link: https://lore.kernel.org/r/20231107085639.3016113-6-davemarchevsky@fb.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    1b121715
verifier.c 619 KB