• Chuyi Zhou's avatar
    bpf: teach the verifier to enforce css_iter and task_iter in RCU CS · dfab99df
    Chuyi Zhou authored
    css_iter and task_iter should be used in rcu section. Specifically, in
    sleepable progs explicit bpf_rcu_read_lock() is needed before use these
    iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to
    use them directly.
    
    This patch adds a new a KF flag KF_RCU_PROTECTED for bpf_iter_task_new and
    bpf_iter_css_new. It means the kfunc should be used in RCU CS. We check
    whether we are in rcu cs before we want to invoke this kfunc. If the rcu
    protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU.
    Once user do rcu_unlock during the iteration, state MEM_RCU of regs would
    be cleared. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED.
    
    It is worth noting that currently, bpf_rcu_read_unlock does not
    clear the state of the STACK_ITER reg, since bpf_for_each_spilled_reg
    only considers STACK_SPILL. This patch also let bpf_for_each_spilled_reg
    search STACK_ITER.
    Signed-off-by: default avatarChuyi Zhou <zhouchuyi@bytedance.com>
    Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
    Link: https://lore.kernel.org/r/20231018061746.111364-6-zhouchuyi@bytedance.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    dfab99df
helpers.c 68.9 KB