• Kent Overstreet's avatar
    bcachefs: rcu_pending · 8e973a4f
    Kent Overstreet authored
    Generic data structure for explicitly tracking pending RCU items,
    allowing items to be dequeued (i.e. allocate from items pending
    freeing). Works with conventional RCU and SRCU, and possibly other RCU
    flavors in the future, meaning this can serve as a more generic
    replacement for SLAB_TYPESAFE_BY_RCU.
    
    Pending items are tracked in radix trees; if memory allocation fails, we
    fall back to linked lists.
    
    A rcu_pending is initialized with a callback, which is invoked when
    pending items's grace periods have expired. Two types of callback
    processing are handled specially:
    
    - RCU_PENDING_KVFREE_FN
    
      New backend for kvfree_rcu(). Slightly faster, and eliminates the
      synchronize_rcu() slowpath in kvfree_rcu_mightsleep() - instead, an
      rcu_head is allocated if we don't have one and can't use the radix
      tree
    
      TODO:
      - add a shrinker (as in the existing kvfree_rcu implementation) so that
        memory reclaim can free expired objects if callback processing isn't
        keeping up, and to expedite a grace period if we're under memory
        pressure and too much memory is stranded by RCU
    
      - add a counter for amount of memory pending
    
    - RCU_PENDING_CALL_RCU_FN
    
      Accelerated backend for call_rcu() - pending callbacks are tracked in
      a radix tree to eliminate linked list overhead.
    
    to serve as replacement backends for kvfree_rcu() and call_rcu(); these
    may be of interest to other uses (e.g. SLAB_TYPESAFE_BY_RCU users).
    
    Note:
    
    Internally, we're using a single rearming call_rcu() callback for
    notifications from the core RCU subsystem for notifications when objects
    are ready to be processed.
    
    Ideally we would be getting a callback every time a grace period
    completes for which we have objects, but that would require multiple
    rcu_heads in flight, and since the number of gp sequence numbers with
    uncompleted callbacks is not bounded, we can't do that yet.
    Signed-off-by: default avatarKent Overstreet <kent.overstreet@linux.dev>
    8e973a4f
Makefile 1.71 KB