• Tejun Heo's avatar
    blkcg: consolidate blkg creation in blkcg_bio_issue_check() · ae118896
    Tejun Heo authored
    blkg (blkcg_gq) currently is created by blkcg policies invoking
    blkg_lookup_create() which ends up repeating about the same code in
    different policies.  Theoretically, this can avoid the overhead of
    looking and/or creating blkg's if blkcg is enabled but no policy is in
    use; however, the cost of blkg lookup / creation is very low
    especially if only the root blkcg is in use which is highly likely if
    no blkcg policy is in active use - it boils down to a single very
    predictable conditional and surrounding RCU protection.
    
    This patch consolidates blkg creation to a new function
    blkcg_bio_issue_check() which is called during bio issue from
    generic_make_request_checks().  blkcg_bio_issue_check() is now the
    only function which tries to create missing blkg's.  The subsequent
    policy and request_list operations just perform blkg_lookup() and if
    missing falls back to the root.
    
    * blk_get_rl() no longer tries to create blkg.  It uses blkg_lookup()
      instead of blkg_lookup_create().
    
    * blk_throtl_bio() is now called from blkcg_bio_issue_check() with rcu
      read locked and blkg already looked up.  Both throtl_lookup_tg() and
      throtl_lookup_create_tg() are dropped.
    
    * cfq is similarly updated.  cfq_lookup_create_cfqg() is replaced with
      cfq_lookup_cfqg()which uses blkg_lookup().
    
    This consolidates blkg handling and avoids unnecessary blkg creation
    retries under memory pressure.  In addition, this provides a common
    bio entry point into blkcg where things like common accounting can be
    performed.
    
    v2: Build fixes for !CONFIG_CFQ_GROUP_IOSCHED and
        !CONFIG_BLK_DEV_THROTTLING.
    Signed-off-by: default avatarTejun Heo <tj@kernel.org>
    Cc: Vivek Goyal <vgoyal@redhat.com>
    Cc: Arianna Avanzini <avanzini.arianna@gmail.com>
    Signed-off-by: default avatarJens Axboe <axboe@fb.com>
    ae118896
cfq-iosched.c 121 KB