• Kalesh Singh's avatar
    Multi-gen LRU: avoid race in inc_min_seq() · bb5e7f23
    Kalesh Singh authored
    inc_max_seq() will try to inc_min_seq() if nr_gens == MAX_NR_GENS. This
    is because the generations are reused (the last oldest now empty
    generation will become the next youngest generation).
    
    inc_min_seq() is retried until successful, dropping the lru_lock
    and yielding the CPU on each failure, and retaking the lock before
    trying again:
    
            while (!inc_min_seq(lruvec, type, can_swap)) {
                    spin_unlock_irq(&lruvec->lru_lock);
                    cond_resched();
                    spin_lock_irq(&lruvec->lru_lock);
            }
    
    However, the initial condition that required incrementing the min_seq
    (nr_gens == MAX_NR_GENS) is not retested. This can change by another
    call to inc_max_seq() from run_aging() with force_scan=true from the
    debugfs interface.
    
    Since the eviction stalls when the nr_gens == MIN_NR_GENS, avoid
    unnecessarily incrementing the min_seq by rechecking the number of
    generations before each attempt.
    
    This issue was uncovered in previous discussion on the list by Yu Zhao
    and Aneesh Kumar [1].
    
    [1] https://lore.kernel.org/linux-mm/CAOUHufbO7CaVm=xjEb1avDhHVvnC8pJmGyKcFf2iY_dpf+zR3w@mail.gmail.com/
    
    Link: https://lkml.kernel.org/r/20230802025606.346758-2-kaleshsingh@google.com
    Fixes: d6c3af7d ("mm: multi-gen LRU: debugfs interface")
    Signed-off-by: default avatarKalesh Singh <kaleshsingh@google.com>
    Tested-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com> [mediatek]
    Tested-by: default avatarCharan Teja Kalla <quic_charante@quicinc.com>
    Cc: Yu Zhao <yuzhao@google.com>
    Cc: Aneesh Kumar K V <aneesh.kumar@linux.ibm.com>
    Cc: Barry Song <baohua@kernel.org>
    Cc: Brian Geffon <bgeffon@google.com>
    Cc: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
    Cc: Lecopzer Chen <lecopzer.chen@mediatek.com>
    Cc: Matthias Brugger <matthias.bgg@gmail.com>
    Cc: Oleksandr Natalenko <oleksandr@natalenko.name>
    Cc: Qi Zheng <zhengqi.arch@bytedance.com>
    Cc: Steven Barrett <steven@liquorix.net>
    Cc: Suleiman Souhlal <suleiman@google.com>
    Cc: Suren Baghdasaryan <surenb@google.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    bb5e7f23
vmscan.c 222 KB