• Alex Shi's avatar
    mm/lru: introduce TestClearPageLRU() · d25b5bd8
    Alex Shi authored
    Currently lru_lock still guards both lru list and page's lru bit, that's
    ok.  but if we want to use specific lruvec lock on the page, we need to
    pin down the page's lruvec/memcg during locking.  Just taking lruvec lock
    first may be undermined by the page's memcg charge/migration.  To fix this
    problem, we will clear the lru bit out of locking and use it as pin down
    action to block the page isolation in memcg changing.
    
    So now a standard steps of page isolation is following:
    	1, get_page(); 	       #pin the page avoid to be free
    	2, TestClearPageLRU(); #block other isolation like memcg change
    	3, spin_lock on lru_lock; #serialize lru list access
    	4, delete page from lru list;
    
    This patch start with the first part: TestClearPageLRU, which combines
    PageLRU check and ClearPageLRU into a macro func TestClearPageLRU.  This
    function will be used as page isolation precondition to prevent other
    isolations some where else.  Then there are may !PageLRU page on lru list,
    need to remove BUG() checking accordingly.
    
    There 2 rules for lru bit now:
    1, the lru bit still indicate if a page on lru list, just in some
       temporary moment(isolating), the page may have no lru bit when
       it's on lru list.  but the page still must be on lru list when the
       lru bit set.
    2, have to remove lru bit before delete it from lru list.
    
    As Andrew Morton mentioned this change would dirty cacheline for a page
    which isn't on the LRU.  But the loss would be acceptable in Rong Chen
    <rong.a.chen@intel.com> report:
    https://lore.kernel.org/lkml/20200304090301.GB5972@shao2-debian/
    
    Link: https://lkml.kernel.org/r/1604566549-62481-15-git-send-email-alex.shi@linux.alibaba.comSuggested-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Signed-off-by: default avatarAlex Shi <alex.shi@linux.alibaba.com>
    Acked-by: default avatarHugh Dickins <hughd@google.com>
    Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
    Cc: Alexander Duyck <alexander.duyck@gmail.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
    Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
    Cc: "Huang, Ying" <ying.huang@intel.com>
    Cc: Jann Horn <jannh@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Kirill A. Shutemov <kirill@shutemov.name>
    Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Mika Penttilä <mika.penttila@nextfour.com>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Shakeel Butt <shakeelb@google.com>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Wei Yang <richard.weiyang@gmail.com>
    Cc: Yang Shi <yang.shi@linux.alibaba.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    d25b5bd8
mlock.c 22.1 KB