1. 18 Apr, 2023 3 commits
    • Mathieu Desnoyers's avatar
      mm: fix memory leak on mm_init error handling · b20b0368
      Mathieu Desnoyers authored
      commit f1a79412 ("mm: convert mm's rss stats into percpu_counter")
      introduces a memory leak by missing a call to destroy_context() when a
      percpu_counter fails to allocate.
      
      Before introducing the per-cpu counter allocations, init_new_context() was
      the last call that could fail in mm_init(), and thus there was no need to
      ever invoke destroy_context() in the error paths.  Adding the following
      percpu counter allocations adds error paths after init_new_context(),
      which means its associated destroy_context() needs to be called when
      percpu counters fail to allocate.
      
      Link: https://lkml.kernel.org/r/20230330133822.66271-1-mathieu.desnoyers@efficios.com
      Fixes: f1a79412 ("mm: convert mm's rss stats into percpu_counter")
      Signed-off-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Acked-by: default avatarShakeel Butt <shakeelb@google.com>
      Cc: Marek Szyprowski <m.szyprowski@samsung.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      b20b0368
    • Tetsuo Handa's avatar
      mm/page_alloc: fix potential deadlock on zonelist_update_seq seqlock · 1007843a
      Tetsuo Handa authored
      syzbot is reporting circular locking dependency which involves
      zonelist_update_seq seqlock [1], for this lock is checked by memory
      allocation requests which do not need to be retried.
      
      One deadlock scenario is kmalloc(GFP_ATOMIC) from an interrupt handler.
      
        CPU0
        ----
        __build_all_zonelists() {
          write_seqlock(&zonelist_update_seq); // makes zonelist_update_seq.seqcount odd
          // e.g. timer interrupt handler runs at this moment
            some_timer_func() {
              kmalloc(GFP_ATOMIC) {
                __alloc_pages_slowpath() {
                  read_seqbegin(&zonelist_update_seq) {
                    // spins forever because zonelist_update_seq.seqcount is odd
                  }
                }
              }
            }
          // e.g. timer interrupt handler finishes
          write_sequnlock(&zonelist_update_seq); // makes zonelist_update_seq.seqcount even
        }
      
      This deadlock scenario can be easily eliminated by not calling
      read_seqbegin(&zonelist_update_seq) from !__GFP_DIRECT_RECLAIM allocation
      requests, for retry is applicable to only __GFP_DIRECT_RECLAIM allocation
      requests.  But Michal Hocko does not know whether we should go with this
      approach.
      
      Another deadlock scenario which syzbot is reporting is a race between
      kmalloc(GFP_ATOMIC) from tty_insert_flip_string_and_push_buffer() with
      port->lock held and printk() from __build_all_zonelists() with
      zonelist_update_seq held.
      
        CPU0                                   CPU1
        ----                                   ----
        pty_write() {
          tty_insert_flip_string_and_push_buffer() {
                                               __build_all_zonelists() {
                                                 write_seqlock(&zonelist_update_seq);
                                                 build_zonelists() {
                                                   printk() {
                                                     vprintk() {
                                                       vprintk_default() {
                                                         vprintk_emit() {
                                                           console_unlock() {
                                                             console_flush_all() {
                                                               console_emit_next_record() {
                                                                 con->write() = serial8250_console_write() {
            spin_lock_irqsave(&port->lock, flags);
            tty_insert_flip_string() {
              tty_insert_flip_string_fixed_flag() {
                __tty_buffer_request_room() {
                  tty_buffer_alloc() {
                    kmalloc(GFP_ATOMIC | __GFP_NOWARN) {
                      __alloc_pages_slowpath() {
                        zonelist_iter_begin() {
                          read_seqbegin(&zonelist_update_seq); // spins forever because zonelist_update_seq.seqcount is odd
                                                                   spin_lock_irqsave(&port->lock, flags); // spins forever because port->lock is held
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
            spin_unlock_irqrestore(&port->lock, flags);
                                                                   // message is printed to console
                                                                   spin_unlock_irqrestore(&port->lock, flags);
                                                                 }
                                                               }
                                                             }
                                                           }
                                                         }
                                                       }
                                                     }
                                                   }
                                                 }
                                                 write_sequnlock(&zonelist_update_seq);
                                               }
          }
        }
      
      This deadlock scenario can be eliminated by
      
        preventing interrupt context from calling kmalloc(GFP_ATOMIC)
      
      and
      
        preventing printk() from calling console_flush_all()
      
      while zonelist_update_seq.seqcount is odd.
      
      Since Petr Mladek thinks that __build_all_zonelists() can become a
      candidate for deferring printk() [2], let's address this problem by
      
        disabling local interrupts in order to avoid kmalloc(GFP_ATOMIC)
      
      and
      
        disabling synchronous printk() in order to avoid console_flush_all()
      
      .
      
      As a side effect of minimizing duration of zonelist_update_seq.seqcount
      being odd by disabling synchronous printk(), latency at
      read_seqbegin(&zonelist_update_seq) for both !__GFP_DIRECT_RECLAIM and
      __GFP_DIRECT_RECLAIM allocation requests will be reduced.  Although, from
      lockdep perspective, not calling read_seqbegin(&zonelist_update_seq) (i.e.
      do not record unnecessary locking dependency) from interrupt context is
      still preferable, even if we don't allow calling kmalloc(GFP_ATOMIC)
      inside
      write_seqlock(&zonelist_update_seq)/write_sequnlock(&zonelist_update_seq)
      section...
      
      Link: https://lkml.kernel.org/r/8796b95c-3da3-5885-fddd-6ef55f30e4d3@I-love.SAKURA.ne.jp
      Fixes: 3d36424b ("mm/page_alloc: fix race condition between build_all_zonelists and page allocation")
      Link: https://lkml.kernel.org/r/ZCrs+1cDqPWTDFNM@alley [2]
      Reported-by: default avatarsyzbot <syzbot+223c7461c58c58a4cb10@syzkaller.appspotmail.com>
        Link: https://syzkaller.appspot.com/bug?extid=223c7461c58c58a4cb10 [1]
      Signed-off-by: default avatarTetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
      Acked-by: default avatarMichal Hocko <mhocko@suse.com>
      Acked-by: default avatarMel Gorman <mgorman@techsingularity.net>
      Cc: Petr Mladek <pmladek@suse.com>
      Cc: David Hildenbrand <david@redhat.com>
      Cc: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
      Cc: John Ogness <john.ogness@linutronix.de>
      Cc: Patrick Daly <quic_pdaly@quicinc.com>
      Cc: Sergey Senozhatsky <senozhatsky@chromium.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      1007843a
    • Ondrej Mosnacek's avatar
      kernel/sys.c: fix and improve control flow in __sys_setres[ug]id() · 659c0ce1
      Ondrej Mosnacek authored
      Linux Security Modules (LSMs) that implement the "capable" hook will
      usually emit an access denial message to the audit log whenever they
      "block" the current task from using the given capability based on their
      security policy.
      
      The occurrence of a denial is used as an indication that the given task
      has attempted an operation that requires the given access permission, so
      the callers of functions that perform LSM permission checks must take care
      to avoid calling them too early (before it is decided if the permission is
      actually needed to perform the requested operation).
      
      The __sys_setres[ug]id() functions violate this convention by first
      calling ns_capable_setid() and only then checking if the operation
      requires the capability or not.  It means that any caller that has the
      capability granted by DAC (task's capability set) but not by MAC (LSMs)
      will generate a "denied" audit record, even if is doing an operation for
      which the capability is not required.
      
      Fix this by reordering the checks such that ns_capable_setid() is checked
      last and -EPERM is returned immediately if it returns false.
      
      While there, also do two small optimizations:
      * move the capability check before prepare_creds() and
      * bail out early in case of a no-op.
      
      Link: https://lkml.kernel.org/r/20230217162154.837549-1-omosnace@redhat.com
      Fixes: 1da177e4 ("Linux-2.6.12-rc2")
      Signed-off-by: default avatarOndrej Mosnacek <omosnace@redhat.com>
      Cc: Eric W. Biederman <ebiederm@xmission.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      659c0ce1
  2. 16 Apr, 2023 11 commits
    • Peter Xu's avatar
      Revert "userfaultfd: don't fail on unrecognized features" · 2ff559f3
      Peter Xu authored
      This is a proposal to revert commit 914eedcb.
      
      I found this when writing a simple UFFDIO_API test to be the first unit
      test in this set.  Two things breaks with the commit:
      
        - UFFDIO_API check was lost and missing.  According to man page, the
        kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa.  This
        check is needed if the api version will be extended in the future, or
        user app won't be able to identify which is a new kernel.
      
        - Feature flags checks were removed, which means UFFDIO_API with a
        feature that does not exist will also succeed.  According to the man
        page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
        unknown features passed in.
      
      Link: https://lore.kernel.org/r/20220722201513.1624158-1-axelrasmussen@google.com
      Link: https://lkml.kernel.org/r/20230412163922.327282-2-peterx@redhat.com
      Fixes: 914eedcb ("userfaultfd: don't fail on unrecognized features")
      Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
      Acked-by: default avatarDavid Hildenbrand <david@redhat.com>
      Cc: Axel Rasmussen <axelrasmussen@google.com>
      Cc: Dmitry Safonov <0x7f454c46@gmail.com>
      Cc: Mike Kravetz <mike.kravetz@oracle.com>
      Cc: Mike Rapoport (IBM) <rppt@kernel.org>
      Cc: Zach O'Keefe <zokeefe@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      2ff559f3
    • Baokun Li's avatar
      writeback, cgroup: fix null-ptr-deref write in bdi_split_work_to_wbs · 1ba1199e
      Baokun Li authored
      KASAN report null-ptr-deref:
      ==================================================================
      BUG: KASAN: null-ptr-deref in bdi_split_work_to_wbs+0x5c5/0x7b0
      Write of size 8 at addr 0000000000000000 by task sync/943
      CPU: 5 PID: 943 Comm: sync Tainted: 6.3.0-rc5-next-20230406-dirty #461
      Call Trace:
       <TASK>
       dump_stack_lvl+0x7f/0xc0
       print_report+0x2ba/0x340
       kasan_report+0xc4/0x120
       kasan_check_range+0x1b7/0x2e0
       __kasan_check_write+0x24/0x40
       bdi_split_work_to_wbs+0x5c5/0x7b0
       sync_inodes_sb+0x195/0x630
       sync_inodes_one_sb+0x3a/0x50
       iterate_supers+0x106/0x1b0
       ksys_sync+0x98/0x160
      [...]
      ==================================================================
      
      The race that causes the above issue is as follows:
      
                 cpu1                     cpu2
      -------------------------|-------------------------
      inode_switch_wbs
       INIT_WORK(&isw->work, inode_switch_wbs_work_fn)
       queue_rcu_work(isw_wq, &isw->work)
       // queue_work async
        inode_switch_wbs_work_fn
         wb_put_many(old_wb, nr_switched)
          percpu_ref_put_many
           ref->data->release(ref)
           cgwb_release
            queue_work(cgwb_release_wq, &wb->release_work)
            // queue_work async
             &wb->release_work
             cgwb_release_workfn
                                  ksys_sync
                                   iterate_supers
                                    sync_inodes_one_sb
                                     sync_inodes_sb
                                      bdi_split_work_to_wbs
                                       kmalloc(sizeof(*work), GFP_ATOMIC)
                                       // alloc memory failed
              percpu_ref_exit
               ref->data = NULL
               kfree(data)
                                       wb_get(wb)
                                        percpu_ref_get(&wb->refcnt)
                                         percpu_ref_get_many(ref, 1)
                                          atomic_long_add(nr, &ref->data->count)
                                           atomic64_add(i, v)
                                           // trigger null-ptr-deref
      
      bdi_split_work_to_wbs() traverses &bdi->wb_list to split work into all
      wbs.  If the allocation of new work fails, the on-stack fallback will be
      used and the reference count of the current wb is increased afterwards. 
      If cgroup writeback membership switches occur before getting the reference
      count and the current wb is released as old_wd, then calling wb_get() or
      wb_put() will trigger the null pointer dereference above.
      
      This issue was introduced in v4.3-rc7 (see fix tag1).  Both
      sync_inodes_sb() and __writeback_inodes_sb_nr() calls to
      bdi_split_work_to_wbs() can trigger this issue.  For scenarios called via
      sync_inodes_sb(), originally commit 7fc5854f ("writeback: synchronize
      sync(2) against cgroup writeback membership switches") reduced the
      possibility of the issue by adding wb_switch_rwsem, but in v5.14-rc1 (see
      fix tag2) removed the "inode_io_list_del_locked(inode, old_wb)" from
      inode_switch_wbs_work_fn() so that wb->state contains WB_has_dirty_io,
      thus old_wb is not skipped when traversing wbs in bdi_split_work_to_wbs(),
      and the issue becomes easily reproducible again.
      
      To solve this problem, percpu_ref_exit() is called under RCU protection to
      avoid race between cgwb_release_workfn() and bdi_split_work_to_wbs(). 
      Moreover, replace wb_get() with wb_tryget() in bdi_split_work_to_wbs(),
      and skip the current wb if wb_tryget() fails because the wb has already
      been shutdown.
      
      Link: https://lkml.kernel.org/r/20230410130826.1492525-1-libaokun1@huawei.com
      Fixes: b817525a ("writeback: bdi_writeback iteration must not skip dying ones")
      Signed-off-by: default avatarBaokun Li <libaokun1@huawei.com>
      Reviewed-by: default avatarJan Kara <jack@suse.cz>
      Acked-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Alexander Viro <viro@zeniv.linux.org.uk>
      Cc: Andreas Dilger <adilger.kernel@dilger.ca>
      Cc: Christian Brauner <brauner@kernel.org>
      Cc: Dennis Zhou <dennis@kernel.org>
      Cc: Hou Tao <houtao1@huawei.com>
      Cc: yangerkun <yangerkun@huawei.com>
      Cc: Zhang Yi <yi.zhang@huawei.com>
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      1ba1199e
    • Peng Zhang's avatar
      maple_tree: fix a potential memory leak, OOB access, or other unpredictable bug · 1f5f12ec
      Peng Zhang authored
      In mas_alloc_nodes(), "node->node_count = 0" means to initialize the
      node_count field of the new node, but the node may not be a new node.  It
      may be a node that existed before and node_count has a value, setting it
      to 0 will cause a memory leak.  At this time, mas->alloc->total will be
      greater than the actual number of nodes in the linked list, which may
      cause many other errors.  For example, out-of-bounds access in
      mas_pop_node(), and mas_pop_node() may return addresses that should not be
      used.  Fix it by initializing node_count only for new nodes.
      
      Also, by the way, an if-else statement was removed to simplify the code.
      
      Link: https://lkml.kernel.org/r/20230411041005.26205-1-zhangpeng.00@bytedance.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarPeng Zhang <zhangpeng.00@bytedance.com>
      Reviewed-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      1f5f12ec
    • Steve Chou's avatar
      tools/mm/page_owner_sort.c: fix TGID output when cull=tg is used · 92357568
      Steve Chou authored
      When using cull option with 'tg' flag, the fprintf is using pid instead
      of tgid. It should use tgid instead.
      
      Link: https://lkml.kernel.org/r/20230411034929.2071501-1-steve_chou@pesi.com.tw
      Fixes: 9c8a0a8e ("tools/vm/page_owner_sort.c: support for user-defined culling rules")
      Signed-off-by: default avatarSteve Chou <steve_chou@pesi.com.tw>
      Cc: Jiajian Ye <yejiajian2018@email.szu.edu.cn>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      92357568
    • Jonathan Toppins's avatar
      mailmap: update jtoppins' entry to reference correct email · d2c115ba
      Jonathan Toppins authored
      Link: https://lkml.kernel.org/r/d79bc6eaf65e68bd1c2a1e1510ab6291ce5926a6.1681162487.git.jtoppins@redhat.comSigned-off-by: default avatarJonathan Toppins <jtoppins@redhat.com>
      Cc: Colin Ian King <colin.i.king@gmail.com>
      Cc: Jakub Kicinski <kuba@kernel.org>
      Cc: Kirill Tkhai <tkhai@ya.ru>
      Cc: Konrad Dybcio <konrad.dybcio@linaro.org>
      Cc: Qais Yousef <qyousef@layalina.io>
      Cc: Stephen Hemminger <stephen@networkplumber.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      d2c115ba
    • Liam R. Howlett's avatar
      mm/mempolicy: fix use-after-free of VMA iterator · f4e9e0e6
      Liam R. Howlett authored
      set_mempolicy_home_node() iterates over a list of VMAs and calls
      mbind_range() on each VMA, which also iterates over the singular list of
      the VMA passed in and potentially splits the VMA.  Since the VMA iterator
      is not passed through, set_mempolicy_home_node() may now point to a stale
      node in the VMA tree.  This can result in a UAF as reported by syzbot.
      
      Avoid the stale maple tree node by passing the VMA iterator through to the
      underlying call to split_vma().
      
      mbind_range() is also overly complicated, since there are two calling
      functions and one already handles iterating over the VMAs.  Simplify
      mbind_range() to only handle merging and splitting of the VMAs.
      
      Align the new loop in do_mbind() and existing loop in
      set_mempolicy_home_node() to use the reduced mbind_range() function.  This
      allows for a single location of the range calculation and avoids
      constantly looking up the previous VMA (since this is a loop over the
      VMAs).
      
      Link: https://lore.kernel.org/linux-mm/000000000000c93feb05f87e24ad@google.com/
      Fixes: 66850be5 ("mm/mempolicy: use vma iterator & maple state instead of vma linked list")
      Signed-off-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Reported-by: syzbot+a7c1ec5b1d71ceaa5186@syzkaller.appspotmail.com
        Link: https://lkml.kernel.org/r/20230410152205.2294819-1-Liam.Howlett@oracle.com
      Tested-by: syzbot+a7c1ec5b1d71ceaa5186@syzkaller.appspotmail.com
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      f4e9e0e6
    • Naoya Horiguchi's avatar
      mm/huge_memory.c: warn with pr_warn_ratelimited instead of VM_WARN_ON_ONCE_FOLIO · 4737edbb
      Naoya Horiguchi authored
      split_huge_page_to_list() WARNs when called for huge zero pages, which
      sounds to me too harsh because it does not imply a kernel bug, but just
      notifies the event to admins.  On the other hand, this is considered as
      critical by syzkaller and makes its testing less efficient, which seems to
      me harmful.
      
      So replace the VM_WARN_ON_ONCE_FOLIO with pr_warn_ratelimited.
      
      Link: https://lkml.kernel.org/r/20230406082004.2185420-1-naoya.horiguchi@linux.dev
      Fixes: 478d134e ("mm/huge_memory: do not overkill when splitting huge_zero_page")
      Signed-off-by: default avatarNaoya Horiguchi <naoya.horiguchi@nec.com>
      Reported-by: syzbot+07a218429c8d19b1fb25@syzkaller.appspotmail.com
        Link: https://lore.kernel.org/lkml/000000000000a6f34a05e6efcd01@google.com/Reviewed-by: default avatarYang Shi <shy828301@gmail.com>
      Cc: Miaohe Lin <linmiaohe@huawei.com>
      Cc: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
      Cc: Xu Yu <xuyu@linux.alibaba.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      4737edbb
    • Liam R. Howlett's avatar
      mm/mprotect: fix do_mprotect_pkey() return on error · 82f95134
      Liam R. Howlett authored
      When the loop over the VMA is terminated early due to an error, the return
      code could be overwritten with ENOMEM.  Fix the return code by only
      setting the error on early loop termination when the error is not set.
      
      User-visible effects include: attempts to run mprotect() against a
      special mapping or with a poorly-aligned hugetlb address should return
      -EINVAL, but they presently return -ENOMEM.  In other cases an -EACCESS
      should be returned.
      
      Link: https://lkml.kernel.org/r/20230406193050.1363476-1-Liam.Howlett@oracle.com
      Fixes: 2286a691 ("mm: change mprotect_fixup to vma iterator")
      Signed-off-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      82f95134
    • Peter Xu's avatar
      mm/khugepaged: check again on anon uffd-wp during isolation · dd47ac42
      Peter Xu authored
      Khugepaged collapse an anonymous thp in two rounds of scans.  The 2nd
      round done in __collapse_huge_page_isolate() after
      hpage_collapse_scan_pmd(), during which all the locks will be released
      temporarily.  It means the pgtable can change during this phase before 2nd
      round starts.
      
      It's logically possible some ptes got wr-protected during this phase, and
      we can errornously collapse a thp without noticing some ptes are
      wr-protected by userfault.  e1e267c7 wanted to avoid it but it only
      did that for the 1st phase, not the 2nd phase.
      
      Since __collapse_huge_page_isolate() happens after a round of small page
      swapins, we don't need to worry on any !present ptes - if it existed
      khugepaged will already bail out.  So we only need to check present ptes
      with uffd-wp bit set there.
      
      This is something I found only but never had a reproducer, I thought it
      was one caused a bug in Muhammad's recent pagemap new ioctl work, but it
      turns out it's not the cause of that but an userspace bug.  However this
      seems to still be a real bug even with a very small race window, still
      worth to have it fixed and copy stable.
      
      Link: https://lkml.kernel.org/r/20230405155120.3608140-1-peterx@redhat.com
      Fixes: e1e267c7 ("khugepaged: skip collapse if uffd-wp detected")
      Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
      Reviewed-by: default avatarDavid Hildenbrand <david@redhat.com>
      Reviewed-by: default avatarYang Shi <shy828301@gmail.com>
      Cc: Andrea Arcangeli <aarcange@redhat.com>
      Cc: Axel Rasmussen <axelrasmussen@google.com>
      Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
      Cc: Nadav Amit <nadav.amit@gmail.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      dd47ac42
    • David Hildenbrand's avatar
      mm/userfaultfd: fix uffd-wp handling for THP migration entries · 24bf08c4
      David Hildenbrand authored
      Looks like what we fixed for hugetlb in commit 44f86392 ("mm/hugetlb:
      fix uffd-wp handling for migration entries in
      hugetlb_change_protection()") similarly applies to THP.
      
      Setting/clearing uffd-wp on THP migration entries is not implemented
      properly.  Further, while removing migration PMDs considers the uffd-wp
      bit, inserting migration PMDs does not consider the uffd-wp bit.
      
      We have to set/clear independently of the migration entry type in
      change_huge_pmd() and properly copy the uffd-wp bit in
      set_pmd_migration_entry().
      
      Verified using a simple reproducer that triggers migration of a THP, that
      the set_pmd_migration_entry() no longer loses the uffd-wp bit.
      
      Link: https://lkml.kernel.org/r/20230405160236.587705-2-david@redhat.com
      Fixes: f45ec5ff ("userfaultfd: wp: support swap and page migration")
      Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
      Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
      Cc: <stable@vger.kernel.org>
      Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      24bf08c4
    • Qi Zheng's avatar
      mm: swap: fix performance regression on sparsetruncate-tiny · 998ad18b
      Qi Zheng authored
      The ->percpu_pvec_drained was originally introduced by commit d9ed0d08
      ("mm: only drain per-cpu pagevecs once per pagevec usage") to drain
      per-cpu pagevecs only once per pagevec usage.  But after converting the
      swap code to be more folio-based, the commit c2bc1681 ("mm/swap: add
      folio_batch_move_lru()") breaks this logic, which would cause
      ->percpu_pvec_drained to be reset to false, that means per-cpu pagevecs
      will be drained multiple times per pagevec usage.
      
      In theory, there should be no functional changes when converting code to
      be more folio-based.  We should call folio_batch_reinit() in
      folio_batch_move_lru() instead of folio_batch_init().  And to verify that
      we still need ->percpu_pvec_drained, I ran mmtests/sparsetruncate-tiny and
      got the following data:
      
                                   baseline                   with
                                  baseline/                 patch/
      Min       Time      326.00 (   0.00%)      328.00 (  -0.61%)
      1st-qrtle Time      334.00 (   0.00%)      336.00 (  -0.60%)
      2nd-qrtle Time      338.00 (   0.00%)      341.00 (  -0.89%)
      3rd-qrtle Time      343.00 (   0.00%)      347.00 (  -1.17%)
      Max-1     Time      326.00 (   0.00%)      328.00 (  -0.61%)
      Max-5     Time      327.00 (   0.00%)      330.00 (  -0.92%)
      Max-10    Time      328.00 (   0.00%)      331.00 (  -0.91%)
      Max-90    Time      350.00 (   0.00%)      357.00 (  -2.00%)
      Max-95    Time      395.00 (   0.00%)      390.00 (   1.27%)
      Max-99    Time      508.00 (   0.00%)      434.00 (  14.57%)
      Max       Time      547.00 (   0.00%)      476.00 (  12.98%)
      Amean     Time      344.61 (   0.00%)      345.56 *  -0.28%*
      Stddev    Time       30.34 (   0.00%)       19.51 (  35.69%)
      CoeffVar  Time        8.81 (   0.00%)        5.65 (  35.87%)
      BAmean-99 Time      342.38 (   0.00%)      344.27 (  -0.55%)
      BAmean-95 Time      338.58 (   0.00%)      341.87 (  -0.97%)
      BAmean-90 Time      336.89 (   0.00%)      340.26 (  -1.00%)
      BAmean-75 Time      335.18 (   0.00%)      338.40 (  -0.96%)
      BAmean-50 Time      332.54 (   0.00%)      335.42 (  -0.87%)
      BAmean-25 Time      329.30 (   0.00%)      332.00 (  -0.82%)
      
      From the above it can be seen that we get similar data to when
      ->percpu_pvec_drained was introduced, so we still need it.  Let's call
      folio_batch_reinit() in folio_batch_move_lru() to restore the original
      logic.
      
      Link: https://lkml.kernel.org/r/20230405161854.6931-1-zhengqi.arch@bytedance.com
      Fixes: c2bc1681 ("mm/swap: add folio_batch_move_lru()")
      Signed-off-by: default avatarQi Zheng <zhengqi.arch@bytedance.com>
      Reviewed-by: default avatarMatthew Wilcox (Oracle) <willy@infradead.org>
      Acked-by: default avatarMel Gorman <mgorman@suse.de>
      Cc: Lorenzo Stoakes <lstoakes@gmail.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      998ad18b
  3. 06 Apr, 2023 20 commits
    • Peng Zhang's avatar
      maple_tree: fix a potential concurrency bug in RCU mode · c45ea315
      Peng Zhang authored
      There is a concurrency bug that may cause the wrong value to be loaded
      when a CPU is modifying the maple tree.
      
      CPU1:
      mtree_insert_range()
        mas_insert()
          mas_store_root()
            ...
            mas_root_expand()
              ...
              rcu_assign_pointer(mas->tree->ma_root, mte_mk_root(mas->node));
              ma_set_meta(node, maple_leaf_64, 0, slot);    <---IP
      
      CPU2:
      mtree_load()
        mtree_lookup_walk()
          ma_data_end();
      
      When CPU1 is about to execute the instruction pointed to by IP, the
      ma_data_end() executed by CPU2 may return the wrong end position, which
      will cause the value loaded by mtree_load() to be wrong.
      
      An example of triggering the bug:
      
      Add mdelay(100) between rcu_assign_pointer() and ma_set_meta() in
      mas_root_expand().
      
      static DEFINE_MTREE(tree);
      int work(void *p) {
      	unsigned long val;
      	for (int i = 0 ; i< 30; ++i) {
      		val = (unsigned long)mtree_load(&tree, 8);
      		mdelay(5);
      		pr_info("%lu",val);
      	}
      	return 0;
      }
      
      mt_init_flags(&tree, MT_FLAGS_USE_RCU);
      mtree_insert(&tree, 0, (void*)12345, GFP_KERNEL);
      run_thread(work)
      mtree_insert(&tree, 1, (void*)56789, GFP_KERNEL);
      
      In RCU mode, mtree_load() should always return the value before or after
      the data structure is modified, and in this example mtree_load(&tree, 8)
      may return 56789 which is not expected, it should always return NULL.  Fix
      it by put ma_set_meta() before rcu_assign_pointer().
      
      Link: https://lkml.kernel.org/r/20230314124203.91572-4-zhangpeng.00@bytedance.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarPeng Zhang <zhangpeng.00@bytedance.com>
      Reviewed-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      c45ea315
    • Peng Zhang's avatar
      maple_tree: fix get wrong data_end in mtree_lookup_walk() · ec07967d
      Peng Zhang authored
      if (likely(offset > end))
      	max = pivots[offset];
      
      The above code should be changed to if (likely(offset < end)), which is
      correct.  This affects the correctness of ma_data_end().  Now it seems
      that the final result will not be wrong, but it is best to change it. 
      This patch does not change the code as above, because it simplifies the
      code by the way.
      
      Link: https://lkml.kernel.org/r/20230314124203.91572-1-zhangpeng.00@bytedance.com
      Link: https://lkml.kernel.org/r/20230314124203.91572-2-zhangpeng.00@bytedance.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarPeng Zhang <zhangpeng.00@bytedance.com>
      Reviewed-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      ec07967d
    • Rongwei Wang's avatar
      mm/swap: fix swap_info_struct race between swapoff and get_swap_pages() · 6fe7d6b9
      Rongwei Wang authored
      The si->lock must be held when deleting the si from the available list. 
      Otherwise, another thread can re-add the si to the available list, which
      can lead to memory corruption.  The only place we have found where this
      happens is in the swapoff path.  This case can be described as below:
      
      core 0                       core 1
      swapoff
      
      del_from_avail_list(si)      waiting
      
      try lock si->lock            acquire swap_avail_lock
                                   and re-add si into
                                   swap_avail_head
      
      acquire si->lock but missing si already being added again, and continuing
      to clear SWP_WRITEOK, etc.
      
      It can be easily found that a massive warning messages can be triggered
      inside get_swap_pages() by some special cases, for example, we call
      madvise(MADV_PAGEOUT) on blocks of touched memory concurrently, meanwhile,
      run much swapon-swapoff operations (e.g.  stress-ng-swap).
      
      However, in the worst case, panic can be caused by the above scene.  In
      swapoff(), the memory used by si could be kept in swap_info[] after
      turning off a swap.  This means memory corruption will not be caused
      immediately until allocated and reset for a new swap in the swapon path. 
      A panic message caused: (with CONFIG_PLIST_DEBUG enabled)
      
      ------------[ cut here ]------------
      top: 00000000e58a3003, n: 0000000013e75cda, p: 000000008cd4451a
      prev: 0000000035b1e58a, n: 000000008cd4451a, p: 000000002150ee8d
      next: 000000008cd4451a, n: 000000008cd4451a, p: 000000008cd4451a
      WARNING: CPU: 21 PID: 1843 at lib/plist.c:60 plist_check_prev_next_node+0x50/0x70
      Modules linked in: rfkill(E) crct10dif_ce(E)...
      CPU: 21 PID: 1843 Comm: stress-ng Kdump: ... 5.10.134+
      Hardware name: Alibaba Cloud ECS, BIOS 0.0.0 02/06/2015
      pstate: 60400005 (nZCv daif +PAN -UAO -TCO BTYPE=--)
      pc : plist_check_prev_next_node+0x50/0x70
      lr : plist_check_prev_next_node+0x50/0x70
      sp : ffff0018009d3c30
      x29: ffff0018009d3c40 x28: ffff800011b32a98
      x27: 0000000000000000 x26: ffff001803908000
      x25: ffff8000128ea088 x24: ffff800011b32a48
      x23: 0000000000000028 x22: ffff001800875c00
      x21: ffff800010f9e520 x20: ffff001800875c00
      x19: ffff001800fdc6e0 x18: 0000000000000030
      x17: 0000000000000000 x16: 0000000000000000
      x15: 0736076307640766 x14: 0730073007380731
      x13: 0736076307640766 x12: 0730073007380731
      x11: 000000000004058d x10: 0000000085a85b76
      x9 : ffff8000101436e4 x8 : ffff800011c8ce08
      x7 : 0000000000000000 x6 : 0000000000000001
      x5 : ffff0017df9ed338 x4 : 0000000000000001
      x3 : ffff8017ce62a000 x2 : ffff0017df9ed340
      x1 : 0000000000000000 x0 : 0000000000000000
      Call trace:
       plist_check_prev_next_node+0x50/0x70
       plist_check_head+0x80/0xf0
       plist_add+0x28/0x140
       add_to_avail_list+0x9c/0xf0
       _enable_swap_info+0x78/0xb4
       __do_sys_swapon+0x918/0xa10
       __arm64_sys_swapon+0x20/0x30
       el0_svc_common+0x8c/0x220
       do_el0_svc+0x2c/0x90
       el0_svc+0x1c/0x30
       el0_sync_handler+0xa8/0xb0
       el0_sync+0x148/0x180
      irq event stamp: 2082270
      
      Now, si->lock locked before calling 'del_from_avail_list()' to make sure
      other thread see the si had been deleted and SWP_WRITEOK cleared together,
      will not reinsert again.
      
      This problem exists in versions after stable 5.10.y.
      
      Link: https://lkml.kernel.org/r/20230404154716.23058-1-rongwei.wang@linux.alibaba.com
      Fixes: a2468cc9 ("swap: choose swap device according to numa node") 
      Tested-by: default avatarYongchen Yin <wb-yyc939293@alibaba-inc.com>
      Signed-off-by: default avatarRongwei Wang <rongwei.wang@linux.alibaba.com>
      Cc: Bagas Sanjaya <bagasdotme@gmail.com>
      Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
      Cc: Aaron Lu <aaron.lu@intel.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      6fe7d6b9
    • Ryusuke Konishi's avatar
      nilfs2: fix sysfs interface lifetime · 42560f9c
      Ryusuke Konishi authored
      The current nilfs2 sysfs support has issues with the timing of creation
      and deletion of sysfs entries, potentially leading to null pointer
      dereferences, use-after-free, and lockdep warnings.
      
      Some of the sysfs attributes for nilfs2 per-filesystem instance refer to
      metadata file "cpfile", "sufile", or "dat", but
      nilfs_sysfs_create_device_group that creates those attributes is executed
      before the inodes for these metadata files are loaded, and
      nilfs_sysfs_delete_device_group which deletes these sysfs entries is
      called after releasing their metadata file inodes.
      
      Therefore, access to some of these sysfs attributes may occur outside of
      the lifetime of these metadata files, resulting in inode NULL pointer
      dereferences or use-after-free.
      
      In addition, the call to nilfs_sysfs_create_device_group() is made during
      the locking period of the semaphore "ns_sem" of nilfs object, so the
      shrinker call caused by the memory allocation for the sysfs entries, may
      derive lock dependencies "ns_sem" -> (shrinker) -> "locks acquired in
      nilfs_evict_inode()".
      
      Since nilfs2 may acquire "ns_sem" deep in the call stack holding other
      locks via its error handler __nilfs_error(), this causes lockdep to report
      circular locking.  This is a false positive and no circular locking
      actually occurs as no inodes exist yet when
      nilfs_sysfs_create_device_group() is called.  Fortunately, the lockdep
      warnings can be resolved by simply moving the call to
      nilfs_sysfs_create_device_group() out of "ns_sem".
      
      This fixes these sysfs issues by revising where the device's sysfs
      interface is created/deleted and keeping its lifetime within the lifetime
      of the metadata files above.
      
      Link: https://lkml.kernel.org/r/20230330205515.6167-1-konishi.ryusuke@gmail.com
      Fixes: dd70edbd ("nilfs2: integrate sysfs support into driver")
      Signed-off-by: default avatarRyusuke Konishi <konishi.ryusuke@gmail.com>
      Reported-by: syzbot+979fa7f9c0d086fdc282@syzkaller.appspotmail.com
        Link: https://lkml.kernel.org/r/0000000000003414b505f7885f7e@google.com
      Reported-by: syzbot+5b7d542076d9bddc3c6a@syzkaller.appspotmail.com
        Link: https://lkml.kernel.org/r/0000000000006ac86605f5f44eb9@google.com
      Cc: Viacheslav Dubeyko <slava@dubeyko.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      42560f9c
    • Alistair Popple's avatar
      mm: take a page reference when removing device exclusive entries · 7c7b9629
      Alistair Popple authored
      Device exclusive page table entries are used to prevent CPU access to a
      page whilst it is being accessed from a device.  Typically this is used to
      implement atomic operations when the underlying bus does not support
      atomic access.  When a CPU thread encounters a device exclusive entry it
      locks the page and restores the original entry after calling mmu notifiers
      to signal drivers that exclusive access is no longer available.
      
      The device exclusive entry holds a reference to the page making it safe to
      access the struct page whilst the entry is present.  However the fault
      handling code does not hold the PTL when taking the page lock.  This means
      if there are multiple threads faulting concurrently on the device
      exclusive entry one will remove the entry whilst others will wait on the
      page lock without holding a reference.
      
      This can lead to threads locking or waiting on a folio with a zero
      refcount.  Whilst mmap_lock prevents the pages getting freed via munmap()
      they may still be freed by a migration.  This leads to warnings such as
      PAGE_FLAGS_CHECK_AT_FREE due to the page being locked when the refcount
      drops to zero.
      
      Fix this by trying to take a reference on the folio before locking it. 
      The code already checks the PTE under the PTL and aborts if the entry is
      no longer there.  It is also possible the folio has been unmapped, freed
      and re-allocated allowing a reference to be taken on an unrelated folio. 
      This case is also detected by the PTE check and the folio is unlocked
      without further changes.
      
      Link: https://lkml.kernel.org/r/20230330012519.804116-1-apopple@nvidia.com
      Fixes: b756a3b5 ("mm: device exclusive memory access")
      Signed-off-by: default avatarAlistair Popple <apopple@nvidia.com>
      Reviewed-by: default avatarRalph Campbell <rcampbell@nvidia.com>
      Reviewed-by: default avatarJohn Hubbard <jhubbard@nvidia.com>
      Acked-by: default avatarDavid Hildenbrand <david@redhat.com>
      Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
      Cc: Christoph Hellwig <hch@infradead.org>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      7c7b9629
    • Yafang Shao's avatar
      mm: vmalloc: avoid warn_alloc noise caused by fatal signal · f349b15e
      Yafang Shao authored
      There're some suspicious warn_alloc on my test serer, for example,
      
      [13366.518837] warn_alloc: 81 callbacks suppressed
      [13366.518841] test_verifier: vmalloc error: size 4096, page order 0, failed to allocate pages, mode:0x500dc2(GFP_HIGHUSER|__GFP_ZERO|__GFP_ACCOUNT), nodemask=(null),cpuset=/,mems_allowed=0-1
      [13366.522240] CPU: 30 PID: 722463 Comm: test_verifier Kdump: loaded Tainted: G        W  O       6.2.0+ #638
      [13366.524216] Call Trace:
      [13366.524702]  <TASK>
      [13366.525148]  dump_stack_lvl+0x6c/0x80
      [13366.525712]  dump_stack+0x10/0x20
      [13366.526239]  warn_alloc+0x119/0x190
      [13366.526783]  ? alloc_pages_bulk_array_mempolicy+0x9e/0x2a0
      [13366.527470]  __vmalloc_area_node+0x546/0x5b0
      [13366.528066]  __vmalloc_node_range+0xc2/0x210
      [13366.528660]  __vmalloc_node+0x42/0x50
      [13366.529186]  ? bpf_prog_realloc+0x53/0xc0
      [13366.529743]  __vmalloc+0x1e/0x30
      [13366.530235]  bpf_prog_realloc+0x53/0xc0
      [13366.530771]  bpf_patch_insn_single+0x80/0x1b0
      [13366.531351]  bpf_jit_blind_constants+0xe9/0x1c0
      [13366.531932]  ? __free_pages+0xee/0x100
      [13366.532457]  ? free_large_kmalloc+0x58/0xb0
      [13366.533002]  bpf_int_jit_compile+0x8c/0x5e0
      [13366.533546]  bpf_prog_select_runtime+0xb4/0x100
      [13366.534108]  bpf_prog_load+0x6b1/0xa50
      [13366.534610]  ? perf_event_task_tick+0x96/0xb0
      [13366.535151]  ? security_capable+0x3a/0x60
      [13366.535663]  __sys_bpf+0xb38/0x2190
      [13366.536120]  ? kvm_clock_get_cycles+0x9/0x10
      [13366.536643]  __x64_sys_bpf+0x1c/0x30
      [13366.537094]  do_syscall_64+0x38/0x90
      [13366.537554]  entry_SYSCALL_64_after_hwframe+0x72/0xdc
      [13366.538107] RIP: 0033:0x7f78310f8e29
      [13366.538561] Code: 01 00 48 81 c4 80 00 00 00 e9 f1 fe ff ff 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 17 e0 2c 00 f7 d8 64 89 01 48
      [13366.540286] RSP: 002b:00007ffe2a61fff8 EFLAGS: 00000206 ORIG_RAX: 0000000000000141
      [13366.541031] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f78310f8e29
      [13366.541749] RDX: 0000000000000080 RSI: 00007ffe2a6200b0 RDI: 0000000000000005
      [13366.542470] RBP: 00007ffe2a620010 R08: 00007ffe2a6202a0 R09: 00007ffe2a6200b0
      [13366.543183] R10: 00000000000f423e R11: 0000000000000206 R12: 0000000000407800
      [13366.543900] R13: 00007ffe2a620540 R14: 0000000000000000 R15: 0000000000000000
      [13366.544623]  </TASK>
      [13366.545260] Mem-Info:
      [13366.546121] active_anon:81319 inactive_anon:20733 isolated_anon:0
       active_file:69450 inactive_file:5624 isolated_file:0
       unevictable:0 dirty:10 writeback:0
       slab_reclaimable:69649 slab_unreclaimable:48930
       mapped:27400 shmem:12868 pagetables:4929
       sec_pagetables:0 bounce:0
       kernel_misc_reclaimable:0
       free:15870308 free_pcp:142935 free_cma:0
      [13366.551886] Node 0 active_anon:224836kB inactive_anon:33528kB active_file:175692kB inactive_file:13752kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:59248kB dirty:32kB writeback:0kB shmem:18252kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:4616kB pagetables:10664kB sec_pagetables:0kB all_unreclaimable? no
      [13366.555184] Node 1 active_anon:100440kB inactive_anon:49404kB active_file:102108kB inactive_file:8744kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:50352kB dirty:8kB writeback:0kB shmem:33220kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:3896kB pagetables:9052kB sec_pagetables:0kB all_unreclaimable? no
      [13366.558262] Node 0 DMA free:15360kB boost:0kB min:304kB low:380kB high:456kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
      [13366.560821] lowmem_reserve[]: 0 2735 31873 31873 31873
      [13366.561981] Node 0 DMA32 free:2790904kB boost:0kB min:56028kB low:70032kB high:84036kB reserved_highatomic:0KB active_anon:1936kB inactive_anon:20kB active_file:396kB inactive_file:344kB unevictable:0kB writepending:0kB present:3129200kB managed:2801520kB mlocked:0kB bounce:0kB free_pcp:5188kB local_pcp:0kB free_cma:0kB
      [13366.565148] lowmem_reserve[]: 0 0 29137 29137 29137
      [13366.566168] Node 0 Normal free:28533824kB boost:0kB min:596740kB low:745924kB high:895108kB reserved_highatomic:28672KB active_anon:222900kB inactive_anon:33508kB active_file:175296kB inactive_file:13408kB unevictable:0kB writepending:32kB present:30408704kB managed:29837172kB mlocked:0kB bounce:0kB free_pcp:295724kB local_pcp:0kB free_cma:0kB
      [13366.569485] lowmem_reserve[]: 0 0 0 0 0
      [13366.570416] Node 1 Normal free:32141144kB boost:0kB min:660504kB low:825628kB high:990752kB reserved_highatomic:69632KB active_anon:100440kB inactive_anon:49404kB active_file:102108kB inactive_file:8744kB unevictable:0kB writepending:8kB present:33554432kB managed:33025372kB mlocked:0kB bounce:0kB free_pcp:270880kB local_pcp:46860kB free_cma:0kB
      [13366.573403] lowmem_reserve[]: 0 0 0 0 0
      [13366.574015] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
      [13366.575474] Node 0 DMA32: 782*4kB (UME) 756*8kB (UME) 736*16kB (UME) 745*32kB (UME) 694*64kB (UME) 653*128kB (UME) 595*256kB (UME) 552*512kB (UME) 454*1024kB (UME) 347*2048kB (UME) 246*4096kB (UME) = 2790904kB
      [13366.577442] Node 0 Normal: 33856*4kB (UMEH) 51815*8kB (UMEH) 42418*16kB (UMEH) 36272*32kB (UMEH) 22195*64kB (UMEH) 10296*128kB (UMEH) 7238*256kB (UMEH) 5638*512kB (UEH) 5337*1024kB (UMEH) 3506*2048kB (UMEH) 1470*4096kB (UME) = 28533784kB
      [13366.580460] Node 1 Normal: 15776*4kB (UMEH) 37485*8kB (UMEH) 29509*16kB (UMEH) 21420*32kB (UMEH) 14818*64kB (UMEH) 13051*128kB (UMEH) 9918*256kB (UMEH) 7374*512kB (UMEH) 5397*1024kB (UMEH) 3887*2048kB (UMEH) 2002*4096kB (UME) = 32141240kB
      [13366.583027] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
      [13366.584380] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
      [13366.585702] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
      [13366.587042] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
      [13366.588372] 87386 total pagecache pages
      [13366.589266] 0 pages in swap cache
      [13366.590327] Free swap  = 0kB
      [13366.591227] Total swap = 0kB
      [13366.592142] 16777082 pages RAM
      [13366.593057] 0 pages HighMem/MovableOnly
      [13366.594037] 357226 pages reserved
      [13366.594979] 0 pages hwpoisoned
      
      This failure really confuse me as there're still lots of available pages. 
      Finally I figured out it was caused by a fatal signal.  When a process is
      allocating memory via vm_area_alloc_pages(), it will break directly even
      if it hasn't allocated the requested pages when it receives a fatal
      signal.  In that case, we shouldn't show this warn_alloc, as it is
      useless.  We only need to show this warning when there're really no enough
      pages.
      
      Link: https://lkml.kernel.org/r/20230330162625.13604-1-laoar.shao@gmail.comSigned-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Reviewed-by: default avatarLorenzo Stoakes <lstoakes@gmail.com>
      Cc: Christoph Hellwig <hch@infradead.org>
      Cc: Uladzislau Rezki (Sony) <urezki@gmail.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      f349b15e
    • Tetsuo Handa's avatar
      nilfs2: initialize "struct nilfs_binfo_dat"->bi_pad field · 73970316
      Tetsuo Handa authored
      nilfs_btree_assign_p() and nilfs_direct_assign_p() are not initializing
      "struct nilfs_binfo_dat"->bi_pad field, causing uninit-value reports when
      being passed to CRC function.
      
      Link: https://lkml.kernel.org/r/20230326152146.15872-1-konishi.ryusuke@gmail.comReported-by: default avatarsyzbot <syzbot+048585f3f4227bb2b49b@syzkaller.appspotmail.com>
        Link: https://syzkaller.appspot.com/bug?extid=048585f3f4227bb2b49bReported-by: default avatarDipanjan Das <mail.dipanjan.das@gmail.com>
        Link: https://lkml.kernel.org/r/CANX2M5bVbzRi6zH3PTcNE_31TzerstOXUa9Bay4E6y6dX23_pg@mail.gmail.comSigned-off-by: default avatarTetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
      Signed-off-by: default avatarRyusuke Konishi <konishi.ryusuke@gmail.com>
      Cc: Alexander Potapenko <glider@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      73970316
    • Ryusuke Konishi's avatar
      nilfs2: fix potential UAF of struct nilfs_sc_info in nilfs_segctor_thread() · 6be49d10
      Ryusuke Konishi authored
      The finalization of nilfs_segctor_thread() can race with
      nilfs_segctor_kill_thread() which terminates that thread, potentially
      causing a use-after-free BUG as KASAN detected.
      
      At the end of nilfs_segctor_thread(), it assigns NULL to "sc_task" member
      of "struct nilfs_sc_info" to indicate the thread has finished, and then
      notifies nilfs_segctor_kill_thread() of this using waitqueue
      "sc_wait_task" on the struct nilfs_sc_info.
      
      However, here, immediately after the NULL assignment to "sc_task", it is
      possible that nilfs_segctor_kill_thread() will detect it and return to
      continue the deallocation, freeing the nilfs_sc_info structure before the
      thread does the notification.
      
      This fixes the issue by protecting the NULL assignment to "sc_task" and
      its notification, with spinlock "sc_state_lock" of the struct
      nilfs_sc_info.  Since nilfs_segctor_kill_thread() does a final check to
      see if "sc_task" is NULL with "sc_state_lock" locked, this can eliminate
      the race.
      
      Link: https://lkml.kernel.org/r/20230327175318.8060-1-konishi.ryusuke@gmail.com
      Reported-by: syzbot+b08ebcc22f8f3e6be43a@syzkaller.appspotmail.com
      Link: https://lkml.kernel.org/r/00000000000000660d05f7dfa877@google.comSigned-off-by: default avatarRyusuke Konishi <konishi.ryusuke@gmail.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      6be49d10
    • Sergey Senozhatsky's avatar
      zsmalloc: document freeable stats · 618a8a91
      Sergey Senozhatsky authored
      When freeable class stat was added to classes file (back in 2016) we
      forgot to update zsmalloc documentation.  Fix that.
      
      Link: https://lkml.kernel.org/r/20230325024631.2817153-3-senozhatsky@chromium.org
      Fixes: 1120ed54 ("mm/zsmalloc: add `freeable' column to pool stat")
      Signed-off-by: default avatarSergey Senozhatsky <senozhatsky@chromium.org>
      Cc: Minchan Kim <minchan@kernel.org>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      618a8a91
    • Sergey Senozhatsky's avatar
      zsmalloc: document new fullness grouping · 119b57ea
      Sergey Senozhatsky authored
      Patch series "zsmalloc: minor documentation updates".
      
      Two minor patches that bring zsmalloc documentation up to date.
      
      
      This patch (of 2):
      
      Update documentation and reflect new zspages fullness grouping (we don't
      use almost_empty and almost_full anymore).
      
      Link: https://lkml.kernel.org/r/20230325024631.2817153-1-senozhatsky@chromium.org
      Link: https://lkml.kernel.org/r/20230325024631.2817153-2-senozhatsky@chromium.orgSigned-off-by: default avatarSergey Senozhatsky <senozhatsky@chromium.org>
      Fixes: 67e157eb3639 ("zsmalloc: show per fullness group class stats")
      Cc: Minchan Kim <minchan@kernel.org>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      119b57ea
    • Shiyang Ruan's avatar
      fsdax: force clear dirty mark if CoW · f76b3a32
      Shiyang Ruan authored
      XFS allows CoW on non-shared extents to combat fragmentation[1].  The old
      non-shared extent could be mwrited before, its dax entry is marked dirty. 
      
      This results in a WARNing:
      
      [   28.512349] ------------[ cut here ]------------
      [   28.512622] WARNING: CPU: 2 PID: 5255 at fs/dax.c:390 dax_insert_entry+0x342/0x390
      [   28.513050] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace fscache netfs nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables
      [   28.515462] CPU: 2 PID: 5255 Comm: fsstress Kdump: loaded Not tainted 6.3.0-rc1-00001-g85e1481e19c1-dirty #117
      [   28.515902] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Arch Linux 1.16.1-1-1 04/01/2014
      [   28.516307] RIP: 0010:dax_insert_entry+0x342/0x390
      [   28.516536] Code: 30 5b 5d 41 5c 41 5d 41 5e 41 5f c3 cc cc cc cc 48 8b 45 20 48 83 c0 01 e9 e2 fe ff ff 48 8b 45 20 48 83 c0 01 e9 cd fe ff ff <0f> 0b e9 53 ff ff ff 48 8b 7c 24 08 31 f6 e8 1b 61 a1 00 eb 8c 48
      [   28.517417] RSP: 0000:ffffc9000845fb18 EFLAGS: 00010086
      [   28.517721] RAX: 0000000000000053 RBX: 0000000000000155 RCX: 000000000018824b
      [   28.518113] RDX: 0000000000000000 RSI: ffffffff827525a6 RDI: 00000000ffffffff
      [   28.518515] RBP: ffffea00062092c0 R08: 0000000000000000 R09: ffffc9000845f9c8
      [   28.518905] R10: 0000000000000003 R11: ffffffff82ddb7e8 R12: 0000000000000155
      [   28.519301] R13: 0000000000000000 R14: 000000000018824b R15: ffff88810cfa76b8
      [   28.519703] FS:  00007f14a0c94740(0000) GS:ffff88817bd00000(0000) knlGS:0000000000000000
      [   28.520148] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [   28.520472] CR2: 00007f14a0c8d000 CR3: 000000010321c004 CR4: 0000000000770ee0
      [   28.520863] PKRU: 55555554
      [   28.521043] Call Trace:
      [   28.521219]  <TASK>
      [   28.521368]  dax_fault_iter+0x196/0x390
      [   28.521595]  dax_iomap_pte_fault+0x19b/0x3d0
      [   28.521852]  __xfs_filemap_fault+0x234/0x2b0
      [   28.522116]  __do_fault+0x30/0x130
      [   28.522334]  do_fault+0x193/0x340
      [   28.522586]  __handle_mm_fault+0x2d3/0x690
      [   28.522975]  handle_mm_fault+0xe6/0x2c0
      [   28.523259]  do_user_addr_fault+0x1bc/0x6f0
      [   28.523521]  exc_page_fault+0x60/0x140
      [   28.523763]  asm_exc_page_fault+0x22/0x30
      [   28.524001] RIP: 0033:0x7f14a0b589ca
      [   28.524225] Code: c5 fe 7f 07 c5 fe 7f 47 20 c5 fe 7f 47 40 c5 fe 7f 47 60 c5 f8 77 c3 66 0f 1f 84 00 00 00 00 00 40 0f b6 c6 48 89 d1 48 89 fa <f3> aa 48 89 d0 c5 f8 77 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90
      [   28.525198] RSP: 002b:00007fff1dea1c98 EFLAGS: 00010202
      [   28.525505] RAX: 000000000000001e RBX: 000000000014a000 RCX: 0000000000006046
      [   28.525895] RDX: 00007f14a0c82000 RSI: 000000000000001e RDI: 00007f14a0c8d000
      [   28.526290] RBP: 000000000000006f R08: 0000000000000004 R09: 000000000014a000
      [   28.526681] R10: 0000000000000008 R11: 0000000000000246 R12: 028f5c28f5c28f5c
      [   28.527067] R13: 8f5c28f5c28f5c29 R14: 0000000000011046 R15: 00007f14a0c946c0
      [   28.527449]  </TASK>
      [   28.527600] ---[ end trace 0000000000000000 ]---
      
      
      To be able to delete this entry, clear its dirty mark before
      invalidate_inode_pages2_range().
      
      [1] https://lore.kernel.org/linux-xfs/20230321151339.GA11376@frogsfrogsfrogs/
      
      Link: https://lkml.kernel.org/r/1679653680-2-1-git-send-email-ruansy.fnst@fujitsu.com
      Fixes: f80e1668 ("fsdax: invalidate pages when CoW")
      Signed-off-by: default avatarShiyang Ruan <ruansy.fnst@fujitsu.com>
      Cc: Dan Williams <dan.j.williams@intel.com>
      Cc: Darrick J. Wong <djwong@kernel.org>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      f76b3a32
    • Peter Xu's avatar
      mm/hugetlb: fix uffd wr-protection for CoW optimization path · 60d5b473
      Peter Xu authored
      This patch fixes an issue that a hugetlb uffd-wr-protected mapping can be
      writable even with uffd-wp bit set.  It only happens with hugetlb private
      mappings, when someone firstly wr-protects a missing pte (which will
      install a pte marker), then a write to the same page without any prior
      access to the page.
      
      Userfaultfd-wp trap for hugetlb was implemented in hugetlb_fault() before
      reaching hugetlb_wp() to avoid taking more locks that userfault won't
      need.  However there's one CoW optimization path that can trigger
      hugetlb_wp() inside hugetlb_no_page(), which will bypass the trap.
      
      This patch skips hugetlb_wp() for CoW and retries the fault if uffd-wp bit
      is detected.  The new path will only trigger in the CoW optimization path
      because generic hugetlb_fault() (e.g.  when a present pte was
      wr-protected) will resolve the uffd-wp bit already.  Also make sure
      anonymous UNSHARE won't be affected and can still be resolved, IOW only
      skip CoW not CoR.
      
      This patch will be needed for v5.19+ hence copy stable.
      
      [peterx@redhat.com: v2]
        Link: https://lkml.kernel.org/r/ZBzOqwF2wrHgBVZb@x1n
      [peterx@redhat.com: v3]
        Link: https://lkml.kernel.org/r/20230324142620.2344140-1-peterx@redhat.com
      Link: https://lkml.kernel.org/r/20230321191840.1897940-1-peterx@redhat.com
      Fixes: 166f3ecc ("mm/hugetlb: hook page faults for uffd write protection")
      Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
      Reported-by: default avatarMuhammad Usama Anjum <usama.anjum@collabora.com>
      Tested-by: default avatarMuhammad Usama Anjum <usama.anjum@collabora.com>
      Acked-by: default avatarDavid Hildenbrand <david@redhat.com>
      Reviewed-by: default avatarMike Kravetz <mike.kravetz@oracle.com>
      Cc: Andrea Arcangeli <aarcange@redhat.com>
      Cc: Axel Rasmussen <axelrasmussen@google.com>
      Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
      Cc: Nadav Amit <nadav.amit@gmail.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      60d5b473
    • Liam R. Howlett's avatar
      mm: enable maple tree RCU mode by default · 3dd44325
      Liam R. Howlett authored
      Use the maple tree in RCU mode for VMA tracking.
      
      The maple tree tracks the stack and is able to update the pivot
      (lower/upper boundary) in-place to allow the page fault handler to write
      to the tree while holding just the mmap read lock.  This is safe as the
      writes to the stack have a guard VMA which ensures there will always be a
      NULL in the direction of the growth and thus will only update a pivot.
      
      It is possible, but not recommended, to have VMAs that grow up/down
      without guard VMAs.  syzbot has constructed a testcase which sets up a VMA
      to grow and consume the empty space.  Overwriting the entire NULL entry
      causes the tree to be altered in a way that is not safe for concurrent
      readers; the readers may see a node being rewritten or one that does not
      match the maple state they are using.
      
      Enabling RCU mode allows the concurrent readers to see a stable node and
      will return the expected result.
      
      [Liam.Howlett@Oracle.com: we don't need to free the nodes with RCU[
      Link: https://lore.kernel.org/linux-mm/000000000000b0a65805f663ace6@google.com/
      Link: https://lkml.kernel.org/r/20230227173632.3292573-9-surenb@google.com
      Fixes: d4af56c5 ("mm: start tracking VMAs with maple tree")
      Signed-off-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Reported-by: syzbot+8d95422d3537159ca390@syzkaller.appspotmail.com
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      3dd44325
    • Liam R. Howlett's avatar
      maple_tree: add RCU lock checking to rcu callback functions · 790e1fa8
      Liam R. Howlett authored
      Dereferencing RCU objects within the RCU callback without the RCU check
      has caused lockdep to complain.  Fix the RCU dereferencing by using the
      RCU callback lock to ensure the operation is safe.
      
      Also stop creating a new lock to use for dereferencing during destruction
      of the tree or subtree.  Instead, pass through a pointer to the tree that
      has the lock that is held for RCU dereferencing checking.  It also does
      not make sense to use the maple state in the freeing scenario as the tree
      walk is a special case where the tree no longer has the normal encodings
      and parent pointers.
      
      Link: https://lkml.kernel.org/r/20230227173632.3292573-8-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Reported-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      790e1fa8
    • Liam R. Howlett's avatar
      maple_tree: add smp_rmb() to dead node detection · 0a2b18d9
      Liam R. Howlett authored
      Add an smp_rmb() before reading the parent pointer to ensure that anything
      read from the node prior to the parent pointer hasn't been reordered ahead
      of this check.
      
      The is necessary for RCU mode.
      
      Link: https://lkml.kernel.org/r/20230227173632.3292573-7-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      0a2b18d9
    • Liam R. Howlett's avatar
      maple_tree: fix write memory barrier of nodes once dead for RCU mode · c13af03d
      Liam R. Howlett authored
      During the development of the maple tree, the strategy of freeing multiple
      nodes changed and, in the process, the pivots were reused to store
      pointers to dead nodes.  To ensure the readers see accurate pivots, the
      writers need to mark the nodes as dead and call smp_wmb() to ensure any
      readers can identify the node as dead before using the pivot values.
      
      There were two places where the old method of marking the node as dead
      without smp_wmb() were being used, which resulted in RCU readers seeing
      the wrong pivot value before seeing the node was dead.  Fix this race
      condition by using mte_set_node_dead() which has the smp_wmb() call to
      ensure the race is closed.
      
      Add a WARN_ON() to the ma_free_rcu() call to ensure all nodes being freed
      are marked as dead to ensure there are no other call paths besides the two
      updated paths.
      
      This is necessary for the RCU mode of the maple tree.
      
      Link: https://lkml.kernel.org/r/20230227173632.3292573-6-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam R. Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      c13af03d
    • Liam Howlett's avatar
      maple_tree: remove extra smp_wmb() from mas_dead_leaves() · 8372f4d8
      Liam Howlett authored
      The call to mte_set_dead_node() before the smp_wmb() already calls
      smp_wmb() so this is not needed.  This is an optimization for the RCU mode
      of the maple tree.
      
      Link: https://lkml.kernel.org/r/20230227173632.3292573-5-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      8372f4d8
    • Liam Howlett's avatar
      maple_tree: fix freeing of nodes in rcu mode · 2e5b4921
      Liam Howlett authored
      The walk to destroy the nodes was not always setting the node type and
      would result in a destroy method potentially using the values as nodes. 
      Avoid this by setting the correct node types.  This is necessary for the
      RCU mode of the maple tree.
      
      Link: https://lkml.kernel.org/r/20230227173632.3292573-4-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      2e5b4921
    • Liam Howlett's avatar
      maple_tree: detect dead nodes in mas_start() · a7b92d59
      Liam Howlett authored
      When initially starting a search, the root node may already be in the
      process of being replaced in RCU mode.  Detect and restart the walk if
      this is the case.  This is necessary for RCU mode of the maple tree.
      
      Link: https://lkml.kernel.org/r/20230227173632.3292573-3-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      a7b92d59
    • Liam Howlett's avatar
      maple_tree: be more cautious about dead nodes · 39d0bd86
      Liam Howlett authored
      Patch series "Fix VMA tree modification under mmap read lock".
      
      Syzbot reported a BUG_ON in mm/mmap.c which was found to be caused by an
      inconsistency between threads walking the VMA maple tree.  The
      inconsistency is caused by the page fault handler modifying the maple tree
      while holding the mmap_lock for read.
      
      This only happens for stack VMAs.  We had thought this was safe as it only
      modifies a single pivot in the tree.  Unfortunately, syzbot constructed a
      test case where the stack had no guard page and grew the stack to abut the
      next VMA.  This causes us to delete the NULL entry between the two VMAs
      and rewrite the node.
      
      We considered several options for fixing this, including dropping the
      mmap_lock, then reacquiring it for write; and relaxing the definition of
      the tree to permit a zero-length NULL entry in the node.  We decided the
      best option was to backport some of the RCU patches from -next, which
      solve the problem by allocating a new node and RCU-freeing the old node. 
      Since the problem exists in 6.1, we preferred a solution which is similar
      to the one we intended to merge next merge window.
      
      These patches have been in -next since next-20230301, and have received
      intensive testing in Android as part of the RCU page fault patchset.  They
      were also sent as part of the "Per-VMA locks" v4 patch series.  Patches 1
      to 7 are bug fixes for RCU mode of the tree and patch 8 enables RCU mode
      for the tree.
      
      Performance v6.3-rc3 vs patched v6.3-rc3: Running these changes through
      mmtests showed there was a 15-20% performance decrease in
      will-it-scale/brk1-processes.  This tests creating and inserting a single
      VMA repeatedly through the brk interface and isn't representative of any
      real world applications.
      
      
      This patch (of 8):
      
      ma_pivots() and ma_data_end() may be called with a dead node.  Ensure to
      that the node isn't dead before using the returned values.
      
      This is necessary for RCU mode of the maple tree.
      
      Link: https://lkml.kernel.org/r/20230327185532.2354250-1-Liam.Howlett@oracle.com
      Link: https://lkml.kernel.org/r/20230227173632.3292573-1-surenb@google.com
      Link: https://lkml.kernel.org/r/20230227173632.3292573-2-surenb@google.com
      Fixes: 54a611b6 ("Maple Tree: add new data structure")
      Signed-off-by: default avatarLiam Howlett <Liam.Howlett@oracle.com>
      Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Arjun Roy <arjunroy@google.com>
      Cc: Axel Rasmussen <axelrasmussen@google.com>
      Cc: Chris Li <chriscli@google.com>
      Cc: David Hildenbrand <david@redhat.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Davidlohr Bueso <dave@stgolabs.net>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Eric Dumazet <edumazet@google.com>
      Cc: freak07 <michalechner92@googlemail.com>
      Cc: Greg Thelen <gthelen@google.com>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Jann Horn <jannh@google.com>
      Cc: Joel Fernandes <joelaf@google.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Kent Overstreet <kent.overstreet@linux.dev>
      Cc: Laurent Dufour <ldufour@linux.ibm.com>
      Cc: Lorenzo Stoakes <lstoakes@gmail.com>
      Cc: Matthew Wilcox <willy@infradead.org>
      Cc: Mel Gorman <mgorman@techsingularity.net>
      Cc: Michal Hocko <mhocko@suse.com>
      Cc: Mike Rapoport <rppt@kernel.org>
      Cc: Minchan Kim <minchan@google.com>
      Cc: Paul E. McKenney <paulmck@kernel.org>
      Cc: Peter Oskolkov <posk@google.com>
      Cc: Peter Xu <peterx@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Punit Agrawal <punit.agrawal@bytedance.com>
      Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
      Cc: Shakeel Butt <shakeelb@google.com>
      Cc: Soheil Hassas Yeganeh <soheil@google.com>
      Cc: Song Liu <songliubraving@fb.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Will Deacon <will@kernel.org>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      39d0bd86
  4. 28 Mar, 2023 6 commits