1. 11 Oct, 2019 1 commit
    • Linus Torvalds's avatar
      Merge tag 'for-linus-20191010' of git://git.kernel.dk/linux-block · 297cbccc
      Linus Torvalds authored
      Pull block fixes from Jens Axboe:
      
       - Fix wbt performance regression introduced with the blk-rq-qos
         refactoring (Harshad)
      
       - Fix io_uring fileset removal inadvertently killing the workqueue (me)
      
       - Fix io_uring typo in linked command nonblock submission (Pavel)
      
       - Remove spurious io_uring wakeups on request free (Pavel)
      
       - Fix null_blk zoned command error return (Keith)
      
       - Don't use freezable workqueues for backing_dev, also means we can
         revert a previous libata hack (Mika)
      
       - Fix nbd sysfs mutex dropped too soon at removal time (Xiubo)
      
      * tag 'for-linus-20191010' of git://git.kernel.dk/linux-block:
        nbd: fix possible sysfs duplicate warning
        null_blk: Fix zoned command return code
        io_uring: only flush workqueues on fileset removal
        io_uring: remove wait loop spurious wakeups
        blk-wbt: fix performance regression in wbt scale_up/scale_down
        Revert "libata, freezer: avoid block device removal while system is frozen"
        bdi: Do not use freezable workqueue
        io_uring: fix reversed nonblock flag for link submission
      297cbccc
  2. 10 Oct, 2019 11 commits
  3. 09 Oct, 2019 6 commits
    • Jens Axboe's avatar
      io_uring: only flush workqueues on fileset removal · 8a997340
      Jens Axboe authored
      We should not remove the workqueue, we just need to ensure that the
      workqueues are synced. The workqueues are torn down on ctx removal.
      
      Cc: stable@vger.kernel.org
      Fixes: 6b06314c ("io_uring: add file set registration")
      Reported-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      8a997340
    • Linus Torvalds's avatar
      Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma · 8a8c600d
      Linus Torvalds authored
      Pull rdma fixes from Jason Gunthorpe:
       "The usual collection of driver bug fixes, and a few regressions from
        the merge window. Nothing particularly worrisome.
      
         - Various missed memory frees and error unwind bugs
      
         - Fix regressions in a few iwarp drivers from 5.4 patches
      
         - A few regressions added in past kernels
      
         - Squash a number of races in mlx5 ODP code"
      
      * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma:
        RDMA/mlx5: Add missing synchronize_srcu() for MW cases
        RDMA/mlx5: Put live in the correct place for ODP MRs
        RDMA/mlx5: Order num_pending_prefetch properly with synchronize_srcu
        RDMA/odp: Lift umem_mutex out of ib_umem_odp_unmap_dma_pages()
        RDMA/mlx5: Fix a race with mlx5_ib_update_xlt on an implicit MR
        RDMA/mlx5: Do not allow rereg of a ODP MR
        IB/core: Fix wrong iterating on ports
        RDMA/nldev: Reshuffle the code to avoid need to rebind QP in error path
        RDMA/cxgb4: Do not dma memory off of the stack
        RDMA/cm: Fix memory leak in cm_add/remove_one
        RDMA/core: Fix an error handling path in 'res_get_common_doit()'
        RDMA/i40iw: Associate ibdev to netdev before IB device registration
        RDMA/iwcm: Fix a lock inversion issue
        RDMA/iw_cxgb4: fix SRQ access from dump_qp()
        RDMA/hfi1: Prevent memory leak in sdma_init
        RDMA/core: Fix use after free and refcnt leak on ndev in_device in iwarp_query_port
        RDMA/siw: Fix serialization issue in write_space()
        RDMA/vmw_pvrdma: Free SRQ only once
      8a8c600d
    • Linus Torvalds's avatar
      Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux · e60329c9
      Linus Torvalds authored
      Pull arm64 fixes from Will Deacon:
       "A larger-than-usual batch of arm64 fixes for -rc3.
      
        The bulk of the fixes are dealing with a bunch of issues with the
        build system from the compat vDSO, which unfortunately led to some
        significant Makefile rework to manage the horrible combinations of
        toolchains that we can end up needing to drive simultaneously.
      
        We came close to disabling the thing entirely, but Vincenzo was quick
        to spin up some patches and I ended up picking up most of the bits
        that were left [*]. Future work will look at disentangling the header
        files properly.
      
        Other than that, we have some important fixes all over, including one
        papering over the miscompilation fallout from forcing
        CONFIG_OPTIMIZE_INLINING=y, which I'm still unhappy about. Harumph.
      
        We've still got a couple of open issues, so I'm expecting to have some
        more fixes later this cycle.
      
        Summary:
      
         - Numerous fixes to the compat vDSO build system, especially when
           combining gcc and clang
      
         - Fix parsing of PAR_EL1 in spurious kernel fault detection
      
         - Partial workaround for Neoverse-N1 erratum #1542419
      
         - Fix IRQ priority masking on entry from compat syscalls
      
         - Fix advertisment of FRINT HWCAP to userspace
      
         - Attempt to workaround inlining breakage with '__always_inline'
      
         - Fix accidental freeing of parent SVE state on fork() error path
      
         - Add some missing NULL pointer checks in instruction emulation init
      
         - Some formatting and comment fixes"
      
      [*] Will's final fixes were
      Reviewed-by: default avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      Tested-by: default avatarVincenzo Frascino <vincenzo.frascino@arm.com>
      
          but they were already in linux-next by then and he didn't rebase
          just to add those.
      
      * tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (21 commits)
        arm64: armv8_deprecated: Checking return value for memory allocation
        arm64: Kconfig: Make CONFIG_COMPAT_VDSO a proper Kconfig option
        arm64: vdso32: Rename COMPATCC to CC_COMPAT
        arm64: vdso32: Pass '--target' option to clang via VDSO_CAFLAGS
        arm64: vdso32: Don't use KBUILD_CPPFLAGS unconditionally
        arm64: vdso32: Move definition of COMPATCC into vdso32/Makefile
        arm64: Default to building compat vDSO with clang when CONFIG_CC_IS_CLANG
        lib: vdso: Remove CROSS_COMPILE_COMPAT_VDSO
        arm64: vdso32: Remove jump label config option in Makefile
        arm64: vdso32: Detect binutils support for dmb ishld
        arm64: vdso: Remove stale files from old assembly implementation
        arm64: vdso32: Fix broken compat vDSO build warnings
        arm64: mm: fix spurious fault detection
        arm64: ftrace: Ensure synchronisation in PLT setup for Neoverse-N1 #1542419
        arm64: Fix incorrect irqflag restore for priority masking for compat
        arm64: mm: avoid virt_to_phys(init_mm.pgd)
        arm64: cpufeature: Effectively expose FRINT capability to userspace
        arm64: Mark functions using explicit register variables as '__always_inline'
        docs: arm64: Fix indentation and doc formatting
        arm64/sve: Fix wrong free for task->thread.sve_state
        ...
      e60329c9
    • Brian Foster's avatar
      xfs: move local to extent inode logging into bmap helper · aeea4b75
      Brian Foster authored
      The callers of xfs_bmap_local_to_extents_empty() log the inode
      external to the function, yet this function is where the on-disk
      format value is updated. Push the inode logging down into the
      function itself to help prevent future mistakes.
      
      Note that internal bmap callers track the inode logging flags
      independently and thus may log the inode core twice due to this
      change. This is harmless, so leave this code around for consistency
      with the other attr fork conversion functions.
      Signed-off-by: default avatarBrian Foster <bfoster@redhat.com>
      Reviewed-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
      Signed-off-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
      aeea4b75
    • Brian Foster's avatar
      xfs: remove broken error handling on failed attr sf to leaf change · 603efebd
      Brian Foster authored
      xfs_attr_shortform_to_leaf() attempts to put the shortform fork back
      together after a failed attempt to convert from shortform to leaf
      format. While this code reallocates and copies back the shortform
      attr fork data, it never resets the inode format field back to local
      format. Further, now that the inode is properly logged after the
      initial switch from local format, any error that triggers the
      recovery code will eventually abort the transaction and shutdown the
      fs. Therefore, remove the broken and unnecessary error handling
      code.
      Signed-off-by: default avatarBrian Foster <bfoster@redhat.com>
      Reviewed-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
      Signed-off-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
      603efebd
    • Brian Foster's avatar
      xfs: log the inode on directory sf to block format change · 0b10d8a8
      Brian Foster authored
      When a directory changes from shortform (sf) to block format, the sf
      format is copied to a temporary buffer, the inode format is modified
      and the updated format filled with the dentries from the temporary
      buffer. If the inode format is modified and attempt to grow the
      inode fails (due to I/O error, for example), it is possible to
      return an error while leaving the directory in an inconsistent state
      and with an otherwise clean transaction. This results in corruption
      of the associated directory and leads to xfs_dabuf_map() errors as
      subsequent lookups cannot accurately determine the format of the
      directory. This problem is reproduced occasionally by generic/475.
      
      The fundamental problem is that xfs_dir2_sf_to_block() changes the
      on-disk inode format without logging the inode. The inode is
      eventually logged by the bmapi layer in the common case, but error
      checking introduces the possibility of failing the high level
      request before this happens.
      
      Update both of the dir2 and attr callers of
      xfs_bmap_local_to_extents_empty() to log the inode core as
      consistent with the bmap local to extent format change codepath.
      This ensures that any subsequent errors after the format has changed
      cause the transaction to abort.
      Signed-off-by: default avatarBrian Foster <bfoster@redhat.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
      Signed-off-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
      0b10d8a8
  4. 08 Oct, 2019 9 commits
  5. 07 Oct, 2019 13 commits
    • Linus Torvalds's avatar
      Merge branch 'akpm' (patches from Andrew) · eda57a0e
      Linus Torvalds authored
      Merge misc fixes from Andrew Morton:
       "The usual shower of hotfixes.
      
        Chris's memcg patches aren't actually fixes - they're mature but a few
        niggling review issues were late to arrive.
      
        The ocfs2 fixes are quite old - those took some time to get reviewer
        attention.
      
        Subsystems affected by this patch series: ocfs2, hotfixes, mm/memcg,
        mm/slab-generic"
      
      * emailed patches from Andrew Morton <akpm@linux-foundation.org>:
        mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two)
        mm, sl[ou]b: improve memory accounting
        mm, memcg: make scan aggression always exclude protection
        mm, memcg: make memory.emin the baseline for utilisation determination
        mm, memcg: proportional memory.{low,min} reclaim
        mm/vmpressure.c: fix a signedness bug in vmpressure_register_event()
        mm/page_alloc.c: fix a crash in free_pages_prepare()
        mm/z3fold.c: claim page in the beginning of free
        kernel/sysctl.c: do not override max_threads provided by userspace
        memcg: only record foreign writebacks with dirty pages when memcg is not disabled
        mm: fix -Wmissing-prototypes warnings
        writeback: fix use-after-free in finish_writeback_work()
        mm/memremap: drop unused SECTION_SIZE and SECTION_MASK
        panic: ensure preemption is disabled during panic()
        fs: ocfs2: fix a possible null-pointer dereference in ocfs2_info_scan_inode_alloc()
        fs: ocfs2: fix a possible null-pointer dereference in ocfs2_write_end_nolock()
        fs: ocfs2: fix possible null-pointer dereferences in ocfs2_xa_prepare_entry()
        ocfs2: clear zero in unaligned direct IO
      eda57a0e
    • Vlastimil Babka's avatar
      mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two) · 59bb4798
      Vlastimil Babka authored
      In most configurations, kmalloc() happens to return naturally aligned
      (i.e.  aligned to the block size itself) blocks for power of two sizes.
      
      That means some kmalloc() users might unknowingly rely on that
      alignment, until stuff breaks when the kernel is built with e.g.
      CONFIG_SLUB_DEBUG or CONFIG_SLOB, and blocks stop being aligned.  Then
      developers have to devise workaround such as own kmem caches with
      specified alignment [1], which is not always practical, as recently
      evidenced in [2].
      
      The topic has been discussed at LSF/MM 2019 [3].  Adding a
      'kmalloc_aligned()' variant would not help with code unknowingly relying
      on the implicit alignment.  For slab implementations it would either
      require creating more kmalloc caches, or allocate a larger size and only
      give back part of it.  That would be wasteful, especially with a generic
      alignment parameter (in contrast with a fixed alignment to size).
      
      Ideally we should provide to mm users what they need without difficult
      workarounds or own reimplementations, so let's make the kmalloc()
      alignment to size explicitly guaranteed for power-of-two sizes under all
      configurations.  What this means for the three available allocators?
      
      * SLAB object layout happens to be mostly unchanged by the patch.  The
        implicitly provided alignment could be compromised with
        CONFIG_DEBUG_SLAB due to redzoning, however SLAB disables redzoning for
        caches with alignment larger than unsigned long long.  Practically on at
        least x86 this includes kmalloc caches as they use cache line alignment,
        which is larger than that.  Still, this patch ensures alignment on all
        arches and cache sizes.
      
      * SLUB layout is also unchanged unless redzoning is enabled through
        CONFIG_SLUB_DEBUG and boot parameter for the particular kmalloc cache.
        With this patch, explicit alignment is guaranteed with redzoning as
        well.  This will result in more memory being wasted, but that should be
        acceptable in a debugging scenario.
      
      * SLOB has no implicit alignment so this patch adds it explicitly for
        kmalloc().  The potential downside is increased fragmentation.  While
        pathological allocation scenarios are certainly possible, in my testing,
        after booting a x86_64 kernel+userspace with virtme, around 16MB memory
        was consumed by slab pages both before and after the patch, with
        difference in the noise.
      
      [1] https://lore.kernel.org/linux-btrfs/c3157c8e8e0e7588312b40c853f65c02fe6c957a.1566399731.git.christophe.leroy@c-s.fr/
      [2] https://lore.kernel.org/linux-fsdevel/20190225040904.5557-1-ming.lei@redhat.com/
      [3] https://lwn.net/Articles/787740/
      
      [akpm@linux-foundation.org: documentation fixlet, per Matthew]
      Link: http://lkml.kernel.org/r/20190826111627.7505-3-vbabka@suse.czSigned-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Reviewed-by: default avatarMatthew Wilcox (Oracle) <willy@infradead.org>
      Acked-by: default avatarMichal Hocko <mhocko@suse.com>
      Acked-by: default avatarKirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Acked-by: default avatarChristoph Hellwig <hch@lst.de>
      Cc: David Sterba <dsterba@suse.cz>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Pekka Enberg <penberg@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ming Lei <ming.lei@redhat.com>
      Cc: Dave Chinner <david@fromorbit.com>
      Cc: "Darrick J . Wong" <darrick.wong@oracle.com>
      Cc: Christoph Hellwig <hch@lst.de>
      Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      59bb4798
    • Vlastimil Babka's avatar
      mm, sl[ou]b: improve memory accounting · 6a486c0a
      Vlastimil Babka authored
      Patch series "guarantee natural alignment for kmalloc()", v2.
      
      This patch (of 2):
      
      SLOB currently doesn't account its pages at all, so in /proc/meminfo the
      Slab field shows zero.  Modifying a counter on page allocation and
      freeing should be acceptable even for the small system scenarios SLOB is
      intended for.  Since reclaimable caches are not separated in SLOB,
      account everything as unreclaimable.
      
      SLUB currently doesn't account kmalloc() and kmalloc_node() allocations
      larger than order-1 page, that are passed directly to the page
      allocator.  As they also don't appear in /proc/slabinfo, it might look
      like a memory leak.  For consistency, account them as well.  (SLAB
      doesn't actually use page allocator directly, so no change there).
      
      Ideally SLOB and SLUB would be handled in separate patches, but due to
      the shared kmalloc_order() function and different kfree()
      implementations, it's easier to patch both at once to prevent
      inconsistencies.
      
      Link: http://lkml.kernel.org/r/20190826111627.7505-2-vbabka@suse.czSigned-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Pekka Enberg <penberg@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ming Lei <ming.lei@redhat.com>
      Cc: Dave Chinner <david@fromorbit.com>
      Cc: Matthew Wilcox <willy@infradead.org>
      Cc: "Darrick J . Wong" <darrick.wong@oracle.com>
      Cc: Christoph Hellwig <hch@lst.de>
      Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      6a486c0a
    • Chris Down's avatar
      mm, memcg: make scan aggression always exclude protection · 1bc63fb1
      Chris Down authored
      This patch is an incremental improvement on the existing
      memory.{low,min} relative reclaim work to base its scan pressure
      calculations on how much protection is available compared to the current
      usage, rather than how much the current usage is over some protection
      threshold.
      
      This change doesn't change the experience for the user in the normal
      case too much.  One benefit is that it replaces the (somewhat arbitrary)
      100% cutoff with an indefinite slope, which makes it easier to ballpark
      a memory.low value.
      
      As well as this, the old methodology doesn't quite apply generically to
      machines with varying amounts of physical memory.  Let's say we have a
      top level cgroup, workload.slice, and another top level cgroup,
      system-management.slice.  We want to roughly give 12G to
      system-management.slice, so on a 32GB machine we set memory.low to 20GB
      in workload.slice, and on a 64GB machine we set memory.low to 52GB.
      However, because these are relative amounts to the total machine size,
      while the amount of memory we want to generally be willing to yield to
      system.slice is absolute (12G), we end up putting more pressure on
      system.slice just because we have a larger machine and a larger workload
      to fill it, which seems fairly unintuitive.  With this new behaviour, we
      don't end up with this unintended side effect.
      
      Previously the way that memory.low protection works is that if you are
      50% over a certain baseline, you get 50% of your normal scan pressure.
      This is certainly better than the previous cliff-edge behaviour, but it
      can be improved even further by always considering memory under the
      currently enforced protection threshold to be out of bounds.  This means
      that we can set relatively low memory.low thresholds for variable or
      bursty workloads while still getting a reasonable level of protection,
      whereas with the previous version we may still trivially hit the 100%
      clamp.  The previous 100% clamp is also somewhat arbitrary, whereas this
      one is more concretely based on the currently enforced protection
      threshold, which is likely easier to reason about.
      
      There is also a subtle issue with the way that proportional reclaim
      worked previously -- it promotes having no memory.low, since it makes
      pressure higher during low reclaim.  This happens because we base our
      scan pressure modulation on how far memory.current is between memory.min
      and memory.low, but if memory.low is unset, we only use the overage
      method.  In most cromulent configurations, this then means that we end
      up with *more* pressure than with no memory.low at all when we're in low
      reclaim, which is not really very usable or expected.
      
      With this patch, memory.low and memory.min affect reclaim pressure in a
      more understandable and composable way.  For example, from a user
      standpoint, "protected" memory now remains untouchable from a reclaim
      aggression standpoint, and users can also have more confidence that
      bursty workloads will still receive some amount of guaranteed
      protection.
      
      Link: http://lkml.kernel.org/r/20190322160307.GA3316@chrisdown.nameSigned-off-by: default avatarChris Down <chris@chrisdown.name>
      Reviewed-by: default avatarRoman Gushchin <guro@fb.com>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Acked-by: default avatarMichal Hocko <mhocko@kernel.org>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: Dennis Zhou <dennis@kernel.org>
      Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      1bc63fb1
    • Chris Down's avatar
      mm, memcg: make memory.emin the baseline for utilisation determination · 9de7ca46
      Chris Down authored
      Roman points out that when when we do the low reclaim pass, we scale the
      reclaim pressure relative to position between 0 and the maximum
      protection threshold.
      
      However, if the maximum protection is based on memory.elow, and
      memory.emin is above zero, this means we still may get binary behaviour
      on second-pass low reclaim.  This is because we scale starting at 0, not
      starting at memory.emin, and since we don't scan at all below emin, we
      end up with cliff behaviour.
      
      This should be a fairly uncommon case since usually we don't go into the
      second pass, but it makes sense to scale our low reclaim pressure
      starting at emin.
      
      You can test this by catting two large sparse files, one in a cgroup
      with emin set to some moderate size compared to physical RAM, and
      another cgroup without any emin.  In both cgroups, set an elow larger
      than 50% of physical RAM.  The one with emin will have less page
      scanning, as reclaim pressure is lower.
      
      Rebase on top of and apply the same idea as what was applied to handle
      cgroup_memory=disable properly for the original proportional patch
      http://lkml.kernel.org/r/20190201045711.GA18302@chrisdown.name ("mm,
      memcg: Handle cgroup_disable=memory when getting memcg protection").
      
      Link: http://lkml.kernel.org/r/20190201051810.GA18895@chrisdown.nameSigned-off-by: default avatarChris Down <chris@chrisdown.name>
      Suggested-by: default avatarRoman Gushchin <guro@fb.com>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Cc: Michal Hocko <mhocko@kernel.org>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: Dennis Zhou <dennis@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      9de7ca46
    • Chris Down's avatar
      mm, memcg: proportional memory.{low,min} reclaim · 9783aa99
      Chris Down authored
      cgroup v2 introduces two memory protection thresholds: memory.low
      (best-effort) and memory.min (hard protection).  While they generally do
      what they say on the tin, there is a limitation in their implementation
      that makes them difficult to use effectively: that cliff behaviour often
      manifests when they become eligible for reclaim.  This patch implements
      more intuitive and usable behaviour, where we gradually mount more
      reclaim pressure as cgroups further and further exceed their protection
      thresholds.
      
      This cliff edge behaviour happens because we only choose whether or not
      to reclaim based on whether the memcg is within its protection limits
      (see the use of mem_cgroup_protected in shrink_node), but we don't vary
      our reclaim behaviour based on this information.  Imagine the following
      timeline, with the numbers the lruvec size in this zone:
      
      1. memory.low=1000000, memory.current=999999. 0 pages may be scanned.
      2. memory.low=1000000, memory.current=1000000. 0 pages may be scanned.
      3. memory.low=1000000, memory.current=1000001. 1000001* pages may be
         scanned. (?!)
      
      * Of course, we won't usually scan all available pages in the zone even
        without this patch because of scan control priority, over-reclaim
        protection, etc.  However, as shown by the tests at the end, these
        techniques don't sufficiently throttle such an extreme change in input,
        so cliff-like behaviour isn't really averted by their existence alone.
      
      Here's an example of how this plays out in practice.  At Facebook, we are
      trying to protect various workloads from "system" software, like
      configuration management tools, metric collectors, etc (see this[0] case
      study).  In order to find a suitable memory.low value, we start by
      determining the expected memory range within which the workload will be
      comfortable operating.  This isn't an exact science -- memory usage deemed
      "comfortable" will vary over time due to user behaviour, differences in
      composition of work, etc, etc.  As such we need to ballpark memory.low,
      but doing this is currently problematic:
      
      1. If we end up setting it too low for the workload, it won't have
         *any* effect (see discussion above).  The group will receive the full
         weight of reclaim and won't have any priority while competing with the
         less important system software, as if we had no memory.low configured
         at all.
      
      2. Because of this behaviour, we end up erring on the side of setting
         it too high, such that the comfort range is reliably covered.  However,
         protected memory is completely unavailable to the rest of the system,
         so we might cause undue memory and IO pressure there when we *know* we
         have some elasticity in the workload.
      
      3. Even if we get the value totally right, smack in the middle of the
         comfort zone, we get extreme jumps between no pressure and full
         pressure that cause unpredictable pressure spikes in the workload due
         to the current binary reclaim behaviour.
      
      With this patch, we can set it to our ballpark estimation without too much
      worry.  Any undesirable behaviour, such as too much or too little reclaim
      pressure on the workload or system will be proportional to how far our
      estimation is off.  This means we can set memory.low much more
      conservatively and thus waste less resources *without* the risk of the
      workload falling off a cliff if we overshoot.
      
      As a more abstract technical description, this unintuitive behaviour
      results in having to give high-priority workloads a large protection
      buffer on top of their expected usage to function reliably, as otherwise
      we have abrupt periods of dramatically increased memory pressure which
      hamper performance.  Having to set these thresholds so high wastes
      resources and generally works against the principle of work conservation.
      In addition, having proportional memory reclaim behaviour has other
      benefits.  Most notably, before this patch it's basically mandatory to set
      memory.low to a higher than desirable value because otherwise as soon as
      you exceed memory.low, all protection is lost, and all pages are eligible
      to scan again.  By contrast, having a gradual ramp in reclaim pressure
      means that you now still get some protection when thresholds are exceeded,
      which means that one can now be more comfortable setting memory.low to
      lower values without worrying that all protection will be lost.  This is
      important because workingset size is really hard to know exactly,
      especially with variable workloads, so at least getting *some* protection
      if your workingset size grows larger than you expect increases user
      confidence in setting memory.low without a huge buffer on top being
      needed.
      
      Thanks a lot to Johannes Weiner and Tejun Heo for their advice and
      assistance in thinking about how to make this work better.
      
      In testing these changes, I intended to verify that:
      
      1. Changes in page scanning become gradual and proportional instead of
         binary.
      
         To test this, I experimented stepping further and further down
         memory.low protection on a workload that floats around 19G workingset
         when under memory.low protection, watching page scan rates for the
         workload cgroup:
      
         +------------+-----------------+--------------------+--------------+
         | memory.low | test (pgscan/s) | control (pgscan/s) | % of control |
         +------------+-----------------+--------------------+--------------+
         |        21G |               0 |                  0 | N/A          |
         |        17G |             867 |               3799 | 23%          |
         |        12G |            1203 |               3543 | 34%          |
         |         8G |            2534 |               3979 | 64%          |
         |         4G |            3980 |               4147 | 96%          |
         |          0 |            3799 |               3980 | 95%          |
         +------------+-----------------+--------------------+--------------+
      
         As you can see, the test kernel (with a kernel containing this
         patch) ramps up page scanning significantly more gradually than the
         control kernel (without this patch).
      
      2. More gradual ramp up in reclaim aggression doesn't result in
         premature OOMs.
      
         To test this, I wrote a script that slowly increments the number of
         pages held by stress(1)'s --vm-keep mode until a production system
         entered severe overall memory contention.  This script runs in a highly
         protected slice taking up the majority of available system memory.
         Watching vmstat revealed that page scanning continued essentially
         nominally between test and control, without causing forward reclaim
         progress to become arrested.
      
      [0]: https://facebookmicrosites.github.io/cgroup2/docs/overview.html#case-study-the-fbtax2-project
      
      [akpm@linux-foundation.org: reflow block comments to fit in 80 cols]
      [chris@chrisdown.name: handle cgroup_disable=memory when getting memcg protection]
        Link: http://lkml.kernel.org/r/20190201045711.GA18302@chrisdown.name
      Link: http://lkml.kernel.org/r/20190124014455.GA6396@chrisdown.nameSigned-off-by: default avatarChris Down <chris@chrisdown.name>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Reviewed-by: default avatarRoman Gushchin <guro@fb.com>
      Cc: Michal Hocko <mhocko@kernel.org>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: Dennis Zhou <dennis@kernel.org>
      Cc: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      9783aa99
    • Dan Carpenter's avatar
      mm/vmpressure.c: fix a signedness bug in vmpressure_register_event() · 518a8671
      Dan Carpenter authored
      The "mode" and "level" variables are enums and in this context GCC will
      treat them as unsigned ints so the error handling is never triggered.
      
      I also removed the bogus initializer because it isn't required any more
      and it's sort of confusing.
      
      [akpm@linux-foundation.org: reduce implicit and explicit typecasting]
      [akpm@linux-foundation.org: fix return value, add comment, per Matthew]
      Link: http://lkml.kernel.org/r/20190925110449.GO3264@mwanda
      Fixes: 3cadfa2b ("mm/vmpressure.c: convert to use match_string() helper")
      Signed-off-by: default avatarDan Carpenter <dan.carpenter@oracle.com>
      Reviewed-by: default avatarAndy Shevchenko <andriy.shevchenko@linux.intel.com>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Reviewed-by: default avatarMatthew Wilcox <willy@infradead.org>
      Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Enrico Weigelt <info@metux.net>
      Cc: Kate Stewart <kstewart@linuxfoundation.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      518a8671
    • Qian Cai's avatar
      mm/page_alloc.c: fix a crash in free_pages_prepare() · 234fdce8
      Qian Cai authored
      On architectures like s390, arch_free_page() could mark the page unused
      (set_page_unused()) and any access later would trigger a kernel panic.
      Fix it by moving arch_free_page() after all possible accessing calls.
      
       Hardware name: IBM 2964 N96 400 (z/VM 6.4.0)
       Krnl PSW : 0404e00180000000 0000000026c2b96e (__free_pages_ok+0x34e/0x5d8)
                  R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
       Krnl GPRS: 0000000088d43af7 0000000000484000 000000000000007c 000000000000000f
                  000003d080012100 000003d080013fc0 0000000000000000 0000000000100000
                  00000000275cca48 0000000000000100 0000000000000008 000003d080010000
                  00000000000001d0 000003d000000000 0000000026c2b78a 000000002717fdb0
       Krnl Code: 0000000026c2b95c: ec1100b30659 risbgn %r1,%r1,0,179,6
                  0000000026c2b962: e32014000036 pfd 2,1024(%r1)
                 #0000000026c2b968: d7ff10001000 xc 0(256,%r1),0(%r1)
                 >0000000026c2b96e: 41101100  la %r1,256(%r1)
                  0000000026c2b972: a737fff8  brctg %r3,26c2b962
                  0000000026c2b976: d7ff10001000 xc 0(256,%r1),0(%r1)
                  0000000026c2b97c: e31003400004 lg %r1,832
                  0000000026c2b982: ebff1430016a asi 5168(%r1),-1
       Call Trace:
       __free_pages_ok+0x16a/0x5d8)
       memblock_free_all+0x206/0x290
       mem_init+0x58/0x120
       start_kernel+0x2b0/0x570
       startup_continue+0x6a/0xc0
       INFO: lockdep is turned off.
       Last Breaking-Event-Address:
       __free_pages_ok+0x372/0x5d8
       Kernel panic - not syncing: Fatal exception: panic_on_oops
       00: HCPGIR450W CP entered; disabled wait PSW 00020001 80000000 00000000 26A2379C
      
      In the past, only kernel_poison_pages() would trigger this but it needs
      "page_poison=on" kernel cmdline, and I suspect nobody tested that on
      s390.  Recently, kernel_init_free_pages() (commit 6471384a ("mm:
      security: introduce init_on_alloc=1 and init_on_free=1 boot options"))
      was added and could trigger this as well.
      
      [akpm@linux-foundation.org: add comment]
      Link: http://lkml.kernel.org/r/1569613623-16820-1-git-send-email-cai@lca.pw
      Fixes: 8823b1db ("mm/page_poison.c: enable PAGE_POISONING as a separate option")
      Fixes: 6471384a ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options")
      Signed-off-by: default avatarQian Cai <cai@lca.pw>
      Reviewed-by: default avatarHeiko Carstens <heiko.carstens@de.ibm.com>
      Acked-by: default avatarChristian Borntraeger <borntraeger@de.ibm.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.com>
      Cc: "Kirill A. Shutemov" <kirill@shutemov.name>
      Cc: Vasily Gorbik <gor@linux.ibm.com>
      Cc: Alexander Duyck <alexander.duyck@gmail.com>
      Cc: <stable@vger.kernel.org>	[5.3+]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      234fdce8
    • Vitaly Wool's avatar
      mm/z3fold.c: claim page in the beginning of free · 5b6807de
      Vitaly Wool authored
      There's a really hard to reproduce race in z3fold between z3fold_free()
      and z3fold_reclaim_page().  z3fold_reclaim_page() can claim the page
      after z3fold_free() has checked if the page was claimed and
      z3fold_free() will then schedule this page for compaction which may in
      turn lead to random page faults (since that page would have been
      reclaimed by then).
      
      Fix that by claiming page in the beginning of z3fold_free() and not
      forgetting to clear the claim in the end.
      
      [vitalywool@gmail.com: v2]
        Link: http://lkml.kernel.org/r/20190928113456.152742cf@bigdell
      Link: http://lkml.kernel.org/r/20190926104844.4f0c6efa1366b8f5741eaba9@gmail.comSigned-off-by: default avatarVitaly Wool <vitalywool@gmail.com>
      Reported-by: default avatarMarkus Linnala <markus.linnala@gmail.com>
      Cc: Dan Streetman <ddstreet@ieee.org>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Henry Burns <henrywolfeburns@gmail.com>
      Cc: Shakeel Butt <shakeelb@google.com>
      Cc: Markus Linnala <markus.linnala@gmail.com>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      5b6807de
    • Michal Hocko's avatar
      kernel/sysctl.c: do not override max_threads provided by userspace · b0f53dbc
      Michal Hocko authored
      Partially revert 16db3d3f ("kernel/sysctl.c: threads-max observe
      limits") because the patch is causing a regression to any workload which
      needs to override the auto-tuning of the limit provided by kernel.
      
      set_max_threads is implementing a boot time guesstimate to provide a
      sensible limit of the concurrently running threads so that runaways will
      not deplete all the memory.  This is a good thing in general but there
      are workloads which might need to increase this limit for an application
      to run (reportedly WebSpher MQ is affected) and that is simply not
      possible after the mentioned change.  It is also very dubious to
      override an admin decision by an estimation that doesn't have any direct
      relation to correctness of the kernel operation.
      
      Fix this by dropping set_max_threads from sysctl_max_threads so any
      value is accepted as long as it fits into MAX_THREADS which is important
      to check because allowing more threads could break internal robust futex
      restriction.  While at it, do not use MIN_THREADS as the lower boundary
      because it is also only a heuristic for automatic estimation and admin
      might have a good reason to stop new threads to be created even when
      below this limit.
      
      This became more severe when we switched x86 from 4k to 8k kernel
      stacks.  Starting since 6538b8ea ("x86_64: expand kernel stack to
      16K") (3.16) we use THREAD_SIZE_ORDER = 2 and that halved the auto-tuned
      value.
      
      In the particular case
      
        3.12
        kernel.threads-max = 515561
      
        4.4
        kernel.threads-max = 200000
      
      Neither of the two values is really insane on 32GB machine.
      
      I am not sure we want/need to tune the max_thread value further.  If
      anything the tuning should be removed altogether if proven not useful in
      general.  But we definitely need a way to override this auto-tuning.
      
      Link: http://lkml.kernel.org/r/20190922065801.GB18814@dhcp22.suse.cz
      Fixes: 16db3d3f ("kernel/sysctl.c: threads-max observe limits")
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.com>
      Reviewed-by: default avatar"Eric W. Biederman" <ebiederm@xmission.com>
      Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
      Cc: <stable@vger.kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      b0f53dbc
    • Baoquan He's avatar
      memcg: only record foreign writebacks with dirty pages when memcg is not disabled · 08d1d0e6
      Baoquan He authored
      In kdump kernel, memcg usually is disabled with 'cgroup_disable=memory'
      for saving memory.  Now kdump kernel will always panic when dump vmcore
      to local disk:
      
        BUG: kernel NULL pointer dereference, address: 0000000000000ab8
        Oops: 0000 [#1] SMP NOPTI
        CPU: 0 PID: 598 Comm: makedumpfile Not tainted 5.3.0+ #26
        Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 10/02/2018
        RIP: 0010:mem_cgroup_track_foreign_dirty_slowpath+0x38/0x140
        Call Trace:
         __set_page_dirty+0x52/0xc0
         iomap_set_page_dirty+0x50/0x90
         iomap_write_end+0x6e/0x270
         iomap_write_actor+0xce/0x170
         iomap_apply+0xba/0x11e
         iomap_file_buffered_write+0x62/0x90
         xfs_file_buffered_aio_write+0xca/0x320 [xfs]
         new_sync_write+0x12d/0x1d0
         vfs_write+0xa5/0x1a0
         ksys_write+0x59/0xd0
         do_syscall_64+0x59/0x1e0
         entry_SYSCALL_64_after_hwframe+0x44/0xa9
      
      And this will corrupt the 1st kernel too with 'cgroup_disable=memory'.
      
      Via the trace and with debugging, it is pointing to commit 97b27821
      ("writeback, memcg: Implement foreign dirty flushing") which introduced
      this regression.  Disabling memcg causes the null pointer dereference at
      uninitialized data in function mem_cgroup_track_foreign_dirty_slowpath().
      
      Fix it by returning directly if memcg is disabled, but not trying to
      record the foreign writebacks with dirty pages.
      
      Link: http://lkml.kernel.org/r/20190924141928.GD31919@MiWiFi-R3L-srv
      Fixes: 97b27821 ("writeback, memcg: Implement foreign dirty flushing")
      Signed-off-by: default avatarBaoquan He <bhe@redhat.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: Jens Axboe <axboe@kernel.dk>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      08d1d0e6
    • Yi Wang's avatar
      mm: fix -Wmissing-prototypes warnings · 758b8db4
      Yi Wang authored
      We get two warnings when build kernel W=1:
      
        mm/shuffle.c:36:12: warning: no previous prototype for `shuffle_show' [-Wmissing-prototypes]
        mm/sparse.c:220:6: warning: no previous prototype for `subsection_mask_set' [-Wmissing-prototypes]
      
      Make the functions static to fix this.
      
      Link: http://lkml.kernel.org/r/1566978161-7293-1-git-send-email-wang.yi59@zte.com.cnSigned-off-by: default avatarYi Wang <wang.yi59@zte.com.cn>
      Reviewed-by: default avatarDavid Hildenbrand <david@redhat.com>
      Reviewed-by: default avatarKees Cook <keescook@chromium.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      758b8db4
    • Tejun Heo's avatar
      writeback: fix use-after-free in finish_writeback_work() · 8e00c4e9
      Tejun Heo authored
      finish_writeback_work() reads @done->waitq after decrementing
      @done->cnt.  However, once @done->cnt reaches zero, @done may be freed
      (from stack) at any moment and @done->waitq can contain something
      unrelated by the time finish_writeback_work() tries to read it.  This
      led to the following crash.
      
        "BUG: kernel NULL pointer dereference, address: 0000000000000002"
        #PF: supervisor write access in kernel mode
        #PF: error_code(0x0002) - not-present page
        PGD 0 P4D 0
        Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
        CPU: 40 PID: 555153 Comm: kworker/u98:50 Kdump: loaded Not tainted
        ...
        Workqueue: writeback wb_workfn (flush-btrfs-1)
        RIP: 0010:_raw_spin_lock_irqsave+0x10/0x30
        Code: 48 89 d8 5b c3 e8 50 db 6b ff eb f4 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 9c 5b fa 31 c0 ba 01 00 00 00 <f0> 0f b1 17 75 05 48 89 d8 5b c3 89 c6 e8 fe ca 6b ff eb f2 66 90
        RSP: 0018:ffffc90049b27d98 EFLAGS: 00010046
        RAX: 0000000000000000 RBX: 0000000000000246 RCX: 0000000000000000
        RDX: 0000000000000001 RSI: 0000000000000003 RDI: 0000000000000002
        RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000001
        R10: ffff889fff407600 R11: ffff88ba9395d740 R12: 000000000000e300
        R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000000000
        FS:  0000000000000000(0000) GS:ffff88bfdfa00000(0000) knlGS:0000000000000000
        CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
        CR2: 0000000000000002 CR3: 0000000002409005 CR4: 00000000001606e0
        Call Trace:
         __wake_up_common_lock+0x63/0xc0
         wb_workfn+0xd2/0x3e0
         process_one_work+0x1f5/0x3f0
         worker_thread+0x2d/0x3d0
         kthread+0x111/0x130
         ret_from_fork+0x1f/0x30
      
      Fix it by reading and caching @done->waitq before decrementing
      @done->cnt.
      
      Link: http://lkml.kernel.org/r/20190924010631.GH2233839@devbig004.ftw2.facebook.com
      Fixes: 5b9cce4c ("writeback: Generalize and expose wb_completion")
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Debugged-by: default avatarChris Mason <clm@fb.com>
      Reviewed-by: default avatarJens Axboe <axboe@kernel.dk>
      Cc: Jan Kara <jack@suse.cz>
      Cc: <stable@vger.kernel.org>	[5.2+]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      8e00c4e9