1. 23 Sep, 2015 8 commits
    • Ming Lei's avatar
      block: loop: support DIO & AIO · bc07c10a
      Ming Lei authored
      There are at least 3 advantages to use direct I/O and AIO on
      read/write loop's backing file:
      
      1) double cache can be avoided, then memory usage gets
      decreased a lot
      
      2) not like user space direct I/O, there isn't cost of
      pinning pages
      
      3) avoid context switch for obtaining good throughput
      - in buffered file read, random I/O top throughput is often obtained
      only if they are submitted concurrently from lots of tasks; but for
      sequential I/O, most of times they can be hit from page cache, so
      concurrent submissions often introduce unnecessary context switch
      and can't improve throughput much. There was such discussion[1]
      to use non-blocking I/O to improve the problem for application.
      - with direct I/O and AIO, concurrent submissions can be
      avoided and random read throughput can't be affected meantime
      
      xfstests(-g auto, ext4) is basically passed when running with
      direct I/O(aio), one exception is generic/232, but it failed in
      loop buffered I/O(4.2-rc6-next-20150814) too.
      
      Follows the fio test result for performance purpose:
      	4 jobs fio test inside ext4 file system over loop block
      
      1) How to run
      	- KVM: 4 VCPUs, 2G RAM
      	- linux kernel: 4.2-rc6-next-20150814(base) with the patchset
      	- the loop block is over one image on SSD.
      	- linux psync, 4 jobs, size 1500M, ext4 over loop block
      	- test result: IOPS from fio output
      
      2) Throughput(IOPS) becomes a bit better with direct I/O(aio)
              -------------------------------------------------------------
              test cases          |randread   |read   |randwrite  |write  |
              -------------------------------------------------------------
              base                |8015       |113811 |67442      |106978
              -------------------------------------------------------------
              base+loop aio       |8136       |125040 |67811      |111376
              -------------------------------------------------------------
      
      - somehow, it should be caused by more page cache avaiable for
      application or one extra page copy is avoided in case of direct I/O
      
      3) context switch
              - context switch decreased by ~50% with loop direct I/O(aio)
      	compared with loop buffered I/O(4.2-rc6-next-20150814)
      
      4) memory usage from /proc/meminfo
              -------------------------------------------------------------
                                         | Buffers       | Cached
              -------------------------------------------------------------
              base                       | > 760MB       | ~950MB
              -------------------------------------------------------------
              base+loop direct I/O(aio)  | < 5MB         | ~1.6GB
              -------------------------------------------------------------
      
      - so there are much more page caches available for application with
      direct I/O
      
      [1] https://lwn.net/Articles/612483/Signed-off-by: default avatarMing Lei <ming.lei@canonical.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      bc07c10a
    • Ming Lei's avatar
      block: loop: introduce ioctl command of LOOP_SET_DIRECT_IO · ab1cb278
      Ming Lei authored
      If loop block is mounted via 'mount -o loop', it isn't easy
      to pass file descriptor opened as O_DIRECT, so this patch
      introduces a new command to support direct IO for this case.
      
      Cc: linux-api@vger.kernel.org
      Signed-off-by: default avatarMing Lei <ming.lei@canonical.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      ab1cb278
    • Ming Lei's avatar
      block: loop: prepare for supporing direct IO · 2e5ab5f3
      Ming Lei authored
      This patches provides one interface for enabling direct IO
      from user space:
      
      	- userspace(such as losetup) can pass 'file' which is
      	opened/fcntl as O_DIRECT
      
      Also __loop_update_dio() is introduced to check if direct I/O
      can be used on current loop setting.
      
      The last big change is to introduce LO_FLAGS_DIRECT_IO flag
      for userspace to know if direct IO is used to access backing
      file.
      
      Cc: linux-api@vger.kernel.org
      Signed-off-by: default avatarMing Lei <ming.lei@canonical.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      2e5ab5f3
    • Ming Lei's avatar
      block: loop: use kthread_work · e03a3d7a
      Ming Lei authored
      The following patch will use dio/aio to submit IO to backing file,
      then it needn't to schedule IO concurrently from work, so
      use kthread_work for decreasing context switch cost a lot.
      
      For non-AIO case, single thread has been used for long long time,
      and it was just converted to work in v4.0, which has caused performance
      regression for fedora live booting already. In discussion[1], even
      though submitting I/O via work concurrently can improve random read IO
      throughput, meantime it might hurt sequential read IO performance, so
      better to restore to single thread behaviour.
      
      For the following AIO support, it is better to use multi hw-queue
      with per-hwq kthread than current work approach suppose there is so
      high performance requirement for loop.
      
      [1] http://marc.info/?t=143082678400002&r=1&w=2Signed-off-by: default avatarMing Lei <ming.lei@canonical.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      e03a3d7a
    • Ming Lei's avatar
      block: loop: set QUEUE_FLAG_NOMERGES for request queue of loop · 5b5e20f4
      Ming Lei authored
      It doesn't make sense to enable merge because the I/O
      submitted to backing file is handled page by page.
      Signed-off-by: default avatarMing Lei <ming.lei@canonical.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      5b5e20f4
    • Ming Lei's avatar
      fs: direct-io: don't dirtying pages for ITER_BVEC/ITER_KVEC direct read · 53cbf3b1
      Ming Lei authored
      When direct read IO is submitted from kernel, it is often
      unnecessary to dirty pages, for example of loop, dirtying pages
      have been considered in the upper filesystem(over loop) side
      already, and they don't need to be dirtied again.
      
      So this patch doesn't dirtying pages for ITER_BVEC/ITER_KVEC
      direct read, and loop should be the 1st case to use ITER_BVEC/ITER_KVEC
      for direct read I/O.
      
      The patch is based on previous Dave's patch.
      Reviewed-by: default avatarDave Kleikamp <dave.kleikamp@oracle.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarMing Lei <ming.lei@canonical.com>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      53cbf3b1
    • Roman Pen's avatar
      fs/mpage.c: forgotten WRITE_SYNC in case of data integrity write · 5948edbc
      Roman Pen authored
      In case of wbc->sync_mode == WB_SYNC_ALL we need to do data integrity
      write, thus mark request as WRITE_SYNC.
      
      akpm: afaict this change will cause the data integrity write bios to be
      placed onto the second queue in cfq_io_cq.cfqq[], which presumably results
      in special treatment.  The documentation for REQ_SYNC is horrid.
      Signed-off-by: default avatarRoman Pen <r.peniaev@gmail.com>
      Reviewed-by: default avatarJan Kara <jack@suse.cz>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Reviewed-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      5948edbc
    • Catalin Marinas's avatar
      block: kmemleak: Track the page allocations for struct request · f75782e4
      Catalin Marinas authored
      The pages allocated for struct request contain pointers to other slab
      allocations (via ops->init_request). Since kmemleak does not track/scan
      page allocations, the slab objects will be reported as leaks (false
      positives). This patch adds kmemleak callbacks to allow tracking of such
      pages.
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      Reported-by: default avatarBart Van Assche <bart.vanassche@sandisk.com>
      Tested-by: Bart Van Assche<bart.vanassche@sandisk.com>
      Cc: Christoph Hellwig <hch@infradead.org>
      Cc: Jens Axboe <axboe@kernel.dk>
      Signed-off-by: default avatarJens Axboe <axboe@fb.com>
      f75782e4
  2. 22 Sep, 2015 1 commit
    • Linus Torvalds's avatar
      Merge branch 'for-4.3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup · bcee19f4
      Linus Torvalds authored
      Pull cgroup fixes from Tejun Heo:
       "The threadgroup locking changes which went in during 4.2 devel cycle
        added write locking of a percpu_rwsem in cgroup task migration path;
        unfortunately, that involved expedited rcu syncing which turned out to
        be too slow and heavy for certain workloads.  The patchset which is
        dependent on this one didn't get committed during that devel cycle, so
        these two patches can be reverted safely.
      
        Oleg reworked percpu_rwsem for 4.4 so that the writer path is a lot
        lighter.  The reported issue goes away with Oleg's reworked
        percpu_rwsem and I'll reapply these patches on the for-4.4 branch so
        that they can land together with Oleg's changes"
      
      * 'for-4.3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
        Revert "sched, cgroup: replace signal_struct->group_rwsem with a global percpu_rwsem"
        Revert "cgroup: simplify threadgroup locking"
      bcee19f4
  3. 21 Sep, 2015 3 commits
  4. 20 Sep, 2015 10 commits
    • Linus Torvalds's avatar
      Linux 4.3-rc2 · 1f93e4a9
      Linus Torvalds authored
      1f93e4a9
    • Linus Torvalds's avatar
      Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm · 99bc7215
      Linus Torvalds authored
      Pull ARM fixes from Russell King:
       "Three fixes and a resulting cleanup for -rc2:
      
         - Andre Przywara reported that he was seeing a warning with the new
           cast inside DMA_ERROR_CODE's definition, and fixed the incorrect
           use.
      
         - Doug Anderson noticed that kgdb causes a "scheduling while atomic"
           bug.
      
         - OMAP5 folk noticed that their Thumb-2 compiled X servers crashed
           when enabling support to cover ARMv6 CPUs due to a kernel bug
           leaking some conditional context into the signal handler"
      
      * 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm:
        ARM: 8425/1: kgdb: Don't try to stop the machine when setting breakpoints
        ARM: 8437/1: dma-mapping: fix build warning with new DMA_ERROR_CODE definition
        ARM: get rid of needless #if in signal handling code
        ARM: fix Thumb2 signal handling when ARMv6 is enabled
      99bc7215
    • Linus Torvalds's avatar
      Merge tag 'linux-kselftest-4.3-rc2' of... · 30ec5682
      Linus Torvalds authored
      Merge tag 'linux-kselftest-4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
      
      Pull kselftest fixes from Shuah Khan:
       "This update contains 7 fixes for problems ranging from build failurs
        to incorrect error reporting"
      
      * tag 'linux-kselftest-4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
        selftests: exec: revert to default emit rule
        selftests: change install command to rsync
        selftests: mqueue: simplify the Makefile
        selftests: mqueue: allow extra cflags
        selftests: rename jump label to static_keys
        selftests/seccomp: add support for s390
        seltests/zram: fix syntax error
      30ec5682
    • Linus Torvalds's avatar
      Merge tag 'pm+acpi-4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm · 009884f3
      Linus Torvalds authored
      Pull power management and ACPI updates from Rafael Wysocki:
       "Included are: a somewhat late devfreq update which however is mostly
        fixes and cleanups with one new thing only (the PPMUv2 support on
        Exynos5433), an ACPI cpufreq driver fixup and two ACPI core cleanups
        related to preprocessor directives.
      
        Specifics:
      
         - Fix a memory allocation size in the devfreq core (Xiaolong Ye).
      
         - Fix a mistake in the exynos-ppmu DT binding (Javier Martinez
           Canillas).
      
         - Add support for PPMUv2 ((Platform Performance Monitoring Unit
           version 2.0) on the Exynos5433 SoCs (Chanwoo Choi).
      
         - Fix a type casting bug in the Exynos PPMU code (MyungJoo Ham).
      
         - Assorted devfreq code cleanups and optimizations (Javi Merino,
           MyungJoo Ham, Viresh Kumar).
      
         - Fix up the ACPI cpufreq driver to use a more lightweight way to get
           to its private data in the ->get() callback (Rafael J Wysocki).
      
         - Fix a CONFIG_ prefix bug in one of the ACPI drivers and make the
           ACPI subsystem use IS_ENABLED() instead of #ifdefs in function
           bodies (Sudeep Holla)"
      
      * tag 'pm+acpi-4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
        cpufreq: acpi-cpufreq: Use cpufreq_cpu_get_raw() in ->get()
        ACPI: Eliminate CONFIG_.*{, _MODULE} #ifdef in favor of IS_ENABLED()
        ACPI: int340x_thermal: add missing CONFIG_ prefix
        PM / devfreq: Fix incorrect type issue.
        PM / devfreq: tegra: Update governor to use devfreq_update_stats()
        PM / devfreq: comments for get_dev_status usage updated
        PM / devfreq: drop comment about thermal setting max_freq
        PM / devfreq: cache the last call to get_dev_status()
        PM / devfreq: Drop unlikely before IS_ERR(_OR_NULL)
        PM / devfreq: exynos-ppmu: bit-wise operation bugfix.
        PM / devfreq: exynos-ppmu: Update documentation to support PPMUv2
        PM / devfreq: exynos-ppmu: Add the support of PPMUv2 for Exynos5433
        PM / devfreq: event: Remove incorrect property in exynos-ppmu DT binding
      009884f3
    • Linus Torvalds's avatar
      Merge tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux · d590b2d4
      Linus Torvalds authored
      Pull clk fixes from Stephen Boyd:
       "A few driver fixes for tegra, rockchip, and st SoCs and a two-liner in
        the framework to avoid oops when get_parent ops return out of range
        values on tegra platforms"
      
      * tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
        drivers: clk: st: Rename st_pll3200c32_407_c0_x into st_pll3200c32_cx_x
        clk: check for invalid parent index of orphans in __clk_init()
        clk: tegra: dfll: Properly protect OPP list
        clk: rockchip: add critical clock for rk3368
      d590b2d4
    • Linus Torvalds's avatar
      Merge tag 'led-fixes-for-v4.3-rc2' of... · e6827baf
      Linus Torvalds authored
      Merge tag 'led-fixes-for-v4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/j.anaszewski/linux-leds
      
      Pull LED fixes from Jacek Anaszewski:
       - fix module autoload for six OF platform drivers (aat1290, bcm6328,
         bcm6358, ktd2692, max77693, ns2)
       - aat1290: add missing static modifier
       - ipaq-micro: add missing LEDS_CLASS dependency
       - lp55xx: correct Kconfig dependecy for f/w user helper
      
      * tag 'led-fixes-for-v4.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/j.anaszewski/linux-leds:
        leds:lp55xx: Correct Kconfig dependency for f/w user helper
        leds: leds-ipaq-micro: Add LEDS_CLASS dependency
        leds: aat1290: add 'static' modifier to init_mm_current_scale
        leds: leds-ns2: Fix module autoload for OF platform driver
        leds: max77693: Fix module autoload for OF platform driver
        leds: ktd2692: Fix module autoload for OF platform driver
        leds: bcm6358: Fix module autoload for OF platform driver
        leds: bcm6328: Fix module autoload for OF platform driver
        leds: aat1290: Fix module autoload for OF platform driver
      e6827baf
    • Linus Torvalds's avatar
      Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma · dc847d5b
      Linus Torvalds authored
      Pull rdma fixes from Doug Ledford:
       "The new hfi1 driver in staging/rdma has had a number of fixup patches
        since being added to the tree.  This is the first batch of those fixes"
      
      * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma:
        IB/hfi: Properly set permissions for user device files
        IB/hfi1: mask vs shift confusion
        IB/hfi1: clean up some defines
        IB/hfi1: info leak in get_ctxt_info()
        IB/hfi1: fix a locking bug
        IB/hfi1: checking for NULL instead of IS_ERR
        IB/hfi1: fix sdma_descq_cnt parameter parsing
        IB/hfi1: fix copy_to/from_user() error handling
        IB/hfi1: fix pstateinfo from returning improperly byteswapped value
      dc847d5b
    • Linus Torvalds's avatar
      Merge branch 'libnvdimm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm · 2673ee56
      Linus Torvalds authored
      Pull libnvdimm fixes from Dan Williams:
      
       - a boot regression (since v4.2) fix for some ARM configurations from
         Tyler
      
       - regression (since v4.1) fixes for mkfs.xfs on a DAX enabled device
         from Jeff.  These are tagged for -stable.
      
       - a pair of locking fixes from Axel that are hidden from lockdep since
         they involve device_lock().  The "btt" one is tagged for -stable, the
         other only applies to the new "pfn" mechanism in v4.3.
      
       - a fix for the pmem ->rw_page() path to use wmb_pmem() from Ross.
      
      * 'libnvdimm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm:
        mm: fix type cast in __pfn_to_phys()
        pmem: add proper fencing to pmem_rw_page()
        libnvdimm: pfn_devs: Fix locking in namespace_store
        libnvdimm: btt_devs: Fix locking in namespace_store
        blockdev: don't set S_DAX for misaligned partitions
        dax: fix O_DIRECT I/O to the last block of a blockdev
      2673ee56
    • Linus Torvalds's avatar
      Merge branch 'for-linus' of git://git.kernel.dk/linux-block · 133bb595
      Linus Torvalds authored
      Pull block updates from Jens Axboe:
       "This is a bit bigger than it should be, but I could (did) not want to
        send it off last week due to both wanting extra testing, and expecting
        a fix for the bounce regression as well.  In any case, this contains:
      
         - Fix for the blk-merge.c compilation warning on gcc 5.x from me.
      
         - A set of back/front SG gap merge fixes, from me and from Sagi.
           This ensures that we honor SG gapping for integrity payloads as
           well.
      
         - Two small fixes for null_blk from Matias, fixing a leak and a
           capacity propagation issue.
      
         - A blkcg fix from Tejun, fixing a NULL dereference.
      
         - A fast clone optimization from Ming, fixing a performance
           regression since the arbitrarily sized bio's were introduced.
      
         - Also from Ming, a regression fix for bouncing IOs"
      
      * 'for-linus' of git://git.kernel.dk/linux-block:
        block: fix bounce_end_io
        block: blk-merge: fast-clone bio when splitting rw bios
        block: blkg_destroy_all() should clear q->root_blkg and ->root_rl.blkg
        block: Copy a user iovec if it includes gaps
        block: Refuse adding appending a gapped integrity page to a bio
        block: Refuse request/bio merges with gaps in the integrity payload
        block: Check for gaps on front and back merges
        null_blk: fix wrong capacity when bs is not 512 bytes
        null_blk: fix memory leak on cleanup
        block: fix bogus compiler warnings in blk-merge.c
      133bb595
    • Chris Mason's avatar
      fs-writeback: unplug before cond_resched in writeback_sb_inodes · 590dca3a
      Chris Mason authored
      Commit 505a666e ("writeback: plug writeback in wb_writeback() and
      writeback_inodes_wb()") has us holding a plug during writeback_sb_inodes,
      which increases the merge rate when relatively contiguous small files
      are written by the filesystem.  It helps both on flash and spindles.
      
      For an fs_mark workload creating 4K files in parallel across 8 drives,
      this commit improves performance ~9% more by unplugging before calling
      cond_resched().  cond_resched() doesn't trigger an implicit unplug, so
      explicitly getting the IO down to the device before scheduling reduces
      latencies for anyone waiting on clean pages.
      
      It also cuts down on how often we use kblockd to unplug, which means
      less work bouncing from one workqueue to another.
      
      Many more details about how we got here:
      
        https://lkml.org/lkml/2015/9/11/570Signed-off-by: default avatarChris Mason <clm@fb.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      590dca3a
  5. 19 Sep, 2015 1 commit
  6. 18 Sep, 2015 17 commits