1. 23 Jan, 2014 10 commits
    • Linus Torvalds's avatar
      Merge tag 'modules-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux · 0dc3fd02
      Linus Torvalds authored
      Pull module updates from Rusty Russell.
      
      * tag 'modules-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux:
        module: Add missing newline in printk call.
        module: fix coding style
        export: declare ksymtab symbols
        module.h: Remove unnecessary semicolon
        params: improve standard definitions
        Add Documentation/module-signing.txt file
      0dc3fd02
    • Linus Torvalds's avatar
      Merge tag 'virtio-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux · 93b05cba
      Linus Torvalds authored
      Pull virtio update from Rusty Russell:
       "A few simple fixes.  Quiet cycle"
      
      * tag 'virtio-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux:
        drivers: virtio: Mark function virtballoon_migratepage() as static in virtio_balloon.c
        virtio-scsi: Fix hotcpu_notifier use-after-free with virtscsi_freeze
        virtio: pci: remove unnecessary pci_set_drvdata()
      93b05cba
    • Linus Torvalds's avatar
      Merge tag 'stable/for-linus-3.14-rc0-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip · 84621c9b
      Linus Torvalds authored
      Pull Xen updates from Konrad Rzeszutek Wilk:
       "Two major features that Xen community is excited about:
      
        The first is event channel scalability by David Vrabel - we switch
        over from an two-level per-cpu bitmap of events (IRQs) - to an FIFO
        queue with priorities.  This lets us be able to handle more events,
        have lower latency, and better scalability.  Good stuff.
      
        The other is PVH by Mukesh Rathor.  In short, PV is a mode where the
        kernel lets the hypervisor program page-tables, segments, etc.  With
        EPT/NPT capabilities in current processors, the overhead of doing this
        in an HVM (Hardware Virtual Machine) container is much lower than the
        hypervisor doing it for us.
      
        In short we let a PV guest run without doing page-table, segment,
        syscall, etc updates through the hypervisor - instead it is all done
        within the guest container.  It is a "hybrid" PV - hence the 'PVH'
        name - a PV guest within an HVM container.
      
        The major benefits are less code to deal with - for example we only
        use one function from the the pv_mmu_ops (which has 39 function
        calls); faster performance for syscall (no context switches into the
        hypervisor); less traps on various operations; etc.
      
        It is still being baked - the ABI is not yet set in stone.  But it is
        pretty awesome and we are excited about it.
      
        Lastly, there are some changes to ARM code - you should get a simple
        conflict which has been resolved in #linux-next.
      
        In short, this pull has awesome features.
      
        Features:
         - FIFO event channels.  Key advantages: support for over 100,000
           events (2^17), 16 different event priorities, improved fairness in
           event latency through the use of FIFOs.
         - Xen PVH support.  "It’s a fully PV kernel mode, running with
           paravirtualized disk and network, paravirtualized interrupts and
           timers, no emulated devices of any kind (and thus no qemu), no BIOS
           or legacy boot — but instead of requiring PV MMU, it uses the HVM
           hardware extensions to virtualize the pagetables, as well as system
           calls and other privileged operations." (from "The
           Paravirtualization Spectrum, Part 2: From poles to a spectrum")
      
        Bug-fixes:
         - Fixes in balloon driver (refactor and make it work under ARM)
         - Allow xenfb to be used in HVM guests.
         - Allow xen_platform_pci=0 to work properly.
         - Refactors in event channels"
      
      * tag 'stable/for-linus-3.14-rc0-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip: (52 commits)
        xen/pvh: Set X86_CR0_WP and others in CR0 (v2)
        MAINTAINERS: add git repository for Xen
        xen/pvh: Use 'depend' instead of 'select'.
        xen: delete new instances of __cpuinit usage
        xen/fb: allow xenfb initialization for hvm guests
        xen/evtchn_fifo: fix error return code in evtchn_fifo_setup()
        xen-platform: fix error return code in platform_pci_init()
        xen/pvh: remove duplicated include from enlighten.c
        xen/pvh: Fix compile issues with xen_pvh_domain()
        xen: Use dev_is_pci() to check whether it is pci device
        xen/grant-table: Force to use v1 of grants.
        xen/pvh: Support ParaVirtualized Hardware extensions (v3).
        xen/pvh: Piggyback on PVHVM XenBus.
        xen/pvh: Piggyback on PVHVM for grant driver (v4)
        xen/grant: Implement an grant frame array struct (v3).
        xen/grant-table: Refactor gnttab_init
        xen/grants: Remove gnttab_max_grant_frames dependency on gnttab_init.
        xen/pvh: Piggyback on PVHVM for event channels (v2)
        xen/pvh: Update E820 to work with PVH (v2)
        xen/pvh: Secondary VCPU bringup (non-bootup CPUs)
        ...
      84621c9b
    • Linus Torvalds's avatar
      Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm · 7ebd3faa
      Linus Torvalds authored
      Pull KVM updates from Paolo Bonzini:
       "First round of KVM updates for 3.14; PPC parts will come next week.
      
        Nothing major here, just bugfixes all over the place.  The most
        interesting part is the ARM guys' virtualized interrupt controller
        overhaul, which lets userspace get/set the state and thus enables
        migration of ARM VMs"
      
      * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (67 commits)
        kvm: make KVM_MMU_AUDIT help text more readable
        KVM: s390: Fix memory access error detection
        KVM: nVMX: Update guest activity state field on L2 exits
        KVM: nVMX: Fix nested_run_pending on activity state HLT
        KVM: nVMX: Clean up handling of VMX-related MSRs
        KVM: nVMX: Add tracepoints for nested_vmexit and nested_vmexit_inject
        KVM: nVMX: Pass vmexit parameters to nested_vmx_vmexit
        KVM: nVMX: Leave VMX mode on clearing of feature control MSR
        KVM: VMX: Fix DR6 update on #DB exception
        KVM: SVM: Fix reading of DR6
        KVM: x86: Sync DR7 on KVM_SET_DEBUGREGS
        add support for Hyper-V reference time counter
        KVM: remove useless write to vcpu->hv_clock.tsc_timestamp
        KVM: x86: fix tsc catchup issue with tsc scaling
        KVM: x86: limit PIT timer frequency
        KVM: x86: handle invalid root_hpa everywhere
        kvm: Provide kvm_vcpu_eligible_for_directed_yield() stub
        kvm: vfio: silence GCC warning
        KVM: ARM: Remove duplicate include
        arm/arm64: KVM: relax the requirements of VMA alignment for THP
        ...
      7ebd3faa
    • Linus Torvalds's avatar
      Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial · bb1281f2
      Linus Torvalds authored
      Pull trivial tree updates from Jiri Kosina:
       "Usual rocket science stuff from trivial.git"
      
      * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (39 commits)
        neighbour.h: fix comment
        sched: Fix warning on make htmldocs caused by wait.h
        slab: struct kmem_cache is protected by slab_mutex
        doc: Fix typo in USB Gadget Documentation
        of/Kconfig: Spelling s/one/once/
        mkregtable: Fix sscanf handling
        lp5523, lp8501: comment improvements
        thermal: rcar: comment spelling
        treewide: fix comments and printk msgs
        IXP4xx: remove '1 &&' from a condition check in ixp4xx_restart()
        Documentation: update /proc/uptime field description
        Documentation: Fix size parameter for snprintf
        arm: fix comment header and macro name
        asm-generic: uaccess: Spelling s/a ny/any/
        mtd: onenand: fix comment header
        doc: driver-model/platform.txt: fix a typo
        drivers: fix typo in DEVTMPFS_MOUNT Kconfig help text
        doc: Fix typo (acces_process_vm -> access_process_vm)
        treewide: Fix typos in printk
        drivers/gpu/drm/qxl/Kconfig: reformat the help text
        ...
      bb1281f2
    • Linus Torvalds's avatar
      Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/hid · 4988abf1
      Linus Torvalds authored
      Pull HID updates from Jiri Kosina:
      
       - quite some work on hid-sony driver in order to have DualShock 4
         device properly supported, from Frank Praznik
      
       - fixed support for suspending I2C conntected devices, from Mika
         Westerberg
      
       - regression fix for 0xff05 usage on Microsoft Ergonomy, from Jiri
         Kosina
      
       - support for Synaptics HD touchscreen, from AceLan Kao
      
       - workaround for USB 3.0 problem for logitech-dj connected devices,
         from Benjamin Tisssoires
      
       - support for Logitech Dual Action pads, from Vitaly Katraew
      
       - quite a few other assorted fixes and device ID additions
      
      * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/hid: (33 commits)
        HID: sony: Use colors for the Dualshock 4 LED names
        HID: sony: Add annotated HID descriptor for the Dualshock 4
        HID: sony: Cache the output report for the Dualshock 4
        HID: sony: Map gyroscopes and accelerometers to axes
        HID: sony: Fix spacing in the device definitions.
        HID: sony: Use standard output reports instead of raw reports to send data to the Dualshock 4.
        HID: sony: Use separate identifiers for USB and Bluetooth connected Dualshock 4 controllers.
        HID: hid-holtek-mouse: add new a070 mouse
        HID: hid-sensor-hub: Fix buggy report descriptors
        HID: logitech-dj: Fix USB 3.0 issue
        HID: sony: Rename worker function
        HID: sony: Add LED controls for the Dualshock 4
        HID: sony: Add force-feedback support for the Dualshock 4
        HID: hidraw: make comment more accurate and nicer
        HID: sony: fix error return code
        HID: input: fix input sysfs path for hid devices
        HID: debug: add labels for some new buttons
        HID: remove SIS entries from hid_have_special_driver[]
        HID: microsoft: no fallthrough in MS ergonomy 0xff05 usage
        HID: add support for SiS multitouch panel in the touch monitor LG 23ET83V
        ...
      4988abf1
    • Linus Torvalds's avatar
      Merge tag 'dm-3.14-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm · fe41c2c0
      Linus Torvalds authored
      Pull device-mapper changes from Mike Snitzer:
       "A lot of attention was paid to improving the thin-provisioning
        target's handling of metadata operation failures and running out of
        space.  A new 'error_if_no_space' feature was added to allow users to
        error IOs rather than queue them when either the data or metadata
        space is exhausted.
      
        Additional fixes/features include:
         - a few fixes to properly support thin metadata device resizing
         - a solution for reliably waiting for a DM device's embedded kobject
           to be released before destroying the device
         - old dm-snapshot is updated to use the dm-bufio interface to take
           advantage of readahead capabilities that improve snapshot
           activation
         - new dm-cache target tunables to control how quickly data is
           promoted to the cache (fast) device
         - improved write efficiency of cluster mirror target by combining
           userspace flush and mark requests"
      
      * tag 'dm-3.14-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (35 commits)
        dm log userspace: allow mark requests to piggyback on flush requests
        dm space map metadata: fix bug in resizing of thin metadata
        dm cache: add policy name to status output
        dm thin: fix pool feature parsing
        dm sysfs: fix a module unload race
        dm snapshot: use dm-bufio prefetch
        dm snapshot: use dm-bufio
        dm snapshot: prepare for switch to using dm-bufio
        dm snapshot: use GFP_KERNEL when initializing exceptions
        dm cache: add block sizes and total cache blocks to status output
        dm btree: add dm_btree_find_lowest_key
        dm space map metadata: fix extending the space map
        dm space map common: make sure new space is used during extend
        dm: wait until embedded kobject is released before destroying a device
        dm: remove pointless kobject comparison in dm_get_from_kobject
        dm snapshot: call destroy_work_on_stack() to pair with INIT_WORK_ONSTACK()
        dm cache policy mq: introduce three promotion threshold tunables
        dm cache policy mq: use list_del_init instead of list_del + INIT_LIST_HEAD
        dm thin: fix set_pool_mode exposed pool operation races
        dm thin: eliminate the no_free_space flag
        ...
      fe41c2c0
    • Linus Torvalds's avatar
      Merge tag 'scsi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi · 194e57fd
      Linus Torvalds authored
      Pull SCSI updates from James Bottomley:
       "This patch set is a lot of driver updates for qla4xxx, bfa, hpsa,
        qla2xxx.  It also removes the aic7xxx_old driver (which has been
        deprecated for nearly a decade) and adds support for deadlines in
        error handling"
      
      * tag 'scsi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (75 commits)
        [SCSI] hpsa: allow SCSI mid layer to handle unit attention
        [SCSI] hpsa: do not require board "not ready" status after hard reset
        [SCSI] hpsa: enable unit attention reporting
        [SCSI] hpsa: rename scsi prefetch field
        [SCSI] hpsa: use workqueue instead of kernel thread for lockup detection
        [SCSI] ipr: increase dump size in ipr driver
        [SCSI] mac_scsi: Fix crash on out of memory
        [SCSI] st: fix enlarge_buffer
        [SCSI] qla1280: Annotate timer on stack so object debug does not complain
        [SCSI] qla4xxx: Update driver version to 5.04.00-k3
        [SCSI] qla4xxx: Recreate chap data list during get chap operation
        [SCSI] qla4xxx: Add support for ISCSI_PARAM_LOCAL_IPADDR sysfs attr
        [SCSI] libiscsi: Add local_ipaddr parameter in iscsi_conn struct
        [SCSI] scsi_transport_iscsi: Export ISCSI_PARAM_LOCAL_IPADDR attr for iscsi_connection
        [SCSI] qla4xxx: Add host statistics support
        [SCSI] scsi_transport_iscsi: Add host statistics support
        [SCSI] qla4xxx: Added support for Diagnostics MBOX command
        [SCSI] bfa: Driver version upgrade to 3.2.23.0
        [SCSI] bfa: change FC_ELS_TOV to 20sec
        [SCSI] bfa: Observed auto D-port mode instead of manual
        ...
      194e57fd
    • Linus Torvalds's avatar
      Merge tag 'pci-v3.14-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci · e1ba8459
      Linus Torvalds authored
      Pull PCI updates from Bjorn Helgaas:
       "PCI changes for the v3.14 merge window:
      
        Resource management
          - Change pci_bus_region addresses to dma_addr_t (Bjorn Helgaas)
          - Support 64-bit AGP BARs (Bjorn Helgaas, Yinghai Lu)
          - Add pci_bus_address() to get bus address of a BAR (Bjorn Helgaas)
          - Use pci_resource_start() for CPU address of AGP BARs (Bjorn Helgaas)
          - Enforce bus address limits in resource allocation (Yinghai Lu)
          - Allocate 64-bit BARs above 4G when possible (Yinghai Lu)
          - Convert pcibios_resource_to_bus() to take pci_bus, not pci_dev (Yinghai Lu)
      
        PCI device hotplug
          - Major rescan/remove locking update (Rafael J. Wysocki)
          - Make ioapic builtin only (not modular) (Yinghai Lu)
          - Fix release/free issues (Yinghai Lu)
          - Clean up pciehp (Bjorn Helgaas)
          - Announce pciehp slot info during enumeration (Bjorn Helgaas)
      
        MSI
          - Add pci_msi_vec_count(), pci_msix_vec_count() (Alexander Gordeev)
          - Add pci_enable_msi_range(), pci_enable_msix_range() (Alexander Gordeev)
          - Deprecate "tri-state" interfaces: fail/success/fail+info (Alexander Gordeev)
          - Export MSI mode using attributes, not kobjects (Greg Kroah-Hartman)
          - Drop "irq" param from *_restore_msi_irqs() (DuanZhenzhong)
      
        SR-IOV
          - Clear NumVFs when disabling SR-IOV in sriov_init() (ethan.zhao)
      
        Virtualization
          - Add support for save/restore of extended capabilities (Alex Williamson)
          - Add Virtual Channel to save/restore support (Alex Williamson)
          - Never treat a VF as a multifunction device (Alex Williamson)
          - Add pci_try_reset_function(), et al (Alex Williamson)
      
        AER
          - Ignore non-PCIe error sources (Betty Dall)
          - Support ACPI HEST error sources for domains other than 0 (Betty Dall)
          - Consolidate HEST error source parsers (Bjorn Helgaas)
          - Add a TLP header print helper (Borislav Petkov)
      
        Freescale i.MX6
          - Remove unnecessary code (Fabio Estevam)
          - Make reset-gpio optional (Marek Vasut)
          - Report "link up" only after link training completes (Marek Vasut)
          - Start link in Gen1 before negotiating for Gen2 mode (Marek Vasut)
          - Fix PCIe startup code (Richard Zhu)
      
        Marvell MVEBU
          - Remove duplicate of_clk_get_by_name() call (Andrew Lunn)
          - Drop writes to bridge Secondary Status register (Jason Gunthorpe)
          - Obey bridge PCI_COMMAND_MEM and PCI_COMMAND_IO bits (Jason Gunthorpe)
          - Support a bridge with no IO port window (Jason Gunthorpe)
          - Use max_t() instead of max(resource_size_t,) (Jingoo Han)
          - Remove redundant of_match_ptr (Sachin Kamat)
          - Call pci_ioremap_io() at startup instead of dynamically (Thomas Petazzoni)
      
        NVIDIA Tegra
          - Disable Gen2 for Tegra20 and Tegra30 (Eric Brower)
      
        Renesas R-Car
          - Add runtime PM support (Valentine Barshak)
          - Fix rcar_pci_probe() return value check (Wei Yongjun)
      
        Synopsys DesignWare
          - Fix crash in dw_msi_teardown_irq() (Bjørn Erik Nilsen)
          - Remove redundant call to pci_write_config_word() (Bjørn Erik Nilsen)
          - Fix missing MSI IRQs (Harro Haan)
          - Add dw_pcie prefix before cfg_read/write (Pratyush Anand)
          - Fix I/O transfers by using CPU (not realio) address (Pratyush Anand)
          - Whitespace cleanup (Jingoo Han)
      
        EISA
          - Call put_device() if device_register() fails (Levente Kurusa)
          - Revert EISA initialization breakage ((Bjorn Helgaas)
      
        Miscellaneous
          - Remove unused code, including PCIe 3.0 interfaces (Stephen Hemminger)
          - Prevent bus conflicts while checking for bridge apertures (Bjorn Helgaas)
          - Stop clearing bridge Secondary Status when setting up I/O aperture (Bjorn Helgaas)
          - Use dev_is_pci() to identify PCI devices (Yijing Wang)
          - Deprecate DEFINE_PCI_DEVICE_TABLE (Joe Perches)
          - Update documentation 00-INDEX (Erik Ekman)"
      
      * tag 'pci-v3.14-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (119 commits)
        Revert "EISA: Initialize device before its resources"
        Revert "EISA: Log device resources in dmesg"
        vfio-pci: Use pci "try" reset interface
        PCI: Check parent kobject in pci_destroy_dev()
        xen/pcifront: Use global PCI rescan-remove locking
        powerpc/eeh: Use global PCI rescan-remove locking
        PCI: Fix pci_check_and_unmask_intx() comment typos
        PCI: Add pci_try_reset_function(), pci_try_reset_slot(), pci_try_reset_bus()
        MPT / PCI: Use pci_stop_and_remove_bus_device_locked()
        platform / x86: Use global PCI rescan-remove locking
        PCI: hotplug: Use global PCI rescan-remove locking
        pcmcia: Use global PCI rescan-remove locking
        ACPI / hotplug / PCI: Use global PCI rescan-remove locking
        ACPI / PCI: Use global PCI rescan-remove locking in PCI root hotplug
        PCI: Add global pci_lock_rescan_remove()
        PCI: Cleanup pci.h whitespace
        PCI: Reorder so actual code comes before stubs
        PCI/AER: Support ACPI HEST AER error sources for PCI domains other than 0
        ACPICA: Add helper macros to extract bus/segment numbers from HEST table.
        PCI: Make local functions static
        ...
      e1ba8459
    • Linus Torvalds's avatar
      Merge tag 'trace-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace · 60eaa019
      Linus Torvalds authored
      Pull tracing updates from Steven Rostedt:
       "This pull request has a new feature to ftrace, namely the trace event
        triggers by Tom Zanussi.  A trigger is a way to enable an action when
        an event is hit.  The actions are:
      
         o  trace on/off - enable or disable tracing
         o  snapshot     - save the current trace buffer in the snapshot
         o  stacktrace   - dump the current stack trace to the ringbuffer
         o  enable/disable events - enable or disable another event
      
        Namhyung Kim added updates to the tracing uprobes code.  Having the
        uprobes add support for fetch methods.
      
        The rest are various bug fixes with the new code, and minor ones for
        the old code"
      
      * tag 'trace-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (38 commits)
        tracing: Fix buggered tee(2) on tracing_pipe
        tracing: Have trace buffer point back to trace_array
        ftrace: Fix synchronization location disabling and freeing ftrace_ops
        ftrace: Have function graph only trace based on global_ops filters
        ftrace: Synchronize setting function_trace_op with ftrace_trace_function
        tracing: Show available event triggers when no trigger is set
        tracing: Consolidate event trigger code
        tracing: Fix counter for traceon/off event triggers
        tracing: Remove double-underscore naming in syscall trigger invocations
        tracing/kprobes: Add trace event trigger invocations
        tracing/probes: Fix build break on !CONFIG_KPROBE_EVENT
        tracing/uprobes: Add @+file_offset fetch method
        uprobes: Allocate ->utask before handler_chain() for tracing handlers
        tracing/uprobes: Add support for full argument access methods
        tracing/uprobes: Fetch args before reserving a ring buffer
        tracing/uprobes: Pass 'is_return' to traceprobe_parse_probe_arg()
        tracing/probes: Implement 'memory' fetch method for uprobes
        tracing/probes: Add fetch{,_size} member into deref fetch method
        tracing/probes: Move 'symbol' fetch method to kprobes
        tracing/probes: Implement 'stack' fetch method for uprobes
        ...
      60eaa019
  2. 22 Jan, 2014 30 commits
    • James Bottomley's avatar
      Merge branch 'misc' into for-linus · 4b1a9a5e
      James Bottomley authored
      4b1a9a5e
    • Jiri Kosina's avatar
      Merge branches 'for-3.13/upstream-fixes', 'for-3.14/i2c-hid',... · 62813858
      Jiri Kosina authored
      Merge branches 'for-3.13/upstream-fixes', 'for-3.14/i2c-hid', 'for-3.14/sensor-hub', 'for-3.14/sony' and 'for-3.14/upstream' into for-linus
      62813858
    • Li Zhong's avatar
      neighbour.h: fix comment · c04e7da0
      Li Zhong authored
      Signed-off-by: default avatarLi Zhong <zhong@linux.vnet.ibm.com>
      Signed-off-by: default avatarJiri Kosina <jkosina@suse.cz>
      c04e7da0
    • Masanari Iida's avatar
      sched: Fix warning on make htmldocs caused by wait.h · f434f7af
      Masanari Iida authored
      Missing "@" in include/linux/wait.h cause "make htmldocs" failed
      with following warning messages.
      
      Warning(/home/iida/Repo/linux-next//include/linux/wait.h:304):
      No description found for parameter 'cmd1'
      Warning(/home/iida/Repo/linux-next//include/linux/wait.h:304):
      No description found for parameter 'cmd2'
      Signed-off-by: default avatarMasanari Iida <standby24x7@gmail.com>
      Signed-off-by: default avatarJiri Kosina <jkosina@suse.cz>
      f434f7af
    • Dongmao Zhang's avatar
      dm log userspace: allow mark requests to piggyback on flush requests · 5066a4df
      Dongmao Zhang authored
      In the cluster evironment, cluster write has poor performance because
      userspace_flush() has to contact a userspace program (cmirrord) for
      clear/mark/flush requests.  But both mark and flush requests require
      cmirrord to communicate the message to all the cluster nodes for each
      flush call.  This behaviour is really slow.
      
      To address this we now merge mark and flush requests together to reduce
      the kernel-userspace-kernel time.  We allow a new directive,
      "integrated_flush" that can be used to instruct the kernel log code to
      combine flush and mark requests when directed by userspace.  If not
      directed by userspace (due to an older version of the userspace code
      perhaps), the kernel will function as it did previously - preserving
      backwards compatibility.  Additionally, flush requests are performed
      lazily when only clear requests exist.
      Signed-off-by: default avatarDongmao Zhang <dmzhang@suse.com>
      Signed-off-by: default avatarJonathan Brassow <jbrassow@redhat.com>
      Signed-off-by: default avatarAlasdair G Kergon <agk@redhat.com>
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      5066a4df
    • Linus Torvalds's avatar
      Merge branch 'akpm' (incoming from Andrew) · df32e43a
      Linus Torvalds authored
      Merge first patch-bomb from Andrew Morton:
      
       - a couple of misc things
      
       - inotify/fsnotify work from Jan
      
       - ocfs2 updates (partial)
      
       - about half of MM
      
      * emailed patches from Andrew Morton <akpm@linux-foundation.org>: (117 commits)
        mm/migrate: remove unused function, fail_migrate_page()
        mm/migrate: remove putback_lru_pages, fix comment on putback_movable_pages
        mm/migrate: correct failure handling if !hugepage_migration_support()
        mm/migrate: add comment about permanent failure path
        mm, page_alloc: warn for non-blockable __GFP_NOFAIL allocation failure
        mm: compaction: reset scanner positions immediately when they meet
        mm: compaction: do not mark unmovable pageblocks as skipped in async compaction
        mm: compaction: detect when scanners meet in isolate_freepages
        mm: compaction: reset cached scanner pfn's before reading them
        mm: compaction: encapsulate defer reset logic
        mm: compaction: trace compaction begin and end
        memcg, oom: lock mem_cgroup_print_oom_info
        sched: add tracepoints related to NUMA task migration
        mm: numa: do not automatically migrate KSM pages
        mm: numa: trace tasks that fail migration due to rate limiting
        mm: numa: limit scope of lock for NUMA migrate rate limiting
        mm: numa: make NUMA-migrate related functions static
        lib/show_mem.c: show num_poisoned_pages when oom
        mm/hwpoison: add '#' to hwpoison_inject
        mm/memblock: use WARN_ONCE when MAX_NUMNODES passed as input parameter
        ...
      df32e43a
    • Linus Torvalds's avatar
      Merge branch 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/libata · fbd918a2
      Linus Torvalds authored
      Pull libata updates from Tejun Heo:
       "Support for some new embedded controllers.
      
        A couple late (<= a week) fixes have stable cc'd and one patch ("SATA:
        MV: Add support for the optional PHYs") got committed yesterday
        because otherwise the resulting kernel would fail boot on an embedded
        board due to interdependent changes in its platform tree.
      
        Other than that, nothing too noteworthy"
      
      * 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/libata:
        SATA: MV: Add support for the optional PHYs
        sata-highbank: Remove unnecessary ahci_platform.h include
        libata: disable LPM for some WD SATA-I devices
        ARM: mvebu: update the SATA compatible string for Armada 370/XP
        ata: sata_mv: fix disk hotplug for Armada 370/XP SoCs
        ata: sata_mv: introduce compatible string "marvell, armada-370-sata"
        ata: pata_samsung_cf: Remove unused macros
        ata: pata_samsung_cf: Use devm_ioremap_resource()
        ata: pata_samsung_cf: Merge pata_samsung_cf.h into pata_samsung_cf.c
        ata: pata_samsung_cf: Move plat/regs-ata.h to drivers/ata
        drivers: ata: Mark the function as static in libahci.c
        drivers: ata: Mark the function ahci_init_interrupts() as static in ahci.c
        ahci: imx: fix the error handling in imx_ahci_probe()
        ahci: imx: ahci_imx_softreset() can be static
        ahci: imx: Add i.MX53 support
        ahci: imx: Pull out the clock enable/disable calls
        libata, dt: Document sata_rcar bindings
        sata_rcar: Add R-Car Gen2 SATA PHY support
        ahci: mcp89: enter AHCI mode under Apple BIOS emulation
        ata: libata-eh: Remove unnecessary snprintf arithmetic
      fbd918a2
    • Linus Torvalds's avatar
      Merge branch 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup · f075e0f6
      Linus Torvalds authored
      Pull cgroup updates from Tejun Heo:
       "The bulk of changes are cleanups and preparations for the upcoming
        kernfs conversion.
      
         - cgroup_event mechanism which is and will be used only by memcg is
           moved to memcg.
      
         - pidlist handling is updated so that it can be served by seq_file.
      
           Also, the list is not sorted if sane_behavior.  cgroup
           documentation explicitly states that the file is not sorted but it
           has been for quite some time.
      
         - All cgroup file handling now happens on top of seq_file.  This is
           to prepare for kernfs conversion.  In addition, all operations are
           restructured so that they map 1-1 to kernfs operations.
      
         - Other cleanups and low-pri fixes"
      
      * 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (40 commits)
        cgroup: trivial style updates
        cgroup: remove stray references to css_id
        doc: cgroups: Fix typo in doc/cgroups
        cgroup: fix fail path in cgroup_load_subsys()
        cgroup: fix missing unlock on error in cgroup_load_subsys()
        cgroup: remove for_each_root_subsys()
        cgroup: implement for_each_css()
        cgroup: factor out cgroup_subsys_state creation into create_css()
        cgroup: combine css handling loops in cgroup_create()
        cgroup: reorder operations in cgroup_create()
        cgroup: make for_each_subsys() useable under cgroup_root_mutex
        cgroup: css iterations and css_from_dir() are safe under cgroup_mutex
        cgroup: unify pidlist and other file handling
        cgroup: replace cftype->read_seq_string() with cftype->seq_show()
        cgroup: attach cgroup_open_file to all cgroup files
        cgroup: generalize cgroup_pidlist_open_file
        cgroup: unify read path so that seq_file is always used
        cgroup: unify cgroup_write_X64() and cgroup_write_string()
        cgroup: remove cftype->read(), ->read_map() and ->write()
        hugetlb_cgroup: convert away from cftype->read()
        ...
      f075e0f6
    • Linus Torvalds's avatar
      Merge branch 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu · 5cb7398c
      Linus Torvalds authored
      Pull percpu changes from Tejun Heo:
       "Two trivial changes - addition of WARN_ONCE() in lib/percpu-refcount.c
        and use of VMALLOC_TOTAL instead of END - START in percpu.c"
      
      * 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu:
        percpu: use VMALLOC_TOTAL instead of VMALLOC_END - VMALLOC_START
        percpu-refcount: Add a WARN() for ref going negative
      5cb7398c
    • Linus Torvalds's avatar
      Merge branch 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq · 4a2829b9
      Linus Torvalds authored
      Pull workqueue update from Tejun Heo:
       "Just one patch to add destroy_work_on_stack() annotations to help
        debugobj debugging"
      
      * 'for-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
        workqueue: Calling destroy_work_on_stack() to pair with INIT_WORK_ONSTACK()
      4a2829b9
    • Linus Torvalds's avatar
      Merge tag 'dlm-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/teigland/linux-dlm · ff0bc6cc
      Linus Torvalds authored
      Pull dlm update from David Teigland:
       "A single change to speed up recovery times when using SCTP
        connections"
      
      * tag 'dlm-3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/teigland/linux-dlm:
        dlm: set zero linger time on sctp socket
      ff0bc6cc
    • Linus Torvalds's avatar
      Merge tag 'gfs2-merge-window' of git://git.kernel.org/pub/scm/linux/kernel/git/steve/gfs2-3.0-nmw · 2182c815
      Linus Torvalds authored
      Pull GFS2 updates from Steven Whitehouse:
       "The main topics this time are allocation, in the form of Bob's
        improvements when searching resource groups and several updates to
        quotas which should increase scalability.  The quota changes follow on
        from those in the last merge window, and there will likely be further
        work to come in this area in due course.
      
        There are also a few patches which help to improve efficiency of
        adding entries into directories, and clean up some of that code.
      
        One on-disk change is included this time, which is to write some
        additional information which should be useful to fsck and also
        potentially for debugging.
      
        Other than that, its just a few small random bug fixes and clean ups"
      
      * tag 'gfs2-merge-window' of git://git.kernel.org/pub/scm/linux/kernel/git/steve/gfs2-3.0-nmw: (24 commits)
        GFS2: revert "GFS2: d_splice_alias() can't return error"
        GFS2: Small cleanup
        GFS2: Don't use ENOBUFS when ENOMEM is the correct error code
        GFS2: Fix kbuild test robot reported warning
        GFS2: Move quota bitmap operations under their own lock
        GFS2: Clean up quota slot allocation
        GFS2: Only run logd and quota when mounted read/write
        GFS2: Use RCU/hlist_bl based hash for quotas
        GFS2: No need to invalidate pages for a dio read
        GFS2: Add initialization for address space in super block
        GFS2: Add hints to directory leaf blocks
        GFS2: For exhash conversion, only one block is needed
        GFS2: Increase i_writecount during gfs2_setattr_chown
        GFS2: Remember directory insert point
        GFS2: Consolidate transaction blocks calculation for dir add
        GFS2: Add directory addition info structure
        GFS2: Use only a single address space for rgrps
        GFS2: Use range based functions for rgrp sync/invalidation
        GFS2: Remove test which is always true
        GFS2: Remove gfs2_quota_change_host structure
        ...
      2182c815
    • Joonsoo Kim's avatar
      mm/migrate: remove unused function, fail_migrate_page() · 78d5506e
      Joonsoo Kim authored
      fail_migrate_page() isn't used anywhere, so remove it.
      Signed-off-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
      Acked-by: default avatarChristoph Lameter <cl@linux.com>
      Reviewed-by: default avatarNaoya Horiguchi <n-horiguchi@ah.jp.nec.com>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Rafael Aquini <aquini@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      78d5506e
    • Joonsoo Kim's avatar
      mm/migrate: remove putback_lru_pages, fix comment on putback_movable_pages · 59c82b70
      Joonsoo Kim authored
      Some part of putback_lru_pages() and putback_movable_pages() is
      duplicated, so it could confuse us what we should use.  We can remove
      putback_lru_pages() since it is not really needed now.  This makes us
      undestand and maintain the code more easily.
      
      And comment on putback_movable_pages() is stale now, so fix it.
      Signed-off-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
      Cc: Rafael Aquini <aquini@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      59c82b70
    • Joonsoo Kim's avatar
      mm/migrate: correct failure handling if !hugepage_migration_support() · 32665f2b
      Joonsoo Kim authored
      We should remove the page from the list if we fail with ENOSYS, since
      migrate_pages() consider error cases except -ENOMEM and -EAGAIN as
      permanent failure and it assumes that the page would be removed from the
      list.  Without this patch, we could overcount number of failure.
      
      In addition, we should put back the new hugepage if
      !hugepage_migration_support().  If not, we would leak hugepage memory.
      Signed-off-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
      Acked-by: default avatarChristoph Lameter <cl@linux.com>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Reviewed-by: default avatarNaoya Horiguchi <n-horiguchi@ah.jp.nec.com>
      Cc: Rafael Aquini <aquini@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      32665f2b
    • Naoya Horiguchi's avatar
      mm/migrate: add comment about permanent failure path · 354a3363
      Naoya Horiguchi authored
      Let's add a comment about where the failed page goes to, which makes
      code more readable.
      Signed-off-by: default avatarNaoya Horiguchi <n-horiguchi@ah.jp.nec.com>
      Signed-off-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
      Acked-by: default avatarChristoph Lameter <cl@linux.com>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Acked-by: default avatarRafael Aquini <aquini@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Vlastimil Babka <vbabka@suse.cz>
      Cc: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      354a3363
    • David Rientjes's avatar
      mm, page_alloc: warn for non-blockable __GFP_NOFAIL allocation failure · aed0a0e3
      David Rientjes authored
      __GFP_NOFAIL may return NULL when coupled with GFP_NOWAIT or GFP_ATOMIC.
      
      Luckily, nothing currently does such craziness.  So instead of causing
      such allocations to loop (potentially forever), we maintain the current
      behavior and also warn about the new users of the deprecated flag.
      Suggested-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Michal Hocko <mhocko@suse.cz>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      aed0a0e3
    • Vlastimil Babka's avatar
      mm: compaction: reset scanner positions immediately when they meet · 55b7c4c9
      Vlastimil Babka authored
      Compaction used to start its migrate and free page scaners at the zone's
      lowest and highest pfn, respectively.  Later, caching was introduced to
      remember the scanners' progress across compaction attempts so that
      pageblocks are not re-scanned uselessly.  Additionally, pageblocks where
      isolation failed are marked to be quickly skipped when encountered again
      in future compactions.
      
      Currently, both the reset of cached pfn's and clearing of the pageblock
      skip information for a zone is done in __reset_isolation_suitable().
      This function gets called when:
      
       - compaction is restarting after being deferred
       - compact_blockskip_flush flag is set in compact_finished() when the scanners
         meet (and not again cleared when direct compaction succeeds in allocation)
         and kswapd acts upon this flag before going to sleep
      
      This behavior is suboptimal for several reasons:
      
       - when direct sync compaction is called after async compaction fails (in the
         allocation slowpath), it will effectively do nothing, unless kswapd
         happens to process the compact_blockskip_flush flag meanwhile. This is racy
         and goes against the purpose of sync compaction to more thoroughly retry
         the compaction of a zone where async compaction has failed.
         The restart-after-deferring path cannot help here as deferring happens only
         after the sync compaction fails. It is also done only for the preferred
         zone, while the compaction might be done for a fallback zone.
      
       - the mechanism of marking pageblock to be skipped has little value since the
         cached pfn's are reset only together with the pageblock skip flags. This
         effectively limits pageblock skip usage to parallel compactions.
      
      This patch changes compact_finished() so that cached pfn's are reset
      immediately when the scanners meet.  Clearing pageblock skip flags is
      unchanged, as well as the other situations where cached pfn's are reset.
      This allows the sync-after-async compaction to retry pageblocks not
      marked as skipped, such as blocks !MIGRATE_MOVABLE blocks that async
      compactions now skips without marking them.
      Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Cc: Rik van Riel <riel@redhat.com>
      Acked-by: default avatarMel Gorman <mgorman@suse.de>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      55b7c4c9
    • Vlastimil Babka's avatar
      mm: compaction: do not mark unmovable pageblocks as skipped in async compaction · 50b5b094
      Vlastimil Babka authored
      Compaction temporarily marks pageblocks where it fails to isolate pages
      as to-be-skipped in further compactions, in order to improve efficiency.
      One of the reasons to fail isolating pages is that isolation is not
      attempted in pageblocks that are not of MIGRATE_MOVABLE (or CMA) type.
      
      The problem is that blocks skipped due to not being MIGRATE_MOVABLE in
      async compaction become skipped due to the temporary mark also in future
      sync compaction.  Moreover, this may follow quite soon during
      __alloc_page_slowpath, without much time for kswapd to clear the
      pageblock skip marks.  This goes against the idea that sync compaction
      should try to scan these blocks more thoroughly than the async
      compaction.
      
      The fix is to ensure in async compaction that these !MIGRATE_MOVABLE
      blocks are not marked to be skipped.  Note this should not affect
      performance or locking impact of further async compactions, as skipping
      a block due to being !MIGRATE_MOVABLE is done soon after skipping a
      block marked to be skipped, both without locking.
      Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Cc: Rik van Riel <riel@redhat.com>
      Acked-by: default avatarMel Gorman <mgorman@suse.de>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      50b5b094
    • Vlastimil Babka's avatar
      mm: compaction: detect when scanners meet in isolate_freepages · 7ed695e0
      Vlastimil Babka authored
      Compaction of a zone is finished when the migrate scanner (which begins
      at the zone's lowest pfn) meets the free page scanner (which begins at
      the zone's highest pfn).  This is detected in compact_zone() and in the
      case of direct compaction, the compact_blockskip_flush flag is set so
      that kswapd later resets the cached scanner pfn's, and a new compaction
      may again start at the zone's borders.
      
      The meeting of the scanners can happen during either scanner's activity.
      However, it may currently fail to be detected when it occurs in the free
      page scanner, due to two problems.  First, isolate_freepages() keeps
      free_pfn at the highest block where it isolated pages from, for the
      purposes of not missing the pages that are returned back to allocator
      when migration fails.  Second, failing to isolate enough free pages due
      to scanners meeting results in -ENOMEM being returned by
      migrate_pages(), which makes compact_zone() bail out immediately without
      calling compact_finished() that would detect scanners meeting.
      
      This failure to detect scanners meeting might result in repeated
      attempts at compaction of a zone that keep starting from the cached
      pfn's close to the meeting point, and quickly failing through the
      -ENOMEM path, without the cached pfns being reset, over and over.  This
      has been observed (through additional tracepoints) in the third phase of
      the mmtests stress-highalloc benchmark, where the allocator runs on an
      otherwise idle system.  The problem was observed in the DMA32 zone,
      which was used as a fallback to the preferred Normal zone, but on the
      4GB system it was actually the largest zone.  The problem is even
      amplified for such fallback zone - the deferred compaction logic, which
      could (after being fixed by a previous patch) reset the cached scanner
      pfn's, is only applied to the preferred zone and not for the fallbacks.
      
      The problem in the third phase of the benchmark was further amplified by
      commit 81c0a2bb ("mm: page_alloc: fair zone allocator policy") which
      resulted in a non-deterministic regression of the allocation success
      rate from ~85% to ~65%.  This occurs in about half of benchmark runs,
      making bisection problematic.  It is unlikely that the commit itself is
      buggy, but it should put more pressure on the DMA32 zone during phases 1
      and 2, which may leave it more fragmented in phase 3 and expose the bugs
      that this patch fixes.
      
      The fix is to make scanners meeting in isolate_freepage() stay that way,
      and to check in compact_zone() for scanners meeting when migrate_pages()
      returns -ENOMEM.  The result is that compact_finished() also detects
      scanners meeting and sets the compact_blockskip_flush flag to make
      kswapd reset the scanner pfn's.
      
      The results in stress-highalloc benchmark show that the "regression" by
      commit 81c0a2bb in phase 3 no longer occurs, and phase 1 and 2
      allocation success rates are also significantly improved.
      Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      7ed695e0
    • Vlastimil Babka's avatar
      mm: compaction: reset cached scanner pfn's before reading them · d3132e4b
      Vlastimil Babka authored
      Compaction caches pfn's for its migrate and free scanners to avoid
      scanning the whole zone each time.  In compact_zone(), the cached values
      are read to set up initial values for the scanners.  There are several
      situations when these cached pfn's are reset to the first and last pfn
      of the zone, respectively.  One of these situations is when a compaction
      has been deferred for a zone and is now being restarted during a direct
      compaction, which is also done in compact_zone().
      
      However, compact_zone() currently reads the cached pfn's *before*
      resetting them.  This means the reset doesn't affect the compaction that
      performs it, and with good chance also subsequent compactions, as
      update_pageblock_skip() is likely to be called and update the cached
      pfn's to those being processed.  Another chance for a successful reset
      is when a direct compaction detects that migration and free scanners
      meet (which has its own problems addressed by another patch) and sets
      update_pageblock_skip flag which kswapd uses to do the reset because it
      goes to sleep.
      
      This is clearly a bug that results in non-deterministic behavior, so
      this patch moves the cached pfn reset to be performed *before* the
      values are read.
      Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Acked-by: default avatarMel Gorman <mgorman@suse.de>
      Acked-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      d3132e4b
    • Vlastimil Babka's avatar
      mm: compaction: encapsulate defer reset logic · de6c60a6
      Vlastimil Babka authored
      Currently there are several functions to manipulate the deferred
      compaction state variables.  The remaining case where the variables are
      touched directly is when a successful allocation occurs in direct
      compaction, or is expected to be successful in the future by kswapd.
      Here, the lowest order that is expected to fail is updated, and in the
      case of successful allocation, the deferred status and counter is reset
      completely.
      
      Create a new function compaction_defer_reset() to encapsulate this
      functionality and make it easier to understand the code.  No functional
      change.
      Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Acked-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      de6c60a6
    • Mel Gorman's avatar
      mm: compaction: trace compaction begin and end · 0eb927c0
      Mel Gorman authored
      The broad goal of the series is to improve allocation success rates for
      huge pages through memory compaction, while trying not to increase the
      compaction overhead.  The original objective was to reintroduce
      capturing of high-order pages freed by the compaction, before they are
      split by concurrent activity.  However, several bugs and opportunities
      for simple improvements were found in the current implementation, mostly
      through extra tracepoints (which are however too ugly for now to be
      considered for sending).
      
      The patches mostly deal with two mechanisms that reduce compaction
      overhead, which is caching the progress of migrate and free scanners,
      and marking pageblocks where isolation failed to be skipped during
      further scans.
      
      Patch 1 (from mgorman) adds tracepoints that allow calculate time spent in
              compaction and potentially debug scanner pfn values.
      
      Patch 2 encapsulates the some functionality for handling deferred compactions
              for better maintainability, without a functional change
              type is not determined without being actually needed.
      
      Patch 3 fixes a bug where cached scanner pfn's are sometimes reset only after
              they have been read to initialize a compaction run.
      
      Patch 4 fixes a bug where scanners meeting is sometimes not properly detected
              and can lead to multiple compaction attempts quitting early without
              doing any work.
      
      Patch 5 improves the chances of sync compaction to process pageblocks that
              async compaction has skipped due to being !MIGRATE_MOVABLE.
      
      Patch 6 improves the chances of sync direct compaction to actually do anything
              when called after async compaction fails during allocation slowpath.
      
      The impact of patches were validated using mmtests's stress-highalloc
      benchmark with mmtests's stress-highalloc benchmark on a x86_64 machine
      with 4GB memory.
      
      Due to instability of the results (mostly related to the bugs fixed by
      patches 2 and 3), 10 iterations were performed, taking min,mean,max
      values for success rates and mean values for time and vmstat-based
      metrics.
      
      First, the default GFP_HIGHUSER_MOVABLE allocations were tested with the
      patches stacked on top of v3.13-rc2.  Patch 2 is OK to serve as baseline
      due to no functional changes in 1 and 2.  Comments below.
      
      stress-highalloc
                                   3.13-rc2              3.13-rc2              3.13-rc2              3.13-rc2              3.13-rc2
                                    2-nothp               3-nothp               4-nothp               5-nothp               6-nothp
      Success 1 Min          9.00 (  0.00%)       10.00 (-11.11%)       43.00 (-377.78%)       43.00 (-377.78%)       33.00 (-266.67%)
      Success 1 Mean        27.50 (  0.00%)       25.30 (  8.00%)       45.50 (-65.45%)       45.90 (-66.91%)       46.30 (-68.36%)
      Success 1 Max         36.00 (  0.00%)       36.00 (  0.00%)       47.00 (-30.56%)       48.00 (-33.33%)       52.00 (-44.44%)
      Success 2 Min         10.00 (  0.00%)        8.00 ( 20.00%)       46.00 (-360.00%)       45.00 (-350.00%)       35.00 (-250.00%)
      Success 2 Mean        26.40 (  0.00%)       23.50 ( 10.98%)       47.30 (-79.17%)       47.60 (-80.30%)       48.10 (-82.20%)
      Success 2 Max         34.00 (  0.00%)       33.00 (  2.94%)       48.00 (-41.18%)       50.00 (-47.06%)       54.00 (-58.82%)
      Success 3 Min         65.00 (  0.00%)       63.00 (  3.08%)       85.00 (-30.77%)       84.00 (-29.23%)       85.00 (-30.77%)
      Success 3 Mean        76.70 (  0.00%)       70.50 (  8.08%)       86.20 (-12.39%)       85.50 (-11.47%)       86.00 (-12.13%)
      Success 3 Max         87.00 (  0.00%)       86.00 (  1.15%)       88.00 ( -1.15%)       87.00 (  0.00%)       87.00 (  0.00%)
      
                  3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2
                   2-nothp     3-nothp     4-nothp     5-nothp     6-nothp
      User         6437.72     6459.76     5960.32     5974.55     6019.67
      System       1049.65     1049.09     1029.32     1031.47     1032.31
      Elapsed      1856.77     1874.48     1949.97     1994.22     1983.15
      
                                    3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2
                                     2-nothp     3-nothp     4-nothp     5-nothp     6-nothp
      Minor Faults                 253952267   254581900   250030122   250507333   250157829
      Major Faults                       420         407         506         530         530
      Swap Ins                             4           9           9           6           6
      Swap Outs                          398         375         345         346         333
      Direct pages scanned            197538      189017      298574      287019      299063
      Kswapd pages scanned           1809843     1801308     1846674     1873184     1861089
      Kswapd pages reclaimed         1806972     1798684     1844219     1870509     1858622
      Direct pages reclaimed          197227      188829      298380      286822      298835
      Kswapd efficiency                  99%         99%         99%         99%         99%
      Kswapd velocity                953.382     970.449     952.243     934.569     922.286
      Direct efficiency                  99%         99%         99%         99%         99%
      Direct velocity                104.058     101.832     153.961     143.200     148.205
      Percentage direct scans             9%          9%         13%         13%         13%
      Zone normal velocity           347.289     359.676     348.063     339.933     332.983
      Zone dma32 velocity            710.151     712.605     758.140     737.835     737.507
      Zone dma velocity                0.000       0.000       0.000       0.000       0.000
      Page writes by reclaim         557.600     429.000     353.600     426.400     381.800
      Page writes file                   159          53           7          79          48
      Page writes anon                   398         375         345         346         333
      Page reclaim immediate             825         644         411         575         420
      Sector Reads                   2781750     2769780     2878547     2939128     2910483
      Sector Writes                 12080843    12083351    12012892    12002132    12010745
      Page rescued immediate               0           0           0           0           0
      Slabs scanned                  1575654     1545344     1778406     1786700     1794073
      Direct inode steals               9657       10037       15795       14104       14645
      Kswapd inode steals              46857       46335       50543       50716       51796
      Kswapd skipped wait                  0           0           0           0           0
      THP fault alloc                     97          91          81          71          77
      THP collapse alloc                 456         506         546         544         565
      THP splits                           6           5           5           4           4
      THP fault fallback                   0           1           0           0           0
      THP collapse fail                   14          14          12          13          12
      Compaction stalls                 1006         980        1537        1536        1548
      Compaction success                 303         284         562         559         578
      Compaction failures                702         696         974         976         969
      Page migrate success           1177325     1070077     3927538     3781870     3877057
      Page migrate failure                 0           0           0           0           0
      Compaction pages isolated      2547248     2306457     8301218     8008500     8200674
      Compaction migrate scanned    42290478    38832618   153961130   154143900   159141197
      Compaction free scanned       89199429    79189151   356529027   351943166   356326727
      Compaction cost                   1566        1426        5312        5156        5294
      NUMA PTE updates                     0           0           0           0           0
      NUMA hint faults                     0           0           0           0           0
      NUMA hint local faults               0           0           0           0           0
      NUMA hint local percent            100         100         100         100         100
      NUMA pages migrated                  0           0           0           0           0
      AutoNUMA cost                        0           0           0           0           0
      
      Observations:
      
      - The "Success 3" line is allocation success rate with system idle
        (phases 1 and 2 are with background interference).  I used to get stable
        values around 85% with vanilla 3.11.  The lower min and mean values came
        with 3.12.  This was bisected to commit 81c0a2bb ("mm: page_alloc: fair
        zone allocator policy") As explained in comment for patch 3, I don't
        think the commit is wrong, but that it makes the effect of compaction
        bugs worse.  From patch 3 onwards, the results are OK and match the 3.11
        results.
      
      - Patch 4 also clearly helps phases 1 and 2, and exceeds any results
        I've seen with 3.11 (I didn't measure it that thoroughly then, but it
        was never above 40%).
      
      - Compaction cost and number of scanned pages is higher, especially due
        to patch 4.  However, keep in mind that patches 3 and 4 fix existing
        bugs in the current design of compaction overhead mitigation, they do
        not change it.  If overhead is found unacceptable, then it should be
        decreased differently (and consistently, not due to random conditions)
        than the current implementation does.  In contrast, patches 5 and 6
        (which are not strictly bug fixes) do not increase the overhead (but
        also not success rates).  This might be a limitation of the
        stress-highalloc benchmark as it's quite uniform.
      
      Another set of results is when configuring stress-highalloc t allocate
      with similar flags as THP uses:
       (GFP_HIGHUSER_MOVABLE|__GFP_NOMEMALLOC|__GFP_NORETRY|__GFP_NO_KSWAPD)
      
      stress-highalloc
                                   3.13-rc2              3.13-rc2              3.13-rc2              3.13-rc2              3.13-rc2
                                      2-thp                 3-thp                 4-thp                 5-thp                 6-thp
      Success 1 Min          2.00 (  0.00%)        7.00 (-250.00%)       18.00 (-800.00%)       19.00 (-850.00%)       26.00 (-1200.00%)
      Success 1 Mean        19.20 (  0.00%)       17.80 (  7.29%)       29.20 (-52.08%)       29.90 (-55.73%)       32.80 (-70.83%)
      Success 1 Max         27.00 (  0.00%)       29.00 ( -7.41%)       35.00 (-29.63%)       36.00 (-33.33%)       37.00 (-37.04%)
      Success 2 Min          3.00 (  0.00%)        8.00 (-166.67%)       21.00 (-600.00%)       21.00 (-600.00%)       32.00 (-966.67%)
      Success 2 Mean        19.30 (  0.00%)       17.90 (  7.25%)       32.20 (-66.84%)       32.60 (-68.91%)       35.70 (-84.97%)
      Success 2 Max         27.00 (  0.00%)       30.00 (-11.11%)       36.00 (-33.33%)       37.00 (-37.04%)       39.00 (-44.44%)
      Success 3 Min         62.00 (  0.00%)       62.00 (  0.00%)       85.00 (-37.10%)       75.00 (-20.97%)       64.00 ( -3.23%)
      Success 3 Mean        66.30 (  0.00%)       65.50 (  1.21%)       85.60 (-29.11%)       83.40 (-25.79%)       83.50 (-25.94%)
      Success 3 Max         70.00 (  0.00%)       69.00 (  1.43%)       87.00 (-24.29%)       86.00 (-22.86%)       87.00 (-24.29%)
      
                  3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2
                     2-thp       3-thp       4-thp       5-thp       6-thp
      User         6547.93     6475.85     6265.54     6289.46     6189.96
      System       1053.42     1047.28     1043.23     1042.73     1038.73
      Elapsed      1835.43     1821.96     1908.67     1912.74     1956.38
      
                                    3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2    3.13-rc2
                                       2-thp       3-thp       4-thp       5-thp       6-thp
      Minor Faults                 256805673   253106328   253222299   249830289   251184418
      Major Faults                       395         375         423         434         448
      Swap Ins                            12          10          10          12           9
      Swap Outs                          530         537         487         455         415
      Direct pages scanned             71859       86046      153244      152764      190713
      Kswapd pages scanned           1900994     1870240     1898012     1892864     1880520
      Kswapd pages reclaimed         1897814     1867428     1894939     1890125     1877924
      Direct pages reclaimed           71766       85908      153167      152643      190600
      Kswapd efficiency                  99%         99%         99%         99%         99%
      Kswapd velocity               1029.000    1067.782    1000.091     991.049     951.218
      Direct efficiency                  99%         99%         99%         99%         99%
      Direct velocity                 38.897      49.127      80.747      79.983      96.468
      Percentage direct scans             3%          4%          7%          7%          9%
      Zone normal velocity           351.377     372.494     348.910     341.689     335.310
      Zone dma32 velocity            716.520     744.414     731.928     729.343     712.377
      Zone dma velocity                0.000       0.000       0.000       0.000       0.000
      Page writes by reclaim         669.300     604.000     545.700     538.900     429.900
      Page writes file                   138          66          58          83          14
      Page writes anon                   530         537         487         455         415
      Page reclaim immediate             806         655         772         548         517
      Sector Reads                   2711956     2703239     2811602     2818248     2839459
      Sector Writes                 12163238    12018662    12038248    11954736    11994892
      Page rescued immediate               0           0           0           0           0
      Slabs scanned                  1385088     1388364     1507968     1513292     1558656
      Direct inode steals               1739        2564        4622        5496        6007
      Kswapd inode steals              47461       46406       47804       48013       48466
      Kswapd skipped wait                  0           0           0           0           0
      THP fault alloc                    110          82          84          69          70
      THP collapse alloc                 445         482         467         462         539
      THP splits                           6           5           4           5           3
      THP fault fallback                   3           0           0           0           0
      THP collapse fail                   15          14          14          14          13
      Compaction stalls                  659         685        1033        1073        1111
      Compaction success                 222         225         410         427         456
      Compaction failures                436         460         622         646         655
      Page migrate success            446594      439978     1085640     1095062     1131716
      Page migrate failure                 0           0           0           0           0
      Compaction pages isolated      1029475     1013490     2453074     2482698     2565400
      Compaction migrate scanned     9955461    11344259    24375202    27978356    30494204
      Compaction free scanned       27715272    28544654    80150615    82898631    85756132
      Compaction cost                    552         555        1344        1379        1436
      NUMA PTE updates                     0           0           0           0           0
      NUMA hint faults                     0           0           0           0           0
      NUMA hint local faults               0           0           0           0           0
      NUMA hint local percent            100         100         100         100         100
      NUMA pages migrated                  0           0           0           0           0
      AutoNUMA cost                        0           0           0           0           0
      
      There are some differences from the previous results for THP-like allocations:
      
      - Here, the bad result for unpatched kernel in phase 3 is much more
        consistent to be between 65-70% and not related to the "regression" in
        3.12.  Still there is the improvement from patch 4 onwards, which brings
        it on par with simple GFP_HIGHUSER_MOVABLE allocations.
      
      - Compaction costs have increased, but nowhere near as much as the
        non-THP case.  Again, the patches should be worth the gained
        determininsm.
      
      - Patches 5 and 6 somewhat increase the number of migrate-scanned pages.
         This is most likely due to __GFP_NO_KSWAPD flag, which means the cached
        pfn's and pageblock skip bits are not reset by kswapd that often (at
        least in phase 3 where no concurrent activity would wake up kswapd) and
        the patches thus help the sync-after-async compaction.  It doesn't
        however show that the sync compaction would help so much with success
        rates, which can be again seen as a limitation of the benchmark
        scenario.
      
      This patch (of 6):
      
      Add two tracepoints for compaction begin and end of a zone.  Using this it
      is possible to calculate how much time a workload is spending within
      compaction and potentially debug problems related to cached pfns for
      scanning.  In combination with the direct reclaim and slab trace points it
      should be possible to estimate most allocation-related overhead for a
      workload.
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      0eb927c0
    • Michal Hocko's avatar
      memcg, oom: lock mem_cgroup_print_oom_info · 947b3dd1
      Michal Hocko authored
      mem_cgroup_print_oom_info uses a static buffer (memcg_name) to store the
      name of the cgroup.  This is not safe as pointed out by David Rientjes
      because memcg oom is locked only for its hierarchy and nothing prevents
      another parallel hierarchy to trigger oom as well and overwrite the
      already in-use buffer.
      
      This patch introduces oom_info_lock hidden inside
      mem_cgroup_print_oom_info which is held throughout the function.  It
      makes access to memcg_name safe and as a bonus it also prevents parallel
      memcg ooms to interleave their statistics which would make the printed
      data hard to analyze otherwise.
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      947b3dd1
    • Mel Gorman's avatar
      sched: add tracepoints related to NUMA task migration · 286549dc
      Mel Gorman authored
      This patch adds three tracepoints
       o trace_sched_move_numa	when a task is moved to a node
       o trace_sched_swap_numa	when a task is swapped with another task
       o trace_sched_stick_numa	when a numa-related migration fails
      
      The tracepoints allow the NUMA scheduler activity to be monitored and the
      following high-level metrics can be calculated
      
       o NUMA migrated stuck	 nr trace_sched_stick_numa
       o NUMA migrated idle	 nr trace_sched_move_numa
       o NUMA migrated swapped nr trace_sched_swap_numa
       o NUMA local swapped	 trace_sched_swap_numa src_nid == dst_nid (should never happen)
       o NUMA remote swapped	 trace_sched_swap_numa src_nid != dst_nid (should == NUMA migrated swapped)
       o NUMA group swapped	 trace_sched_swap_numa src_ngid == dst_ngid
      			 Maybe a small number of these are acceptable
      			 but a high number would be a major surprise.
      			 It would be even worse if bounces are frequent.
       o NUMA avg task migs.	 Average number of migrations for tasks
       o NUMA stddev task mig	 Self-explanatory
       o NUMA max task migs.	 Maximum number of migrations for a single task
      
      In general the intent of the tracepoints is to help diagnose problems
      where automatic NUMA balancing appears to be doing an excessive amount
      of useless work.
      
      [akpm@linux-foundation.org: remove semicolon-after-if, repair coding-style]
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Alex Thorlton <athorlton@sgi.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      286549dc
    • Mel Gorman's avatar
      mm: numa: do not automatically migrate KSM pages · 64a9a34e
      Mel Gorman authored
      KSM pages can be shared between tasks that are not necessarily related
      to each other from a NUMA perspective.  This patch causes those pages to
      be ignored by automatic NUMA balancing so they do not migrate and do not
      cause unrelated tasks to be grouped together.
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Alex Thorlton <athorlton@sgi.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      64a9a34e
    • Mel Gorman's avatar
      mm: numa: trace tasks that fail migration due to rate limiting · af1839d7
      Mel Gorman authored
      A low local/remote numa hinting fault ratio is potentially explained by
      failed migrations.  This patch adds a tracepoint that fires when
      migration fails due to migration rate limitation.
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Alex Thorlton <athorlton@sgi.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      af1839d7
    • Mel Gorman's avatar
      mm: numa: limit scope of lock for NUMA migrate rate limiting · 1c5e9c27
      Mel Gorman authored
      NUMA migrate rate limiting protects a migration counter and window using
      a lock but in some cases this can be a contended lock.  It is not
      critical that the number of pages be perfect, lost updates are
      acceptable.  Reduce the importance of this lock.
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Alex Thorlton <athorlton@sgi.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      1c5e9c27
    • Mel Gorman's avatar
      mm: numa: make NUMA-migrate related functions static · 1c30e017
      Mel Gorman authored
      numamigrate_update_ratelimit and numamigrate_isolate_page only have
      callers in mm/migrate.c.  This patch makes them static.
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Alex Thorlton <athorlton@sgi.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      1c30e017
    • Xishi Qiu's avatar
      lib/show_mem.c: show num_poisoned_pages when oom · 25487d73
      Xishi Qiu authored
      Show num_poisoned_pages when oom, it is a little helpful to find the
      reason.  Also it will be emitted anytime show_mem() is called.
      Signed-off-by: default avatarXishi Qiu <qiuxishi@huawei.com>
      Suggested-by: default avatarNaoya Horiguchi <n-horiguchi@ah.jp.nec.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Acked-by: default avatarKOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      25487d73