Once we have determined that we will not have all of our desired msix
vectors there is no point in attempting a single msix allocation.  The
driver will already need to read registers to determine the source of
the interrupt the fact that it is msix is moot.  Fallback directly to
msi.
Reported-by: Alexander Gordeev <agordeev@redhat.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

With 24 disks and an ioatdma instance with 16 source support there is a
corner case where the driver needs to be careful to account for the
number of implied sources in the continuation case.

Also bump the default case to test more than 16 sources now that it
triggers different paths in offload drivers.

Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use a single cache for all sed allocations.  No need to make it per
channel.  This also avoids the slub_debug warnings for multiple caches
with the same name.

Switching to dmam_pool_create() to fix leaking the dma pools on
initialization failure and lets us kill ioat3_dma_remove().

Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

When performing continuations there are implied sources that need to be
added to the source count. Quoting dma_set_maxpq:

/* dma_maxpq - reduce maxpq in the face of continued operations
 * @dma - dma device with PQ capability
 * @flags - to check if DMA_PREP_CONTINUE and DMA_PREP_PQ_DISABLE_P are set
 *
 * When an engine does not support native continuation we need 3 extra
 * source slots to reuse P and Q with the following coefficients:
 * 1/ {00} * P : remove P from Q', but use it as a source for P'
 * 2/ {01} * Q : use Q to continue Q' calculation
 * 3/ {00} * Q : subtract Q from P' to cancel (2)
 *
 * In the case where P is disabled we only need 1 extra source:
 * 1/ {01} * Q : use Q to continue Q' calculation
 */

...fix the selection of the 16 source path to take these implied sources
into account.

Note this also kills the BUG_ON(src_cnt < 9) check in
__ioat3_prep_pq16_lock().  Besides not accounting for implied sources
the check is redundant given we already made the path selection.

Cc: <stable@vger.kernel.org>
Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

The array to lookup the sed pool based on the number of sources
(pq16_idx_to_sedi) is 16 entries and expects a max source index.
However, we pass the total source count which runs off the end of the
array when src_cnt == 16.  The minimal fix is to just pass src_cnt-1,
but given we know the source count is > 8 we can just calculate the sed
pool by (src_cnt - 2) >> 3.

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: <stable@vger.kernel.org>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Commit 48a9db46 (3.11) removed the memset op in the xor selftest for ioatdma.
The issue is that with the removal of that op, it never replaced the memset
with a CPU memset. The memory being operated on is expected to be zeroes but
was not. This is causing the xor selftest to fail.

Cc: <stable@vger.kernel.org>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Verbose mode turns on test success messages, by default we only output
test summaries and failure results.

Also cleaned up some stray quotes, leftover from putting the result
message format string all on one line.

Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Remove the open coded unmap and add coverage for this core functionality
to dmatest.

Also fixes up a couple places where we leaked dma mappings.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Allows for scripting test runs by module load / unload.  Prevent module
load from returning until 'iterations' (finite) tests have completed, or
cause reads of the 'wait' parameter in sysfs to pause until the tests
are done.

Also killed the local waitqueue since we can just let the thread exit
naturally as long as we hold a reference.

Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Add iops and throughput to the summary output.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Towards enabling dmatest to checkout performance add a 'noverify' mode.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

There is no need for dmatest to drain the entropy pool.

It would be nice to one day have repeatable runs, but would need a
larger rework to synchronize and order calls to the rng across test
threads.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Currently we only test raid channels that happen to also have 'copy'
capability.  Search for capable channels that do not have DMA_MEMCPY.

Note the return value from run_threaded_test never really made sense
because it could return errors after successfully starting tests.  We
already have the test results per channel so missing channels can be
detected at that time.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

1/ move 'run' control to a module parameter so we can do:
   modprobe dmatest run=1.  With this moved the rest of the debugfs
   boilerplate can go.

2/ Fix parameter initialization.  Previously the test was being started
   without taking the parameters into account in the built-in case.

Also killed off the '__' version of some routines.  The new rule is just
hold the lock when calling a *threaded_test() routine.
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

...now that we have a common pr_fmt.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

For long running tests the tracking results in a memory leak for the "ok"
results, and for the failures the kernel log should be sufficient.  Provide a
uniform format for error messages so they can be easily parsed and remove the
debugfs file.

Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

This reverts commit d86b2f29.

The kernel log buffer is sufficient for collecting test results.  The
current logging OOMs the machine on long running tests, and usually only
the first error is relevant.  It is better to stop on error and parse
the kernel output.  If output volume becomes an issue we can always
investigate using trace messages.

Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Remove no longer needed DMA unmap flags:
- DMA_COMPL_SKIP_SRC_UNMAP
- DMA_COMPL_SKIP_DEST_UNMAP
- DMA_COMPL_SRC_UNMAP_SINGLE
- DMA_COMPL_DEST_UNMAP_SINGLE

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Jon Mason <jon.mason@intel.com>
Acked-by: Mark Brown <broonie@linaro.org>
[djbw: clean up straggling skip unmap flags in ntb]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Remove support for DMA unmapping from drivers as it is no longer
needed (DMA core code is now handling it).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
[djbw: fix up chan2parent() unused warning in drivers/dma/dw/core.c]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Jon Mason <jon.mason@intel.com>
[djbw: fix up unmap len, and GFP flags]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: keep temporary dma_dest array in do_async_gen_syndrome()]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: keep temporary dma_dest array in async_mult()]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: minor cleanups]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Later we can push this unmap object up to the raid layer and get rid of
the 'scribble' parameter.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: minor cleanups]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: add missing unmap->len initialization]
[bzolnier: fix whitespace damage]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
[djbw: add DMA_ENGINE=n support]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Hang a common 'unmap' object off of dma descriptors for the purpose of
providing a unified unmapping interface.  The lifetime of a mapping may
span multiple descriptors, so these unmap objects are reference counted
by related descriptor.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: fix IS_ENABLED() check]
[bzolnier: fix release ordering in dmaengine_destroy_unmap_pool()]
[bzolnier: fix check for success in dmaengine_init_unmap_pool()]
[bzolnier: use mempool_free() instead of kmem_cache_free()]
[bzolnier: add missing unmap->len initializations]
[bzolnier: add __init tag to dmaengine_init_unmap_pool()]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
[djbw: move DMAENGINE=n support to this patch for async_tx]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Add a hook for a common dma unmap implementation to enable removal of
the per driver custom unmap code.  (A reworked version of Bartlomiej
Zolnierkiewicz's patches to remove the custom callbacks and the size
increase of dma_async_tx_descriptor for drivers that don't care about
raid).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: prepare pl330 driver for adding missing unmap while at it]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Copying from page to page (dma_async_memcpy_pg_to_pg) is the superset,
make the other two apis use that one in preparation for providing a
common dma unmap implementation.  The common implementation just wants
to assume all buffers are mapped with dma_map_page().

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Make the driver DMA unmap also source buffers by itself
(currently it DMA unmaps only destination buffers) as
a preparation for introducing generic 'ummap' data.

Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Pull btrfs fix from Chris Mason:
 "Sage hit a deadlock with ceph on btrfs, and Josef tracked it down to a
  regression in our initial rc1 pull.  When doing nocow writes we were
  sometimes starting a transaction with locks held"

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs:
  Btrfs: release path before starting transaction in can_nocow_extent

Pull ACPI and power management fixes from Rafael Wysocki:

 - intel_pstate fix for misbehavior after system resume if sysfs
   attributes are set in a specific way before the corresponding suspend
   from Dirk Brandewie.

 - A recent intel_pstate fix has no effect if unsigned long is 32-bit,
   so fix it up to cover that case as well.

 - The s3c64xx cpufreq driver was not updated when the index field of
   struct cpufreq_frequency_table was replaced with driver_data, so
   update it now.  From Charles Keepax.

 - The Kconfig help text for ACPI_BUTTON still refers to
   /proc/acpi/event that has been dropped recently, so modify it to
   remove that reference.  From Krzysztof Mazur.

 - A Lan Tianyu's change adds a missing mutex unlock to an error code
   path in acpi_resume_power_resources().

 - Some code related to ACPI power resources, whose very purpose is
   questionable to put it lightly, turns out to cause problems to happen
   during testing on real systems, so remove it completely (we may
   revisit that in the future if there's a compelling enough reason).
   From Rafael J Wysocki and Aaron Lu.

* tag 'pm+acpi-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
  ACPI / PM: Drop two functions that are not used any more
  ATA / ACPI: remove power dependent device handling
  cpufreq: s3c64xx: Rename index to driver_data
  ACPI / power: Drop automaitc resume of power resource dependent devices
  intel_pstate: Fix type mismatch warning
  cpufreq / intel_pstate: Fix max_perf_pct on resume
  ACPI: remove /proc/acpi/event from ACPI_BUTTON help
  ACPI / power: Release resource_lock after acpi_power_get_state() return error

Pull x86 fixes from Ingo Molnar:
 "Two fixlets:

   - fix a (rare-config) build bug
   - fix a next-gen SGI/UV hw/firmware enumeration bug"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86: Update UV3 hub revision ID
  x86/microcode: Correct Kconfig dependencies

We can't be holding tree locks while we try to start a transaction, we will
deadlock.  Thanks,
Reported-by: Sage Weil <sage@inktank.com>
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>

* acpi-fixes:
  ACPI / PM: Drop two functions that are not used any more
  ATA / ACPI: remove power dependent device handling
  ACPI / power: Drop automaitc resume of power resource dependent devices
  ACPI: remove /proc/acpi/event from ACPI_BUTTON help
  ACPI / power: Release resource_lock after acpi_power_get_state() return error

* pm-fixes:
  cpufreq: s3c64xx: Rename index to driver_data
  intel_pstate: Fix type mismatch warning
  cpufreq / intel_pstate: Fix max_perf_pct on resume

Pull CIFS fixes from Steve French:
 "Five small cifs fixes (includes fixes for: unmount hang, 2 security
  related, symlink, large file writes)"

* 'for-linus' of git://git.samba.org/sfrench/cifs-2.6:
  cifs: ntstatus_to_dos_map[] is not terminated
  cifs: Allow LANMAN auth method for servers supporting unencapsulated authentication methods
  cifs: Fix inability to write files >2GB to SMB2/3 shares
  cifs: Avoid umount hangs with smb2 when server is unresponsive
  do not treat non-symlink reparse points as valid symlinks

Merge tag 'driver-core-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core

Pull driver core fix from Greg KH:
 "Here is one fix for the hotplug memory path that resolves a regression
  when removing memory that showed up in 3.12-rc1"

* tag 'driver-core-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core:
  driver core: Release device_hotplug_lock when store_mem_state returns EINVAL

Pull USB fixes from Greg KH:
 "Here are some USB fixes and new device ids for 3.12-rc6

  The largest change here is a bunch of new device ids for the option
  USB serial driver for new Huawei devices.  Other than that, just some
  small bug fixes for issues that people have reported (run-time and
  build-time), nothing major"

* tag 'usb-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
  usb: usb_phy_gen: refine conditional declaration of usb_nop_xceiv_register
  usb: misc: usb3503: Fix compile error due to incorrect regmap depedency
  usb/chipidea: fix oops on memory allocation failure
  usb-storage: add quirk for mandatory READ_CAPACITY_16
  usb: serial: option: blacklist Olivetti Olicard200
  USB: quirks: add touchscreen that is dazzeled by remote wakeup
  Revert "usb: musb: gadget: fix otg active status flag"
  USB: quirks.c: add one device that cannot deal with suspension
  USB: serial: option: add support for Inovia SEW858 device
  USB: serial: ti_usb_3410_5052: add Abbott strip port ID to combined table as well.
  USB: support new huawei devices in option.c
  usb: musb: start musb on the udc side, too
  xhci: Fix spurious wakeups after S5 on Haswell
  xhci: fix write to USB3_PSSEN and XUSB2PRM pci config registers
  xhci: quirk for extra long delay for S4
  xhci: Don't enable/disable RWE on bus suspend/resume.