Warning message in case of linking between paRAM slots in different eDMA
controllers.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

edma_write_slot() is for writing an entire paRAM slot.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

We have access to dev, so it is better to use the dev_dbg for debug prints.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Be consistent and do not mix the use of dev, &pdev->dev, etc in the
functions.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

When allocating a memory for number of items it is better (looks better)
to use devm_kcalloc.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Instead of using defines to specify the size of different arrays and
bitmaps, allocate the memory for them based on the information we get from
the HW itself.
Since these defines are set based on the worst case, there are devices
where they are not valid.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Move the code out from arch/arm/common and merge it inside of the dmaengine
driver.
This change is done with as minimal (if eny) functional change to the code
as possible to avoid introducing regression.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The upcoming change to merge the arch/arm/common/edma.c into
drivers/dma/edma.c will need this change when booting daVinci devices in
no DT mode.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Sekhar Nori <nsekhar@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Convert the eDMA platform device creation to use
struct platform_device_info XXXXXX __initconst and
platform_device_register_full()
This will allow us to cleanly specify the dma_mask for the devices in an
upcoming patch.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Sekhar Nori <nsekhar@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Since the driver stack no longer depends on lookup with id number in a
global array of pointers, the limitation for the number of eDMAs are no
longer needed. We can handle as many eDMAs in legacy and DT boot as we have
memory for them to allocate the needed structures.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Instead of relying on indexes pointing to edma private date in the global
pointer array, pass the private data pointer via the public API.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Merge the iomem into the 'struct edma' and change the internal (static)
functions to use pointer to the edma_cc instead of the ctlr number.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

If the of_dma_controller is registered in the non dmaengine driver we could
have race condition:
the of_dma_controller has been registered, but the dmaengine driver is not
yet probed. Drivers requesting DMA channels during this window will fail
since we do not yet have dmaengine drivers registered.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Currently we have one device created to handle all (maximum 2) eDMAs in the
system.
With this change all eDMA instance will have it's own device/driver.
This change is needed for further cleanups in the eDMA driver stack since
the one device/driver to handle all eDMAs in the system was not flexible
enough and prevents the upcoming work.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Sekhar Nori <nsekhar@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The code path in edma_execute() and edma_callback() can be simplified
and make it more optimal.
There is not need to call in to edma_execute() when the transfer
has been finished for example.
Also the handling of missed/first or next batch of paRAMs can
be done in a more optimal way.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

We no longer have users for these functions so they can be removed.
Remove also unused enums from the header file.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

In case when the interrupt happened for the second eDMA the channel
number was incorrectly passed to the client driver.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
CC: <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Pull CRIS updates from Jesper Nilsson:
 "Mostly removal of old cruft of which we can use a generic version, or
  fixes for code not commonly run in the cris port, but also additions
  to enable some good debug"

* tag 'cris-for-4.3' of git://git.kernel.org/pub/scm/linux/kernel/git/jesper/cris: (25 commits)
  CRISv10: delete unused lib/dmacopy.c
  CRISv10: delete unused lib/old_checksum.c
  CRIS: fix switch_mm() lockdep splat
  CRISv32: enable LOCKDEP_SUPPORT
  CRIS: add STACKTRACE_SUPPORT
  CRISv32: annotate irq enable in idle loop
  CRISv32: add support for irqflags tracing
  CRIS: UAPI: use generic types.h
  CRIS: UAPI: use generic shmbuf.h
  CRIS: UAPI: use generic msgbuf.h
  CRIS: UAPI: use generic socket.h
  CRIS: UAPI: use generic sembuf.h
  CRIS: UAPI: use generic sockios.h
  CRIS: UAPI: use generic auxvec.h
  CRIS: UAPI: use generic headers via Kbuild
  CRIS: UAPI: fix elf.h export
  CRIS: don't make asm/elf.h depend on asm/user.h
  CRIS: UAPI: fix ptrace.h
  CRISv32: Squash compile warnings for axisflashmap
  CRISv32: Add GPIO driver to the default configs
  ...

rq_data_dir() returns either READ or WRITE (0 == READ, 1 == WRITE), not
a boolean value.

Now, admittedly the "!= 0" doesn't really change the value (0 stays as
zero, 1 stays as one), but it's not only redundant, it confuses gcc, and
causes gcc to warn about the construct

    switch (rq_data_dir(req)) {
        case READ:
            ...
        case WRITE:
            ...

that we have in a few drivers.

Now, the gcc warning is silly and stupid (it seems to warn not about the
switch value having a different type from the case statements, but about
_any_ boolean switch value), but in this case the code itself is silly
and stupid too, so let's just change it, and get rid of warnings like
this:

  drivers/block/hd.c: In function ‘hd_request’:
  drivers/block/hd.c:630:11: warning: switch condition has boolean value [-Wswitch-bool]
     switch (rq_data_dir(req)) {

The odd '!= 0' came in when "cmd_flags" got turned into a "u64" in
commit 5953316d ("block: make rq->cmd_flags be 64-bit") and is
presumably because the old code (that just did a logical 'and' with 1)
would then end up making the type of rq_data_dir() be u64 too.

But if we want to retain the old regular integer type, let's just cast
the result to 'int' rather than use that rather odd '!= 0'.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Fix up the writeback plugging introduced in commit d353d758
("writeback: plug writeback at a high level") that then caused problems
due to the unplug happening with a spinlock held.

* writeback-plugging:
  writeback: plug writeback in wb_writeback() and writeback_inodes_wb()
  Revert "writeback: plug writeback at a high level"

We had to revert the pluggin in writeback_sb_inodes() because the
wb->list_lock is held, but we could easily plug at a higher level before
taking that lock, and unplug after releasing it.  This does that.

Chris will run performance numbers, just to verify that this approach is
comparable to the alternative (we could just drop and re-take the lock
around the blk_finish_plug() rather than these two commits.

I'd have preferred waiting for actual performance numbers before picking
one approach over the other, but I don't want to release rc1 with the
known "sleeping function called from invalid context" issue, so I'll
pick this cleanup version for now.  But if the numbers show that we
really want to plug just at the writeback_sb_inodes() level, and we
should just play ugly games with the spinlock, we'll switch to that.

Cc: Chris Mason <clm@fb.com>
Cc: Josef Bacik <jbacik@fb.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Neil Brown <neilb@suse.de>
Cc: Jan Kara <jack@suse.cz>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

I didn't notice this when merging the thermal code from Zhang, but his
merge (commit 5a924a07: "Merge branches 'thermal-core' and
'thermal-intel' of .git into next") of the thermal-core and
thermal-intel branches was wrong.

In thermal-core, commit 17e8351a ("thermal: consistently use int for
temperatures") converted the thermal layer to use "int" for
temperatures.

But in parallel, in the thermal-intel branch commit d0a12625
("thermal: Add Intel PCH thermal driver") added support for the intel
PCH thermal sensor using the old interfaces that used "unsigned long"
pointers.

This resulted in warnings like this:

  drivers/thermal/intel_pch_thermal.c:184:14: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
    .get_temp = pch_thermal_get_temp,
                ^
  drivers/thermal/intel_pch_thermal.c:184:14: note: (near initialization for ‘tzd_ops.get_temp’)
  drivers/thermal/intel_pch_thermal.c:186:19: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
    .get_trip_temp = pch_get_trip_temp,
                     ^
  drivers/thermal/intel_pch_thermal.c:186:19: note: (near initialization for ‘tzd_ops.get_trip_temp’)

This fixes it.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Merge fourth patch-bomb from Andrew Morton:

 - sys_membarier syscall

 - seq_file interface changes

 - a few misc fixups

* emailed patches from Andrew Morton <akpm@linux-foundation.org>:
  revert "ocfs2/dlm: use list_for_each_entry instead of list_for_each"
  mm/early_ioremap: add explicit #include of asm/early_ioremap.h
  fs/seq_file: convert int seq_vprint/seq_printf/etc... returns to void
  selftests: enhance membarrier syscall test
  selftests: add membarrier syscall test
  sys_membarrier(): system-wide memory barrier (generic, x86)
  MODSIGN: fix a compilation warning in extract-cert

Newer bitfiles needs the reduced clk even for SMP builds

Cc: <stable@vger.kernel.org>  #4.2
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Pull NTB fixes from Jon Mason:
 "NTB bug and documentation fixes, new device IDs, performance
  improvements, and adding a mailing list to MAINTAINERS for NTB"

* tag 'ntb-4.3' of git://github.com/jonmason/ntb:
  NTB: Fix range check on memory window index
  NTB: Improve index handling in B2B MW workaround
  NTB: Fix documentation for ntb_peer_db_clear.
  NTB: Fix documentation for ntb_link_is_up
  NTB: Use unique DMA channels for TX and RX
  NTB: Remove dma_sync_wait from ntb_async_rx
  NTB: Clean up QP stats info
  NTB: Make the transport list in order of discovery
  NTB: Add PCI Device IDs for Broadwell Xeon
  NTB: Add flow control to the ntb_netdev
  NTB: Add list to MAINTAINERS

Pull more input updates from Dmitry Torokhov:
 "Second round of updates for the input subsystem.

  This introduces two brand new touchscreen drivers (Colibri and
  imx6ul_tsc), some small driver fixes, and we are no longer report
  errors from evdev_flush() as users do not really have a way of
  handling errors, error codes that we were returning were not on the
  list of errors supposed to be returned by close(), and errors were
  causing issues with one of older versions of systemd"

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input:
  Input: imx_keypad - remove obsolete comment
  Input: touchscreen - add imx6ul_tsc driver support
  Input: Add touchscreen support for Colibri VF50
  Input: i8042 - lower log level for "no controller" message
  Input: evdev - do not report errors form flush()
  Input: elants_i2c - extend the calibration timeout to 12 seconds
  Input: sparcspkr - fix module autoload for OF platform drivers
  Input: regulator-haptic - fix module autoload for OF platform driver
  Input: pwm-beeper - fix module autoload for OF platform driver
  Input: ab8500-ponkey - Fix module autoload for OF platform driver
  Input: cyttsp - remove unnecessary MODULE_ALIAS()
  Input: elan_i2c - add ACPI ID "ELAN1000"

Pull more power management and ACPI updates from Rafael Wysocki:
 "These are mostly fixes and cleanups on top of the previous PM+ACPI
  pull request (cpufreq core and drivers, cpuidle, generic power domains
  framework).  Some of them didn't make to that pull request and some
  fix issues introduced by it.

  The only really new thing is the support for suspend frequency in the
  cpufreq-dt driver, but it is needed to fix an issue with Exynos
  platforms.

  Specifics:

   - build fix for the new Mediatek MT8173 cpufreq driver (Guenter
     Roeck).

   - generic power domains framework fixes (power on error code path,
     subdomain removal) and cleanup of a deprecated API user (Geert
     Uytterhoeven, Jon Hunter, Ulf Hansson).

   - cpufreq-dt driver fixes including two fixes for bugs related to the
     new Operating Performance Points Device Tree bindings introduced
     recently (Viresh Kumar).

   - suspend frequency support for the cpufreq-dt driver (Bartlomiej
     Zolnierkiewicz, Viresh Kumar).

   - cpufreq core cleanups (Viresh Kumar).

   - intel_pstate driver fixes (Chen Yu, Kristen Carlson Accardi).

   - additional sanity check in the cpuidle core (Xunlei Pang).

   - fix for a comment related to CPU power management (Lina Iyer)"

* tag 'pm+acpi-4.3-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
  intel_pstate: fix PCT_TO_HWP macro
  intel_pstate: Fix user input of min/max to legal policy region
  PM / OPP: Return suspend_opp only if it is enabled
  cpufreq-dt: add suspend frequency support
  cpufreq: allow cpufreq_generic_suspend() to work without suspend frequency
  PM / OPP: add dev_pm_opp_get_suspend_opp() helper
  staging: board: Migrate away from __pm_genpd_name_add_device()
  cpufreq: Use __func__ to print function's name
  cpufreq: staticize cpufreq_cpu_get_raw()
  PM / Domains: Ensure subdomain is not in use before removing
  cpufreq: Add ARM_MT8173_CPUFREQ dependency on THERMAL
  cpuidle/coupled: Add sanity check for safe_state_index
  PM / Domains: Try power off masters in error path of __pm_genpd_poweron()
  cpufreq: dt: Tolerance applies on both sides of target voltage
  cpufreq: dt: Print error on failing to mark OPPs as shared
  cpufreq: dt: Check OPP count before marking them shared
  kernel/cpu_pm: fix cpu_cluster_pm_exit comment

Pull SCSI target updates from Nicholas Bellinger:
 "Here are the outstanding target-pending updates for v4.3-rc1.

  Mostly bug-fixes and minor changes this round.  The fallout from the
  big v4.2-rc1 RCU conversion have (thus far) been minimal.

  The highlights this round include:

   - Move sense handling routines into scsi_common code (Sagi)

   - Return ABORTED_COMMAND sense key for PI errors (Sagi)

   - Add tpg_enabled_sendtargets attribute for disabled iscsi-target
     discovery (David)

   - Shrink target struct se_cmd by rearranging fields (Roland)

   - Drop iSCSI use of mutex around max_cmd_sn increment (Roland)

   - Replace iSCSI __kernel_sockaddr_storage with sockaddr_storage (Andy +
     Chris)

   - Honor fabric max_data_sg_nents I/O transfer limit (Arun + Himanshu +
     nab)

   - Fix EXTENDED_COPY >= v4.1 regression OOPsen (Alex + nab)"

* 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending: (37 commits)
  target: use stringify.h instead of own definition
  target/user: Fix UFLAG_UNKNOWN_OP handling
  target: Remove no-op conditional
  target/user: Remove unused variable
  target: Fix max_cmd_sn increment w/o cmdsn mutex regressions
  target: Attach EXTENDED_COPY local I/O descriptors to xcopy_pt_sess
  target/qla2xxx: Honor max_data_sg_nents I/O transfer limit
  target/iscsi: Replace __kernel_sockaddr_storage with sockaddr_storage
  target/iscsi: Replace conn->login_ip with login_sockaddr
  target/iscsi: Keep local_ip as the actual sockaddr
  target/iscsi: Fix np_ip bracket issue by removing np_ip
  target: Drop iSCSI use of mutex around max_cmd_sn increment
  qla2xxx: Update tcm_qla2xxx module description to 24xx+
  iscsi-target: Add tpg_enabled_sendtargets for disabled discovery
  drivers: target: Drop unlikely before IS_ERR(_OR_NULL)
  target: check DPO/FUA usage for COMPARE AND WRITE
  target: Shrink struct se_cmd by rearranging fields
  target: Remove cmd->se_ordered_id (unused except debug log lines)
  target: add support for START_STOP_UNIT SCSI opcode
  target: improve unsupported opcode message
  ...

Pull second round of SCSI updates from James Bottomley:
 "There's one late arriving patch here (added today), fixing a build
  issue which the scsi_dh patch set in here uncovered.  Other than that,
  everything has been incubated in -next and the checkers for a week.

  The major pieces of this patch are a set patches facilitating better
  integration between scsi and scsi_dh (the device handling layer used
  by multi-path; all the dm parts are acked by Mike Snitzer).

  This also includes driver updates for mp3sas, scsi_debug and an
  assortment of bug fixes"

* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (50 commits)
  scsi_dh: fix randconfig build error
  scsi: fix scsi_error_handler vs. scsi_host_dev_release race
  fcoe: Convert use of __constant_htons to htons
  mpt2sas: setpci reset kernel oops fix
  pm80xx: Don't override ts->stat on IO_OPEN_CNX_ERROR_HW_RESOURCE_BUSY
  lpfc: Fix possible use-after-free and double free in lpfc_mbx_cmpl_rdp_page_a2()
  bfa: Fix incorrect de-reference of pointer
  bfa: Fix indentation
  scsi_transport_sas: Remove check for SAS expander when querying bay/enclosure IDs.
  scsi_debug: resp_request: remove unused variable
  scsi_debug: fix REPORT LUNS Well Known LU
  scsi_debug: schedule_resp fix input variable check
  scsi_debug: make dump_sector static
  scsi_debug: vfree is null safe so drop the check
  scsi_debug: use SCSI_W_LUN_REPORT_LUNS instead of SAM2_WLUN_REPORT_LUNS;
  scsi_debug: define pr_fmt() for consistent logging
  mpt2sas: Refcount fw_events and fix unsafe list usage
  mpt2sas: Refcount sas_device objects and fix unsafe list usage
  scsi_dh: return SCSI_DH_NOTCONN in scsi_dh_activate()
  scsi_dh: don't allow to detach device handlers at runtime
  ...

Pull media updates from Mauro Carvalho Chehab:
 "A series of patches that move part of the code used to allocate memory
  from the media subsystem to the mm subsystem"

[ The mm parts have been acked by VM people, and the series was
  apparently in -mm for a while   - Linus ]

* tag 'media/v4.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media:
  [media] drm/exynos: Convert g2d_userptr_get_dma_addr() to use get_vaddr_frames()
  [media] media: vb2: Remove unused functions
  [media] media: vb2: Convert vb2_dc_get_userptr() to use frame vector
  [media] media: vb2: Convert vb2_vmalloc_get_userptr() to use frame vector
  [media] media: vb2: Convert vb2_dma_sg_get_userptr() to use frame vector
  [media] vb2: Provide helpers for mapping virtual addresses
  [media] media: omap_vout: Convert omap_vout_uservirt_to_phys() to use get_vaddr_pfns()
  [media] mm: Provide new get_vaddr_frames() helper
  [media] vb2: Push mmap_sem down to memops

Pull edac updates from Mauro Carvalho Chehab:
 "Two EDAC fixes for Intel systems (Haswell and Ivy Bridge)"

* tag 'edac/v4.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-edac:
  sb_edac: correctly fetch DIMM width on Ivy Bridge and Haswell
  sb_edac: look harder for DDRIO on Haswell systems

Pull thermal updates from Zhang Rui:

 - use int instead of unsigned long to represent temperature to avoid
   bogus overheat detection when negative temperature reported.  From
   Sascha Hauer.

 - export available thermal governors information to user space via
   sysfs.  From Wei Ni.

 - introduce new thermal driver for Wildcat Point platform controller
   hub, which uses PCH thermal sensor and associated critical and hot
   trip points.  From Tushar Dave.

 - add suuport for Intel Skylake and Denlow platforms in powerclamp
   driver.

 - some small cleanups in thermal core.

* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/rzhang/linux:
  thermal: Add Intel PCH thermal driver
  thermal: Add comment explaining test for critical temperature
  thermal: Use IS_ENABLED instead of #ifdef
  thermal: remove unnecessary call to thermal_zone_device_set_polling
  thermal: trivial: fix typo in comment
  thermal: consistently use int for temperatures
  thermal: add available policies sysfs attribute
  thermal/powerclamp: add cpu id for denlow platform
  thermal/powerclamp: add cpu id for Skylake u/y
  thermal/powerclamp: add cpu id for skylake h/s

Revert commit f83c7b5e ("ocfs2/dlm: use list_for_each_entry instead
of list_for_each").

list_for_each_entry() will dereference its `pos' argument, which can be
NULL in dlm_process_recovery_data().
Reported-by: Julia Lawall <julia.lawall@lip6.fr>
Reported-by: Fengguang Wu <fengguang.wu@gmail.com>
Cc: Joseph Qi <joseph.qi@huawei.com>
Cc: Mark Fasheh <mfasheh@suse.de>
Cc: Joel Becker <jlbec@evilplan.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Commit 6b0f68e3 ("mm: add utility for early copy from unmapped ram")
introduces a function copy_from_early_mem() into mm/early_ioremap.c
which itself calls early_memremap()/early_memunmap().  However, since
early_memunmap() has not been declared yet at this point in the .c file,
nor by any explicitly included header files, we are depending on a
transitive include of asm/early_ioremap.h to declare it, which is
fragile.

So instead, include this header explicitly.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Mark Salter <msalter@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

The seq_<foo> function return values were frequently misused.

See: commit 1f33c41c ("seq_file: Rename seq_overflow() to
     seq_has_overflowed() and make public")

All uses of these return values have been removed, so convert the
return types to void.

Miscellanea:

o Move seq_put_decimal_<type> and seq_escape prototypes closer the
  other seq_vprintf prototypes
o Reorder seq_putc and seq_puts to return early on overflow
o Add argument names to seq_vprintf and seq_printf
o Update the seq_escape kernel-doc
o Convert a couple of leading spaces to tabs in seq_escape
Signed-off-by: Joe Perches <joe@perches.com>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Update the membarrier syscall self-test to match the membarrier
interface.  Extend coverage of the interface.  Consider ENOSYS as a
"SKIP" test, since it is a valid configuration, but does not allow
testing the system call.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Pranith Kumar <bobby.prani@gmail.com>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Add a self test for the membarrier system call.
Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Here is an implementation of a new system call, sys_membarrier(), which
executes a memory barrier on all threads running on the system.  It is
implemented by calling synchronize_sched().  It can be used to
distribute the cost of user-space memory barriers asymmetrically by
transforming pairs of memory barriers into pairs consisting of
sys_membarrier() and a compiler barrier.  For synchronization primitives
that distinguish between read-side and write-side (e.g.  userspace RCU
[1], rwlocks), the read-side can be accelerated significantly by moving
the bulk of the memory barrier overhead to the write-side.

The existing applications of which I am aware that would be improved by
this system call are as follows:

* Through Userspace RCU library (http://urcu.so)
  - DNS server (Knot DNS) https://www.knot-dns.cz/
  - Network sniffer (http://netsniff-ng.org/)
  - Distributed object storage (https://sheepdog.github.io/sheepdog/)
  - User-space tracing (http://lttng.org)
  - Network storage system (https://www.gluster.org/)
  - Virtual routers (https://events.linuxfoundation.org/sites/events/files/slides/DPDK_RCU_0MQ.pdf)
  - Financial software (https://lkml.org/lkml/2015/3/23/189)

Those projects use RCU in userspace to increase read-side speed and
scalability compared to locking.  Especially in the case of RCU used by
libraries, sys_membarrier can speed up the read-side by moving the bulk of
the memory barrier cost to synchronize_rcu().

* Direct users of sys_membarrier
  - core dotnet garbage collector (https://github.com/dotnet/coreclr/issues/198)

Microsoft core dotnet GC developers are planning to use the mprotect()
side-effect of issuing memory barriers through IPIs as a way to implement
Windows FlushProcessWriteBuffers() on Linux.  They are referring to
sys_membarrier in their github thread, specifically stating that
sys_membarrier() is what they are looking for.

To explain the benefit of this scheme, let's introduce two example threads:

Thread A (non-frequent, e.g. executing liburcu synchronize_rcu())
Thread B (frequent, e.g. executing liburcu
rcu_read_lock()/rcu_read_unlock())

In a scheme where all smp_mb() in thread A are ordering memory accesses
with respect to smp_mb() present in Thread B, we can change each
smp_mb() within Thread A into calls to sys_membarrier() and each
smp_mb() within Thread B into compiler barriers "barrier()".

Before the change, we had, for each smp_mb() pairs:

Thread A                    Thread B
previous mem accesses       previous mem accesses
smp_mb()                    smp_mb()
following mem accesses      following mem accesses

After the change, these pairs become:

Thread A                    Thread B
prev mem accesses           prev mem accesses
sys_membarrier()            barrier()
follow mem accesses         follow mem accesses

As we can see, there are two possible scenarios: either Thread B memory
accesses do not happen concurrently with Thread A accesses (1), or they
do (2).

1) Non-concurrent Thread A vs Thread B accesses:

Thread A                    Thread B
prev mem accesses
sys_membarrier()
follow mem accesses
                            prev mem accesses
                            barrier()
                            follow mem accesses

In this case, thread B accesses will be weakly ordered. This is OK,
because at that point, thread A is not particularly interested in
ordering them with respect to its own accesses.

2) Concurrent Thread A vs Thread B accesses

Thread A                    Thread B
prev mem accesses           prev mem accesses
sys_membarrier()            barrier()
follow mem accesses         follow mem accesses

In this case, thread B accesses, which are ensured to be in program
order thanks to the compiler barrier, will be "upgraded" to full
smp_mb() by synchronize_sched().

* Benchmarks

On Intel Xeon E5405 (8 cores)
(one thread is calling sys_membarrier, the other 7 threads are busy
looping)

1000 non-expedited sys_membarrier calls in 33s =3D 33 milliseconds/call.

* User-space user of this system call: Userspace RCU library

Both the signal-based and the sys_membarrier userspace RCU schemes
permit us to remove the memory barrier from the userspace RCU
rcu_read_lock() and rcu_read_unlock() primitives, thus significantly
accelerating them. These memory barriers are replaced by compiler
barriers on the read-side, and all matching memory barriers on the
write-side are turned into an invocation of a memory barrier on all
active threads in the process. By letting the kernel perform this
synchronization rather than dumbly sending a signal to every process
threads (as we currently do), we diminish the number of unnecessary wake
ups and only issue the memory barriers on active threads. Non-running
threads do not need to execute such barrier anyway, because these are
implied by the scheduler context switches.

Results in liburcu:

Operations in 10s, 6 readers, 2 writers:

memory barriers in reader:    1701557485 reads, 2202847 writes
signal-based scheme:          9830061167 reads,    6700 writes
sys_membarrier:               9952759104 reads,     425 writes
sys_membarrier (dyn. check):  7970328887 reads,     425 writes

The dynamic sys_membarrier availability check adds some overhead to
the read-side compared to the signal-based scheme, but besides that,
sys_membarrier slightly outperforms the signal-based scheme. However,
this non-expedited sys_membarrier implementation has a much slower grace
period than signal and memory barrier schemes.

Besides diminishing the number of wake-ups, one major advantage of the
membarrier system call over the signal-based scheme is that it does not
need to reserve a signal. This plays much more nicely with libraries,
and with processes injected into for tracing purposes, for which we
cannot expect that signals will be unused by the application.

An expedited version of this system call can be added later on to speed
up the grace period. Its implementation will likely depend on reading
the cpu_curr()->mm without holding each CPU's rq lock.

This patch adds the system call to x86 and to asm-generic.

[1] http://urcu.so

membarrier(2) man page:

MEMBARRIER(2)              Linux Programmer's Manual             MEMBARRIER(2)

NAME
       membarrier - issue memory barriers on a set of threads

SYNOPSIS
       #include <linux/membarrier.h>

       int membarrier(int cmd, int flags);

DESCRIPTION
       The cmd argument is one of the following:

       MEMBARRIER_CMD_QUERY
              Query  the  set  of  supported commands. It returns a bitmask of
              supported commands.

       MEMBARRIER_CMD_SHARED
              Execute a memory barrier on all threads running on  the  system.
              Upon  return from system call, the caller thread is ensured that
              all running threads have passed through a state where all memory
              accesses  to  user-space  addresses  match program order between
              entry to and return from the system  call  (non-running  threads
              are de facto in such a state). This covers threads from all pro=E2=80=90
              cesses running on the system.  This command returns 0.

       The flags argument needs to be 0. For future extensions.

       All memory accesses performed  in  program  order  from  each  targeted
       thread is guaranteed to be ordered with respect to sys_membarrier(). If
       we use the semantic "barrier()" to represent a compiler barrier forcing
       memory  accesses  to  be performed in program order across the barrier,
       and smp_mb() to represent explicit memory barriers forcing full  memory
       ordering  across  the barrier, we have the following ordering table for
       each pair of barrier(), sys_membarrier() and smp_mb():

       The pair ordering is detailed as (O: ordered, X: not ordered):

                              barrier()   smp_mb() sys_membarrier()
              barrier()          X           X            O
              smp_mb()           X           O            O
              sys_membarrier()   O           O            O

RETURN VALUE
       On success, these system calls return zero.  On error, -1 is  returned,
       and errno is set appropriately. For a given command, with flags
       argument set to 0, this system call is guaranteed to always return the
       same value until reboot.

ERRORS
       ENOSYS System call is not implemented.

       EINVAL Invalid arguments.

Linux                             2015-04-15                     MEMBARRIER(2)
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Nicholas Miell <nmiell@comcast.net>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Alan Cox <gnomes@lxorguk.ukuu.org.uk>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Stephen Hemminger <stephen@networkplumber.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: David Howells <dhowells@redhat.com>
Cc: Pranith Kumar <bobby.prani@gmail.com>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Fix the following warning when compiling extract-cert:

  scripts/extract-cert.c: In function `write_cert':
  scripts/extract-cert.c:89:2: warning: format not a string literal and no format arguments [-Wformat-security]
    ERR(!i2d_X509_bio(wb, x509), cert_dst);
    ^

whereby the ERR() macro is taking cert_dst as the format string.  "%s"
should be used as the format string as the path could contain special
characters.
Signed-off-by: David Howells <dhowells@redhat.com>
Reported-by: Jim Davis <jim.epost@gmail.com>
Acked-by : David Woodhouse <david.woodhouse@intel.com>
Cc: James Morris <jmorris@namei.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>