To incorporate --grow feature executed on one node, other nodes need to
acknowledge the change in number of disks. Call update_raid_disks()
to update internal data structures.

This leads to call check_reshape() -> md_allow_write() -> md_update_sb(),
this results in a deadlock. This is done so it can safely allocate memory
(which might trigger writeback which might write to raid1). This is
not required for md with a bitmap.

In the clustered case, we don't perform md_update_sb() in md_allow_write(),
but in do_md_run(). Also we disable safemode for clustered mode.

mddev->recovery_cp need not be set in check_sb_changes() because this
is required only when a node reads another node's bitmap. mddev->recovery_cp
(which is read from sb->resync_offset), is set only if mddev is in_sync.
Since we disabled safemode, in_sync is set to zero.
In a clustered environment, the MD may not be in sync because another
node could be writing to it. So make sure that in_sync is not set in
case of clustered node in __md_stop_writes().
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

The arg isn't used, so its presence is only confusing.
Signed-off-by: NeilBrown <neilb@suse.com>

It is common practice in the kernel to leave out this case.
It isn't needed and adds little if any value.
Signed-off-by: NeilBrown <neilb@suse.com>

Signed-off-by: NeilBrown <neilb@suse.com>

This patches fixes sparse warnings like incorrect type in assignment
(different base types), cast to restricted __le64.
Reported-by: kbuild test robot <fengguang.wu@intel.com>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

As cmsg.raid_slot is le32, comparing for >0 is not meaningful.

So introduce cpu-endian 'raid_slot' and only assign to cmsg.raid_slot
when we know value is valid.
Reported-by: kbuild test robot <fengguang.wu@intel.com>
Signed-off-by: NeilBrown <neilb@suse.com>

md-cluster: A better way for METADATA_UPDATED processing

The processing of METADATA_UPDATED message is too simple and prone to
errors. Besides, it would not update the internal data structures as
required.

This set of patches reads the superblock from one of the device of the MD
and checks for changes in the in-memory data structures. If there is a change,
it performs the necessary actions to keep the internal data structures
as it would be in the primary node.

An example is if a devices turns faulty. The algorithm is:

1. The initiator node marks the device as faulty and updates the superblock
2. The initiator node sends METADATA_UPDATED with an advisory  device number to the rest of the nodes.
3. The receiving node on receiving the METADATA_UPDATED message
  3.1 Reads the superblock
  3.2 Detects a device has failed by comparing with memory structure
  3.3 Calls the necessary functions to record the failure and get the device out of the active array.
  3.4 Acknowledges the message.

The patch series also fixes adding the disk which was impacted because of
the changes.

Patches can also be found at
https://github.com/goldwynr/linux branch md-next

Changes since V2:
 - Fix status synchrnoization after --add and --re-add operations
 - Included Guoqing's patches on endian correctness, zeroing cmsg etc
 - Restructure add_new_disk() and cancel()

We shouldn't run related funs of md_cluster_ops in case
metadata_update_start returned failure.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

For cluster raid, we should not kick it from array if the disk can't be
remove from array successfully.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

During the past test, the node occasionally received the msg which is
sent from itself, this case should not happen in theory, but it is
better to avoid it in case something wrong happened.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Since slot will be set within _sendmsg, we can remove
the redundant code in resync_info_update.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

The receive daemon prints kernel messages for every network message
received. This would fill the kernel message log with unnecessary messages.
Remove the pr_info() messages.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Adding the disk worked incorrectly with the new reload code. Fix it:

 - No operation should be performed on rdev marked as Candidate
 - After a metadata update operation, kick disk if role is 0xfffe
   else clear Candidate bit and continue with the regular change check.
 - Saving the mode of the lock resource to check if token lock is already
   locked, because it can be called twice while adding a disk. However,
   unlock_comm() must be called only once.
 - add_new_disk() is called by the node initiating the --add operation.
   If it needs to be canceled, call add_new_disk_cancel(). The operation
   is completed by md_update_sb() which will write and unlock the
   communication.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Resync or recovery must be performed by only one node at a time.
A DLM lock resource, resync_lockres provides the mutual exclusion
so that only one node performs the recovery/resync at a time.

If a node is unable to get the resync_lockres, because recovery is
being performed by another node, it set MD_RECOVER_NEEDED so as
to schedule recovery in the future.

Remove the debug message in resync_info_update()
used during development.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

In a clustered environment, a change such as marking a device faulty,
can be recorded by any of the nodes. This is communicated to all the
nodes and re-recording such a change is unnecessary, and quite often
pretty disruptive.

With this patch, just before the update, we detect for the changes
and if the changes are already in superblock, we abort the update
after clearing all the flags
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

md_reload_sb is too simplistic and it explicitly needs to determine
the changes made by the writing node. However, there are multiple areas
where a simple reload could fail.

Instead, read the superblock of one of the "good" rdevs and update
the necessary information:

- read the superblock into a newly allocated page, by temporarily
  swapping out rdev->sb_page and calling ->load_super.
- if that fails return
- if it succeeds, call check_sb_changes
  1. iterates over list of active devices and checks the matching
   dev_roles[] value.
   	If that is 'faulty', the device must be  marked as faulty
	 - call md_error to mark the device as faulty. Make sure
	   not to set CHANGE_DEVS and wakeup mddev->thread or else
	   it would initiate a resync process, which is the responsibility
	   of the "primary" node.
	 - clear the Blocked bit
	 - Call remove_and_add_spares() to hot remove the device.
	If the device is 'spare':
	 - call remove_and_add_spares() to get the number of spares
	   added in this operation.
	 - Reduce mddev->degraded to mark the array as not degraded.
  2. reset recovery_cp
- read the rest of the rdevs to update recovery_offset. If recovery_offset
  is equal to MaxSector, call spare_active() to set it In_sync

This required that recovery_offset be initialized to MaxSector, as
opposed to zero so as to communicate the end of sync for a rdev.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

remove_and_add_spares() checks for all devices to activate spare.
Change it to activate a specific device if a non-null rdev
argument is passed.

remove_and_add_spares() can be used to activate spares in
slot_store() as well.

For hot_remove_disk(), check if rdev->raid_disk == -1 before
calling remove_and_add_spares()
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

When the suspended_area is deleted, the suspended processes
must be woken up in order to complete their I/O.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Previously, BITMAP_NEEDS_SYNC message is sent when the resyc
aborts, but it could abort for different reasons, and not all
of reasons require another node to take over the resync ownship.

It is better make BITMAP_NEEDS_SYNC message only be sent when
the node is leaving cluster with dirty bitmap. And we also need
to ensure dlm connection is ok.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

Suspending the entire device for resync could take too long. Resync
in small chunks.

cluster's resync window (32M) is maintained in r1conf as
cluster_sync_low and cluster_sync_high and processed in
raid1's sync_request(). If the current resync is outside the cluster
resync window:

1. Set the cluster_sync_low to curr_resync_completed.
2. Check if the sync will fit in the new window, if not issue a
   wait_barrier() and set cluster_sync_low to sector_nr.
3. Set cluster_sync_high to cluster_sync_low + resync_window.
4. Send a message to all nodes so they may add it in their suspension
   list.

bitmap_cond_end_sync is modified to allow to force a sync inorder
to get the curr_resync_completed uptodate with the sector passed.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.de>

Add BITMAP_MAJOR_CLUSTERED as 5, in order to prevent older kernels
to assemble a clustered device.

In order to maximize compatibility, the major version is set to
BITMAP_MAJOR_CLUSTERED *only* if the bitmap is clustered.

Added MD_FEATURE_CLUSTERED in order to return error for older
kernels which would assemble MD even if the bitmap is corrupted.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

process_suspend_info - which handles the RESYNCING request - must not
reply until all writes which were initiated before the request arrived,
have completed.

As a by-product, all process_* functions now take mddev as their
first arguement making it uniform.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

Pull scheduler fix from Thomas Gleixner:
 "Fix a long standing state race in finish_task_switch()"

* 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  sched/core: Fix TASK_DEAD race in finish_task_switch()

Pull perf fix from Thomas Glexiner:
 "Fix build breakage on powerpc in perf tools"

* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf tools: Fix build break on powerpc due to sample_reg_masks

Pull maintainer email update from Thomas Gleixner:
 "Change Matt Fleming's email address in the maintainers file"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  MAINTAINERS: Change Matt Fleming's email address

Pull irq fixes from Thomas Gleixner:
 "Three trivial commits:

   - Fix a kerneldoc regression

   - Export handle_bad_irq to unbreak a driver in next

   - Add an accessor for the of_node field so refactoring in next does
     not depend on merge ordering"

* 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  irqdomain: Add an accessor for the of_node field
  genirq: Fix handle_bad_irq kerneldoc comment
  genirq: Export handle_bad_irq

Pull SCSI fixes from James Bottomley:
 "This is a set of three bug fixes, two of which are regressions from
  recent updates (the 3ware one from 4.1 and the device handler fixes
  from 4.2)"

* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
  3w-9xxx: don't unmap bounce buffered commands
  scsi_dh: Use the correct module name when loading device handler
  libiscsi: Fix iscsi_check_transport_timeouts possible infinite loop

Pull md bugfix from Neil Brown:
 "One bug fix for raid1/raid10.

  Very careless bug earler in 4.3-rc, now fixed :-)"

* tag 'md/4.3-rc4-fix' of git://neil.brown.name/md:
  crash in md-raid1 and md-raid10 due to incorrect list manipulation

My Intel email address will soon expire. Replace it with my
personal address so people still know where to send patches.
Signed-off-by: Matt Fleming <matt.fleming@intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1444494136-10333-1-git-send-email-matt@codeblueprint.co.ukSigned-off-by: Ingo Molnar <mingo@kernel.org>

Pull USB fixes from Greg KH:
 "Here are some small USB and PHY fixes and quirk updates for 4.3-rc5.

  Nothing major here, full details in the shortlog, and all of these
  have been in linux-next for a while"

* tag 'usb-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
  usb: Add device quirk for Logitech PTZ cameras
  USB: chaoskey read offset bug
  USB: Add reset-resume quirk for two Plantronics usb headphones.
  usb: renesas_usbhs: Add support for R-Car H3
  usb: renesas_usbhs: fix build warning if 64-bit architecture
  usb: gadget: bdc: fix memory leak
  phy: berlin-sata: Fix module autoload for OF platform driver
  phy: rockchip-usb: power down phy when rockchip phy probe
  phy: qcom-ufs: fix build error when the component is built as a module

Pull tty/serial driver fixes from Greg KH:
 "Here are a few bug fixes for the tty core that resolve reported
  issues, and some serial driver fixes as well (including the
  much-reported imx driver problem)

  All of these have been in linux-next with no reported problems"

* tag 'tty-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty:
  drivers/tty: require read access for controlling terminal
  serial: 8250: add uart_config entry for PORT_RT2880
  tty: fix data race on tty_buffer.commit
  tty: fix data race in tty_buffer_flush
  tty: fix data race in flush_to_ldisc
  tty: fix stall caused by missing memory barrier in drivers/tty/n_tty.c
  serial: atmel: fix error path of probe function
  tty: don't leak cdev in tty_cdev_add()
  Revert "serial: imx: remove unbalanced clk_prepare"

Pull staging fixes from Greg KH:
 "Here are two tiny staging tree fixes for 4.3-rc5.

  One fixes the broken speakup subsystem as reported by a user, and the
  other removes an entry in the MAINTAINERS file for a developer that
  doesn't want to be listed anymore"

* tag 'staging-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging:
  staging: speakup: fix speakup-r regression
  MAINTAINERS: Remove myself as nvec co-maintainer

Pull char/misc driver fixes from Greg KH:
 "Here are some small fixes for some misc drivers that resolve some
  reported issues.  All of these have been linux-next for a while"

* tag 'char-misc-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
  mcb: Fix error handling in mcb_pci_probe()
  mei: hbm: fix error in state check logic
  nvmem: sunxi: Check for memory allocation failure
  nvmem: core: Fix memory leak in nvmem_cell_write
  nvmem: core: Handle shift bits in-place if cell->nbits is non-zero
  nvmem: core: fix the out-of-range leak in read/write()

Pull MIPS fixes from Ralf Baechle:

 - MIPS didn't define the new ioremap_uc.  Defined it as an alias for
   ioremap_uncached.

 - Replace workaround for MIPS16 build issue with a correct one.

* git://git.linux-mips.org/pub/scm/ralf/upstream-linus:
  MIPS: Define ioremap_uc
  MIPS: UAPI: Ignore __arch_swab{16,32,64} when using MIPS16
  Revert "MIPS: UAPI: Fix unrecognized opcode WSBH/DSBH/DSHD when using MIPS16."

Pull swiotlb fixlet from Konrad Rzeszutek Wilk:
 "Enable the SWIOTLB under 32-bit PAE kernels.

  Nowadays most distros enable this due to CONFIG_HYPERVISOR|XEN=y which
  select SWIOTLB.  But for those that are not interested in
  virtualization and wanting to use 32-bit PAE kernels and wanting to
  have working DMA operations - this configures it for them"

* 'stable/for-linus-4.3' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/swiotlb:
  swiotlb: Enable it under x86 PAE

Leandro Awa writes:
 "After switching to version 4.1.6, our parallelized and distributed
  workflows now fail consistently with errors of the form:

  T34: ./regex.c:39:22: error: config.h: No such file or directory

  From our 'git bisect' testing, the following commit appears to be the
  possible cause of the behavior we've been seeing: commit 766c4cbf"

Al Viro says:
 "What happens is that 766c4cbf got the things subtly wrong.

  We used to treat d_is_negative() after lookup_fast() as "fall with
  ENOENT".  That was wrong - checking ->d_flags outside of ->d_seq
  protection is unreliable and failing with hard error on what should've
  fallen back to non-RCU pathname resolution is a bug.

  Unfortunately, we'd pulled the test too far up and ran afoul of
  another kind of staleness.  The dentry might have been absolutely
  stable from the RCU point of view (and we might be on UP, etc), but
  stale from the remote fs point of view.  If ->d_revalidate() returns
  "it's actually stale", dentry gets thrown away and the original code
  wouldn't even have looked at its ->d_flags.

  What we need is to check ->d_flags where 766c4cbf does (prior to
  ->d_seq validation) but only use the result in cases where we do not
  discard this dentry outright"
Reported-by: Leandro Awa <lawa@nvidia.com>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=104911
Fixes: 766c4cbf ("namei: d_is_negative() should be checked...")
Tested-by: Leandro Awa <lawa@nvidia.com>
Cc: stable@vger.kernel.org # v4.1+
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>