md-cluster: A better way for METADATA_UPDATED processing

The processing of METADATA_UPDATED message is too simple and prone to
errors. Besides, it would not update the internal data structures as
required.

This set of patches reads the superblock from one of the device of the MD
and checks for changes in the in-memory data structures. If there is a change,
it performs the necessary actions to keep the internal data structures
as it would be in the primary node.

An example is if a devices turns faulty. The algorithm is:

1. The initiator node marks the device as faulty and updates the superblock
2. The initiator node sends METADATA_UPDATED with an advisory  device number to the rest of the nodes.
3. The receiving node on receiving the METADATA_UPDATED message
  3.1 Reads the superblock
  3.2 Detects a device has failed by comparing with memory structure
  3.3 Calls the necessary functions to record the failure and get the device out of the active array.
  3.4 Acknowledges the message.

The patch series also fixes adding the disk which was impacted because of
the changes.

Patches can also be found at
https://github.com/goldwynr/linux branch md-next

Changes since V2:
 - Fix status synchrnoization after --add and --re-add operations
 - Included Guoqing's patches on endian correctness, zeroing cmsg etc
 - Restructure add_new_disk() and cancel()

We shouldn't run related funs of md_cluster_ops in case
metadata_update_start returned failure.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

For cluster raid, we should not kick it from array if the disk can't be
remove from array successfully.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

During the past test, the node occasionally received the msg which is
sent from itself, this case should not happen in theory, but it is
better to avoid it in case something wrong happened.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Since slot will be set within _sendmsg, we can remove
the redundant code in resync_info_update.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

Signed-off-by: Guoqing Jiang <gqjiang@suse.com>

The receive daemon prints kernel messages for every network message
received. This would fill the kernel message log with unnecessary messages.
Remove the pr_info() messages.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Adding the disk worked incorrectly with the new reload code. Fix it:

 - No operation should be performed on rdev marked as Candidate
 - After a metadata update operation, kick disk if role is 0xfffe
   else clear Candidate bit and continue with the regular change check.
 - Saving the mode of the lock resource to check if token lock is already
   locked, because it can be called twice while adding a disk. However,
   unlock_comm() must be called only once.
 - add_new_disk() is called by the node initiating the --add operation.
   If it needs to be canceled, call add_new_disk_cancel(). The operation
   is completed by md_update_sb() which will write and unlock the
   communication.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Resync or recovery must be performed by only one node at a time.
A DLM lock resource, resync_lockres provides the mutual exclusion
so that only one node performs the recovery/resync at a time.

If a node is unable to get the resync_lockres, because recovery is
being performed by another node, it set MD_RECOVER_NEEDED so as
to schedule recovery in the future.

Remove the debug message in resync_info_update()
used during development.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

In a clustered environment, a change such as marking a device faulty,
can be recorded by any of the nodes. This is communicated to all the
nodes and re-recording such a change is unnecessary, and quite often
pretty disruptive.

With this patch, just before the update, we detect for the changes
and if the changes are already in superblock, we abort the update
after clearing all the flags
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

md_reload_sb is too simplistic and it explicitly needs to determine
the changes made by the writing node. However, there are multiple areas
where a simple reload could fail.

Instead, read the superblock of one of the "good" rdevs and update
the necessary information:

- read the superblock into a newly allocated page, by temporarily
  swapping out rdev->sb_page and calling ->load_super.
- if that fails return
- if it succeeds, call check_sb_changes
  1. iterates over list of active devices and checks the matching
   dev_roles[] value.
   	If that is 'faulty', the device must be  marked as faulty
	 - call md_error to mark the device as faulty. Make sure
	   not to set CHANGE_DEVS and wakeup mddev->thread or else
	   it would initiate a resync process, which is the responsibility
	   of the "primary" node.
	 - clear the Blocked bit
	 - Call remove_and_add_spares() to hot remove the device.
	If the device is 'spare':
	 - call remove_and_add_spares() to get the number of spares
	   added in this operation.
	 - Reduce mddev->degraded to mark the array as not degraded.
  2. reset recovery_cp
- read the rest of the rdevs to update recovery_offset. If recovery_offset
  is equal to MaxSector, call spare_active() to set it In_sync

This required that recovery_offset be initialized to MaxSector, as
opposed to zero so as to communicate the end of sync for a rdev.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

remove_and_add_spares() checks for all devices to activate spare.
Change it to activate a specific device if a non-null rdev
argument is passed.

remove_and_add_spares() can be used to activate spares in
slot_store() as well.

For hot_remove_disk(), check if rdev->raid_disk == -1 before
calling remove_and_add_spares()
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

When the suspended_area is deleted, the suspended processes
must be woken up in order to complete their I/O.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>

Previously, BITMAP_NEEDS_SYNC message is sent when the resyc
aborts, but it could abort for different reasons, and not all
of reasons require another node to take over the resync ownship.

It is better make BITMAP_NEEDS_SYNC message only be sent when
the node is leaving cluster with dirty bitmap. And we also need
to ensure dlm connection is ok.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

Suspending the entire device for resync could take too long. Resync
in small chunks.

cluster's resync window (32M) is maintained in r1conf as
cluster_sync_low and cluster_sync_high and processed in
raid1's sync_request(). If the current resync is outside the cluster
resync window:

1. Set the cluster_sync_low to curr_resync_completed.
2. Check if the sync will fit in the new window, if not issue a
   wait_barrier() and set cluster_sync_low to sector_nr.
3. Set cluster_sync_high to cluster_sync_low + resync_window.
4. Send a message to all nodes so they may add it in their suspension
   list.

bitmap_cond_end_sync is modified to allow to force a sync inorder
to get the curr_resync_completed uptodate with the sector passed.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.de>

Add BITMAP_MAJOR_CLUSTERED as 5, in order to prevent older kernels
to assemble a clustered device.

In order to maximize compatibility, the major version is set to
BITMAP_MAJOR_CLUSTERED *only* if the bitmap is clustered.

Added MD_FEATURE_CLUSTERED in order to return error for older
kernels which would assemble MD even if the bitmap is corrupted.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

process_suspend_info - which handles the RESYNCING request - must not
reply until all writes which were initiated before the request arrived,
have completed.

As a by-product, all process_* functions now take mddev as their
first arguement making it uniform.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>

Pull scheduler fix from Thomas Gleixner:
 "Fix a long standing state race in finish_task_switch()"

* 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  sched/core: Fix TASK_DEAD race in finish_task_switch()

Pull perf fix from Thomas Glexiner:
 "Fix build breakage on powerpc in perf tools"

* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf tools: Fix build break on powerpc due to sample_reg_masks

Pull maintainer email update from Thomas Gleixner:
 "Change Matt Fleming's email address in the maintainers file"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  MAINTAINERS: Change Matt Fleming's email address

Pull irq fixes from Thomas Gleixner:
 "Three trivial commits:

   - Fix a kerneldoc regression

   - Export handle_bad_irq to unbreak a driver in next

   - Add an accessor for the of_node field so refactoring in next does
     not depend on merge ordering"

* 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  irqdomain: Add an accessor for the of_node field
  genirq: Fix handle_bad_irq kerneldoc comment
  genirq: Export handle_bad_irq

Pull SCSI fixes from James Bottomley:
 "This is a set of three bug fixes, two of which are regressions from
  recent updates (the 3ware one from 4.1 and the device handler fixes
  from 4.2)"

* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
  3w-9xxx: don't unmap bounce buffered commands
  scsi_dh: Use the correct module name when loading device handler
  libiscsi: Fix iscsi_check_transport_timeouts possible infinite loop

Pull md bugfix from Neil Brown:
 "One bug fix for raid1/raid10.

  Very careless bug earler in 4.3-rc, now fixed :-)"

* tag 'md/4.3-rc4-fix' of git://neil.brown.name/md:
  crash in md-raid1 and md-raid10 due to incorrect list manipulation

My Intel email address will soon expire. Replace it with my
personal address so people still know where to send patches.
Signed-off-by: Matt Fleming <matt.fleming@intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1444494136-10333-1-git-send-email-matt@codeblueprint.co.ukSigned-off-by: Ingo Molnar <mingo@kernel.org>

Pull USB fixes from Greg KH:
 "Here are some small USB and PHY fixes and quirk updates for 4.3-rc5.

  Nothing major here, full details in the shortlog, and all of these
  have been in linux-next for a while"

* tag 'usb-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
  usb: Add device quirk for Logitech PTZ cameras
  USB: chaoskey read offset bug
  USB: Add reset-resume quirk for two Plantronics usb headphones.
  usb: renesas_usbhs: Add support for R-Car H3
  usb: renesas_usbhs: fix build warning if 64-bit architecture
  usb: gadget: bdc: fix memory leak
  phy: berlin-sata: Fix module autoload for OF platform driver
  phy: rockchip-usb: power down phy when rockchip phy probe
  phy: qcom-ufs: fix build error when the component is built as a module

Pull tty/serial driver fixes from Greg KH:
 "Here are a few bug fixes for the tty core that resolve reported
  issues, and some serial driver fixes as well (including the
  much-reported imx driver problem)

  All of these have been in linux-next with no reported problems"

* tag 'tty-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty:
  drivers/tty: require read access for controlling terminal
  serial: 8250: add uart_config entry for PORT_RT2880
  tty: fix data race on tty_buffer.commit
  tty: fix data race in tty_buffer_flush
  tty: fix data race in flush_to_ldisc
  tty: fix stall caused by missing memory barrier in drivers/tty/n_tty.c
  serial: atmel: fix error path of probe function
  tty: don't leak cdev in tty_cdev_add()
  Revert "serial: imx: remove unbalanced clk_prepare"

Pull staging fixes from Greg KH:
 "Here are two tiny staging tree fixes for 4.3-rc5.

  One fixes the broken speakup subsystem as reported by a user, and the
  other removes an entry in the MAINTAINERS file for a developer that
  doesn't want to be listed anymore"

* tag 'staging-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging:
  staging: speakup: fix speakup-r regression
  MAINTAINERS: Remove myself as nvec co-maintainer

Pull char/misc driver fixes from Greg KH:
 "Here are some small fixes for some misc drivers that resolve some
  reported issues.  All of these have been linux-next for a while"

* tag 'char-misc-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
  mcb: Fix error handling in mcb_pci_probe()
  mei: hbm: fix error in state check logic
  nvmem: sunxi: Check for memory allocation failure
  nvmem: core: Fix memory leak in nvmem_cell_write
  nvmem: core: Handle shift bits in-place if cell->nbits is non-zero
  nvmem: core: fix the out-of-range leak in read/write()

Pull MIPS fixes from Ralf Baechle:

 - MIPS didn't define the new ioremap_uc.  Defined it as an alias for
   ioremap_uncached.

 - Replace workaround for MIPS16 build issue with a correct one.

* git://git.linux-mips.org/pub/scm/ralf/upstream-linus:
  MIPS: Define ioremap_uc
  MIPS: UAPI: Ignore __arch_swab{16,32,64} when using MIPS16
  Revert "MIPS: UAPI: Fix unrecognized opcode WSBH/DSBH/DSHD when using MIPS16."

Pull swiotlb fixlet from Konrad Rzeszutek Wilk:
 "Enable the SWIOTLB under 32-bit PAE kernels.

  Nowadays most distros enable this due to CONFIG_HYPERVISOR|XEN=y which
  select SWIOTLB.  But for those that are not interested in
  virtualization and wanting to use 32-bit PAE kernels and wanting to
  have working DMA operations - this configures it for them"

* 'stable/for-linus-4.3' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/swiotlb:
  swiotlb: Enable it under x86 PAE

Leandro Awa writes:
 "After switching to version 4.1.6, our parallelized and distributed
  workflows now fail consistently with errors of the form:

  T34: ./regex.c:39:22: error: config.h: No such file or directory

  From our 'git bisect' testing, the following commit appears to be the
  possible cause of the behavior we've been seeing: commit 766c4cbf"

Al Viro says:
 "What happens is that 766c4cbf got the things subtly wrong.

  We used to treat d_is_negative() after lookup_fast() as "fall with
  ENOENT".  That was wrong - checking ->d_flags outside of ->d_seq
  protection is unreliable and failing with hard error on what should've
  fallen back to non-RCU pathname resolution is a bug.

  Unfortunately, we'd pulled the test too far up and ran afoul of
  another kind of staleness.  The dentry might have been absolutely
  stable from the RCU point of view (and we might be on UP, etc), but
  stale from the remote fs point of view.  If ->d_revalidate() returns
  "it's actually stale", dentry gets thrown away and the original code
  wouldn't even have looked at its ->d_flags.

  What we need is to check ->d_flags where 766c4cbf does (prior to
  ->d_seq validation) but only use the result in cases where we do not
  discard this dentry outright"
Reported-by: Leandro Awa <lawa@nvidia.com>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=104911
Fixes: 766c4cbf ("namei: d_is_negative() should be checked...")
Tested-by: Leandro Awa <lawa@nvidia.com>
Cc: stable@vger.kernel.org # v4.1+
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Pull power management and ACPI fixes from Rafael Wysocki:
 "These are four fixes for bugs in the devfreq and cpufreq subsystems,
  including two regression fixes (one for a recent regression and one
  for a problem introduced in 4.2).

  Specifics:

   - Two fixes for cpufreq regressions, an acpi-cpufreq driver one
     introduced during the 4.2 cycle when we started to preserve cpufreq
     directories for offline CPUs and a general one introduced recently
     (Srinivas Pandruvada).

   - Two devfreq fixes, one for a double kfree() in an error code path
     and one for a confusing sysfs-related failure (Geliang Tang, Tobias
     Jakobi)"

* tag 'pm+acpi-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
  cpufreq: prevent lockup on reading scaling_available_frequencies
  cpufreq: acpi_cpufreq: prevent crash on reading freqdomain_cpus
  PM / devfreq: fix double kfree
  PM / devfreq: Fix governor_store()

Pull strscpy powerpc fix from Chris Metcalf.

Fix powerpc big-endian build.

* 'strscpy' of git://git.kernel.org/pub/scm/linux/kernel/git/cmetcalf/linux-tile:
  arch/powerpc: provide zero_bytemask() for big-endian

Pull sound fixes from Takashi Iwai:
 "We see various small fixes, but nothing looks too scary, all are small
  gentle bug fixes:

   - Most of changes are for ASoC codecs: Realtek, SGTL5000, TAS2552,
     TLV320, WM8962

   - A couple of dwc and imx-ssi fixes

   - Usual oneliner HD-audio quirks

   - An old emux synth code fix"

* tag 'sound-4.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound:
  MAINTAINERS: Remove wm97xx entry
  ASoC: tas2552: fix dBscale-min declaration
  ALSA: synth: Fix conflicting OSS device registration on AWE32
  ALSA: hda - Disable power_save_node for IDT 92HD73xx chips
  ALSA: hda - Apply SPDIF pin ctl to MacBookPro 12,1
  ALSA: hda: Add dock support for ThinkPad T550
  ASoC: dwc: fix dma stop transferring issue
  ASoC: dwc: correct irq clear method
  ASoC: tlv320aic3x: Prevent writing reserved registers on tlv320aic3104 CODECs
  ASoC: rt5645: Correct the naming and setting of ADC Boost Volume Control
  ASoC: sgtl5000: fix wrong register MIC_BIAS_VOLTAGE setup on probe
  ASoC: wm8962: balance pm_runtime_enable
  ASoC: imx-ssi: Fix DAI hardware signal inversions
  ASoC: sgtl5000: fix error message output for MicBias voltage
  ASoC: db1200: Fix DAI link format for db1300 and db1550

Pull clk fixes from Stephen Boyd:
 "A Samsung driver fix and a handful of TI driver fixes"

* tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
  clk: ti: dflt: fix enable_reg validity check
  clk: ti: fix dual-registration of uart4_ick
  clk: ti: clk-7xx: Remove hardwired ABE clock configuration
  clk: samsung: fix cpu clock's flags checking

Pull MMC fixes from Ulf Hansson:
 "MMC host:
   - omap_hsmmc: Fix boot regressions for omap4430
   - omap_hsmmc: Fix legacy boot regression for omap3
   - sdhci-pxav3: Fix some clock issues for Armada 38x
   - sdhci-pxav3: Fix error handling at probe
   - sdhci-of-at91: Fix clock stabilization problem"

* tag 'mmc-v4.3-rc4' of git://git.linaro.org/people/ulf.hansson/mmc:
  mmc: sdhci-of-at91: use SDHCI_QUIRK2_NEED_DELAY_AFTER_INT_CLK_RST quirk
  mmc: sdhci: add quirk SDHCI_QUIRK2_NEED_DELAY_AFTER_INT_CLK_RST
  mmc: sdhci-pxav3: fix error handling of armada_38x_quirks
  mmc: sdhci-pxav3: disable clock inversion for HS MMC cards
  mmc: sdhci-pxav3: remove broken clock base quirk for Armada 38x sdhci driver
  mmc: host: omap_hsmmc: Fix MMC for omap3 legacy booting
  Revert "mmc: host: omap_hsmmc: use regulator_is_enabled to find pbias status"

Pull dm fixes from Mike Snitzer:
 "Three stable fixes:

   - DM core AB-BA deadlock fix in the device destruction path (vs
     device creation's DM table swap).

   - DM raid fix to properly round up the region_size to the next
     power-of-2.

   - DM cache fix for a NULL pointer seen while switching from the
     "cleaner" cache policy.

  Two fixes for regressions introduced during the 4.3 merge:

   - request-based DM error propagation regressed due to incorrect
     changes introduced when adding the bi_error field to bio.

   - DM snapshot fix to only support snapshots that overflow if the
     client (e.g. lvm2) is prepared to deal with the associated
     snapshot status interface change"

* tag 'dm-4.3-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm:
  dm snapshot: add new persistent store option to support overflow
  dm cache: fix NULL pointer when switching from cleaner policy
  dm: fix request-based dm error reporting
  dm raid: fix round up of default region size
  dm: fix AB-BA deadlock in __dm_destroy()