1. 28 Jul, 2011 31 commits
    • NeilBrown's avatar
      md/raid10: avoid writing to known bad blocks on known bad drives. · d4432c23
      NeilBrown authored
      Writing to known bad blocks on drives that have seen a write error
      is asking for trouble.  So try to avoid these blocks.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      d4432c23
    • NeilBrown's avatar
      md/raid10 record bad blocks as needed during recovery. · e875ecea
      NeilBrown authored
      When recovering one or more devices, if all the good devices have
      bad blocks we should record a bad block on the device being rebuilt.
      
      If this fails, we need to abort the recovery.
      
      To ensure we don't think that we aborted later than we actually did,
      we need to move the check for MD_RECOVERY_INTR earlier in md_do_sync,
      in particular before mddev->curr_resync is updated.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      e875ecea
    • NeilBrown's avatar
      md/raid10: avoid reading known bad blocks during resync/recovery. · 40c356ce
      NeilBrown authored
      During resync/recovery limit the size of the request to avoid
      reading into a bad block that does not start at-or-before the current
      read address.
      
      Similarly if there is a bad block at this address, don't allow the
      current request to extend beyond the end of that bad block.
      
      Now that we don't ever read from known bad blocks, it is safe to allow
      devices with those blocks into the array.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      40c356ce
    • NeilBrown's avatar
      md/raid10 - avoid reading from known bad blocks - part 3 · 8dbed5ce
      NeilBrown authored
      When attempting to repair a read error, don't read from
      devices with a known bad block.
      
      As we are only reading PAGE_SIZE blocks, we don't try to
      narrow down to smaller regions in the hope that only part of this
      page is bad - it isn't worth the effort.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      8dbed5ce
    • NeilBrown's avatar
      md/raid10: avoid reading from known bad blocks - part 2 · 7399c31b
      NeilBrown authored
      When redirecting a read error to a different device, we must
      again avoid bad blocks and possibly split the request.
      
      Spin_lock typo fixed thanks to Dan Carpenter <error27@gmail.com>
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      7399c31b
    • NeilBrown's avatar
      md/raid10: avoid reading from known bad blocks - part 1 · 856e08e2
      NeilBrown authored
      This patch just covers the basic read path:
       1/ read_balance needs to check for badblocks, and return not only
          the chosen slot, but also how many good blocks are available
          there.
       2/ read submission must be ready to issue multiple reads to
          different devices as different bad blocks on different devices
          could mean that a single large read cannot be served by any one
          device, but can still be served by the array.
          This requires keeping count of the number of outstanding requests
          per bio.  This count is stored in 'bi_phys_segments'
      
      On read error we currently just fail the request if another target
      cannot handle the whole request.  Next patch refines that a bit.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      856e08e2
    • NeilBrown's avatar
      md/raid10: Split handle_read_error out from raid10d. · 560f8e55
      NeilBrown authored
      raid10d() is too big and is about to get bigger, so split
      handle_read_error() out as a separate function.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      560f8e55
    • NeilBrown's avatar
      md/raid10: simplify/reindent some loops. · 1294b9c9
      NeilBrown authored
      When a loop ends with a large if, it can be neater to change the
      if to invert the condition and just 'continue'.
      Then the body of the if can be indented to a lower level.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      1294b9c9
    • NeilBrown's avatar
      md/raid5: Clear bad blocks on successful write. · b84db560
      NeilBrown authored
      On a successful write to a known bad block, flag the sh
      so that raid5d can remove the known bad block from the list.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      b84db560
    • NeilBrown's avatar
      md/raid5. Don't write to known bad block on doubtful devices. · 73e92e51
      NeilBrown authored
      If a device has seen write errors, don't write to any known
      bad blocks on that device.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      73e92e51
    • NeilBrown's avatar
      md/raid5: write errors should be recorded as bad blocks if possible. · bc2607f3
      NeilBrown authored
      When a write error is detected, don't mark the device as failed
      immediately but rather record the fact for handle_stripe to deal with.
      
      Handle_stripe then attempts to record a bad block.  Only if that fails
      does the device get marked as faulty.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      bc2607f3
    • NeilBrown's avatar
      md/raid5: use bad-block log to improve handling of uncorrectable read errors. · 7f0da59b
      NeilBrown authored
      If we get an uncorrectable read error - record a bad block rather than
      failing the device.
      And if these errors (which may be due to known bad blocks) cause
      recovery to be impossible, record a bad block on the recovering
      devices, or abort the recovery.
      
      As we might abort a recovery without failing a device we need to teach
      RAID5 about recovery_disabled handling.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      7f0da59b
    • NeilBrown's avatar
      md/raid5: avoid reading from known bad blocks. · 31c176ec
      NeilBrown authored
      There are two times that we might read in raid5:
      1/ when a read request fits within a chunk on a single
         working device.
         In this case, if there is any bad block in the range of
         the read, we simply fail the cache-bypass read and
         perform the read though the stripe cache.
      
      2/ when reading into the stripe cache.  In this case we
         mark as failed any device which has a bad block in that
         strip (1 page wide).
         Note that we will both avoid reading and avoid writing.
         This is correct (as we will never read from the block, there
         is no point writing), but not optimal (as writing could 'fix'
         the error) - that will be addressed later.
      
      If we have not seen any write errors on the device yet, we treat a bad
      block like a recent read error.  This will encourage an attempt to fix
      the read error which will either generate a write error, or will
      ensure good data is stored there.  We don't yet forget the bad block
      in that case.  That comes later.
      
      Now that we honour bad blocks when reading we can allow devices with
      bad blocks into the array.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      31c176ec
    • NeilBrown's avatar
      md/raid1: factor several functions out or raid1d() · 62096bce
      NeilBrown authored
      raid1d is too big with several deep branches.
      So separate them out into their own functions.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      62096bce
    • NeilBrown's avatar
      md/raid1: improve handling of read failure during recovery. · 3a9f28a5
      NeilBrown authored
      If we cannot read a block from anywhere during recovery, there is
      now a better approach than just giving up.
      We can record a bad block on each device and keep going - being
      careful not to clear the bad block when a write succeeds as it might -
      it will be a write of incorrect data.
      
      We have now reached the state where - for raid1 - we only call
      md_error if md_set_badblocks has failed.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      3a9f28a5
    • NeilBrown's avatar
      md/raid1: record badblocks found during resync etc. · d8f05d29
      NeilBrown authored
      If we find a bad block while writing as part of resync/recovery we
      need to report that back to raid1d which must record the bad block,
      or fail the device.
      
      Similarly when fixing a read error, a further error should just
      record a bad block if possible rather than failing the device.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      d8f05d29
    • NeilBrown's avatar
      md/raid1: Handle write errors by updating badblock log. · cd5ff9a1
      NeilBrown authored
      When we get a write error (in the data area, not in metadata),
      update the badblock log rather than failing the whole device.
      
      As the write may well be many blocks, we trying writing each
      block individually and only log the ones which fail.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      cd5ff9a1
    • NeilBrown's avatar
      md/raid1: store behind-write pages in bi_vecs. · 2ca68f5e
      NeilBrown authored
      When performing write-behind we allocate pages to store the data
      during write.
      Previously we just keep a list of pages.  Now we keep a list of
      bi_vec which includes offset and size.
      This means that the r1bio has complete information to create a new
      bio which will be needed for retrying after write errors.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      2ca68f5e
    • NeilBrown's avatar
      md/raid1: clear bad-block record when write succeeds. · 4367af55
      NeilBrown authored
      If we succeed in writing to a block that was recorded as
      being bad, we clear the bad-block record.
      
      This requires some delayed handling as the bad-block-list update has
      to happen in process-context.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      4367af55
    • NeilBrown's avatar
      md/raid1: avoid writing to known-bad blocks on known-bad drives. · 1f68f0c4
      NeilBrown authored
      If we have seen any write error on a drive, then don't write to
      any known-bad blocks on that drive.
      If necessary, we divide the write request up into pieces just
      like we do for reads, so each piece is either all written or
      all not written to any given drive.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      1f68f0c4
    • Namhyung Kim's avatar
      md: update documentation for md/rdev/state sysfs interface · 0b7d8386
      Namhyung Kim authored
      Previous patches in the bad block series extended behavior of
      rdev's 'state' interface but lacked documentation update.
      Fix it.
      Signed-off-by: default avatarNamhyung Kim <namhyung@gmail.com>
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      0b7d8386
    • NeilBrown's avatar
      md: make it easier to wait for bad blocks to be acknowledged. · de393cde
      NeilBrown authored
      It is only safe to choose not to write to a bad block if that bad
      block is safely recorded in metadata - i.e. if it has been
      'acknowledged'.
      
      If it hasn't we need to wait for the acknowledgement.
      
      We support that using rdev->blocked wait and
      md_wait_for_blocked_rdev by introducing a new device flag
      'BlockedBadBlock'.
      
      This flag is only advisory.
      It is cleared whenever we acknowledge a bad block, so that a waiter
      can re-check the particular bad blocks that it is interested it.
      
      It should be set by a caller when they find they need to wait.
      This (set after test) is inherently racy, but as
      md_wait_for_blocked_rdev already has a timeout, losing the race will
      have minimal impact.
      
      When we clear "Blocked" was also clear "BlockedBadBlocks" incase it
      was set incorrectly (see above race).
      
      We also modify the way we manage 'Blocked' to fit better with the new
      handling of 'BlockedBadBlocks' and to make it consistent between
      externally managed and internally managed metadata.   This requires
      that each raidXd loop checks if the metadata needs to be written and
      triggers a write (md_check_recovery) if needed.  Otherwise a queued
      write request might cause raidXd to wait for the metadata to write,
      and only that thread can write it.
      
      Before writing metadata, we set FaultRecorded for all devices that
      are Faulty, then after writing the metadata we clear Blocked for any
      device for which the Fault was certainly Recorded.
      
      The 'faulty' device flag now appears in sysfs if the device is faulty
      *or* it has unacknowledged bad blocks.  So user-space which does not
      understand bad blocks can continue to function correctly.
      User space which does, should not assume a device is faulty until it
      sees the 'faulty' flag, and then sees the list of unacknowledged bad
      blocks is empty.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      de393cde
    • NeilBrown's avatar
      md: add 'write_error' flag to component devices. · d7a9d443
      NeilBrown authored
      If a device has ever seen a write error, we will want to handle
      known-bad-blocks differently.
      So create an appropriate state flag and export it via sysfs.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      d7a9d443
    • NeilBrown's avatar
      md/raid1: avoid reading known bad blocks during resync · 06f60385
      NeilBrown authored
      When performing resync/etc, keep the size of the request
      small enough that it doesn't overlap any known bad blocks.
      Devices with badblocks at the start of the request are completely
      excluded.
      If there is nowhere to read from due to bad blocks, record
      a bad block on each target device.
      
      Now that we never read from known-bad-blocks we can allow devices with
      known-bad-blocks into a RAID1.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      06f60385
    • NeilBrown's avatar
      md/raid1: avoid reading from known bad blocks. · d2eb35ac
      NeilBrown authored
      Now that we have a bad block list, we should not read from those
      blocks.
      There are several main parts to this:
        1/ read_balance needs to check for bad blocks, and return not only
           the chosen device, but also how many good blocks are available
           there.
        2/ fix_read_error needs to avoid trying to read from bad blocks.
        3/ read submission must be ready to issue multiple reads to
           different devices as different bad blocks on different devices
           could mean that a single large read cannot be served by any one
           device, but can still be served by the array.
           This requires keeping count of the number of outstanding requests
           per bio.  This count is stored in 'bi_phys_segments'
        4/ retrying a read needs to also be ready to submit a smaller read
           and queue another request for the rest.
      
      This does not yet handle bad blocks when reading to perform resync,
      recovery, or check.
      
      'md_trim_bio' will also be used for RAID10, so put it in md.c and
      export it.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      d2eb35ac
    • NeilBrown's avatar
      md: Disable bad blocks and v0.90 metadata. · 9f2f3830
      NeilBrown authored
      v0.90 metadata cannot record bad blocks, so when loading metadata
      for such a device, set shift to -1.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      9f2f3830
    • NeilBrown's avatar
      md: load/store badblock list from v1.x metadata · 2699b672
      NeilBrown authored
      Space must have been allocated when array was created.
      A feature flag is set when the badblock list is non-empty, to
      ensure old kernels don't load and trust the whole device.
      
      We only update the on-disk badblocklist when it has changed.
      If the badblocklist (or other metadata) is stored on a bad block, we
      don't cope very well.
      
      If metadata has no room for bad block, flag bad-blocks as disabled,
      and do the same for 0.90 metadata.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      2699b672
    • NeilBrown's avatar
      md: don't allow arrays to contain devices with bad blocks. · 34b343cf
      NeilBrown authored
      As no personality understand bad block lists yet, we must
      reject any device that is known to contain bad blocks.
      As the personalities get taught, these tests can be removed.
      
      This only applies to raid1/raid5/raid10.
      For linear/raid0/multipath/faulty the whole concept of bad blocks
      doesn't mean anything so there is no point adding the checks.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      34b343cf
    • Namhyung Kim's avatar
      md: add documentation for bad block log · 6e0d2d03
      Namhyung Kim authored
      Previous patch in the bad block series added new sysfs interfaces
      ([unacknowledged_]bad_blocks) for each rdev without documentation.
      Add it.
      Signed-off-by: default avatarNamhyung Kim <namhyung@gmail.com>
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      6e0d2d03
    • NeilBrown's avatar
      md/bad-block-log: add sysfs interface for accessing bad-block-log. · 16c791a5
      NeilBrown authored
      This can show the log (providing it fits in one page) and
      allows bad blocks to be 'acknowledged' meaning that they
      have safely been recorded in metadata.
      
      Clearing bad blocks is not allowed via sysfs (except for
      code testing).  A bad block can only be cleared when
      a write to the block succeeds.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      16c791a5
    • NeilBrown's avatar
      md: beginnings of bad block management. · 2230dfe4
      NeilBrown authored
      This the first step in allowing md to track bad-blocks per-device so
      that we can fail individual blocks rather than the whole device.
      
      This patch just adds a data structure for recording bad blocks, with
      routines to add, remove, search the list.
      Signed-off-by: default avatarNeilBrown <neilb@suse.de>
      Reviewed-by: default avatarNamhyung Kim <namhyung@gmail.com>
      2230dfe4
  2. 27 Jul, 2011 9 commits