- 22 Oct, 2010 6 commits
-
-
Lars Ellenberg authored
If we don't rate limit it, and you happen to log err level messages via serial console, an IO error on a disconnected Primary may cause serious unresponsiveness. Signed-off-by:
Philipp Reisner <philipp.reisner@linbit.com> Signed-off-by:
Lars Ellenberg <lars.ellenberg@linbit.com>
-
Lars Ellenberg authored
This codepath used to be called only for failed kmalloc GFP_ATOMIC, but is now also triggered by other things. Signed-off-by:
-
Lars Ellenberg authored
If we get an IO-error during an activity log transaction, if we failed to write the bitmap of the evicted extent, we must not write the transaction itself. If we failed to write the transaction, we must not even submit the corresponding bio, as its extent is not yet marked in the activity log. Otherwise, if this was a disconneted Primary (degraded cluster), which now lost its disk as well, and we later re-attach the same backend storage, we possibly "forget" to resync some parts of the disk that potentially have been changed. On the receiving side, when receiving from a peer with unhealthy disk, checking for pdsk == D_DISKLESS is not enough, we need to set out of sync and do AL transactions for everything pdsk < D_INCONSISTENT on the receiving side. Signed-off-by:
-
Lars Ellenberg authored
If we have contention in drbd_al_begin_iod (heavy randon IO), an administrative request to detach the disk may deadlock for similar reasons as the recently fixed deadlock if detaching because of IO-error. The approach taken here is to either go through the intermediate cleanup state D_FAILED, or first lock out application io, don't just go directly to D_DISKLESS. We need an additional state bit (WAS_IO_ERROR) to distinguish the -> D_FAILED because of IO-error from other failures. Sanitize D_ATTACHING -> D_FAILED to D_ATTACHING -> D_DISKLESS. If only attaching, ldev may be missing still, but would be referenced from within the after_state_ch for -> D_FAILED, potentially dereferencing a NULL pointer. Signed-off-by:
-
Lars Ellenberg authored
If those messages ever get logged, clearly state that they are actually failed ASSERTS, so our regression tests can pick them up from the logs more easily. Signed-off-by:
-
Lars Ellenberg authored
Every code path changing the current UUID needs to get it on stable storage anyways. Flush it to disk right there, remove the now obsolte explicit drbd_md_sync statements in the other code paths. Signed-off-by:
-
- 19 Oct, 2010 1 commit
-
-
Mike Miller authored
cciss: fix PCI IDs for new controllers This patch fixes the botched up PCI IDs of new controllers. Please consider this patch for inclusion. Signed-off-by:
Mike Miller <mike.miller@hp.com> Signed-off-by:
Jens Axboe <jaxboe@fusionio.com>
-
- 15 Oct, 2010 3 commits
-
-
Lars Ellenberg authored
This adds a necessary race breaker to these commits: drbd: fix for possible deadlock on IO error during resync drbd: drop wrong debug asserts, fix recently introduced race What we do is get a refcount, check the state, then depending on the state and the requested minimum disk state, either hold it (success), or give it back immediately (failed "try lock"). Some code paths (flushing of drbd metadata) may still grab and hold a refcount even if we are D_FAILED (application IO won't). So even if we hit local_cnt == 0 once after being D_FAILED, we still need to wait for that again after we changed to D_DISKLESS. Once local_cnt reaches 0 while we are D_DISKLESS, we can be sure that no one will look at the protected members anymore, so only then is it safe to free them. We cannot easily convert to standard locking primitives here, as we want to be able to use it in atomic context (we always do a "try lock"), as well as hold references for a "long time" (from IO submission to completion callback). Signed-off-by: -
Lars Ellenberg authored
Signed-off-by:
-
Philipp Reisner authored
Signed-off-by:
-
- 14 Oct, 2010 30 commits
-
-
Dan Carpenter authored
dt is unsigned so it's never less than zero. We are calculating the elapsed time, and that's never less than zero (unless there is a bug or we invent time travel). The comparison here is just to guard against divide by zero bugs. Signed-off-by:
Dan Carpenter <error27@gmail.com> Signed-off-by:
-
Lars Ellenberg authored
Consolidate the ifdef's for the debug level, accidentally the used both DEBUG and DRBD_DEBUG_MD_SYNC. Default to off. For production, we can safely reduce the grace period for this timer again the the value we used to have. Signed-off-by:
-
Lars Ellenberg authored
It sometimes may take a while for the after state change work to be scheduled, which does drbd_md_sync. At convenient places, we should do explicit drbd_md_sync to have the new state information on disk as soon as possible. Signed-off-by:
-
Lars Ellenberg authored
commit 2372c38caadeaebc68a5ee190782c2a0df01edc3 drbd: fix for possible deadlock on IO error during resync introduced a new ASSERT, which turns out to be wrong. Drop it. Also serialize the state change to D_DISKLESS with the after state change work of the -> D_FAILED transition, don't open a new race. Signed-off-by:
-
Lars Ellenberg authored
Signed-off-by:
-
Lars Ellenberg authored
As we usually update the generation UUIDs here, we should explicitly sync them to disk. So far this has been done only implicitly by related code paths. Signed-off-by:
-
Philipp Reisner authored
This might happen if on the VERIFY_S node the disk gets dropped. Although this is an cluster wide state transition, the VERIFY_T node, updates it connection state first. Then the ack packet for the cluster wide state transition travels back, and the VERIFY_S node stops to produce the P_OV_REQUEST packets. There is absolutely nothing wrong with that. Further, do not log "Can not satisfy peer's..." on the VERIFY_S node in this case, but pretend that they had equal checksum. [Bugz 327] Signed-off-by:
-
Lars Ellenberg authored
Scenario: Something (say, flush-147:0) is in drbd_al_begin_io, holding a local_cnt, waiting for the resync to make progress. Disk fails, worker in after_state_ch does drbd_rs_cancel_all, then waits for local_cnt to drop to zero. flush-147:0 is woken by drbd_rs_cancel_all, needs to write an AL transaction, and queues that on the worker. Deadlock. Fix: do not wait in the worker, have put_ldev() trigger the state change D_FAILED -> D_DISKLESS when necessary. put_ldev() cannot do the state change directly, as it may or may not already hold various spinlocks. We queue a short work instead. Signed-off-by:
-
Lars Ellenberg authored
Various cleanup paths have been incomplete, for the very unlikely case that we cannot allocate enough bios from process context when submitting on behalf of the peer or resync process. Never observed. Signed-off-by:
-
Lars Ellenberg authored
If it was an "empty" resync, the SyncSource may have already "finished" the resync and rotated the UUIDs, before noticing the connection loss (and generating a new uuid, if Primary, rotating again), while the SyncTarget did not change its uuids at all, or only got to the previous sync-uuid. This would then again lead to a full sync on next handshake (see also Bug #251). Fix: Use explicit resync finished notification even for empty resyncs, do not finish an empty resync implicitly on the SyncSource. Signed-off-by:
-
Lars Ellenberg authored
Preparation patch so more drbd_send_state() usage on the peer will not confuse drbd in receive_state(). Signed-off-by:
-
Lars Ellenberg authored
no functional change, just using full state instead of just the .conn part of it for comparisons. Signed-off-by:
-
Lars Ellenberg authored
drbd commit 17c854fea474a5eb3cfa12e4fb019e46debbc4ec drbd: receiving of big packets, for payloads between 64kByte and 4GByte introduced a new on-the-wire packet header format. We must no longer assume either format, but use the result of whatever drbd_recv_header has decoded. Signed-off-by:
-
Lars Ellenberg authored
We used to be16_to_cpu the length field in our received packet header. drbd commit 17c854fea474a5eb3cfa12e4fb019e46debbc4ec drbd: receiving of big packets, for payloads between 64kByte and 4GByte changed this, but forgot to adjust a few places where we relied on h->length being in native byte order. This broke the receiving side of the RLE compressed bitmap exchange. Signed-off-by: -
Philipp Reisner authored
This caused rs_planed to be not in sync with the content of the fifo. That in turn could cause that the resync comes to a complete halt. Signed-off-by:
-
Philipp Reisner authored
Connections through a compressing proxy might have more bits on the fly. 500MByte instead of 50MByte Signed-off-by:
-
Lars Ellenberg authored
If we release the page pointed to by md_io_tmpp, we need to zero out the pointer, too, as that may be used later to decide whether we need to allocate a new page again. Impact: a previously freed page may be used and clobbered. Depending on what that particular page is being used for meanwhile, this may result in silent data corruption of completely unrelated things. Only of concern on devices with logical_block_size != 512 byte, if you re-attach after becoming diskless once. Signed-off-by:
-
Lars Ellenberg authored
Two missing corner cases to the "maximum packet size" handshake. Signed-off-by:
-
Philipp Reisner authored
There are three ways to get IO suspended: * Loss of any access to data * Fence-peer-handler running * User requested to suspend IO Track those in different bits, so that one condition clearing its state bit does not interfere with the other two conditions. Only when the user resumes IO he overrules all three bits. The fact is hidden from the user, he sees only a single suspend bit. Signed-off-by:
-
Lars Ellenberg authored
Signed-off-by:
-
Philipp Reisner authored
Forgot to consider the max size for the resync requests. Signed-off-by:
-
Lars Ellenberg authored
If a synctarget lost connection while being WFSyncUUID, due to "state sanitizing", the attempted state change to SyncTarget looked like an "invalidate" to after_state_ch() later, thus caused a full sync on next handshake (Bug #318). drbd0: PingAck did not arrive in time. drbd0: peer( Primary -> Unknown ) conn( WFSyncUUID -> NetworkFailure ) pdsk( UpToDate -> DUnknown ) from : { cs:NetworkFailure ro:Secondary/Unknown ds:UpToDate/DUnknown r--- } to : { cs:SyncTarget ro:Secondary/Unknown ds:Inconsistent/DUnknown r--- } after sanizising, resulted in state: { cs:NetworkFailure ro:Secondary/Unknown ds:Inconsistent/DUnknown r--- } drbd0: disk( UpToDate -> Inconsistent ) Fix: don't mask state transition errors in "sanitizing", so the requested state change to SyncTarget fails, instead of being implicitly "remaped" to invalidate. Signed-off-by: -
Lars Ellenberg authored
Signed-off-by:
-
Lars Ellenberg authored
If we cannot satisfy a request (because our disk just broke), we still need to drain the payload. Or we'll get a protocol error when interpreting the payload as DRBD packet header. Signed-off-by:
-
Lars Ellenberg authored
BUG trace would look like: lc_find drbd_rs_complete_io got_OVResult drbd_asender Could be triggered by explicit, or IO-error policy based, detach during online-verify. We may only dereference mdev->resync, if we first get_ldev(), as the disk may break any time, causing mdev->resync to disappear once all ldev references have been returned. Already in flight online-verify requests or replies may still come in, which we then need to ignore. Signed-off-by:
-
Lars Ellenberg authored
Signed-off-by:
-
Philipp Reisner authored
Signed-off-by:
-
Lars Ellenberg authored
Just in case we have some pending meta data changes to sync, do it before we call our userland helper, as that may take some time, or even cause a hard reboot. Signed-off-by:
-
Lars Ellenberg authored
addendum to baa33ae4eaa4477b60af7c434c0ddd1d182c1ae7 The race: drbd_md_sync() if (!test_and_clear_bit(MD_DIRTY, &mdev->flags)) return; ==> RACE with drbd_md_mark_dirty() rearming the timer. del_timer(&mdev->md_sync_timer); Fixed by moving the del_timer before the test_and_clear_bit. Additionally only rearm the timer in drbd_md_mark_dirty, if MD_DIRTY was not already set, reduce the grace period from five to one second, and add an ifdef'ed debuging aid to find code paths missing an explicit drbd_md_sync, if any, as those are the only relevant ones for this race. Signed-off-by: -
Philipp Reisner authored
The actual race happened int the drbd_start_resync() function. Where drbd_resync_finished() -> __drbd_set_state() set STOP_SYNC_TIMER and armed the timer. If the timer fired before execution reaches the mod_timer statement at the end of drbd_start_resync() the latter would cause an unexpected call to w_make_resync_request(). Removed the STOP_SYNC_TIMER bit, and base it on the connection state. The STOP_SYNC_TIMER bit probably originates probably the time before the state engine. Signed-off-by:
-
