When we unlock in order to submit IO, the next relock event is likely to
fail if submit_bio() blocked - we shouldn't those events in our _fail
stats, since those are expected events and shouldn't cause test
failures.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This adds a debug mode where we split up the c->writes refcount into
distinct refcounts for every codepath that takes a reference, and adds
sysfs code to print the value of each ref.

This will make it easier to debug shutdown hangs due to refcount leaks.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

The btree_node_write_blocked bit was a later addition to this code,
it only mirrors the state of the b->write_blocked list (empty or
nonempty) - unfortunately, when it was added it wasn't correctly kept in
sync - oops.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This assert is popping - rarely - in the CI, this will help us track it
down from the logs.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

We need to call bch2_trans_update_max_paths() before marking the new
path as allocated, since we're not initializing it yet.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Brett Holman <bholman.devel@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

It's important that in BTREE_ITER_FILTER_SNAPSHOTS mode we always use
peek_upto() and provide an end for the interval we're searching for -
otherwise, when we hit the end of the inode the next inode be in a
different subvolume and not have any keys in the current snapshot, and
we'd iterate over arbitrarily many keys before returning one.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This fixes a (harmless) lockdep splat, due to a lock order violation in
the key cache exit path.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This adds a threshold for the maximum spin time, similar to the rwsem
code, and a flag to the lock itself indicating when we've spun too long
so other threads also refrain from spinning.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This uses the new _ip() interface to six locks and hooks it up to
btree_path->ip_allocated, when available.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This adds _ip variations of the various lock functions that allow an IP
to be passed in, which is used by lockstat.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This replaces various BUG_ON() assertions with panics that tell us where
the restart was done and the restart type.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This ensures that failure to read a btree node error is treated as a
topology error, and returns the correct error so that the topology
repair pass will be run.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This should have been resetting trans->fs_usage_deltas as well.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

On hash collision when we have to check for duplicates or incorrect
hash value, we weren't specifying a snapshot ID to iterate with.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Easy workaround for a lockdep splat - and since bch2_prt_backtrace() is
only used in debug code this is fine.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This improves io_opts() and makes it a non-inline function - it's big
enough that it probably shouldn't be.

Also, bch_io_opts no longer needs fields for whether options are
defined, so we can slim it down a bit.

We'd like to stop passing around the full bch_io_opts, but that'll be
tricky because of bch2_rebalance_add_key().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

We weren't guarding against the alloc key having an invalid data type.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Provide inline versions of some allocation functions
 - bch2_alloc_sectors_done_inlined()
 - bch2_alloc_sectors_append_ptrs_inlined()

and use them in the core IO path.

Also, inline bch2_extent_update_i_size_sectors() and
bch2_bkey_append_ptr().

In the core write path, function call overhead matters - every function
call is a jump to a new location and a potential cache miss.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This separates out the slowpath into a separate function, and inlines
bch2_alloc_v4_mut into bch2_trans_start_alloc_update(), the main place
it's called.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Now we include the return code.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

We were incorrectly retrying after a transaction restart.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

These are invalid.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

When we need to add more replicas to an extent, it might be the case
that we already have a replica on every device, but some of them are
cached.

This patch fixes a bug where we'd spin on that extent because the write
path fails to find a device we can allocate from: we allow allocating
from devices that already have cached replicas on them, and change
bch2_data_update_index_update() to drop the cached replica if needed.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

When errors=panic, we need to dump transaction updates before calling
bch2_inconsistent_error().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

We now have bch2_trans_inconsistent() which generically does the same
thing - dumps pending btree transaction updates.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This fixes a few error paths in debug code that lead to locks failing to
be dropped.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

More error code cleanup, for better error messages and debugability.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

More error code improvements - this gets us more useful error messages.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Old inode formats don't have all the fields of the current inode format:
when unpacking inodes in the current format we can thus skip zeroing out
the destination buffer, but that doesn't work on for the old formats.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

In debug mode, we now track where btree iterators and paths are
initialized/allocated - helpful in tracking down btree path overflows.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

This is something we need to do more widely: instead of bothering with
GFP_NOIO/GFP_NOFS, if we need to allocate memory while holding locks:

 - first attempt the allocation with GFP_NOWAIT
 - if that fails, drop btree locks with bch2_trans_unlock(), then
   retry with GFP_KERNEL.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

We don't need a write lock to check if a key is dirty.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

We need to take a ref on a path while we're traversing it: this fixes a
bug with paths getting reused while being traversed, in the key cache
fill code.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>