1. 19 Apr, 2021 40 commits
    • Josef Bacik's avatar
      btrfs: handle btrfs_update_reloc_root failure in prepare_to_merge · bbae13f8
      Josef Bacik authored
      btrfs_update_reloc_root will will return errors in the future, so handle
      an error properly in prepare_to_merge.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      bbae13f8
    • Josef Bacik's avatar
      btrfs: handle btrfs_update_reloc_root failure in insert_dirty_subvol · 7934133f
      Josef Bacik authored
      btrfs_update_reloc_root will will return errors in the future, so handle
      the error properly in insert_dirty_subvol.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      7934133f
    • Josef Bacik's avatar
      btrfs: change insert_dirty_subvol to return errors · ac54da6c
      Josef Bacik authored
      This will be able to return errors in the future, so change it to return
      an error and handle the errors appropriately.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ac54da6c
    • Josef Bacik's avatar
      btrfs: handle btrfs_update_reloc_root failure in commit_fs_roots · 2dd8298e
      Josef Bacik authored
      btrfs_update_reloc_root will will return errors in the future, so handle
      the error properly in commit_fs_roots.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2dd8298e
    • Josef Bacik's avatar
      btrfs: validate root::reloc_root after recording root in trans · 39200e59
      Josef Bacik authored
      If we fail to setup a root->reloc_root in a different thread that path
      will error out, however it still leaves root->reloc_root NULL but would
      still appear set up in the transaction.  Subsequent calls to
      btrfs_record_root_in_transaction would succeed without attempting to
      create the reloc root, as the transid has already been updated.
      
      Handle this case by making sure we have a root->reloc_root set after a
      btrfs_record_root_in_transaction call so we don't end up dereferencing a
      NULL pointer.
      Reported-by: default avatarZygo Blaxell <ce3g8jdj@umail.furryterror.org>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      39200e59
    • Josef Bacik's avatar
      btrfs: do proper error handling in create_reloc_root · 84c50ba5
      Josef Bacik authored
      We do memory allocations here, read blocks from disk, all sorts of
      operations that could easily fail at any given point.  Instead of
      panicing the box, simply return the error back up the chain, all callers
      at this point have proper error handling.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      84c50ba5
    • Josef Bacik's avatar
      btrfs: have proper error handling in btrfs_init_reloc_root · 00bb36a0
      Josef Bacik authored
      create_reloc_root will return errors in the future, and __add_reloc_root
      can return ENOMEM or EEXIST, so handle these errors properly.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ add comment ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      00bb36a0
    • Josef Bacik's avatar
      btrfs: return an error from btrfs_record_root_in_trans · 03a7e111
      Josef Bacik authored
      We can create a reloc root when we record the root in the trans, which
      can fail for all sorts of different reasons.  Propagate this error up
      the chain of callers.  Future patches will fix the callers of
      btrfs_record_root_in_trans() to handle the error.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarJohannes Thumshirn <johannes.thumshirn@wdc.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      03a7e111
    • Josef Bacik's avatar
      btrfs: handle record_root_in_trans failure in create_pending_snapshot · f0118cb6
      Josef Bacik authored
      record_root_in_trans can currently fail, so handle this failure
      properly.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      f0118cb6
    • Josef Bacik's avatar
      btrfs: handle record_root_in_trans failure in btrfs_record_root_in_trans · 1409e6cc
      Josef Bacik authored
      record_root_in_trans can fail currently, handle this failure properly.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      1409e6cc
    • Josef Bacik's avatar
      btrfs: handle record_root_in_trans failure in qgroup_account_snapshot · 1c442d22
      Josef Bacik authored
      record_root_in_trans can fail currently, so handle this failure
      properly.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      1c442d22
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in start_transaction · 68075ea8
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in start_transaction.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ add comment ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      68075ea8
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in relocate_tree_block · d18c7bd9
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in relocate_tree_block.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      d18c7bd9
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in create_subvol · 221581e4
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in create_subvol.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      221581e4
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in btrfs_recover_log_trees · 2002ae11
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in btrfs_recover_log_trees.
      
      This appears tricky, however we have a reference count on the
      destination root, so if this fails we need to continue on in the loop to
      make sure the proper cleanup is done.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ add comment ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2002ae11
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in btrfs_delete_subvolume · 2731f518
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in btrfs_delete_subvolume.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2731f518
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename · b0fec6fd
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in btrfs_rename.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      b0fec6fd
    • Josef Bacik's avatar
      btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename_exchange · 00aa8e87
      Josef Bacik authored
      btrfs_record_root_in_trans will return errors in the future, so handle
      the error properly in btrfs_rename_exchange.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      00aa8e87
    • Josef Bacik's avatar
      btrfs: do proper error handling in record_reloc_root_in_trans · 404bccbc
      Josef Bacik authored
      Generally speaking this shouldn't ever fail, the corresponding fs root
      for the reloc root will already be in memory, so we won't get ENOMEM
      here.
      
      However if there is no corresponding root for the reloc root then we
      could get ENOMEM when we try to allocate it or we could get ENOENT
      when we look it up and see that it doesn't exist.
      
      Convert these BUG_ON()'s into ASSERT()'s and add proper error handling
      for the case of corruption.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      404bccbc
    • Josef Bacik's avatar
      btrfs: check record_root_in_trans related failures in select_reloc_root · 92de551b
      Josef Bacik authored
      We will record the fs root or the reloc root in the trans in
      select_reloc_root.  These will actually return errors in the following
      patches, so check their return value here and return it up the stack.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      92de551b
    • Josef Bacik's avatar
      btrfs: convert BUG_ON()'s in select_reloc_root() to proper errors · 8ee66afe
      Josef Bacik authored
      We have several BUG_ON()'s in select_reloc_root() that can be tripped if
      there is an extent tree corruption.  Convert these to ASSERT()'s, because
      if we hit it during testing it really is bad, or could indicate a
      problem with the backref walking code.
      
      However if users hit these problems it generally indicates corruption,
      I've hit a few machines in the fleet that trip over these with clearly
      corrupted extent trees, so be nice and print out an error message and
      return an error instead of bringing the whole box down.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      8ee66afe
    • Josef Bacik's avatar
      btrfs: handle errors from select_reloc_root() · cbdc2ebc
      Josef Bacik authored
      Currently select_reloc_root() doesn't return an error, but followup
      patches will make it possible for it to return an error.  We do have
      proper error recovery in do_relocation however, so handle the
      possibility of select_reloc_root() having an error properly instead of
      BUG_ON(!root).
      
      I've also adjusted select_reloc_root() to return ERR_PTR(-ENOENT) if we
      don't find a root, instead of NULL, to make the error case easier to
      deal with.  I've replaced the BUG_ON(!root) with an ASSERT(0) for this
      case as it indicates we messed up the backref walking code, but it could
      also indicate corruption.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      cbdc2ebc
    • Josef Bacik's avatar
      btrfs: convert BUG_ON()'s in relocate_tree_block · 1c7bfa15
      Josef Bacik authored
      We have a couple of BUG_ON()'s in relocate_tree_block() that can be
      tripped if we have file system corruption.  Convert these to ASSERT()'s
      so developers still get yelled at when they break the backref code, but
      error out nicely for users so the whole box doesn't go down.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      1c7bfa15
    • Josef Bacik's avatar
      btrfs: convert some BUG_ON()'s to ASSERT()'s in do_relocation · ffe30dd8
      Josef Bacik authored
      A few of these are checking for correctness, and won't be triggered by
      corrupted file systems, so convert them to ASSERT() instead of BUG_ON()
      and add a comment explaining their existence.
      Reviewed-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ffe30dd8
    • Matthew Wilcox (Oracle)'s avatar
      btrfs: add and use readahead_batch_length · 32c0a6bc
      Matthew Wilcox (Oracle) authored
      Implement readahead_batch_length() to determine the number of bytes in
      the current batch of readahead pages and use it in btrfs. Also use the
      readahead_pos to get the offset.
      Signed-off-by: default avatarMatthew Wilcox (Oracle) <willy@infradead.org>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      32c0a6bc
    • Wan Jiabing's avatar
      btrfs: move forward declarations to the beginning of extent_io.h · 183ebab7
      Wan Jiabing authored
      There are two forward declarations deep in extent_io.h, move them
      to the beginning and remove the duplicate one.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarWan Jiabing <wanjiabing@vivo.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      183ebab7
    • Qu Wenruo's avatar
      btrfs: subpage: add overview comments · 894d1378
      Qu Wenruo authored
      This patch adds an overview how btrfs subpage support works:
      
      - limitations
      - behavior
      - basic implementation points
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      894d1378
    • Qu Wenruo's avatar
      btrfs: make set_btree_ioerr accept extent buffer and be subpage compatible · 5a2c6075
      Qu Wenruo authored
      Current set_btree_ioerr() only accepts @page parameter and grabs extent
      buffer from page::private.  This works fine for sector size == PAGE_SIZE
      case, but not for subpage case.
      
      Add an extra parameter, @eb, for callers to pass extent buffer to this
      function, so that subpage code can reuse this function.
      
      And also add subpage special handling to update
      btrfs_subpage::error_bitmap.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      5a2c6075
    • Qu Wenruo's avatar
      btrfs: make set/clear_extent_buffer_dirty() subpage compatible · 0d27797e
      Qu Wenruo authored
      For set_extent_buffer_dirty() to support subpage sized metadata, just
      call btrfs_page_set_dirty() to handle both cases.
      
      For clear_extent_buffer_dirty(), it needs to clear the page dirty if and
      only if all extent buffers in the page range are no longer dirty.
      Also do the same for page error.
      
      This is pretty different from the existing clear_extent_buffer_dirty()
      routine, so add a new helper function,
      clear_subpage_extent_buffer_dirty() to do this for subpage metadata.
      
      Also since the main part of clearing page dirty code is still the same,
      extract that into btree_clear_page_dirty() so that it can be utilized
      for both cases.
      
      But there is a special race between set_extent_buffer_dirty() and
      clear_extent_buffer_dirty(), where we can clear the page dirty.
      
      [POSSIBLE RACE WINDOW]
      For the race window between clear_subpage_extent_buffer_dirty() and
      set_extent_buffer_dirty(), due to the fact that we can't call
      clear_page_dirty_for_io() under subpage spin lock, we can race like
      below:
      
         T1 (eb1 in the same page)	|  T2 (eb2 in the same page)
       -------------------------------+------------------------------
       set_extent_buffer_dirty()	| clear_extent_buffer_dirty()
       |- was_dirty = false;		| |- clear_subpagE_extent_buffer_dirty()
       |				|    |- btrfs_clear_and_test_dirty()
       |				|    |  Since eb2 is the last dirty page
       |				|    |  we got:
       |				|    |  last == true;
       |				|    |
       |- btrfs_page_set_dirty()	|    |
       |  We set the page dirty and   |    |
       |  subpage dirty bitmap	|    |
       |				|    |- if (last)
       |				|    |  Since we don't have subpage lock
       |				|    |  held, now @last is no longer
       |				|    |  correct
       |				|    |- btree_clear_page_dirty()
       |				|	Now PageDirty == false, even if
       |				|       we have dirty_bitmap not zero.
       |- ASSERT(PageDirty());	|
          ^^^^ CRASH
      
      The solution here is to also lock the eb->pages[0] for subpage case of
      set_extent_buffer_dirty(), to prevent racing with
      clear_extent_buffer_dirty().
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      0d27797e
    • Qu Wenruo's avatar
      btrfs: support page uptodate assertions in subpage mode · b8f95771
      Qu Wenruo authored
      There are quite some assert checks on page uptodate in extent buffer
      write accessors.  They ensure the destination page is already uptodate.
      
      This is fine for regular sector size case, but not for subpage case, as
      for subpage we only mark the page uptodate if the page contains no hole
      and all its extent buffers are uptodate.
      
      So instead of checking PageUptodate(), for subpage case we check the
      uptodate bitmap of btrfs_subpage structure.
      
      To make the check more elegant, introduce a helper,
      assert_eb_page_uptodate() to do the check for both subpage and regular
      sector size cases.
      
      The following functions are involved:
      
      - write_extent_buffer_chunk_tree_uuid()
      - write_extent_buffer_fsid()
      - write_extent_buffer()
      - memzero_extent_buffer()
      - copy_extent_buffer()
      - extent_buffer_test_bit()
      - extent_buffer_bitmap_set()
      - extent_buffer_bitmap_clear()
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      b8f95771
    • Qu Wenruo's avatar
      btrfs: make alloc_extent_buffer() check subpage dirty bitmap · 1e5eb3d6
      Qu Wenruo authored
      In alloc_extent_buffer(), we make sure that the newly allocated page is
      never dirty.
      
      This is fine for sector size == PAGE_SIZE case, but for subpage it's
      possible that one extent buffer in the page is dirty, thus the whole
      page is marked dirty, and could cause false alert.
      
      To support subpage, call btrfs_page_test_dirty() to handle both cases.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      1e5eb3d6
    • Qu Wenruo's avatar
      btrfs: subpage: support metadata checksum calculation at write time · eca0f6f6
      Qu Wenruo authored
      Add a new helper, csum_dirty_subpage_buffers(), to iterate through all
      dirty extent buffers in one bvec.
      
      Also extract the code of calculating csum for one extent buffer into
      csum_one_extent_buffer(), so that both the existing csum_dirty_buffer()
      and the new csum_dirty_subpage_buffers() can reuse the same routine.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      eca0f6f6
    • Qu Wenruo's avatar
      btrfs: subpage: do more sanity checks on metadata page dirtying · 139e8cd3
      Qu Wenruo authored
      For btree_set_page_dirty(), we should also check the extent buffer
      sanity for subpage support.
      
      Unlike the regular sector size case, since one page can contain multiple
      extent buffers, we need to make sure there is at least one dirty extent
      buffer in the page.
      
      So this patch will iterate through the btrfs_subpage::dirty_bitmap
      to get the extent buffers, and check if any dirty extent buffer in the page
      range has EXTENT_BUFFER_DIRTY and proper refs.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      139e8cd3
    • Qu Wenruo's avatar
      btrfs: subpage: introduce helpers for writeback status · 3470da3b
      Qu Wenruo authored
      Introduces the following functions to handle subpage writeback status:
      
      - btrfs_subpage_set_writeback()
      - btrfs_subpage_clear_writeback()
      - btrfs_subpage_test_writeback()
        These helpers can only be called when the range is ensured to be
        inside the page.
      
      - btrfs_page_set_writeback()
      - btrfs_page_clear_writeback()
      - btrfs_page_test_writeback()
        These helpers can handle both regular sector size and subpage without
        problem.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      3470da3b
    • Qu Wenruo's avatar
      btrfs: subpage: introduce helpers for dirty status · d8a5713e
      Qu Wenruo authored
      Introduce the following functions to handle subpage dirty status:
      
      - btrfs_subpage_set_dirty()
      - btrfs_subpage_clear_dirty()
      - btrfs_subpage_test_dirty()
        These helpers can only be called when the range is ensured to be
        inside the page.
      
      - btrfs_page_set_dirty()
      - btrfs_page_clear_dirty()
      - btrfs_page_test_dirty()
        These helpers can handle both regular sector size and subpage without
        problem.
        Thus they would be used to replace PageDirty() related calls in
        later patches.
      
      There is one special point to note here, just like set_page_dirty() and
      clear_page_dirty_for_io(), btrfs_*page_set_dirty() and
      btrfs_*page_clear_dirty() must be called with page locked.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      d8a5713e
    • Qu Wenruo's avatar
      btrfs: remove unnecessary variable shadowing in btrfs_invalidatepage() · d239bcb8
      Qu Wenruo authored
      In btrfs_invalidatepage() we re-declare @tree variable as
      btrfs_ordered_inode_tree.
      
      Since it's only used to do the spinlock, we can grab it from inode
      directly, and remove the unnecessary declaration completely.
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      d239bcb8
    • Qu Wenruo's avatar
      btrfs: use min() to replace open-code in btrfs_invalidatepage() · ac5804eb
      Qu Wenruo authored
      In btrfs_invalidatepage() we introduce a temporary variable, new_len, to
      update ordered->truncated_len.  But we can use min() to replace it
      completely and no need for the variable.
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ac5804eb
    • Qu Wenruo's avatar
      btrfs: add sysfs interface for supported sectorsize · fc57ad8d
      Qu Wenruo authored
      Export supported sector sizes in /sys/fs/btrfs/features/supported_sectorsizes.
      
      Currently all architectures have PAGE_SIZE, There's some disparity
      between read-only and read-write support but that will be unified in the
      future so there's only one file exporting the size.
      
      The read-only support for systems with 64K pages also works for 4K
      sector size.
      
      This new sysfs interface would help eg. mkfs.btrfs to print more
      accurate warnings about potentially incompatible option combinations.
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      fc57ad8d
    • Filipe Manana's avatar
      btrfs: improve btree readahead for full send operations · ace75066
      Filipe Manana authored
      Currently a full send operation uses the standard btree readahead when
      iterating over the subvolume/snapshot btree, which despite bringing good
      performance benefits, it could be improved in a few aspects for use cases
      such as full send operations, which are guaranteed to visit every node
      and leaf of a btree, in ascending and sequential order. The limitations
      of that standard btree readahead implementation are the following:
      
      1) It only triggers readahead for leaves that are physically close
         to the leaf being read, within a 64K range;
      
      2) It only triggers readahead for the next or previous leaves if the
         leaf being read is not currently in memory;
      
      3) It never triggers readahead for nodes.
      
      So add a new readahead mode that addresses all these points and use it
      for full send operations.
      
      The following test script was used to measure the improvement on a box
      using an average, consumer grade, spinning disk and with 16GiB of RAM:
      
        $ cat test.sh
        #!/bin/bash
      
        DEV=/dev/sdj
        MNT=/mnt/sdj
        MKFS_OPTIONS="--nodesize 16384"     # default, just to be explicit
        MOUNT_OPTIONS="-o max_inline=2048"  # default, just to be explicit
      
        mkfs.btrfs -f $MKFS_OPTIONS $DEV > /dev/null
        mount $MOUNT_OPTIONS $DEV $MNT
      
        # Create files with inline data to make it easier and faster to create
        # large btrees.
        add_files()
        {
            local total=$1
            local start_offset=$2
            local number_jobs=$3
            local total_per_job=$(($total / $number_jobs))
      
            echo "Creating $total new files using $number_jobs jobs"
            for ((n = 0; n < $number_jobs; n++)); do
                (
                    local start_num=$(($start_offset + $n * $total_per_job))
                    for ((i = 1; i <= $total_per_job; i++)); do
                        local file_num=$((start_num + $i))
                        local file_path="$MNT/file_${file_num}"
                        xfs_io -f -c "pwrite -S 0xab 0 2000" $file_path > /dev/null
                        if [ $? -ne 0 ]; then
                            echo "Failed creating file $file_path"
                            break
                        fi
                    done
                ) &
                worker_pids[$n]=$!
            done
      
            wait ${worker_pids[@]}
      
            sync
            echo
            echo "btree node/leaf count: $(btrfs inspect-internal dump-tree -t 5 $DEV | egrep '^(node|leaf) ' | wc -l)"
        }
      
        initial_file_count=500000
        add_files $initial_file_count 0 4
      
        echo
        echo "Creating first snapshot..."
        btrfs subvolume snapshot -r $MNT $MNT/snap1
      
        echo
        echo "Adding more files..."
        add_files $((initial_file_count / 4)) $initial_file_count 4
      
        echo
        echo "Updating 1/50th of the initial files..."
        for ((i = 1; i < $initial_file_count; i += 50)); do
            xfs_io -c "pwrite -S 0xcd 0 20" $MNT/file_$i > /dev/null
        done
      
        echo
        echo "Creating second snapshot..."
        btrfs subvolume snapshot -r $MNT $MNT/snap2
      
        umount $MNT
      
        echo 3 > /proc/sys/vm/drop_caches
        blockdev --flushbufs $DEV &> /dev/null
        hdparm -F $DEV &> /dev/null
      
        mount $MOUNT_OPTIONS $DEV $MNT
      
        echo
        echo "Testing full send..."
        start=$(date +%s)
        btrfs send $MNT/snap1 > /dev/null
        end=$(date +%s)
        echo
        echo "Full send took $((end - start)) seconds"
      
        umount $MNT
      
      The durations of the full send operation in seconds were the following:
      
      Before this change:  217 seconds
      After this change:   205 seconds (-5.7%)
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ace75066
    • Filipe Manana's avatar
      btrfs: fix exhaustion of the system chunk array due to concurrent allocations · eafa4fd0
      Filipe Manana authored
      When we are running out of space for updating the chunk tree, that is,
      when we are low on available space in the system space info, if we have
      many task concurrently allocating block groups, via fallocate for example,
      many of them can end up all allocating new system chunks when only one is
      needed. In extreme cases this can lead to exhaustion of the system chunk
      array, which has a size limit of 2048 bytes, and results in a transaction
      abort with errno EFBIG, producing a trace in dmesg like the following,
      which was triggered on a PowerPC machine with a node/leaf size of 64K:
      
        [1359.518899] ------------[ cut here ]------------
        [1359.518980] BTRFS: Transaction aborted (error -27)
        [1359.519135] WARNING: CPU: 3 PID: 16463 at ../fs/btrfs/block-group.c:1968 btrfs_create_pending_block_groups+0x340/0x3c0 [btrfs]
        [1359.519152] Modules linked in: (...)
        [1359.519239] Supported: Yes, External
        [1359.519252] CPU: 3 PID: 16463 Comm: stress-ng Tainted: G               X    5.3.18-47-default #1 SLE15-SP3
        [1359.519274] NIP:  c008000000e36fe8 LR: c008000000e36fe4 CTR: 00000000006de8e8
        [1359.519293] REGS: c00000056890b700 TRAP: 0700   Tainted: G               X     (5.3.18-47-default)
        [1359.519317] MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 48008222  XER: 00000007
        [1359.519356] CFAR: c00000000013e170 IRQMASK: 0
        [1359.519356] GPR00: c008000000e36fe4 c00000056890b990 c008000000e83200 0000000000000026
        [1359.519356] GPR04: 0000000000000000 0000000000000000 0000d52a3b027651 0000000000000007
        [1359.519356] GPR08: 0000000000000003 0000000000000001 0000000000000007 0000000000000000
        [1359.519356] GPR12: 0000000000008000 c00000063fe44600 000000001015e028 000000001015dfd0
        [1359.519356] GPR16: 000000000000404f 0000000000000001 0000000000010000 0000dd1e287affff
        [1359.519356] GPR20: 0000000000000001 c000000637c9a000 ffffffffffffffe5 0000000000000000
        [1359.519356] GPR24: 0000000000000004 0000000000000000 0000000000000100 ffffffffffffffc0
        [1359.519356] GPR28: c000000637c9a000 c000000630e09230 c000000630e091d8 c000000562188b08
        [1359.519561] NIP [c008000000e36fe8] btrfs_create_pending_block_groups+0x340/0x3c0 [btrfs]
        [1359.519613] LR [c008000000e36fe4] btrfs_create_pending_block_groups+0x33c/0x3c0 [btrfs]
        [1359.519626] Call Trace:
        [1359.519671] [c00000056890b990] [c008000000e36fe4] btrfs_create_pending_block_groups+0x33c/0x3c0 [btrfs] (unreliable)
        [1359.519729] [c00000056890ba90] [c008000000d68d44] __btrfs_end_transaction+0xbc/0x2f0 [btrfs]
        [1359.519782] [c00000056890bae0] [c008000000e309ac] btrfs_alloc_data_chunk_ondemand+0x154/0x610 [btrfs]
        [1359.519844] [c00000056890bba0] [c008000000d8a0fc] btrfs_fallocate+0xe4/0x10e0 [btrfs]
        [1359.519891] [c00000056890bd00] [c0000000004a23b4] vfs_fallocate+0x174/0x350
        [1359.519929] [c00000056890bd50] [c0000000004a3cf8] ksys_fallocate+0x68/0xf0
        [1359.519957] [c00000056890bda0] [c0000000004a3da8] sys_fallocate+0x28/0x40
        [1359.519988] [c00000056890bdc0] [c000000000038968] system_call_exception+0xe8/0x170
        [1359.520021] [c00000056890be20] [c00000000000cb70] system_call_common+0xf0/0x278
        [1359.520037] Instruction dump:
        [1359.520049] 7d0049ad 40c2fff4 7c0004ac 71490004 40820024 2f83fffb 419e0048 3c620000
        [1359.520082] e863bcb8 7ec4b378 48010d91 e8410018 <0fe00000> 3c820000 e884bcc8 7ec6b378
        [1359.520122] ---[ end trace d6c186e151022e20 ]---
      
      The following steps explain how we can end up in this situation:
      
      1) Task A is at check_system_chunk(), either because it is allocating a
         new data or metadata block group, at btrfs_chunk_alloc(), or because
         it is removing a block group or turning a block group RO. It does not
         matter why;
      
      2) Task A sees that there is not enough free space in the system
         space_info object, that is 'left' is < 'thresh'. And at this point
         the system space_info has a value of 0 for its 'bytes_may_use'
         counter;
      
      3) As a consequence task A calls btrfs_alloc_chunk() in order to allocate
         a new system block group (chunk) and then reserves 'thresh' bytes in
         the chunk block reserve with the call to btrfs_block_rsv_add(). This
         changes the chunk block reserve's 'reserved' and 'size' counters by an
         amount of 'thresh', and changes the 'bytes_may_use' counter of the
         system space_info object from 0 to 'thresh'.
      
         Also during its call to btrfs_alloc_chunk(), we end up increasing the
         value of the 'total_bytes' counter of the system space_info object by
         8MiB (the size of a system chunk stripe). This happens through the
         call chain:
      
         btrfs_alloc_chunk()
             create_chunk()
                 btrfs_make_block_group()
                     btrfs_update_space_info()
      
      4) After it finishes the first phase of the block group allocation, at
         btrfs_chunk_alloc(), task A unlocks the chunk mutex;
      
      5) At this point the new system block group was added to the transaction
         handle's list of new block groups, but its block group item, device
         items and chunk item were not yet inserted in the extent, device and
         chunk trees, respectively. That only happens later when we call
         btrfs_finish_chunk_alloc() through a call to
         btrfs_create_pending_block_groups();
      
         Note that only when we update the chunk tree, through the call to
         btrfs_finish_chunk_alloc(), we decrement the 'reserved' counter
         of the chunk block reserve as we COW/allocate extent buffers,
         through:
      
         btrfs_alloc_tree_block()
            btrfs_use_block_rsv()
               btrfs_block_rsv_use_bytes()
      
         And the system space_info's 'bytes_may_use' is decremented everytime
         we allocate an extent buffer for COW operations on the chunk tree,
         through:
      
         btrfs_alloc_tree_block()
            btrfs_reserve_extent()
               find_free_extent()
                  btrfs_add_reserved_bytes()
      
         If we end up COWing less chunk btree nodes/leaves than expected, which
         is the typical case since the amount of space we reserve is always
         pessimistic to account for the worst possible case, we release the
         unused space through:
      
         btrfs_create_pending_block_groups()
            btrfs_trans_release_chunk_metadata()
               btrfs_block_rsv_release()
                  block_rsv_release_bytes()
                      btrfs_space_info_free_bytes_may_use()
      
         But before task A gets into btrfs_create_pending_block_groups()...
      
      6) Many other tasks start allocating new block groups through fallocate,
         each one does the first phase of block group allocation in a
         serialized way, since btrfs_chunk_alloc() takes the chunk mutex
         before calling check_system_chunk() and btrfs_alloc_chunk().
      
         However before everyone enters the final phase of the block group
         allocation, that is, before calling btrfs_create_pending_block_groups(),
         new tasks keep coming to allocate new block groups and while at
         check_system_chunk(), the system space_info's 'bytes_may_use' keeps
         increasing each time a task reserves space in the chunk block reserve.
         This means that eventually some other task can end up not seeing enough
         free space in the system space_info and decide to allocate yet another
         system chunk.
      
         This may repeat several times if yet more new tasks keep allocating
         new block groups before task A, and all the other tasks, finish the
         creation of the pending block groups, which is when reserved space
         in excess is released. Eventually this can result in exhaustion of
         system chunk array in the superblock, with btrfs_add_system_chunk()
         returning EFBIG, resulting later in a transaction abort.
      
         Even when we don't reach the extreme case of exhausting the system
         array, most, if not all, unnecessarily created system block groups
         end up being unused since when finishing creation of the first
         pending system block group, the creation of the following ones end
         up not needing to COW nodes/leaves of the chunk tree, so we never
         allocate and deallocate from them, resulting in them never being
         added to the list of unused block groups - as a consequence they
         don't get deleted by the cleaner kthread - the only exceptions are
         if we unmount and mount the filesystem again, which adds any unused
         block groups to the list of unused block groups, if a scrub is
         run, which also adds unused block groups to the unused list, and
         under some circumstances when using a zoned filesystem or async
         discard, which may also add unused block groups to the unused list.
      
      So fix this by:
      
      *) Tracking the number of reserved bytes for the chunk tree per
         transaction, which is the sum of reserved chunk bytes by each
         transaction handle currently being used;
      
      *) When there is not enough free space in the system space_info,
         if there are other transaction handles which reserved chunk space,
         wait for some of them to complete in order to have enough excess
         reserved space released, and then try again. Otherwise proceed with
         the creation of a new system chunk.
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      eafa4fd0