- 07 Oct, 2020 4 commits
-
-
Darrick J. Wong authored
When xfs_defer_capture extracts the deferred ops and transaction state from a transaction, it should record the remaining block reservations so that when we continue the dfops chain, we can reserve the same number of blocks to use. We capture the reservations for both data and realtime volumes. This adds the requirement that every log intent item recovery function must be careful to reserve enough blocks to handle both itself and all defer ops that it can queue. On the other hand, this enables us to do away with the handwaving block estimation nonsense that was going on in xlog_finish_defer_ops. Signed-off-by:
Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Brian Foster <bfoster@redhat.com>
-
Darrick J. Wong authored
When we replay unfinished intent items that have been recovered from the log, it's possible that the replay will cause the creation of more deferred work items. As outlined in commit 50995582 ("xfs: log recovery should replay deferred ops in order"), later work items have an implicit ordering dependency on earlier work items. Therefore, recovery must replay the items (both recovered and created) in the same order that they would have been during normal operation. For log recovery, we enforce this ordering by using an empty transaction to collect deferred ops that get created in the process of recovering a log intent item to prevent them from being committed before the rest of the recovered intent items. After we finish committing all the recovered log items, we allocate a transaction with an enormous block reservation, splice our huge list of created deferred ops into that transaction, and commit it, thereby finishing all those ops. This is /really/ hokey -- it's the one place in XFS where we allow nested transactions; the splicing of the defer ops list is is inelegant and has to be done twice per recovery function; and the broken way we handle inode pointers and block reservations cause subtle use-after-free and allocator problems that will be fixed by this patch and the two patches after it. Therefore, replace the hokey empty transaction with a structure designed to capture each chain of deferred ops that are created as part of recovering a single unfinished log intent. Finally, refactor the loop that replays those chains to do so using one transaction per chain. Signed-off-by:
-
Darrick J. Wong authored
The ->iop_recover method of a log intent item removes the recovered intent item from the AIL by logging an intent done item and committing the transaction, so it's superfluous to have this flag check. Nothing else uses it, so get rid of the flag entirely. Signed-off-by:
-
Darrick J. Wong authored
Remove this one-line helper since the assert is trivially true in one call site and the rest obscures a bitmask operation. Signed-off-by:
Dave Chinner <dchinner@redhat.com> Reviewed-by:
-
- 30 Sep, 2020 1 commit
-
-
Dave Chinner authored
Nathan popped up on #xfs and pointed out that we fail to handle finobt btree blocks in xlog_recover_get_buf_lsn(). This means they always fall through the entire magic number matching code to "recover immediately". Whilst most of the time this is the correct behaviour, occasionally it will be incorrect and could potentially overwrite more recent metadata because we don't check the LSN in the on disk metadata at all. This bug has been present since the finobt was first introduced, and is a potential cause of the occasional xfs_iget_check_free_state() failures we see that indicate that the inode btree state does not match the on disk inode state. Fixes: aafc3c24 ("xfs: support the XFS_BTNUM_FINOBT free inode btree type") Reported-by:
Nathan Scott <nathans@redhat.com> Signed-off-by:
-
- 25 Sep, 2020 13 commits
-
-
Pavel Reichl authored
These optionr were for Irix compatibility, probably for clustered XFS clients in a heterogenous cluster which contained both Irix & Linux machines, so that behavior would be consistent. That doesn't exist anymore and it's no longer needed. Signed-off-by:
Pavel Reichl <preichl@redhat.com> Reviewed-by:
-
Pavel Reichl authored
ikeep/noikeep was a workaround for old DMAPI code which is no longer relevant. attr2/noattr2 - is for controlling upgrade behaviour from fixed attribute fork sizes in the inode (attr1) and dynamic attribute fork sizes (attr2). mkfs has defaulted to setting attr2 since 2007, hence just about every XFS filesystem out there in production right now uses attr2. Signed-off-by:
-
Kaixu Xia authored
The current create and mkdir handlers both call the xfs_vn_mknod() which is a wrapper routine around xfs_generic_create() function. Actually the create and mkdir handlers can directly call xfs_generic_create() function and reduce the call chain. Signed-off-by:
Kaixu Xia <kaixuxia@tencent.com> Reviewed-by:
-
Darrick J. Wong authored
During code review, I noticed that the rmap code uses the (slower) shared mappings rmap functions for any extent of a reflinked file, even if those extents are for the attr fork, which doesn't support sharing. We can speed up rmap a tiny bit by optimizing out this case. Signed-off-by:
Chandan Babu R <chandanrlinux@gmail.com> Reviewed-by:
-
Gao Xiang authored
Since commit 1c1c6ebc ("xfs: Replace per-ag array with a radix tree"), there is no m_peraglock anymore, so it's hard to understand the described situation since per-ag is no longer an array and no need to reallocate, call xfs_filestream_flush() in growfs. In addition, the race condition for shrink feature is quite confusing to me currently as well. Get rid of it instead. Signed-off-by:
Gao Xiang <hsiangkao@redhat.com> Reviewed-by:
-
Kaixu Xia authored
Cleanup the typedef usage, the unnecessary parentheses, the unnecessary backslash and use the open-coded round_up call in xfs_attr_leaf_entsize_{remote,local}. Signed-off-by: -
Kaixu Xia authored
We should do the assert for all the log intent-done items if they appear here. This patch detect intent-done items by the fact that their item ops don't have iop_unpin and iop_push methods and also move the helper xlog_item_is_intent to xfs_trans.h. Signed-off-by:
-
Kaixu Xia authored
Since we never use the second parameter id, so remove it from xfs_qm_dqattach_one() function. Signed-off-by:
-
Kaixu Xia authored
We already check whether the crc feature is enabled before calling xfs_attr3_rmt_verify(), so remove the redundant feature check in that function. Signed-off-by:
-
Kaixu Xia authored
Fix the comments to help people understand the code. Signed-off-by:
-
Kaixu Xia authored
Since the type prid_t and xfs_dqid_t both are uint32_t, seems the type cast is unnecessary, so remove it. Signed-off-by:
-
Kaixu Xia authored
We have already defined the project ID type prid_t, so maybe should use it here. Signed-off-by:
-
Kaixu Xia authored
There are no callers of the SYNCHRONIZE() macro, so remove it. Signed-off-by:
-
- 23 Sep, 2020 8 commits
-
-
Gao Xiang authored
Let's use DIV_ROUND_UP() to calculate log record header blocks as what did in xlog_get_iclog_buffer_size() and wrap up a common helper for log recovery. Reviewed-by:
-
Gao Xiang authored
Currently, crafted h_len has been blocked for the log header of the tail block in commit a70f9fe5 ("xfs: detect and handle invalid iclog size set by mkfs"). However, each log record could still have crafted h_len and cause log record buffer overrun. So let's check h_len vs buffer size for each log record as well. Signed-off-by:
-
Darrick J. Wong authored
Nowadays, log recovery will call ->release on the recovered intent items if recovery fails. Therefore, it's redundant to release them from inside the ->recover functions when they're about to return an error. Signed-off-by:
-
Darrick J. Wong authored
In the bmap intent item recovery code, we must be careful to attach the inode to its dquots (if quotas are enabled) so that a change in the shape of the bmap btree doesn't cause the quota counters to be incorrect. Signed-off-by:
-
Darrick J. Wong authored
During a code inspection, I found a serious bug in the log intent item recovery code when an intent item cannot complete all the work and decides to requeue itself to get that done. When this happens, the item recovery creates a new incore deferred op representing the remaining work and attaches it to the transaction that it allocated. At the end of _item_recover, it moves the entire chain of deferred ops to the dummy parent_tp that xlog_recover_process_intents passed to it, but fail to log a new intent item for the remaining work before committing the transaction for the single unit of work. xlog_finish_defer_ops logs those new intent items once recovery has finished dealing with the intent items that it recovered, but this isn't sufficient. If the log is forced to disk after a recovered log item decides to requeue itself and the system goes down before we call xlog_finish_defer_ops, the second log recovery will never see the new intent item and therefore has no idea that there was more work to do. It will finish recovery leaving the filesystem in a corrupted state. The same logic applies to /any/ deferred ops added during intent item recovery, not just the one handling the remaining work. Signed-off-by:
-
Darrick J. Wong authored
When xchk_da_btree_block is loading a non-root dabtree block, we know that the parent block had to have a (hashval, address) pointer to the block that we just loaded. Check that the hashval in the parent matches the block we just loaded. This was found by fuzzing nbtree[3].hashval = ones in xfs/394. Signed-off-by:
-
Darrick J. Wong authored
When callers pass XFS_BMAPI_REMAP into xfs_bunmapi, they want the extent to be unmapped from the given file fork without the extent being freed. We do this for non-rt files, but we forgot to do this for realtime files. So far this isn't a big deal since nobody makes a bunmapi call to a rt file with the REMAP flag set, but don't leave a logic bomb. Signed-off-by:
-
Chandan Babu R authored
In xfs_growfs_rt(), we enlarge bitmap and summary files by allocating new blocks for both files. For each of the new blocks allocated, we allocate an xfs_buf, zero the payload, log the contents and commit the transaction. Hence these buffers will eventually find themselves appended to list at xfs_ail->ail_buf_list. Later, xfs_growfs_rt() loops across all of the new blocks belonging to the bitmap inode to set the bitmap values to 1. In doing so, it allocates a new transaction and invokes the following sequence of functions, - xfs_rtfree_range() - xfs_rtmodify_range() - xfs_rtbuf_get() We pass '&xfs_rtbuf_ops' as the ops pointer to xfs_trans_read_buf(). - xfs_trans_read_buf() We find the xfs_buf of interest in per-ag hash table, invoke xfs_buf_reverify() which ends up assigning '&xfs_rtbuf_ops' to xfs_buf->b_ops. On the other hand, if xfs_growfs_rt_alloc() had allocated a few blocks for the bitmap inode and returned with an error, all the xfs_bufs corresponding to the new bitmap blocks that have been allocated would continue to be on xfs_ail->ail_buf_list list without ever having a non-NULL value assigned to their b_ops members. An AIL flush operation would then trigger the following warning message to be printed on the console, XFS (loop0): _xfs_buf_ioapply: no buf ops on daddr 0x58 len 8 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ CPU: 3 PID: 449 Comm: xfsaild/loop0 Not tainted 5.8.0-rc4-chandan-00038-g4d8c2b9de9ab-dirty #37 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Call Trace: dump_stack+0x57/0x70 _xfs_buf_ioapply+0x37c/0x3b0 ? xfs_rw_bdev+0x1e0/0x1e0 ? xfs_buf_delwri_submit_buffers+0xd4/0x210 __xfs_buf_submit+0x6d/0x1f0 xfs_buf_delwri_submit_buffers+0xd4/0x210 xfsaild+0x2c8/0x9e0 ? __switch_to_asm+0x42/0x70 ? xfs_trans_ail_cursor_first+0x80/0x80 kthread+0xfe/0x140 ? kthread_park+0x90/0x90 ret_from_fork+0x22/0x30 This message indicates that the xfs_buf had its b_ops member set to NULL. This commit fixes the issue by assigning "&xfs_rtbuf_ops" to b_ops member of each of the xfs_bufs logged by xfs_growfs_rt_alloc(). Signed-off-by:
-
- 21 Sep, 2020 2 commits
-
-
Chandan Babu R authored
The following sequence of commands, mkfs.xfs -f -m reflink=0 -r rtdev=/dev/loop1,size=10M /dev/loop0 mount -o rtdev=/dev/loop1 /dev/loop0 /mnt xfs_growfs /mnt ... causes the following call trace to be printed on the console, XFS: Assertion failed: (bip->bli_flags & XFS_BLI_STALE) || (xfs_blft_from_flags(&bip->__bli_format) > XFS_BLFT_UNKNOWN_BUF && xfs_blft_from_flags(&bip->__bli_format) < XFS_BLFT_MAX_BUF), file: fs/xfs/xfs_buf_item.c, line: 331 Call Trace: xfs_buf_item_format+0x632/0x680 ? kmem_alloc_large+0x29/0x90 ? kmem_alloc+0x70/0x120 ? xfs_log_commit_cil+0x132/0x940 xfs_log_commit_cil+0x26f/0x940 ? xfs_buf_item_init+0x1ad/0x240 ? xfs_growfs_rt_alloc+0x1fc/0x280 __xfs_trans_commit+0xac/0x370 xfs_growfs_rt_alloc+0x1fc/0x280 xfs_growfs_rt+0x1a0/0x5e0 xfs_file_ioctl+0x3fd/0xc70 ? selinux_file_ioctl+0x174/0x220 ksys_ioctl+0x87/0xc0 __x64_sys_ioctl+0x16/0x20 do_syscall_64+0x3e/0x70 entry_SYSCALL_64_after_hwframe+0x44/0xa9 This occurs because the buffer being formatted has the value of XFS_BLFT_UNKNOWN_BUF assigned to the 'type' subfield of bip->bli_formats->blf_flags. This commit fixes the issue by assigning one of XFS_BLFT_RTSUMMARY_BUF and XFS_BLFT_RTBITMAP_BUF to the 'type' subfield of bip->bli_formats->blf_flags before committing the corresponding transaction. Reviewed-by:
-
Brian Foster authored
The inode extent truncate path unmaps extents from the inode block mapping, finishes deferred ops to free the associated extents and then explicitly rolls the transaction before processing the next extent. The latter extent roll is spurious as xfs_defer_finish() always returns a clean transaction and automatically relogs inodes attached to the transaction (with lock_flags == 0). This can unnecessarily increase the number of log ticket regrants that occur during a long running truncate operation. Remove the explicit transaction roll. Signed-off-by:
-
- 16 Sep, 2020 12 commits
-
-
Darrick J. Wong authored
The V4 filesystem format contains known weaknesses in the on-disk format that make metadata verification diffiult. In addition, the format does not support dates past 2038 and will not be upgraded to do so. We should start the process of retiring the old format to close off attack surfaces and to encourage users to migrate onto V5. Therefore, make XFS V4 support a configurable option. For the first period it will be default Y in case some distributors want to withdraw support early; for the second period it will be default N so that anyone who wishes to continue support can do so; and after that, support will be removed from the kernel. Dates for these events have been added to the upstream kernel. Signed-off-by:
Eric Sandeen <sandeen@redhat.com>
-
Darrick J. Wong authored
While running generic/042 with -drtinherit=1 set in MKFS_OPTIONS, I observed that the kernel will gladly set the realtime flag on any file created on the loopback filesystem even though that filesystem doesn't actually have a realtime device attached. This leads to verifier failures and doesn't make any sense, so be smarter about this. Signed-off-by:
-
Darrick J. Wong authored
Hoist the code that propagates di_flags and di_flags2 from a parent to a new child into separate functions. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Make sure that any fallocate operation that requires the range to be block-aligned also checks that the range is aligned to the realtime extent size. Signed-off-by:
-
Darrick J. Wong authored
There's an overflow bug in the realtime allocator. If the rt volume is large enough to handle a single allocation request that is larger than the maximum bmap extent length and the rt bitmap ends exactly on a bitmap block boundary, it's possible that the near allocator will try to check the freeness of a range that extends past the end of the bitmap. This fails with a corruption error and shuts down the fs. Therefore, constrain maxlen so that the range scan cannot run off the end of the rt bitmap. Signed-off-by:
-
Zheng Bin authored
Fixes coccicheck warning: fs/xfs/xfs_icache.c:1214:2-3: Unneeded semicolon Signed-off-by:
Zheng Bin <zhengbin13@huawei.com> Reviewed-by:
-
Darrick J. Wong authored
Commit 5833112d tried to make it so that a remap operation would force the log out to disk if the filesystem is mounted with mandatory synchronous writes. Unfortunately, that commit failed to handle the case where the inode or the file descriptor require mandatory synchronous writes. Refactor the check into into a helper that will look for all three conditions, and now we can treat reflink just like any other synchronous write. Fixes: 5833112d ("xfs: reflink should force the log out if mounted with wsync") Signed-off-by:
-
Carlos Maiolino authored
xfs_attr_sf_totsize() requires access to xfs_inode structure, so, once xfs_attr_shortform_addname() is its only user, move it to xfs_attr.c instead of playing with more #includes. Reviewed-by:
Carlos Maiolino <cmaiolino@redhat.com> Signed-off-by:
-
Carlos Maiolino authored
nameval is a variable-size array, so, define it as it, and remove all the -1 magic number subtractions Reviewed-by:
-
Carlos Maiolino authored
Reviewed-by:
-
Carlos Maiolino authored
Reviewed-by:
-
Carlos Maiolino authored
This patch aims to replace kmem_zalloc_large() with global kernel memory API. So, all its callers are now using kvzalloc() directly, so kmalloc() fallsback to vmalloc() automatically. Signed-off-by:
-
