We only need to return transaction_restart_nested when we're inside a
context that's handling transaction restarts.

Also, add a missing check_subdir_count() call.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>