Drop the whole rcu handling since we are holding the resv lock anyway.

v2: drop all rcu handling instead.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602140359.272601-1-christian.koenig@amd.com