Make sure to never get in trouble due to the backref_ctx
which was on the stack before.

Commit is a result of Arne's review.
Signed-off-by: Alexander Block <ablock84@googlemail.com>