If we keep hold of the mutex here, the process which currently holds the
buffer object will never be able to release it, causing a deadlock.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>