All the dmabufs are torn down when th VGPU is released, so there is
no need for extra refcounting here.

Based on an patch from Jason Gunthorpe.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20220411141403.86980-25-hch@lst.deReviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>