A change in remote port removal introduced a spurious put which can
cause a premature structure teardown. The affects were most notable when
the driver attempted to unload as a null pointer would be hit.

Fix by removing the unnecessary put.
Signed-off-by: James Smart <james.smart@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>