• Harald Freudenberger's avatar
    s390/zcrypt: revisit ap device remove procedure · 01396a37
    Harald Freudenberger authored
    Working with the vfio-ap driver let to some revisit of the way
    how an ap (queue) device is removed from the driver.
    With the current implementation all the cleanup was done before
    the driver even got notified about the removal. Now the ap
    queue removal is done in 3 steps:
    1) A preparation step, all ap messages within the queue
       are flushed and so the driver does 'receive' them.
       Also a new state AP_STATE_REMOVE assigned to the queue
       makes sure there are no new messages queued in.
    2) Now the driver's remove function is invoked and the
       driver should do the job of cleaning up it's internal
       administration lists or whatever. After 2) is done
       it is guaranteed, that the driver is not invoked any
       more. On the other hand the driver has to make sure
       that the APQN is not accessed any more after step 2
       is complete.
    3) Now the ap bus code does the job of total cleanup of the
       APQN. A reset with zero is triggered and the state of
       the queue goes to AP_STATE_UNBOUND.
       After step 3) is complete, the ap queue has no pending
       messages and the APQN is cleared and so there are no
       requests and replies lingering around in the firmware
       queue for this APQN. Also the interrupts are disabled.
    
    After these remove steps the ap queue device may be assigned
    to another driver.
    
    Stress testing this remove/probe procedure showed a problem with the
    correct module reference counting. The actual receive of an reply in
    the driver is done asynchronous with completions. So with a driver
    change on an ap queue the message flush triggers completions but the
    threads waiting for the completions may run at a time where the queue
    already has the new driver assigned. So the module_put() at receive
    time needs to be done on the driver module which queued the ap
    message. This change is also part of this patch.
    Signed-off-by: default avatarHarald Freudenberger <freude@linux.ibm.com>
    Reviewed-by: default avatarIngo Franzki <ifranzki@linux.ibm.com>
    Signed-off-by: default avatarMartin Schwidefsky <schwidefsky@de.ibm.com>
    01396a37
ap_bus.c 41.2 KB