This fixes one cause of an occational problem when unloading
libfc where the exchange manager pool doesn't have all items freed.

The existing WARN_ON(mp->total_exches <= 0) isn't hit.
However, note that total_exches is decremented when the
exchange is completed, and it can be held with a refcnt
for a while after that.

I'm not sure what the offending exchange is, but I suspect
it is an incoming request, because outgoing state machines
should be all stopped at this point.

Note that although receive is stopped before the exchange
manager is freed, there could still be active threads
handling received frames.

This patch flushes the queues by allocating a new skb
and sending it through, and have the thread handle
this new skb specially.  This is similar to the way the work
queues are flushed now by putting work items in them and waiting
until they make it through the queue.

An skb->destructor function is used to inform us of
the completion of the flush, and the fr_dev() is left
NULL to indicate to fcoe_percpu_receive_thread() that
the skb should be just freed.  There's already a check
for the lp being NULL which prints a message.
We skip printing the message if the destructor is for flushing.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>