Here's a patch which fixes the deadlock Brad Fitzpatrick reported here:
http://lkml.org/lkml/2004/11/14/98

The meat of the problem is a locking order reversal between the XFS I/O
lock and the inode semaphore (i_sem).  When we mix a number of threads
doing both direct reads and writes, we hit an ABBA deadlock after a while
because direct-io.c is taking and dropping i_sem after the XFS read path
has already taken its I/O lock.  This is the wrong way around from XFS's
point of view, in particular its the opposite order to the XFS write path. 
So this patch changes the logic for direct reads in the DIO_OWN_LOCKING
case (i.e.  XFS-only case), but leaves things as is for the other two types
of locking.

Not real pretty, but fixes up the lock ordering and deadlock.

Oh, I have tested direct reads and writes on ext3 and block devices as
well, to ensure they still function correctly with the change (i.e.  they
don't regress, should be a no-op there) which covers the other locking
cases in __blockdev_direct_IO.
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>