• Jeff Layton's avatar
    ceph: add buffered/direct exclusionary locking for reads and writes · 321fe13c
    Jeff Layton authored
    xfstest generic/451 intermittently fails. The test does O_DIRECT writes
    to a file, and then reads back the result using buffered I/O, while
    running a separate set of tasks that are also doing buffered reads.
    
    The client will invalidate the cache prior to a direct write, but it's
    easy for one of the other readers' replies to race in and reinstantiate
    the invalidated range with stale data.
    
    To fix this, we must to serialize direct I/O writes and buffered reads.
    We could just sprinkle in some shared locks on the i_rwsem for reads,
    and increase the exclusive footprint on the write side, but that would
    cause O_DIRECT writes to end up serialized vs. other direct requests.
    
    Instead, borrow the scheme used by nfs.ko. Buffered writes take the
    i_rwsem exclusively, but buffered reads take a shared lock, allowing
    them to run in parallel.
    
    O_DIRECT requests also take a shared lock, but we need for them to not
    run in parallel with buffered reads.  A flag on the ceph_inode_info is
    used to indicate whether it's in direct or buffered I/O mode. When a
    conflicting request is submitted, it will block until the inode can be
    flipped to the necessary mode.
    
    Link: https://tracker.ceph.com/issues/40985Signed-off-by: default avatarJeff Layton <jlayton@kernel.org>
    Reviewed-by: default avatar"Yan, Zheng" <zyan@redhat.com>
    Signed-off-by: default avatarIlya Dryomov <idryomov@gmail.com>
    321fe13c
io.h 387 Bytes