• Vivek Goyal's avatar
    Fix regression in direct writes performance due to WRITE_ODIRECT flag removal · d9449ce3
    Vivek Goyal authored
    There seems to be a regression in direct write path due to following
    commit in for-2.6.33 branch of block tree.
    
    commit 1af60fbd
    Author: Jeff Moyer <jmoyer@redhat.com>
    Date:   Fri Oct 2 18:56:53 2009 -0400
    
        block: get rid of the WRITE_ODIRECT flag
    
    Marking direct writes as WRITE_SYNC_PLUG instead of WRITE_ODIRECT, sets
    the NOIDLE flag in bio and hence in request. This tells CFQ to not expect
    more request from the queue and not idle on it (despite the fact that
    queue's think time is less and it is not seeky).
    
    So direct writers lose big time when competing with sequential readers.
    
    Using fio, I have run one direct writer and two sequential readers and
    following are the results with 2.6.32-rc7 kernel and with for-2.6.33
    branch.
    
    Test
    ====
    1 direct writer and 2 sequential reader running simultaneously.
    
    [global]
    directory=/mnt/sdc/fio/
    runtime=10
    
    [seqwrite]
    rw=write
    size=4G
    direct=1
    
    [seqread]
    rw=read
    size=2G
    numjobs=2
    
    2.6.32-rc7
    ==========
    direct writes: aggrb=2,968KB/s
    readers	     : aggrb=101MB/s
    
    for-2.6.33 branch
    =================
    direct write: aggrb=19KB/s
    readers	      aggrb=137MB/s
    
    This patch brings back the WRITE_ODIRECT flag, with the difference that we
    don't set the BIO_RW_UNPLUG flag so that device is not unplugged after
    submission of request and an explicit unplug from submitter is required.
    
    That way we fix the jeff's issue of not enough merging taking place in aio
    path as well as make sure direct writes get their fair share.
    
    After the fix
    =============
    for-2.6.33 + fix
    ----------------
    direct writes: aggrb=2,728KB/s
    reads: aggrb=103MB/s
    
    Thanks
    Vivek
    Signed-off-by: default avatarVivek Goyal <vgoyal@redhat.com>
    Signed-off-by: default avatarJens Axboe <jens.axboe@oracle.com>
    d9449ce3
direct-io.c 34.4 KB