• Song Liu's avatar
    md/r5cache: write-out phase and reclaim support · a39f7afd
    Song Liu authored
    There are two limited resources, stripe cache and journal disk space.
    For better performance, we priotize reclaim of full stripe writes.
    To free up more journal space, we free earliest data on the journal.
    
    In current implementation, reclaim happens when:
    1. Periodically (every R5C_RECLAIM_WAKEUP_INTERVAL, 30 seconds) reclaim
       if there is no reclaim in the past 5 seconds.
    2. when there are R5C_FULL_STRIPE_FLUSH_BATCH (256) cached full stripes,
       or cached stripes is enough for a full stripe (chunk size / 4k)
       (r5c_check_cached_full_stripe)
    3. when there is pressure on stripe cache (r5c_check_stripe_cache_usage)
    4. when there is pressure on journal space (r5l_write_stripe, r5c_cache_data)
    
    r5c_do_reclaim() contains new logic of reclaim.
    
    For stripe cache:
    
    When stripe cache pressure is high (more than 3/4 stripes are cached,
    or there is empty inactive lists), flush all full stripe. If fewer
    than R5C_RECLAIM_STRIPE_GROUP (NR_STRIPE_HASH_LOCKS * 2) full stripes
    are flushed, flush some paritial stripes. When stripe cache pressure
    is moderate (1/2 to 3/4 of stripes are cached), flush all full stripes.
    
    For log space:
    
    To avoid deadlock due to log space, we need to reserve enough space
    to flush cached data. The size of required log space depends on total
    number of cached stripes (stripe_in_journal_count). In current
    implementation, the writing-out phase automatically include pending
    data writes with parity writes (similar to write through case).
    Therefore, we need up to (conf->raid_disks + 1) pages for each cached
    stripe (1 page for meta data, raid_disks pages for all data and
    parity). r5c_log_required_to_flush_cache() calculates log space
    required to flush cache. In the following, we refer to the space
    calculated by r5c_log_required_to_flush_cache() as
    reclaim_required_space.
    
    Two flags are added to r5conf->cache_state: R5C_LOG_TIGHT and
    R5C_LOG_CRITICAL. R5C_LOG_TIGHT is set when free space on the log
    device is less than 3x of reclaim_required_space. R5C_LOG_CRITICAL
    is set when free space on the log device is less than 2x of
    reclaim_required_space.
    
    r5c_cache keeps all data in cache (not fully committed to RAID) in
    a list (stripe_in_journal_list). These stripes are in the order of their
    first appearance on the journal. So the log tail (last_checkpoint)
    should point to the journal_start of the first item in the list.
    
    When R5C_LOG_TIGHT is set, r5l_reclaim_thread starts flushing out
    stripes at the head of stripe_in_journal. When R5C_LOG_CRITICAL is
    set, the state machine only writes data that are already in the
    log device (in stripe_in_journal_list).
    
    This patch includes a fix to improve performance by
    Shaohua Li <shli@fb.com>.
    Signed-off-by: default avatarSong Liu <songliubraving@fb.com>
    Signed-off-by: default avatarShaohua Li <shli@fb.com>
    a39f7afd
raid5.h 28.2 KB