• unknown's avatar
    WL#3072 - Maria recovery. · d72c22de
    unknown authored
    * fix for bitmap vs checkpoint bug which could lead to corrupted
    tables in case of crashes at certain moments: a bitmap could be flushed
    to disk even though it was inconsistent with the log (it could be
    flushed before REDO-UNDO are written to the log). One bug remains, need
    code from others. Tests added. Fix is to pin unflushable bitmap pages,
    and let checkpoint wait for them to be flushable.
    * fix for long_trid!=0 assertion failure at Recovery.
    * less useless wakeups in the background flush|checkpoint thread.
    * store global_trid_generator in checkpoint record.
    
    
    mysql-test/r/maria-recovery.result:
      result update
    mysql-test/t/maria-recovery.test:
      make it easier to locate subtests
    storage/maria/ma_bitmap.c:
      When we send a bitmap to the pagecache, if this bitmap is not in a
      flushable state we keep it pinned and add it to a list, it will be
      unpinned when the bitmap is flushable again.
      A new function _ma_bitmap_flush_all() used by checkpoint.
      A new function _ma_bitmap_flushable() used by block format to signal
      when it starts modifying a bitmap and when it is done with it.
    storage/maria/ma_blockrec.c:
      When starting a row operation (insert/update/delete), mark that
      the bitmap is not flushable (because for example INSERT is going
      to over-allocate in the bitmap to prevent other threads from using
      our data pages). If a checkpoint comes at this moment it will wait
      for the bitmap to be flushable before flushing it.
      When the operation ends, bitmap becomes flushable again; that
      transition is done under the bitmap's mutex (needed for correct
      synchro with a concurrent checkpoint); but for INSERT/UPDATE this
      happens inside _ma_bitmap_release_unused() at a place where it already
      has the mutex, so the only penalty (mutex adding) is in DELETE and UNDO
      of INSERT. In case of errors after setting the bitmap unflushable,
      we must always set it back to flushable or checkpoint would block.
      Debug possibilities to force a sleep while the bitmap is over-allocated.
      In case of error in get_head_or_tail() in allocate_and_write_block_record(),
      we still need to unpin all pages.
      Bugfix: _ma_apply_redo_insert_row_blobs() produced wrong
      data_file_length.
    storage/maria/ma_blockrec.h:
      new bitmap calls.
    storage/maria/ma_checkpoint.c:
      filter_flush_indirect not needed anymore (flushing bitmap
      pages happens in _ma_bitmap_flush_all() now). So
      st_filter_param::is_data_file|pages_covered_by_bitmap not needed.
      Other filter_flush* don't need to flush bitmap anymore.
      Add debug possibility to flush all bitmap pages outside of a checkpoint,
      to simulate pagecache LRU eviction.
      When the background flush/checkpoint thread notices it has nothing
      to flush, it now sleeps directly until the next potential checkpoint
      moment instead of waking up every second.
      When in checkpoint we decide to not store a table in the checkpoint record
      (because it has logged no writes for example), we can also skip flushing
      this table.
    storage/maria/ma_commit.c:
      comment is out-of-date
    storage/maria/ma_key_recover.c:
      comment fix
    storage/maria/ma_loghandler.c:
      comment is out-of-date
    storage/maria/ma_open.c:
      comment is out-of-date
    storage/maria/ma_pagecache.c:
      comment for bug to fix. And we don't take checkpoints at end of REDO
      phase yet so can trust block->type.
    storage/maria/ma_recovery.c:
      Comments. Now-unneeded code for incomplete REDO-UNDO groups removed.
      When we forget about an old transaction we must really forget
      about it with bzero() (fixes the "long_trid!=0 assertion" recovery
      bug). When we delete a row with maria_delete() we turn on
      STATE_NOT_OPTIMIZED_ROWS so we do the same when we see a CLR_END
      for an UNDO_ROW_INSERT or when we execute an UNDO_ROW_INSERT (in both
      cases a row was deleted). Pick up max_long_trid from the checkpoint record.
    storage/maria/maria_chk.c:
      comment
    storage/maria/maria_def.h:
      MARIA_FILE_BITMAP gets new members: 'flushable', 'bitmap_cond' and
      'pinned_pages'.
    storage/maria/trnman.c:
      I used to think that recovery only needs to know the maximum TrID
      of the lists of active and committed transactions. But no, sometimes
      both lists can even be empty and their TrID should not be reused.
      So Checkpoint now saves global_trid_generator in the checkpoint record.
    storage/maria/trnman_public.h:
      macros to read/store a TrID
    mysql-test/r/maria-recovery-bitmap.result:
      result is ok. Without the code fix, we would get a corruption message
      about the bitmap page in CHECK TABLE EXTENDED.
    mysql-test/t/maria-recovery-bitmap-master.opt:
      usual when we crash mysqld in tests
    mysql-test/t/maria-recovery-bitmap.test:
      test of recovery problems specific of the bitmap pages.
    d72c22de
ma_bitmap.c 69.6 KB