added dbug/tests


BitKeeper/etc/ignore:
  added dbug/tests

dbug/dbug.c:
  function/ (a.k.a. SUBDIR) syntax
dbug/tests-t.pl:
  1. add support for test comments
  2. add test comments
  3. move tests around
  4. add SUBDIR tests
dbug/tests.c:
  support code for SUBDIR testing
include/my_dbug.h:
  comments. change in _db_set_ prototype
dbug/user.r:
  negative lists and function/ syntax.

into  janus.mylan:/usr/home/serg/Abk/mysql-maria

into  a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-maria

storage/maria/ma_test_recovery.expected:
  Removed a white space from a list of tests.
  
  This was originally unintentional and change makes it easier to
  port ma_test_recovery to Perl.
storage/maria/ma_test_recovery:
  Removed a white space from a list of tests.

unit tests for dbug


dbug/Makefile.am:
  unit tests for dbug
dbug/dbug.c:
  negative lists (-#-d,info => everything but "info")
include/my_dbug.h:
  negative lists (-#-d,info => everything but "info")
unittest/Makefile.am:
  unit tests for dbug
dbug/tests-t.pl:
  unit tests for dbug
dbug/tests.c:
  unit tests for dbug

Test for BUG#34106 "auto_increment is reset to 1 when table is recovered from crash"
(fixed by Monty yesterday)


mysql-test/r/maria-recovery.result:
  result, which is correct (before pulling Monty's fix for BUG#34106,
  we got a warning about auto_increment in CHECK TABLE (done in
  maria-verify-recovery.inc), no AUTO_INCREMENT clause in
  SHOW CREATE TABLE, and a failure of the last INSERT.
mysql-test/r/maria.result:
  result
mysql-test/t/maria-recovery.test:
  Test for BUG#34106
mysql-test/t/maria.test:
  look at what is reported in SHOW ENGINES
mysys/my_pread.c:
  changed my mind: if Count argument is >4GB, we'll surely see a segfault
  in the pread() call when it tries to read 4GB from memory, so no need
  to print it in ulonglong format (saves a function call).
mysys/my_read.c:
  changed my mind: if Count argument is >4GB, we'll surely see a segfault
  in the pread() call when it tries to read 4GB from memory, so no need
  to print it in ulonglong format (saves a function call).
mysys/my_write.c:
  changed my mind: if Count argument is >4GB, we'll surely see a segfault
  in the pread() call when it tries to read 4GB from memory, so no need
  to print it in ulonglong format (saves a function call).
storage/maria/ha_maria.cc:
  Description representing the current reality. This can be changed later
storage/maria/ma_page.c:
  When reading the new key_del from a page on disk, if there is a bug
  (like BUG#34062) this key_del could be wrong, we try to catch if it's
  out of the key file.
storage/maria/ma_pagecache.c:
  - no truncation of page's number in DBUG_PRINT (useful for BUG#34062)
  - page_korr instead of uint5korr
storage/maria/ma_recovery.c:
  page_korr instead of uint5korr
storage/maria/plug.in:
  Description representing the current reality. This can be changed later.

dbug/CMakeLists.txt:
  Change mode to -rw-rw-r--
BitKeeper/deleted/.del-doinstall.sh:
  Delete: dbug/doinstall.sh
BitKeeper/deleted/.del-install.sh:
  Delete: dbug/install.sh
BitKeeper/deleted/.del-mklintlib.sh:
  Delete: dbug/mklintlib.sh
BitKeeper/deleted/.del-qmake.cmd:
  Delete: dbug/qmake.cmd
BitKeeper/deleted/.del-.cvsignore:
  Delete: dbug/.cvsignore
BitKeeper/deleted/.del-vargs.h:
  Delete: dbug/vargs.h

into  desktop.sanja.is.com.ua:/home/bell/tmp/work-maria-assert


storage/maria/ma_loghandler.c:
  Auto merged

storage/maria/unittest/Makefile.am:
  fixed names of tests.
storage/maria/unittest/ma_pagecache_consist.c:
  Some of pagecache multi-thread tests are not big.
  Fixed reporting of tests end (now under mutex protection).
storage/maria/unittest/ma_test_loghandler-t.c:
  fixed layout to make blocks nof parameters more visible.

Fixed bug where return value 'error' was not set in case of error in pagecache
Documented the open LOAD INDEX bug


KNOWN_BUGS.txt:
  Added the problem with LOAD INDEX as a known bugs. Will fix this bug later this week
storage/maria/ma_pagecache.c:
  Fixed portability issue with comparing thread id
  Fixed bug where return value 'error' was not set in case of error

into  mysql.com:/home/my/mysql-maria


mysys/my_pread.c:
  Auto merged
mysys/my_read.c:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_blockrec.c:
  Auto merged
storage/maria/ma_check.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_page.c:
  Auto merged
storage/maria/ma_write.c:
  Auto merged
storage/maria/maria_chk.c:
  Auto merged
storage/maria/maria_def.h:
  Auto merged

Fixed bug when calculating max_key_length that caused some ALTER TABLE to fail if MAX_ROWS was used.
Use maria_block_size instead of MARIA_MIN_KEY_BLOCK_LENGTH
Fixed bug when scanning table with BLOCK format for repair; If table was > bitmap coverage one page block was read twice which caused a lot of duplicate key errors
Could not repeat Bug#34106 "auto_increment is reset to 1 when table is recovered from crash" after this patch.

NOTE: This is an incompatible change, so one must do maria_chk -r on ones old Maria tables!
Sorry, but this was needed to fix the bug with max_key_length and to be able to handle bigger key files with smaller key references


cmd-line-utils/readline/readline.c:
  Fixed compiler warnings
mysql-test/r/maria.result:
  Added more test of auto-increment handling
mysql-test/t/maria.test:
  Added more test of auto-increment handling
mysys/my_pread.c:
  Fixed wrong test
  Removed not needed tests (error is always 1 if readbytes != Count)
mysys/my_read.c:
  Fixed wrong test
storage/maria/ha_maria.cc:
  Disable LOAD INDEX until I got Sanja's extension to pagecache interface
storage/maria/ma_blockrec.c:
  Ensure that info->last_auto_increment is reset properly
storage/maria/ma_check.c:
  Fixed wrong printing of row number in case of duplicate key for --safe-repair
  Safety fix in recreate table so that Column numbers are given to maria_create() in original order
  Added missing HA_OPEN_FOR_REPAIR to maria_open()
  Fixed bug when scanning table with BLOCK format for repair; If table was > bitmap coverage one page block was read twice which caused a lot of duplicate key errors
storage/maria/ma_create.c:
  Use correct value for how much free space there is on a key page
  Remember some missing table option when doing re-create.
  Removed optimization where last packed fields is unpacked; Caused problems for re-create.
storage/maria/ma_delete.c:
  Ensure that info->last_auto_increment is reset properly
  Fix for update to restore autoincrement value on duplicate key
storage/maria/ma_key_recover.c:
  Moved handling of restoring value of auto-increment in case of duplicate key from clr to undo
  This ensures the restoring works both for insert and update and also that this is symetrical to how the auto_increment value is stored
storage/maria/ma_key_recover.h:
  Added new prototype
storage/maria/ma_loghandler.c:
  Added hook to write_hook_for_undo_key_delete()
storage/maria/ma_open.c:
  Fixed wrong calculation of max_key_file_length
storage/maria/ma_page.c:
  Use maria_block_size instead of MARIA_MIN_KEY_BLOCK_LENGTH
  Increase internal buffer (safety fix)
storage/maria/ma_search.c:
  Use maria_block_size instead of MARIA_MIN_KEY_BLOCK_LENGTH
  Note that this is an incompatible change, so one must do maria_chk -r on ones old Maria tables (sorry)
storage/maria/ma_update.c:
  Ensure that info->last_auto_increment is reset properly
storage/maria/ma_write.c:
  Ensure that info->last_auto_increment is reset properly
  Fix for update to restore autoincrement value on duplicate key
storage/maria/maria_chk.c:
  Allow small page_buffer_size
  Fixed printing for --describe to better fit into 80 characters
storage/maria/maria_def.h:
  Added comments

into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-assert

into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-write


storage/maria/ma_pagecache.c:
  Auto merged

 case of opening and closing loghandler without writing
 sometrhing in it.
Fixed comment (sent_to_file was renamed to sent_to disk)
Write log during shutdown in natural order.

Added support for delete by link.
Aded level ("hits") management functions.


storage/maria/ma_pagecache.h:
  Added support for delete by link.
  Aded level ("hits") management functions.
storage/maria/unittest/ma_pagecache_single.c:
  Test of delete by link.

into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria


storage/maria/ma_loghandler.c:
  Auto merged

storage/maria/ma_loghandler.c:
  Compiler warning fixed.
storage/maria/ma_test1.c:
  Compiler warning fixed.
storage/maria/ma_test2.c:
  Compiler warning fixed.
storage/maria/maria_chk.c:
  Compiler warning fixed.
storage/maria/unittest/ma_control_file-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Compiler warning fixed.

into  gbichot4.local:/home/mysql_src/mysql-maria-monty


mysql-test/t/maria-recovery.test:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged

command-line" and BUG#34062 "Maria table corruption on master".
Use 5 bytes (instead of 4) to store page's number in the checkpoint
record, to allow bigger table (1PB with maria-block-size=1kB).
Help pushbuild not run out of memory by moving the portion of
maria-recovery.test which generates lots of data into a -big.test.


mysql-test/r/maria-recovery.result:
  result moved
mysql-test/t/maria-recovery.test:
  piece which generates much data moved to maria-recovery-big.test
mysys/my_pread.c:
  To fix BUG#34062, where a 1.1TB file was generated due to a wrong
  pwrite offset, it was useful to not lose precision on 'offset' in
  DBUG_PRINT, so that the crazy value is visible.
mysys/my_read.c:
  To fix BUG#34062, where a 1.1TB file was generated due to a wrong
  pwrite offset, it was useful to not lose precision on 'offset' in
  DBUG_PRINT, so that the crazy value is visible.
mysys/my_write.c:
  To fix BUG#34062, where a 1.1TB file was generated due to a wrong
  pwrite offset, it was useful to not lose precision on 'offset' in
  DBUG_PRINT, so that the crazy value is visible.
storage/maria/ha_maria.cc:
  When starting a bulk insert, we throw away dirty index pages from the
  cache. Unique (non disabled) key insertions thus read out-of-date
  pages from the disk leading to BUG#34062 "Maria table corruption on
  master": a DELETE in procedure viewer_sp() had deleted all rows of
  viewer_tbl2 one by one, putting index page 1 into key_del; that page
  was thrown away at start of INSERT SELECT, then the INSERT SELECT
  needed a page to insert keys, looked at key_del, found 1, read page 1
  from disk, and its out-of-date content was used to set the new value of
  key_del (crazy value of 1TB), then a later insertion needed another
  index page, tried to read page at this crazy offset and failed, leading
  to corruption mark.
  The fix is to destroy out-of-date pages and make the state consistent
  with that, i.e. call maria_delete_all_rows().
storage/maria/ma_blockrec.c:
  Special hook for UNDO_BULK_INSERT
storage/maria/ma_blockrec.h:
  special hook for UNDO_BULK_INSERT
storage/maria/ma_check.c:
  Fix for BUG#34114 "maria_chk reports false error when several tables on
  command-line": if the Nth (on the command line) table was BLOCK_RECORD
  it would start checks by using the param->record_checksum computed by
  checks of table N-1.
storage/maria/ma_delete_all.c:
  comment
storage/maria/ma_loghandler.c:
  special hook for UNDO_BULK_INSERT
storage/maria/ma_page.c:
  comment
storage/maria/ma_pagecache.c:
  page number is 5 bytes in checkpoint record now (allows bigger tables)
storage/maria/ma_recovery.c:
  page number is 5 bytes in checkpoint record now
storage/maria/ma_recovery_util.c:
  page number is 5 bytes now
storage/maria/ma_write.c:
  typo
mysql-test/r/maria-recovery-big.result:
  result is correct
mysql-test/t/maria-recovery-big-master.opt:
  usual options for recovery tests
mysql-test/t/maria-recovery-big.test:
  Moving out the big blob test to a -big test (it exhausts memory when
  using /dev/shm on certain machines)

into  janus.mylan:/usr/home/serg/Abk/mysql-maria

storage/maria/unittest/ma_test_all-t:
  Size doesn't matter or ma_test_recovery is not big

into  janus.mylan:/usr/home/serg/Abk/mysql-maria

Makefile.am:
  make test-unit-big
storage/maria/unittest/Makefile.am:
  support big unit tests
storage/maria/unittest/ma_pagecache_consist.c:
  support big unit tests and other changes
storage/maria/unittest/ma_pagecache_single.c:
  support big unit tests
storage/maria/unittest/ma_test_all-t:
  support big unit tests and sub skip()
storage/maria/unittest/ma_test_loghandler-t.c:
  support big unit tests
unittest/unit.pl:
  don't ignore existing $MYTAP_CONFIG

    1. reasonable defaults for maria_path
    2. reasonable TAP-compatible output in non-verbose mode
unittest/Makefile.am
    remove maria_path hack
BitKeeper/etc/ignore@1.269
    added maria_log.00000* ma_test_recovery.output test?.MA?


BitKeeper/etc/ignore:
  added maria_log.00000* ma_test_recovery.output test?.MA?
storage/maria/unittest/ma_test_all-t:
  1. reasonable defaults for maria_path
  2. reasonable TAP-compatible output in non-verbose mode
unittest/Makefile.am:
  remove maria_path hack

into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-logdir

Fixed typo in maria_read_log "help" output.
Compilation warning on Mac OS fixed.


mysql-test/include/maria_empty_logs.inc:
  Added support of moving maria log relatively
    to master data directory.
mysql-test/include/maria_make_snapshot_for_feeding_recovery.inc:
  Added support of moving maria log relatively
    to master data directory.
mysql-test/include/maria_verify_recovery.inc:
  Added support of moving maria log relatively
    to master data directory.
mysql-test/t/maria-recovery-master.opt:
  Check of recovery with logs in other place.
mysql-test/t/maria-recovery.test:
  Check of recovery with logs in other place.
storage/maria/ha_maria.cc:
  Support of moving logs in other directory then data root.
storage/maria/ma_loghandler.c:
  Compilation warning on Mac OS fixed.
storage/maria/ma_static.c:
  Support of moving logs in other directory then data root.
storage/maria/ma_test1.c:
  Support of moving logs in other directory then data root.
storage/maria/maria_def.h:
  Support of moving logs in other directory then data root.
storage/maria/maria_read_log.c:
  Support of moving logs in other directory then data root.
  Fixed typo in maria_read_log "help" output.

Fixed bug when using maria_chk --safe-recover and getting duplicated keys
Fixed bug when using maria_pack --join


storage/maria/ma_check.c:
  Fixed bug when using --safe-recover with packed tables
  Fixed bug when using --safe-recover and getting duplicated keys
storage/maria/maria_pack.c:
  Fixed bug when using --join
storage/maria/unittest/ma_test_all-t:
  Added test to check maria_pack --join

Fixes for the remaining 'not-yet-fixed' bugs found by Guilhem when running all tests with maria engine and one bug in maria_pack reported by Martin:
- Fix that query cache works with Maria
- Fixed wrong calculation if min_key_length which casued some tables to assert in insert if key contained NULL
- Restore auto_increment value if insert statement fails
- Fixed rare bug that caused crash in maria_pack if buffer was flushed at wrong point in time


configure.in:
  Added maria extension to distribution
mysql-test/r/maria.result:
  More test cases to cover bugs found by Guilhem
mysql-test/t/maria.test:
  More test cases to cover bugs found by Guilhem
storage/maria/ha_maria.cc:
  Fix that query cache works with Maria
storage/maria/ma_blockrec.c:
  Removed duplicate functionallity (already done in ma_pagecrc.c)
storage/maria/ma_create.c:
  Fixed wrong calculation if min_key_length which casued some tables to assert in insert if key contained NULL
storage/maria/ma_key_recover.c:
  Restore auto_increment value if insert statement fails
storage/maria/maria_def.h:
  Added variables to track auto_increment changes to be able to restore it in case of duplicate key
storage/maria/maria_pack.c:
  Fixed rare bug that caused crash if buffer was flushed at wrong point in time
storage/maria/unittest/ma_test_all-t:
  Added option --abort-on-error
  Ordered help message

into  mysql.com:/home/my/mysql-maria


mysql-test/r/maria.result:
  Auto merged
mysql-test/t/maria.test:
  Auto merged
sql/sql_table.cc:
  Auto merged
storage/maria/ma_create.c:
  Auto merged
storage/maria/ma_pagecrc.c:
  Auto merged

Fixed error in Maria when using table with only CHAR(0) fields
Fixed valgrind warning


BitKeeper/etc/ignore:
  added storage/maria/maria_dump_log
mysql-test/r/maria.result:
  Testing of table with char(0)
mysql-test/t/maria.test:
  Testing of table with char(0)
sql/sql_table.cc:
  Allow index on 'CHAR(0) NULL' columns
storage/maria/ma_create.c:
  Allow creation of table with 0 record data (for example CHAR(0))
storage/maria/ma_pagecrc.c:
  Fixed valgrind warning

sql/sql_table.cc:
  initialize variables used in 'err:' before going to err.

top-level statement) and fixes for the bugs it finds.
Fix for non-serious Valgrind warning.


sql/sql_insert.cc:
  When CREATE TABLE IF NOT EXISTS finds the table already exists,
  'table' is the existing table. So if that table is temporary we don't
  re-enable transactions which is a bug.
sql/sql_parse.cc:
  verify that at the end of each top-statement transactions have
  been re-enabled. Does not apply to substatements (consider
  CREATE TABLE t1 SELECT stored_func() : the substatements inside
  stored_func() run with transaction disabled).
  I am not putting the assertion into ha_external_lock(F_UNLCK) because
  performance schema tables get closed in the middle of a statement
  sometimes while transaction is disabled.
sql/sql_table.cc:
  copy_data_between_tables() forgot to clean-up several things in error
  conditions (ha_enable_transaction(), free-ing 'copy', etc) as found
  by the assertion added to sql_parse.cc.
storage/maria/ha_maria.cc:
  Comment
storage/maria/ma_blockrec.c:
  fix for Valgrind warning: a temporary table was created, a blob
  page of its was flushed to disk and had random bytes in the checksum
  area ("write of uninitialized bytes in pwrite")
storage/maria/ma_pagecrc.c:
  typo

(fix is keeping the real TRN through a disable_logging/reenable cycle)
- fix for pagecache assertion failure in ps/type_ranges with default
engine=maria (fix is in sql_insert.cc)
- when reenabling logging we must either flush all dirty pages,
or at least verify (in debug build) that there are none. For example
a bulk insert with single UNDO_BULK_INSERT must flush them, no matter
if it uses repair or not (bugfix)
- UNDO_BULK_INSERT_WITH_REPAIR is also used with repair, changes name


mysql-test/r/maria.result:
  tests for bugs fixed
mysql-test/t/maria.test:
  tests for bugs fixed
sql/sql_insert.cc:
  Bugfix: even if select_create::prepare() failed to create the 'table' object
  we still have to re-enable logging.
storage/maria/ha_maria.cc:
  Bugfix: when a transactional table does a bulk insert without
  repair, it still sometimes skips logging of REDOs thus needs a full
  flush and sync at the end. Not if repair is done, as repair does
  it internally already (see end of maria_repair*()).
storage/maria/ha_maria.h:
  variable now can have 3 states not 2
storage/maria/ma_bitmap.c:
  name change
storage/maria/ma_blockrec.c:
  name change
storage/maria/ma_blockrec.h:
  name change
storage/maria/ma_check.c:
  * When maria_repair() re-enables logging it does not need to ask for
  a flush&sync as it did it by itself already a few lines before.
  * the log record of bulk insert can be used even without repair
  * disable logging in maria_zerofill(): without that, it puts LSN pages
  in the cache, so when it flushes them it flushes the log; the change
  makes auto-ha_maria::zerofill-if-moved faster (no log flush).
storage/maria/ma_key_recover.c:
  name change
storage/maria/ma_loghandler.c:
  name change
storage/maria/ma_loghandler.h:
  name change
storage/maria/ma_pagecache.c:
  A function, to check in debug builds that no dirty pages exist for a file.
storage/maria/ma_pagecache.h:
  new function (nothing in non-debug)
storage/maria/ma_recovery.c:
  _ma_tmp_disable_logging() sets info->trn to dummy_transaction_object
  when needed now. The changes done here about info->trn are to allow
  a table to retain its original, real TRN through a disable/reenable
  cycle (see replication scenario in _ma_reenable_logging_for_table()).
  When we reenable, we offer the caller to flush and sync the table;
  if the caller doesn't accept our offer, we verify that it's ok
  (no REDOs => no dirty pages are allowed to exist).
storage/maria/maria_chk.c:
  comment
storage/maria/maria_def.h:
  new names
mysql-test/suite/rpl/r/rpl_stm_maria.result:
  result (it used to crash)
mysql-test/suite/rpl/t/rpl_stm_maria.test:
  Test of replication-specific Maria bug fixed

- fix for bug seen when running test "type_datetime" with Maria
(wrong data_file_length in maria_repair())
- fix for bug seen when running test "repair" with Maria
(myisam_sort_buffer_size was influencing Maria)


sql/handler.cc:
  Sounds illogical to store myisam_sort_buffer_size into a structure
  used by all engines. There are only MyISAM and Maria which used 
  sort_buffer_size: they can get their value from their respective
  system variable (myisam|maria_sort_buffer_size). Using MyISAM's value
  for all engines was wrong (myisam_sort_buffer_size influenced Maria).
sql/handler.h:
  not needed
storage/maria/ha_maria.cc:
  check_opt->sort_buffer_size was myisam_sort_buffer_size; Maria
  must use maria_sort_buffer_size instead.
storage/maria/ma_bitmap.c:
  don't use my_chsize() now that Monty re-explained the problem to me :)
storage/maria/ma_check.c:
  making maria_repair() work like maria_repair_by_sort(): sort_param.filepos
  must be set at start then possibly corrected by create_new_data_handle();
  in the opposite order, filepos is finally set to 0, and if the table
  has no records, it stays 0 and this causes state.data_file_length
  to be 0 which is incorrect for a BLOCK_RECORD table having always
  at least one bitmap page.
storage/maria/ma_pagecache.c:
  Comments
storage/myisam/ha_myisam.cc:
  check_opt->sort_buffer_size is gone

storage/maria/ma_check.c:
  fix for important compiler warning
storage/maria/ma_recovery.c:
  fix for compiler warning