A corrupted compressed table could crash the server and
myisamchk.

The data file of an uncompressed table contains just the records.
There is no header in the data file.

However the data file of a compressed table has a header.
The header describes how the table was compressed. This
information is necessary to extract the records from the
compressed data file.

Part of the compressed data file header are the [de]code tables.
They are numeric representations of the Huffman trees used for
coding and decoding. A Huffman tree is a binary tree. Every
node has two childs. A child can be a leaf or a branch. Leaves
contain the decoded value. Branches point to another tree node.

Since the [de]code table is represented as an array of childs,
the branches need to point at a child within the same array.
The corruption of the compressed data file from the bug report
was a couple of branches that pointed outside their array.
This condition had not been correctly checked.

I added some checks for the pointers in the decode tables.
This type of corruption will no longer crash the server or
myisamchk.

No test case. A corrupted compressed table is required.


myisam/mi_packrec.c:
  Bug#23139 - myisamchk and mysqld crash when trying to access table
  Added some checks for the pointers in the decode tables.
  Added comments, DBUG prints, style fixes.

into  chilla.local:/home/mydev/mysql-4.1-bug8283-one


myisam/mi_check.c:
  Auto merged
myisam/mi_packrec.c:
  Auto merged
myisam/sort.c:
  Auto merged
mysql-test/r/myisam.result:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Manual merge
mysql-test/t/myisam.test:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Manual merge

OPTIMIZE TABLE with myisam_repair_threads > 1 performs a non-quick 
parallel repair. This means that it does not only rebuild all 
indexes, but also the data file.

Non-quick parallel repair works so that there is one thread per 
index. The first of the threads rebuilds also the new data file.

The problem was that all threads shared the read io cache on the
old data file. If there were holes (deleted records) in the table,
the first thread skipped them, writing only contiguous, non-deleted
records to the new data file. Then it built the new index so that
its entries pointed to the correct record positions. But the other
threads didn't know the new record positions, but put the positions
from the old data file into the index.

The new design is so that there is a shared io cache which is filled
by the first thread (the data file writer) with the new contiguous
records and read by the other threads. Now they know the new record
positions.

Another problem was that for the parallel repair of compressed
tables a common bit_buff and rec_buff was used. I changed it so
that thread specific buffers are used for parallel repair.

A similar problem existed for checksum calculation. I made this
multi-thread safe too.


include/my_sys.h:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Redesign of io_cache_share.
include/myisam.h:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Redesign of checksum calculation in mi_check.c.
  'calc_checksum' is now in myisamdef.h:st_mi_sort_param.
myisam/mi_check.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Implemented a new parallel repair design.
  Using a synchronized shared read/write cache.
  Allowed for thread specific bit_buff, rec_buff, and calc_checksum.
myisam/mi_open.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added DBUG output.
myisam/mi_packrec.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Allowed for thread specific bit_buff and rec_buff.
myisam/myisamdef.h:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Commented on checksum calculation variables.
  Allowed for thread specific bit_buff.
  Added DBUG output for better table crash detection.
myisam/sort.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added implications of the new parallel repair design.
  Renamed 'info' -> 'sort_param'.
  Added DBUG output.
mysql-test/r/myisam.result:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added test results.
mysql-test/t/myisam.test:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added test cases.
mysys/mf_iocache.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Redesign of io_cache_share.
  We do now allow a writer to synchronize himself with the
  readers of a shared cache. When all threads join in the lock,
  the writer copies the data from his write buffer to the shared
  read buffer.

into  mysql.com:/home/svoj/devel/mysql/BUG22937/mysql-4.1-engines

This is addition to fix for bug21617. Valgrind reports an error when
opening merge table that has underlying tables with less indexes than
in a merge table itself.

Copy at most min(file->keys, table->key_parts) elements from rec_per_key array.
This fixes problems when merge table and subtables have different number of keys.


sql/ha_myisammrg.cc:
  Copy at most min(file->keys, table->key_parts) elements from rec_per_key array.
  This fixes problems when merge table and subtables have different number of keys.

into  mysql.com:/home/svoj/devel/mysql/BUG21381/mysql-4.1-engines

Though this is not storage engine specific problem, I was able to
repeat this problem with BDB and NDB engines only. That was the
reason to add a test case into ndb_update.test. As a result
different bad things could happen.

BDB has removed duplicate rows which is not expected.
NDB returns an error.

For multi table update notify storage engine about UPDATE IGNORE
as it is done in single table UPDATE.


mysql-test/r/ndb_update.result:
  A test case for bug#21381.
mysql-test/t/ndb_update.test:
  A test case for bug#21381.
sql/sql_update.cc:
  For multi table update notify storage engine about UPDATE IGNORE
  as it is done in single table UPDATE.

into  mysql.com:/home/svoj/devel/mysql/merge/mysql-4.1-engines

into  mysql.com:/home/svoj/devel/mysql/merge/mysql-4.1-engines


mysql-test/r/myisam.result:
  Auto merged
mysql-test/t/myisam.test:
  Auto merged
sql/table.cc:
  Auto merged

into  chilla.local:/home/mydev/mysql-4.0-bug20719

into  chilla.local:/home/mydev/mysql-4.1-bug22384

into  chilla.local:/home/mydev/mysql-4.1-bug22384


myisam/mi_delete.c:
  Auto merged

into  mysql.com:/home/svoj/devel/mysql/BUG21617/mysql-4.1-engines


myisammrg/myrg_open.c:
  Auto merged
mysql-test/r/merge.result:
  Manual merge.
mysql-test/t/merge.test:
  Manual merge.

Crash may happen when selecting from a merge table that has underlying
tables with less indexes than in a merge table itself.

If number of keys in merge table is not bigger than requested key number,
return error.


myisammrg/myrg_open.c:
  Store min(number of keys) in m_info instead of number of keys in last
  underlying table.
myisammrg/myrg_queue.c:
  Return error if inx passed to _myrg_init_queue function is not less
  than number of keys.
mysql-test/r/merge.result:
  A test case for bug#21617.
mysql-test/t/merge.test:
  A test case for bug#21617.
mysys/queues.c:
  Replaced annoying ifndef DBUG_OFF with DBUG_ASSERT, fixed coding style.
  The problem was that having queue overrun in debug build was hidden
  with this ifdef.

into  chilla.local:/home/mydev/mysql-4.1-bug20719-m

Deletes on a big index could crash the index when it needs to
shrink.

Put a forgotten negation operator in.

No test case. It is too big for the test suite. And it does not
work with 4.0, only with higher versions. It is attached to the
bug report.


myisam/mi_delete.c:
  Bug#22384 - DELETE FROM table causes "Incorrect key file for table"
  Put a negation operator ('!') before _mi_get_last_key() in del().
  It returns NULL on error, non-NULL on success.

into  dl145s.mysql.com:/data/bk/team_tree_merge/MERGE/mysql-4.1-opt


sql/sql_select.cc:
  Auto merged

into  mysql.com:/home/svoj/devel/mysql/merge/mysql-4.1-engines

into  chilla.local:/home/mydev/mysql-4.0-bug20719

After merge fix.

into  chilla.local:/home/mydev/mysql-4.1-bug20719-m


myisam/mi_dynrec.c:
  Auto merged

into  mysql.com:/home/hf/work/mysql-4.1.15428

an ALL/ANY quantified subquery in HAVING.
The Item::split_sum_func2 method should not create Item_ref
for objects of any class derived from Item_subselect.


mysql-test/r/subselect.result:
  Added a test case for bug #21853.
mysql-test/t/subselect.test:
  Added a test case for bug #21853.

into  mysql.com:/home/hf/work/mysql-4.1.15428


myisam/mi_check.c:
  Auto merged

into  mysql.com:/home/hf/work/mysql-4.1.15428

  Filter out plain -O and Sun C/C++ style optimization flags, -xO<level>
  Filter out icc specific options from cflags/libs(_r)


scripts/mysql_config.sh:
  Filter out plain -O and Sun C/C++ style optimization flags, -xO<level>

into  chilla.local:/home/mydev/mysql-4.1-bug14400-monty


myisam/mi_rkey.c:
  Bug#14400 - Query joins wrong rows from table which is subject of
              "concurrent insert"
  Merge of after merge fix
mysql-test/r/myisam.result:
  Bug#14400 - Query joins wrong rows from table which is subject of
              "concurrent insert"
  Merge of after merge fix
mysql-test/t/myisam.test:
  Bug#14400 - Query joins wrong rows from table which is subject of
              "concurrent insert"
  Merge of after merge fix

into  mysql.com:/users/lthalmann/bk/MERGE/mysql-4.1-merge


BitKeeper/etc/ignore:
  auto-union
mysql-test/r/myisam.result:
  Auto merged
mysql-test/t/myisam.test:
  Auto merged
sql/sql_select.cc:
  Auto merged

into  mysql.com:/users/lthalmann/bk/MERGE/mysql-4.1-merge


BitKeeper/etc/ignore:
  auto-union
mysql-test/r/myisam.result:
  Auto merged
mysql-test/t/myisam.test:
  Auto merged
sql/sql_select.cc:
  Auto merged

            "concurrent insert"
After merge fix.

into  chilla.local:/home/mydev/mysql-4.0-bug14400


mysql-test/r/myisam.result:
  Auto merged
myisam/mi_rkey.c:
  Bug#14400 - Query joins wrong rows from table which is subject of
              "concurrent insert"
  Manual merge
mysql-test/t/myisam.test:
  Bug#14400 - Query joins wrong rows from table which is subject of
              "concurrent insert"
  Manual merge

into  chilla.local:/home/mydev/mysql-4.0-bug14400


myisam/mi_rkey.c:
  Bug#14400 - Query joins wrong rows from table which is subject of
              "concurrent insert"
  Manual merge

mysql-test/r/rpl_insert_id.result:
  Updated result file

Any default value for a enum fields over UCS2 charsets was corrupted
when we put it into the frm file, as it had been overwritten by its
HEX representation.
To fix it now we save a copy of structure that represents the enum
type and when putting the default values we use this copy. 


mysql-test/r/ctype_ucs.result:
  Added a test case for bug #20108.
mysql-test/t/ctype_ucs.test:
  Added a test case for bug #20108.

into  mysql.com:/users/lthalmann/bk/MERGE/mysql-4.1-merge


sql/sql_yacc.yy:
  Auto merged