Fulltext boolean mode phrase search may crash server on platforms
where size of pointer is not equal to size of unsigned integer
(in other words some 64-bit platforms).

The problem was integer overflow.

Affects 4.1 only.


myisam/ft_boolean_search.c:
  my_match_t::beg is unsigned int, that means type of expression
  (m[0].beg - 1) has unsigned type too. It may happen that instr()
  finds substring in the beggining of passed string, returning
  m[0].beg equal to 0. In this case value of expression (m[0].beg - 1)
  is equal to MAX_UINT.
  
  This is not a problem on platforms where sizeof(pointer) equals to
  sizeof(uint). That means ptr[(uint)-1] = ptr[(uint)MAX_UINT] = ptr - 1.
  
  On some 64-bit platforms where sizeof(pointer) is 8 and sizeof(uint)
  is 4, wrong address gets accessed. In other words ptr[(uint)-1] is
  equal to ptr + MAX_UINT.
mysql-test/r/fulltext.result:
  A test case for BUG#11392.
mysql-test/t/fulltext.test:
  A test case for BUG#11392.

into  mysql.com:/home/svoj/devel/mysql/BUG31159/mysql-4.1-engines

ucs2 doesn't provide required by fulltext ctype array. Crash
happens because fulltext attempts to use unitialized ctype
array.

Fixed by converting ucs2 fields to compatible utf8 analogue.


include/my_sys.h:
  Added a function to find compatible character set with ctype array
  available. Currently used by fulltext search to find compatible
  substitute for ucs2 collations.
mysql-test/r/ctype_ucs.result:
  A test case for BUG#31159.
mysql-test/t/ctype_ucs.test:
  A test case for BUG#31159.
mysys/charset.c:
  Added a function to find compatible character set with ctype array
  available. Currently used by fulltext search to find compatible
  substitute for ucs2 collations.
sql/item_func.cc:
  Convert ucs2 fields to utf8. Fulltext requires ctype array, but
  ucs2 doesn't provide it.

into  anubis.xiphis.org:/usr/home/antony/work/mysql-4.1-engines.merge

into  polly.(none):/home/kaa/src/maint/mysql-4.1-maint

myisam_sort_buffer_size.

An incorrect length of the sort buffer was used when calculating the
maximum number of keys. When myisam_sort_buffer_size is small enough,
this could result in the number of keys < number of
BUFFPEK structures which in turn led to use of uninitialized BUFFPEKs.

Fixed by correcting the buffer length calculation.


myisam/sort.c:
  Use a correct buffer length when calculating the maximum number of keys.
  Assert that for each BUFFPEK structure there is at least one
  corresponding key. Otherwise we would fail earlier and not reach
  merge_buffers().
mysql-test/r/repair.result:
  Added a test case for bug #31174.
mysql-test/t/repair.test:
  Added a test case for bug #31174.

into  polly.(none):/home/kaa/src/maint/mysql-4.1-maint

into  sin.intern.azundris.com:/home/tnurnberg/30444/41-30444

Bug#28878: InnoDB tables with UTF8 character set and indexes cause  wrong result for DML
When making key reference buffers over CHAR fields whitespace (0x20) must be used to fill in the remaining space in the field's buffer. This is what Field_string::store() does. Fixed Field_string::get_key_image() to do the same.


mysql-test/r/innodb_mysql.result:
  Bug#28878: test case
mysql-test/t/innodb_mysql.test:
  Bug#28878: test case
sql/field.cc:
  Bug#28878: Fill with space instead of binary zeros.

into  sin.intern.azundris.com:/home/tnurnberg/30444/41-30444

The combination of --single-transaction and --master-data requires
START TRANSACTION WITH CONSISTENT SNAPSHOT which is available from
mysqld 4.1 on. When trying this against an older server, print
diagnostic, then, if --force is not given, abort.

No test-case given since it would require a mysqld < 4.1.


client/mysqldump.c:
  Bug #30444: 5.0 mysqldump silently allows wrong backup to be taken against a 4.0 database
  
  The combination of --single-transaction and --master-data requires
  START TRANSACTION WITH CONSISTENT SNAPSHOT which is available from
  mysqld 4.1 on. When trying this against an older server, print
  diagnostic, then, if --force is not given, abort.

and for bug #31070: crash during conversion of charsets

Problem: passing a 0 byte length string to some my_mb_wc_XXX() 
functions leads to server crash due to improper argument check.

Fix: properly check arguments passed to my_mb_wc_XXX() functions.


mysql-test/include/ctype_common.inc:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - test case.
mysql-test/r/ctype_big5.result:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - test result.
mysql-test/r/ctype_euckr.result:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - test result.
mysql-test/r/ctype_gb2312.result:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - test result.
mysql-test/r/ctype_gbk.result:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - test result.
mysql-test/r/ctype_uca.result:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - test result.
strings/ctype-big5.c:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - check the string length before testing its first byte.
strings/ctype-cp932.c:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - check the string length before testing its first byte.
strings/ctype-euc_kr.c:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - check the string length before testing its first byte.
strings/ctype-gb2312.c:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - check the string length before testing its first byte.
strings/ctype-sjis.c:
  Fix for bug #31069: crash in 'sounds like'
  and bug #31070: crash during conversion of charsets
    - check the string length before testing its first byte.

into  stella.local:/home2/mydev/mysql-4.1-axmrg

into  trift2.:/MySQL/M41/push-4.1

into  stella.local:/home/mydev/mysql-4.1-axmrg

DELETE query against memory table with btree index may remove
not all matching rows. This happens only when DELETE uses
index read method to find matching rows. E.g. for queries
like DELETE FROM t1 WHERE a=1.

Fixed by reverting fix for BUG9719 and applying proper solution.


heap/hp_delete.c:
  Reverted fix for BUG9719 as it makes queries like
  DELETE FROM t1 WHERE a=1 to remove not all matching
  rows (assuming this is memory table and there is btree
  key over `a`).
  
  This happens because we calculate info->lastkey_len in
  heap_rkey(). When we enter heap_rnext(), info->lastkey_len
  is 0 (set by hp_rb_delete_key()). We need to preserve
  info->lastkey_len in this situation, otherwise
  tree_search_key() will always return smallest value in
  a tree.
heap/hp_rfirst.c:
  If we're performing index_first on a table that was taken from
  table cache, info->lastkey_len is initialized to previous query.
  Thus we set info->lastkey_len to proper value for subsequent
  heap_rnext() calls.
  This is needed for DELETE queries only, otherwise this variable is
  not used.
  Note that the same workaround may be needed for heap_rlast(), but
  for now heap_rlast() is never used for DELETE queries.
heap/hp_rnext.c:
  An optimization for DELETE queries that use index_first()/index_next().
  Use faster tree_search_edge() instead of tree_search_key().
mysql-test/r/heap_btree.result:
  A test case for BUG#30590.
mysql-test/t/heap_btree.test:
  A test case for BUG#30590.

in get_index_for_order(), don't walk over the end of the index key parts
when matching index description and needed ordering.


mysql-test/r/delete.result:
  BUG#30385: Testcase
mysql-test/t/delete.test:
  BUG#30385: Testcase

into  trift2.:/MySQL/M41/push-4.1

  Deleted BitKeeper soft links not needed


BitKeeper/deleted/.del-client_settings.h:
  Delete: libmysql_r/client_settings.h
BitKeeper/deleted/.del-ha_blackhole.cc:
  Delete: libmysqld/ha_blackhole.cc

  Flag changed name in icc 10


scripts/mysql_config.sh:
  Flag changed name in icc 10

'mysqld_wait_started' don't return prematurely because of an old
pidfile

mysql-test/lib/mtr_misc.pl:
  Add function 'mtr_rmtree' it will try 'rmtree' and if that fails (most likely
  due to permission problems we will fun File::find to chmod all files and dirs
  to 0777 and then delete.
mysql-test/mysql-test-run.pl:
  Use 'mtr_rmtree' in favour of 'rmtree'

into  synthia.local:/home/mydev/mysql-4.1-axmrg

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1-build

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1

into  trift2.:/MySQL/M41/push-4.1

into  chilla.local:/home/mydev/mysql-4.1-axmrg

into  production.mysql.com:/usersnfs/jperkin/bk/trees/build/mysql-4.1

  Fix typo in usage.


myisam/myisamchk.c:
  Fix typo in usage.

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  production.mysql.com:/usersnfs/jperkin/bk/mysql-4.1-maint

which does not work.  Removing these attempted privileges makes
this identical to option 5 so remove it completely.  The spirit
of the program appears to be aimed at database privileges, so do
not add another option for granting global privileges as it may
be unexpected.  Fixes bug#14618 (same as previous patch, this
time applied to -maint tree).


scripts/mysql_setpermission.sh:
  Option 6 tries to apply global privileges at the database
  level which does not work - remove it.

When using concurrent insert with parallel index reads, it could
happen that reading sessions found keys that pointed to records
yet to be written to the data file. The result was a report of
a corrupted table. But it was false alert.

When inserting a record in a table with indexes, the keys are
inserted into the indexes before the record is written to the data
file. When the insert happens concurrently to selects, an
index read can find a key that references the record that is not
yet written to the data file. To avoid any access to such record,
the select saves the current end of file position when it starts.
Since concurrent inserts are always appended at end of the data
file, the select can easily ignore any concurrently inserted record.

The problem was that the ignore was only done for non-exact key
searches (partial key or using >, >=, < or <=).

The fix is to ignore concurrently inserted records also for
exact key searches.

No test case. Concurrent inserts cannot be tested with the test
suite. Test cases are attached to the bug report.


myisam/mi_rkey.c:
  Bug#29838 - myisam corruption using concurrent select ... and update
  Fixed mi_rkey() to always ignore records beyond saved eof.

into  mysql.com:/home/hf/work/29717/my41-29717


sql/sql_select.cc:
  Auto merged

 SELECT statement itself returns empty.

As a result of this bug 'SELECT AGGREGATE_FUNCTION(fld) ... GROUP BY'
can return one row instead of an empty result set.

When GROUP BY only has fields of constant tables
(with a single row), the optimizer deletes the group_list.
After that we lose the information about whether we had an
GROUP BY statement. Though it's important
as SELECT min(x) from empty_table; and
   SELECT min(x) from empty_table GROUP BY y; have to return
different results - the first query should return one row,
second - an empty result set.
So here we add the 'group_optimized_away' flag to remember this case
when GROUP BY exists in the query and is removed
by the optimizer, and check this flag in end_send_group()


mysql-test/r/group_by.result:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test result
mysql-test/r/insert_select.result:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test result
mysql-test/t/group_by.test:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  This is additional testcase that is more basic than the
  original bug's testcase and has the same reason.
mysql-test/t/insert_select.test:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test case
sql/sql_select.cc:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  Remember the 'GROUP BY was optimized away' case in the JOIN::group_optimized
  and check this in the end_send_group()
sql/sql_select.h:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  JOIN::group_optimized member added to remember the 'GROUP BY optimied away'
  case

  Backport of correction for Mac OS X build problem, global variable not
  initiated is "common" and can't be used in shared libraries, unless
  special flags are used (bug#26218)


mysys/my_pthread.c:
  Backport of correction for Mac OS X build problem, global variable not
  initiated is "common" and can't be used in shared libraries, unless
  special flags are used (bug#26218)