when index is used

When the table contained TEXT columns with empty contents
('', zero length, but not NULL) _and_ strings starting with
control characters like tabulator or newline, the empty values
were not found in a "records in range" estimate. Hence count(*)
missed these records.

The reason was a different set of search flags used for key
insert and key range estimation.

I decided to fix the set of flags used in range estimation.
Otherwise millions of databases around the world would require
a repair after an upgrade.

The consequence is that the manual must be fixed, which claims
that TEXT columns are compared with "end space padding". This
is true for CHAR/VARCHAR but wrong for TEXT. See also bug 21335.


myisam/mi_range.c:
  Bug#26231 - select count(*) on myisam table returns wrong value
              when index is used
  Added SEARCH_UPDATE to the search flags so that it compares
  like write/update/delete operations do. Only so it expects
  the keys at the place where they have been inserted.
myisam/mi_search.c:
  Bug#26231 - select count(*) on myisam table returns wrong value
              when index is used
  Added some comments to explain how _mi_get_binary_pack_key()
  works.
mysql-test/r/myisam.result:
  Bug#26231 - select count(*) on myisam table returns wrong value
              when index is used
  Added a test.
mysql-test/t/myisam.test:
  Bug#26231 - select count(*) on myisam table returns wrong value
              when index is used
  Added test result.

Using INSERT DELAYED on MERGE tables could lead to table
corruptions.

The manual lists a couple of storage engines, which can be
used with INSERT DELAYED. MERGE is not in this list.

The attempt to try it anyway has not been rejected yet.
This bug was not detected earlier as it can work under
special circumstances. Most notable is low concurrency.

To be safe, this patch rejects any attempt to use INSERT
DELAYED on MERGE tables.


mysql-test/r/merge.result:
  Bug#26464 - insert delayed + update + merge = corruption
  Added test result.
mysql-test/t/merge.test:
  Bug#26464 - insert delayed + update + merge = corruption
  Added test.
sql/ha_myisammrg.h:
  Bug#26464 - insert delayed + update + merge = corruption
  Removed HA_CAN_INSERT_DELAYED flag from table_flags().
  The insert delayed thread upgrades the lock from the first
  entry in MYSQL_LOCK::locks only. Hence it is incapable to
  handle MERGE tables, which have as many entries in this
  array as they have MyISAM sub-tables.

into  bk-internal.mysql.com:/data0/bk/mysql-4.1-opt

into  zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-4.1-maint


sql/item.cc:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/item_cmpfunc.h:
  Auto merged
sql/sql_select.cc:
  Auto merged

into  mysql.com:/home/hf/work/25492/my41-25492

libmysqld/lib_sql.cc:
  code modified to prevent freeing of memory that wasn't malloc-ed.
  Now we check if MYSQL_STMT::result was used.

into  mysql.com:/home/tnurnberg/24660/41-24660


sql/table.cc:
  Auto merged

ENUMs weren't allowed to have character 0xff, a perfectly good character in some locales.
This was circumvented by mapping 0xff in ENUMs to ',', thereby prevent actual commas from
being used. Now if 0xff makes an appearance, we find a character not used in the enum and
use that as a separator. If no such character exists, we throw an error.

Any solution would have broken some sort of existing behaviour. This solution should
serve both fractions (those with 0xff and those with ',' in their enums), but
WILL REQUIRE A DUMP/RESTORE CYCLE FROM THOSE WITH 0xff IN THEIR ENUMS. :-/
That is, mysqldump with their current server, and restore when upgrading to one with
this patch.


mysql-test/r/type_enum.result:
  Bug#24660: "enum" field type definition problem
  
  Show that enums can now contain NAMES_SEP_CHAR (0xff, which is a perfectly respectable
  char in some locales), or ',', or both.
mysql-test/t/type_enum.test:
  Bug#24660: "enum" field type definition problem
  
  Show that enums can now contain NAMES_SEP_CHAR (0xff, which is a perfectly respectable
  char in some locales), or ',', or both.
sql/table.cc:
  Bug#24660: "enum" field type definition problem
  
  Revert fix for Bug#20922.
sql/unireg.cc:
  Bug#24660: "enum" field type definition problem
  
  Use a field-separator for ENUM-values that is not part of those values. If impossible,
  throw error.

into  siva.hindu.god:/home/tsmith/m/bk/maint/41


sql/sql_prepare.cc:
  Manual merge

into  mysql.com:/home/gluh/MySQL/Merge/4.1-opt


sql/item_func.cc:
  Auto merged

into  poseidon.mysql.com:/home/tomas/mysql-4.1-ndb

into  pilot.mysql.com:/home/msvensson/mysql/mysql-4.1-maint

 - strerror might return NULL on some platforms


extra/perror.c:
  Backport fix from 5.0

into  pilot.mysql.com:/home/msvensson/mysql/mysql-4.1-maint


mysys/default.c:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/sql_prepare.cc:
  Auto merged

into  neptunus.(none):/home/msvensson/mysql/mysql-4.1-maint


sql/sql_prepare.cc:
  Auto merged

into  pilot.mysql.com:/home/msvensson/mysql/mysql-4.1-maint

into  pilot.mysql.com:/home/msvensson/mysql/mysql-4.1-maint


sql/sql_prepare.cc:
  Auto merged

 - Backport fix from 5.0


sql/sql_prepare.cc:
  Avoid problems with -no-strict-aliasing in gcc 4.0+

into  mysql.com:/usr/home/ram/work/bug23938/my41-bug23938


sql/item_timefunc.cc:
  Auto merged

mysys/my_getopt.c:
  Fixes problem with getting output lines from
  my_getopt in random places in tests.
sql/sql_prepare.cc:
  Fixes problem with failing mysql_client_test
  on some machines. This is actually a problem
  with prepared statements.

into  mysql.com:/usr/home/ram/work/bug10798/my41-bug10798


sql/slave.cc:
  Auto merged

into  chilla.local:/home/mydev/mysql-4.1-axmrg

added valgrind.supp to EXTRA_SCRIPTS(for pushbuild)

into  mysql.com:/home/ram/work/b26012/b26012.4.1


sql/field.h:
  Auto merged

As we have Field_double::not_fixed we must have Field_double::size_of().


sql/field.h:
  Fix for bug #26012: missed Field_double::size_of()
    - Field_double::size_of() added.

into  dev3-63.(none):/home/zhl/mysql/tree-merge/4.1/mysql-4.1-ndb

into  chilla.local:/home/mydev/mysql-4.1-axmrg


mysql-test/r/symlink.result:
  Auto merged
mysql-test/t/symlink.test:
  Auto merged
sql/mysqld.cc:
  Auto merged

Notes:
This patch doesn't fix all issues in the tree and we need jani's fix for that
This patch shoud not be merged into 5.0


mysql-test/r/ps.result:
  result fix
mysql-test/r/symlink.result:
  result fix
mysql-test/t/ps.test:
  test is moved to 'symlink' test
mysql-test/t/symlink.test:
  test from 'ps' test
sql/field_conv.cc:
  valgrind error fix: backport from 5.0
sql/mysqld.cc:
  release of ssl_context
vio/viosslfactories.c:
  release of ssl_context
mysql-test/valgrind.supp:
  New BitKeeper file ``mysql-test/valgrind.supp''
  This file is backported from 5.0, added suppressing of OpenSSL errors

into  chilla.local:/home/mydev/mysql-4.1-axmrg


include/my_global.h:
  Auto merged
include/my_pthread.h:
  Auto merged
include/thr_alarm.h:
  Auto merged
mysys/default.c:
  Auto merged
mysys/my_pthread.c:
  Auto merged
mysys/thr_alarm.c:
  Auto merged
mysys/my_thr_init.c:
  Manual merged
sql/mysqld.cc:
  Manual merged

into  chilla.local:/home/mydev/mysql-4.1-axmrg

into  zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-4.1-maint


configure.in:
  Auto merged
mysql-test/mysql-test-run.pl:
  Auto merged
scripts/mysqld_multi.sh:
  Auto merged
sql/mysqld.cc:
  Auto merged

into  zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-4.1-maint


configure.in:
  Auto merged
mysql-test/r/range.result:
  Auto merged
mysql-test/t/range.test:
  Auto merged

into  qualinost.(none):/home/mtaylor/src/mysql-4.1-maint

into  mysql.com:/home/svoj/devel/mysql/WL3567/mysql-4.1-engines

When a merge table is opened compare column and key definition of
underlying tables against column and key definition of merge table.

If any of underlying tables have different column/key definition
refuse to open merge table.


mysql-test/r/merge.result:
  A test case for WL#3456.
mysql-test/t/merge.test:
  A test case for WL#3456.
sql/ha_myisam.cc:
  Moved a part of code that converts TABLE object to MyISAM key and column
  definition to a separate function.
  
  Added check_definition() function that compares MyISAM table definitions.
sql/ha_myisammrg.cc:
  Check if underlying table definition conforms to merge table definition,
  and in case underlying table definition differs, refuse to open merge
  table.

into  mysql.com:/home/ram/work/b19690/b19690.4.1


sql/mysql_priv.h:
  Auto merged
sql/mysqld.cc:
  Auto merged

Depending on the queries we use different data processing methods
and can lose some data in case of double (and decimal in 4.1) fields.

The fix consists of two parts:
1. double comparison changed, now double a is equal to double b 
if (a-b) is less than 5*0.1^(1 + max(a->decimals, b->decimals)). 
For example, if a->decimals==1, b->decimals==2, a==b if (a-b)<0.005
2. if we use a temporary table, store double values there as is 
to avoid any data conversion (rounding).


mysql-test/r/type_float.result:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - test result
mysql-test/t/type_float.test:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - test case
sql/field.cc:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - use not_fixed flag instead of dec to check bounds.
sql/field.h:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - Field_Double::not_fixed flag introduced, which is set if dec == NOT_FIXED_DEC
      and is used in the ::store() to check bounds. 
    - new constructor introduced (with not_fixed_arg parameter).
sql/init.cc:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - fill log_01[] array with 0.1 powers.
sql/item_cmpfunc.cc:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - compare_real_fixed() and compare_e_real_fixed() introduced,
      they consider double a == double b if a-b is less than 'precision',
      'precision' is set to 5*0.1^(1 + max(a->decimals, b->decimals)), 
      for example, if a->decimals==1, b->decimals==2, 'precision' is 0.005
    - use the above functions if both arguments are fixed.
sql/item_cmpfunc.h:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - Arg_comparator::presision introduced.
    - Arg_comparator::compare_real_fixed(), Arg_comparator::compare_e_real_fixed() introduced.
sql/mysql_priv.h:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - log_01 array of 0.1 powers added.
sql/mysqld.cc:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - log_01 array of 0.1 powers added.
sql/sql_select.cc:
  fix for bug #19690: ORDER BY eliminates rows from the result
    - if we create double field in a temporary table, set not_fixed flag
      (use proper constructor) to avoid data conversion 
      in the Field_double::store(). Otherwise we can lose some data.