The bug caused memory corruption for some queries with top OR level
in the WHERE condition if they contained equality predicates and 
other sargable predicates in disjunctive parts of the condition.

The corruption happened because the upper bound of the memory
allocated for KEY_FIELD and SARGABLE_PARAM internal structures
containing info about potential lookup keys was calculated incorrectly
in some cases. In particular it was calculated incorrectly when the
WHERE condition was an OR formula with disjuncts being AND formulas
including equalities and other sargable predicates.


mysql-test/r/select.result:
  Added a test case for bug #30396.
mysql-test/t/select.test:
  Added a test case for bug #30396.
sql/item_cmpfunc.h:
  Removed max_members from the COND_EQUAL class as not useful anymore.
sql/sql_base.cc:
  Added the max_equal_elems field to the st_select_lex structure.
sql/sql_lex.cc:
  Added the max_equal_elems field to the st_select_lex structure.
sql/sql_lex.h:
  Added the max_equal_elems field to the st_select_lex structure.
  The field contains the maximal number of elements in multiple equalities
  built for the query conditions.
sql/sql_select.cc:
  Fixed bug #30396.
  The bug caused memory corruption for some queries with top OR level
  in the WHERE condition if they contained equality predicates and 
  other sargable predicates in disjunctive parts of the condition.
  
  The corruption happened because the upper bound of the memory
  allocated for KEY_FIELD and SARGABLE_PARAM internal structures
  containing info about potential lookup keys was calculated incorrectly
  in some cases. In particular it was calculated incorrectly when the
  WHERE condition was an OR formula with disjuncts being AND formulas
  including equalities and other sargable predicates.
   
  The max_equal_elems field to the st_select_lex structure is used now
  to calculate the above mentioned upper bound. The field contains the
  maximal number of elements in multiple equalities built for the query
  conditions.

client/mysqldump.c:
  table type compare is changed to case insensitive
mysql-test/r/information_schema.result:
  test result
mysql-test/r/information_schema_db.result:
  result fix
mysql-test/suite/ndb/r/ndb_alter_table.result:
  result fix
mysql-test/suite/ndb/r/ndb_temporary.result:
  result fix
mysql-test/t/information_schema.test:
  test case
sql/ha_ndbcluster.cc:
  char* variables are changed to LEX_STRING
sql/ha_ndbcluster.h:
  char* variables are changed to LEX_STRING
sql/ha_ndbcluster_binlog.cc:
  char* variables are changed to LEX_STRING
sql/handler.cc:
  char* variables are changed to LEX_STRING
sql/handler.h:
  char* variables are changed to LEX_STRING
sql/sql_base.cc:
  Modified functions which are used during open table process
  according to table opening method and requested_object.
sql/sql_select.cc:
  Add support for I_S tables into select_describe() function
sql/sql_show.cc:
  1. Added initialization of 'open_method' to 'st_field_info' structs.
  2. Added initialization of 'i_s_requested_object' to 'ST_SCHEMA_TABLE' structs.
  3. New function which calculates database name and table name values 
     from 'where' condition if it's possible
     void get_lookup_field_values(THD *thd, COND *cond, TABLE_LIST *table,
                                  LOOKUP_FIELD_VALUES *lookup_field_vals);
  4. New function which set table open method
     setup_table_open_method(TABLE_LIST *tables,
                             ST_SCHEMA_TABLE *schema_table,
                             enum enum_schema_tables schema_table_idx)
  5. New function
     int make_db_list(THD *thd, List<LEX_STRING> *files,
                      LOOKUP_FIELD_VALUES *lookup_field_vals,
                      bool *with_i_schema)
  6. New function
     int make_table_name_list(THD *thd, List<LEX_STRING> *files, LEX *lex,
                              LOOKUP_FIELD_VALUES *lookup_field_vals,
                              bool with_i_schema, LEX_STRING *db_name)
  7. Modified 'get_all_tables' function according to new schema(see wl#3732).
sql/sql_show.h:
  char* variables are changed to LEX_STRING
sql/table.cc:
  Modified functions which are used during open table process
  according to table opening method and requested_object.
sql/table.h:
  1. added new constants(open_method)
  #define SKIP_OPEN_TABLE 0
  #define OPEN_FRM_ONLY   1
  #define OPEN_FULL_TABLE 2
      
  2. Added new field 'open_method' into struct st_field_info;
     uint open_method;
      
  3. Added new field into ST_SCHEMA_TABLE struct
     uint i_s_requested_object;  /* the object we need to open(TABLE | VIEW) */.
  4. Added new field to TABLE_LIST struct.
     uint i_s_requested_object;
     This field is set from ST_SCHEMA_TABLE.i_s_requested_object
     for processed table before opening.
      
  5. Added new fields to TABLE_LIST struct, used for 'explain select' for I_S table
     bool has_db_lookup_value;
     bool has_table_lookup_value;
     uint table_open_method;
sql/unireg.h:
  added new constants

This patch adds cost estimation for the queries with ORDER BY / GROUP BY
and LIMIT. 
If there was a ref/range access to the table whose rows were required
to be ordered in the result set the optimizer always employed this access
though a scan by a different index that was compatible with the required 
order could be cheaper to produce the first L rows of the result set.
Now for such queries the optimizer makes a choice between the cheapest
ref/range accesses not compatible with the given order and index scans
compatible with it.


mysql-test/r/distinct.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/endspace.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/group_by.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/group_min_max.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/innodb.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/innodb_mysql.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/merge.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/order_by.result:
  Added a test case for bug #28404.
mysql-test/r/select_found.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/r/subselect.result:
  Adjusted results for test cases affected fy the fix for bug #28404.
mysql-test/t/distinct.test:
  Changed a test case after adding the fix for bug #28404.
mysql-test/t/order_by.test:
  Added a test case for bug #28404.
sql/sql_select.cc:
  Fixed bug#28404.
  This patch adds cost estimation for the queries with ORDER BY / GROUP BY
  and LIMIT. 
  Now for such queries the optimizer makes a choice between the cheapest
  ref/range accesses not compatible with the given order and index scans
  compatible with it.
  
  Modified the function test_if_skip_sort_order to make the above mentioned
  choice cost based.
sql/sql_select.h:
  Fixed bug#28404.
  This patch adds cost estimation for the queries with ORDER BY / GROUP BY
  and LIMIT. 
  Added a new field fot the JOIN_TAB structure.

  
Use size_t instead of uint when calculating join buffer size, because uint can be overflown on 64-bit platforms and join_buffer_size > 4 GB.
  
The test case for this bug is a part of the test suite for bug #5731.


sql/sql_select.cc:
  Use size_t instead of uint when calculating join buffer size, because uint can be overflown on 64-bit platforms and join_buffer_size > 4G.

- Don't call mysql_select() several times for the select that enumerates
  a temporary table with the results of the UNION. Making this call for
  every subquery execution caused O(#enumerated-rows-in-the-outer-query) 
  memory allocations.
- Instead, call join->reinit() and join->exec(), and
  = disable constant table detection for such joins,  
  = provide special handling for table-less constant subqueries.


sql/sql_select.cc:
  BUG#29582: huge memory consumption with union, subselect, joins:
   - Don't mark tables as constant if JOIN::no_const_tables flag is set
sql/sql_select.h:
  BUG#29582: huge memory consumption with union, subselect, joins:
   - Don't mark tables as constant if JOIN::no_const_tables flag is set
sql/sql_union.cc:
  BUG#29582: huge memory consumption with union, subselect, joins:
  - Don't call mysql_select() several times for the select that enumerates
    a temporary table with UNION results. 
  - Instead, call join->reinit() and join->exec().
  - Provide special handling for table-less constant subqueries.

 SELECT statement itself returns empty.

As a result of this bug 'SELECT AGGREGATE_FUNCTION(fld) ... GROUP BY'
can return one row instead of an empty result set.

When GROUP BY only has fields of constant tables
(with a single row), the optimizer deletes the group_list.
After that we lose the information about whether we had an
GROUP BY statement. Though it's important
as SELECT min(x) from empty_table; and
   SELECT min(x) from empty_table GROUP BY y; have to return
different results - the first query should return one row,
second - an empty result set.
So here we add the 'group_optimized_away' flag to remember this case
when GROUP BY exists in the query and is removed
by the optimizer, and check this flag in end_send_group()


mysql-test/r/group_by.result:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test result
mysql-test/r/insert_select.result:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test result
mysql-test/t/group_by.test:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  This is additional testcase that is more basic than the
  original bug's testcase and has the same reason.
mysql-test/t/insert_select.test:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test case
sql/sql_select.cc:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  Remember the 'GROUP BY was optimized away' case in the JOIN::group_optimized
  and check this in the end_send_group()
sql/sql_select.h:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  JOIN::group_optimized member added to remember the 'GROUP BY optimied away'
  case

--long-query-time is now given in seconds with microseconds as decimals
--min_examined_row_limit added for slow query log
long_query_time user variable is now double with 6 decimals
Added functions to get time in microseconds
Added faster time() functions for system that has gethrtime()  (Solaris)
We now do less time() calls.
Added field->in_read_set() and field->in_write_set() for easier field manipulation by handlers
set_var.cc and my_getopt() can now handle DOUBLE variables.
All time() calls changed to my_time()
my_time() now does retry's if time() call fails.
Added debug function for stopping in mysql_admin_table() when tables are locked
Some trivial function and struct variable renames to avoid merge errors.
Fixed compiler warnings
Initialization of some time variables on windows moved to my_init() 


include/my_getopt.h:
  Added support for double arguments
include/my_sys.h:
  Fixed wrong type to packfrm()
  Added new my_time functions
include/mysql/plugin.h:
  Added support for DOUBLE
libmysql/CMakeLists.txt:
  Added new time functions
libmysql/Makefile.shared:
  Added new time functions
mysql-test/r/variables.result:
  Testing of long_query_time
mysql-test/t/variables.test:
  Testing of long_query_time
mysys/charset.c:
  Fixed compiler warnings
mysys/default_modify.c:
  Fixed compiler warnings
mysys/hash.c:
  Fixed compiler warnings
mysys/mf_getdate.c:
  Use my_time()
mysys/mf_iocache2.c:
  Fixed compiler warnings
mysys/mf_pack.c:
  Fixed compiler warnings
mysys/mf_path.c:
  Fixed compiler warnings
mysys/my_append.c:
  Fixed compiler warnings
mysys/my_compress.c:
  Fixed compiler warnings
mysys/my_copy.c:
  Fixed compiler warnings
mysys/my_gethwaddr.c:
  Fixed compiler warnings
mysys/my_getopt.c:
  Added support for double arguments
mysys/my_getsystime.c:
  Added functions to get time in microseconds.
  Added faster time() functions for system that has gethrtime()  (Solaris)
  Moved windows initialization code to my_init()
mysys/my_init.c:
  Added initializing of variables needed for windows time functions
mysys/my_static.c:
  Added variables needed for windows time functions
mysys/my_static.h:
  Added variables needed for windows time functions
mysys/my_thr_init.c:
  Added THR_LOCK_time, used for faster my_time()
mysys/mysys_priv.h:
  Added THR_LOCK_time, used for faster my_time()
mysys/thr_alarm.c:
  time() -> my_time()
sql/event_data_objects.cc:
  end_time() -> set_current_time()
sql/event_queue.cc:
  end_time() -> set_current_time()
sql/event_scheduler.cc:
  Fixed compiler warnings
sql/field.h:
  Added field->in_read_set() and field->in_write_set() for easier field manipulation by handlers
sql/item.h:
  Added decimal to Item_float(double)
sql/item_cmpfunc.h:
  Added decimal to Item_float(double)
sql/item_timefunc.cc:
  time() -> my_time()
sql/item_xmlfunc.cc:
  Fixed compiler warning
sql/lock.cc:
  lock_time() -> set_time_after_lock()
sql/log.cc:
  Timing in slow query log to file is now done in microseconds
  Changed some while() loops to for() loops.
  Fixed indentation
  time() -> my_time()
sql/log.h:
  Slow query logging is now done based on microseconds
sql/log_event.cc:
  time() -> my_time()
  Fixed arguments to new Item_float()
sql/mysql_priv.h:
  Fixed compiler warnings
  Added opt_log_slow_slave_statements
sql/mysqld.cc:
  Added --log_slow_slave_statements and --min_examined_row_limit
  --long-query-time now takes a double argument with microsecond resolution
  Don't write shutdown message when using --help
  Removed not needed \n
  Thread create time and connect time is now done in microseconds
  time() -> my_time()
  Avoid some time() calls
sql/net_serv.cc:
  Fixed compiler warnings
sql/parse_file.cc:
  time() -> my_time()
sql/set_var.cc:
  Added support for DOUBLE variables
  Added support for variables that are given in seconds with microsecond resolution
sql/set_var.h:
  Added support for variables that are given in seconds with microsecond resolution
sql/slave.cc:
  Allow logging of slave queries to slow query log if 'opt_log_slow_slave_statements' is given
  time() -> my_time()
sql/sql_cache.h:
  Fixed compiler warning()
sql/sql_class.cc:
  Initialize new THD variables
sql/sql_class.h:
  long_query_time is now in microseconds
  Added min_examined_row_limit
  Reordered some THD elements for higher efficency
  Added timers in microseconds (connect_utime, thr_create_utime, start_utime and utime_after_lock)
  Start of query is now recorded both in seconds and in microseconds.
  Following renames was made for more clarity and avoid merge problems from earlier versions:
  connect_time -> connect_utime
  thr_create_time -> thr_create_utime
  end_time()  -> set_current_time()
  lock_time() -> set_time_after_lock()
  
  Added THD::start_utime, which is start of query in microseconds from some arbitary time
  Added function THD::current_utime()
  
  Removed safe_time() as retry's are handled in my_time()
sql/sql_connect.cc:
  User resources are now using microsecond resolution
sql/sql_insert.cc:
  end_time() -> set_current_time()
sql-common/client.c:
  time() -> my_time()
sql/sql_parse.cc:
  Testing if we should print to slow_query_log() is now done with microsecond precission.
  If min_examined_row_limit is given, only log queries to slow query log that has examined more rows than this.
sql/sql_select.cc:
  Simplify code now that Item_float() takes decimals as argument
sql/sql_show.cc:
  time() -> my_time()
  Added support for SYS_DOUBLE
sql/sql_table.cc:
  Added debug function for stopping in mysql_admin_table() when tables are locked
sql/structs.h:
  intime -> reset_utime

 - make ha_berkeley::cmp_ref() take into account that auto-generated PKs 
   are stored in LSB-first order. 
 - Remove the temporary code that made the bugfix work for innodb only 


mysql-test/r/bdb.result:
  Adjust test-results.
sql/ha_berkeley.cc:
  BUG#28591: make the fix work for BDB tables too:
   - make ha_berkeley::cmp_ref() take into account that auto-generated PKs 
     are stored in LSB-first order.
sql/sql_select.cc:
  BUG#28591: Remove "innodb only" clause as the fix now works for BDB too
sql/table.cc:
  BUG#28591: Remove "innodb only" clause as the fix now works for BDB too

  Post-merge fix.


sql/sql_select.cc:
  Post-merge fix.
sql/table.cc:
  Post-merge fix.

  Post-merge fix.


sql/sql_select.cc:
  Post-merge fix.

Now we don't take any mutexes when creating or dropping internal HEAP tables during SELECT.
Change buffer sizes to size_t to make keycache 64 bit safe on platforms where sizeof(ulong) != sizeof(size_t)


BitKeeper/etc/ignore:
  added support-files/mysqld_multi.server
include/heap.h:
  Added 'internal_table' to HP_CREATE_INFO
include/keycache.h:
  Change buffer sizes to size_t to make keycache 64 bit safe
include/my_base.h:
  Added HA_OPEN_INTERNAL_TABLE to mark temporary tables that should be deleted on close
mysys/mf_keycache.c:
  Change buffer sizes to size_t to make keycache 64 bit safe
sql/sql_select.cc:
  Added HA_OPEN_INTERNAL_TABLE to mark temporary tables that should be deleted on close
  Removed not anymore needed call to delete_table()
storage/heap/ha_heap.cc:
  Added support for internal temporary tables that should be deleted on close.
  Internal tables now use dedicated open and close calls to avoid taking mutexes.
  If heap_open() failes, now delete the newly created table. (This fixes a possible memory leak)
  Remove never executed info() in create()
storage/heap/ha_heap.h:
  Added slots needed to handle internal temporary tables
storage/heap/heapdef.h:
  Protect against C++ inclusion
storage/heap/hp_close.c:
  Don't call list_delete() for internal temporary tables (They are not in the list)
storage/heap/hp_create.c:
  Added HP_SHARE ** element to heap_create() to store the SHARE of the newly created table.
  For internal temporary tables: Don't take any mutex and don't put them into the open table list.
storage/heap/hp_open.c:
  Split heap_open() into sub functions to be able to create internal temporary tables without putting them in the heap_share_list.
  Add faster open() functions for when we already know the 'share'.
storage/heap/hp_test1.c:
  Update call to heap_create()
  Initialize all keyinfo members.
storage/heap/hp_test2.c:
  Update call to heap_create()

  Limit the fix for bug 28591 to InnoDB only


sql/sql_select.cc:
  Limit the fix for bug 28591 to InnoDB only
sql/table.cc:
  Limit the fix for bug 28591 to InnoDB only

This bug manifested itself for join queries with GROUP BY and HAVING clauses
whose SELECT lists contained DISTINCT. It occurred when the optimizer could
deduce that the result set would have not more than one row.
The bug could lead to wrong result sets for queries of this type because
HAVING conditions were erroneously ignored in some cases in the function
remove_duplicates.   


mysql-test/r/having.result:
  Added a test case for bug #29911.
mysql-test/t/having.test:
  Added a test case for bug #29911.

ORDER BY primary_key on InnoDB table

Queries that use an InnoDB secondary index to retrieve
data don't need to sort in case of ORDER BY primary key
if the secondary index is compared to constant(s).
They can also skip sorting if ORDER BY contains both the
the secondary key parts and the primary key parts (in
that order).
This is because InnoDB returns the rows in order of the
primary key for rows with the same values of the secondary
key columns.
Fixed by preventing temp table sort for the qualifying 
queries.


mysql-test/r/innodb_mysql.result:
  Bug #28591: test case
mysql-test/t/innodb_mysql.test:
  Bug #28591: test case
sql/sql_select.cc:
  Bug #28591: Use the primary key as suffix when testing
  if the key can be used for ORDER BY on supporting engines.
sql/table.cc:
  Bug #28591: can use the primary key
  as a suffix for the secondary keys

A bug in the restore_prev_nj_state function allowed interleaving 
inner tables of outer join operations with outer tables. With the
current implementation of the nested loops algorithm it could lead
to wrong result sets for queries with nested outer joins.
Another bug in this procedure effectively blocked evaluation of some
valid execution plans for queries with nested outer joins. 


mysql-test/r/join_nested.result:
  Added a test case for bug #29604.
mysql-test/t/join_nested.test:
  Added a test case for bug #29604.

The need arose when working on Bug 26141, where it became
necessary to replace TABLE_LIST with its forward declaration in a few
headers, and this involved a lot of s/TABLE_LIST/st_table_list/.
Although other workarounds exist, this patch is in line
with our general strategy of moving away from typedef-ed names.
Sometime in future we might also rename TABLE_LIST to follow the
coding style, but this is a huge change.


sql/item.cc:
  st_table_list -> TABLE_LIST
sql/item.h:
  st_table_list -> TABLE_LIST
sql/mysql_priv.h:
  st_table_list -> TABLE_LIST
sql/sql_base.cc:
  st_table_list -> TABLE_LIST
sql/sql_lex.cc:
  st_table_list -> TABLE_LIST
sql/sql_lex.h:
  st_table_list -> TABLE_LIST
sql/sql_select.cc:
  st_table_list -> TABLE_LIST
sql/sql_show.cc:
  st_table_list -> TABLE_LIST
sql/sql_udf.h:
  Was not needed.
sql/table.cc:
  st_table_list -> TABLE_LIST
sql/table.h:
  st_table_list -> TABLE_LIST

query / no aggregate of subquery
 The optimizer counts the aggregate functions that 
 appear as top level expressions (in all_fields) in 
 the current subquery. Later it makes a list of these
 that it uses to actually execute the aggregates in
 end_send_group().
 That count is used in several places as a flag whether
 there are aggregates functions.
 While collecting the above info it must not consider
 aggregates that are not aggregated in the current 
 context. It must treat them as normal expressions 
 instead. Not doing that leads to incorrect data about
 the query, e.g. running a query that actually has no
 aggregate functions as if it has some (and hence is
 expected to return only one row).
 Fixed by ignoring the aggregates that are not aggregated
 in the current context. 
 One other smaller omission discovered and fixed in the 
 process : the place of aggregation was not calculated for
 user defined functions. Fixed by calling 
 Item_sum::init_sum_func_check() and 
 Item_sum::check_sum_func() as it's done for the rest of 
 the aggregate functions.


mysql-test/r/subselect.result:
  Bug #27333: test case
mysql-test/t/subselect.test:
  Bug #27333: test case
sql/item_subselect.cc:
  Bug#27333: need select_lex to filter out
   aggregates that are not aggregated in
   the current select.
sql/item_sum.cc:
  Bug#27333: need select_lex to filter out
   aggregates that are not aggregated in
   the current select.
sql/item_sum.h:
  Bug#27333: calculate the place of 
   aggregation for user defined functions.
sql/sql_select.cc:
  Bug#27333: When counting the aggregated functions
   and collecting a list of them we must not consider
   the aggregates that are not aggregated in the local
   context as "local" : i.e. we must treat them as 
   normal functions and not add them to the aggregate
   functions list.
sql/sql_select.h:
  Bug#27333: need select_lex to filter out
   aggregates that are not aggregated in
   the current select.

the loose scan optimization for grouping queries was applied returned 
a wrong result set when the query was used with the SQL_BIG_RESULT
option.

The SQL_BIG_RESULT option forces to use sorting algorithm for grouping
queries instead of employing a suitable index. The current loose scan
optimization is applied only for one table queries when the suitable
index is covering. It does not make sense to use sort algorithm in this
case. However the create_sort_index function does not take into account
the possible choice of the loose scan to implement the DISTINCT operator
which makes sorting unnecessary. Moreover the current implementation of
the loose scan for queries with distinct assumes that sorting will
never happen. Thus in this case create_sort_index should not call
the function filesort.


mysql-test/r/group_min_max.result:
  Added a test case for bug #25602.
mysql-test/t/group_min_max.test:
  Added a test case for bug #25602.

INSERT into table from SELECT from the same table
with ORDER BY and LIMIT was inserting other data
than sole SELECT ... ORDER BY ... LIMIT returns.

One part of the patch for bug #9676 improperly pushed
LIMIT to temporary table in the presence of the ORDER BY
clause.
That part has been removed.


sql/sql_select.cc:
  Fixed bug #29095.
  One part of the patch for bug #9676 improperly pushed
  LIMIT to temporary table in the presence of the ORDER BY
  clause.
  That part has been removed.
mysql-test/t/insert_select.test:
  Expanded the test case for bug #9676.
  Created a test case for bug #29095.
mysql-test/r/insert_select.result:
  Expanded the test case for bug #9676.
  Created a test case for bug #29095.

For a join query with GROUP BY and/or ORDER BY and a view reference
in the FROM list the metadata erroneously showed empty table aliases
and database names for the view columns.



sql/item.h:
  Fixed bug #28898.
  Body of Item_ref::get_tmp_table_item method has been moved
  to item.cc file.
mysql-test/t/metadata.test:
  Updated test case for bug #28898.
sql/item.cc:
  Fixed bug #28898.
  The Item_ref::get_tmp_table_item method has been modified
  to copy pointers to the table alias and database name to the new
  Item_field object created for a field stored in the temporary
  table.
mysql-test/r/metadata.result:
  Updated test case for bug #28898.
sql/sql_select.cc:
  Fixed bug #28898.
  The change_to_use_tmp_fields function has been modified to
  to copy pointers to the table alias and database name from
  the Item_ref objects to the new Item_field objects created
  for fields stored in the temporary table.

The method select_insert::send_error does two things, it rolls back a statement
being executed and outputs an error message. But when a 
nonexistent column is referenced, an error message has been published already and
there is no need to publish another.
Fixed by moving all functionality beyond publishing an error message into 
select_insert::abort() and calling only that function.


mysql-test/r/errors.result:
  Bug#28677: test result
mysql-test/t/errors.test:
  Bug#28677: test case
sql/sql_class.h:
  Bug#28677: overriding abort()
sql/sql_insert.cc:
  Bug#28677: 
  - moved everything beyond producing an error message out of select_insert::send_error 
  and into new override select_insert::abort() 
  - made corresponding move of code from select_create::send_error to select_create::abort
sql/sql_select.cc:
  Bug#28677: No need to pusblish an error here

In create_tmp_table() don't set full table path as a table name. Other code assumes table names to not exceed NAME_LEN bytes.


sql/sql_select.cc:
  In create_tmp_table() don't set full table path as a table name. Other code assumes table names to not exceed NAME_LEN bytes.
mysql-test/r/long_tmpdir.result:
  Added testcase for bug #29015 "Stack overflow in processing temporary table name when tmpdir path is long"
mysql-test/t/long_tmpdir-master.opt:
  Added testcase for bug #29015 "Stack overflow in processing temporary table name when tmpdir path is long"
mysql-test/t/long_tmpdir-master.sh:
  Added testcase for bug #29015 "Stack overflow in processing temporary table name when tmpdir path is long"
mysql-test/t/long_tmpdir.test:
  Added testcase for bug #29015 "Stack overflow in processing temporary table name when tmpdir path is long"

Coding style: classes start with a capital letter.
Rename some classes related to parsing:
create_field -> Create_field
foreign_key -> Foreign_key
key_part_spec -> Key_part_spec


sql/field.cc:
  create_field -> Create_field
sql/field.h:
  create_field -> Create_field
sql/item.h:
  create_field -> Create_field
sql/item_sum.cc:
  create_field -> Create_field
sql/mysql_priv.h:
  create_field -> Create_field
sql/sp_head.cc:
  create_field -> Create_field
sql/sp_head.h:
  create_field -> Create_field
sql/sp_pcontext.cc:
  create_field -> Create_field
sql/sp_pcontext.h:
  create_field -> Create_field
sql/sp_rcontext.cc:
  create_field -> Create_field
sql/sql_class.cc:
  create_field -> Create_field
  key_part_spec -> Key_part_spec
  foreign_key -> Foreign_key
sql/sql_class.h:
  create_field -> Create_field
  key_part_spec -> Key_part_spec
  foreign_key -> Foreign_key
sql/sql_insert.cc:
  create_field -> Create_field
sql/sql_lex.cc:
  Coding style: classes start with a capital, create_field -> Create_field
sql/sql_lex.h:
  create_field -> Create_field
  key_part_spec -> Key_part_spec
sql/sql_parse.cc:
  create_field -> Create_field
  key_part_spec -> Key_part_spec
sql/sql_select.cc:
  create_field -> Create_field
sql/sql_table.cc:
  create_field -> Create_field
sql/sql_yacc.yy:
  create_field -> Create_field
  key_part_spec -> Key_part_spec
  foreign_key -> Foreign_key
sql/unireg.cc:
  create_field -> Create_field

a temporary table has grown out of heap memory reserved for it and 
the remaining disk space is not big enough to store the table as
a MyISAM table.

The crash happens because the function create_myisam_from_heap
does not handle safely the mem_root structure associated 
with the converted table in the case when an error has occurred.


sql/sql_select.cc:
  Fixed bug #28449: a crash may happen at some rare conditions when
  a temporary table has grown out of heap memory reserved for it and 
  the remaining disk space is not big enough to store the table as
  a MyISAM table.
  
  The crash happens because the function create_myisam_from_heap
  does not handle safely the mem_root structure associated 
  with the converted table in the case when an error has occurred.
  
  As it's hard to create a sitiation that would throw an error 
  a special code has been added that raises an error for a newly 
  created test called error_simulation.
mysql-test/r/error_simulation.result:
  New BitKeeper file ``mysql-test/r/error_simulation.result''
  
  Added a test case for bug #28449.
mysql-test/t/error_simulation-master.opt:
  New BitKeeper file ``mysql-test/t/error_simulation-master.opt''
mysql-test/t/error_simulation.test:
  New BitKeeper file ``mysql-test/t/error_simulation.test''
  
  Added a test case for bug #28449.

using a derived table over a grouping subselect.

This crash happens only when materialization of the derived tables 
requires creation of auxiliary temporary table, for example when
a grouping operation is carried out with usage of a temporary table.

The crash happened because EXPLAIN EXTENDED when printing the query
expression made an attempt to use the objects created in the mem_root
of the temporary table which has been already freed by the moment
when printing is called.

This bug appeared after the method Item_field::print() had been 
introduced.    


mysql-test/r/subselect.result:
  Added a test case for bug #28728.
mysql-test/t/subselect.test:
  Added a test case for bug #28728.
sql/sql_select.cc:
  Fixed bug #28728: a crash when executing EXPLAIN EXTENDED for a query
  using a derived table over a grouping subselect.
  The crash happened because EXPLAIN EXTENDED when printing the query
  expression made an attempt to use the objects created in the mem_root
  of the temporary table which has been already freed by the moment
  when printing is accomplished.
  The fix in JOIN::exec() ensures using existing objects when printing 
  subselects for a derived tables by EXPLAIN EXTENDED.

  Patch to eliminate compilation error.


sql/sql_select.cc:
  Patch to eliminate compilation error.

Problem:

HASH indexes on VARCHAR columns with binary collations did not ignore trailing spaces from strings before comparisons. This could result in duplicate records being successfully inserted into a MEMORY table with unique key constraints.

As a direct consequence of the above, internal MEMORY tables used for GROUP BY calculation in testcases for bug #27643 contained duplicate rows which resulted in duplicate key errors when converting those temporary tables to MyISAM. Additionally, that error was incorrectly converted to the 'table is full' error.

Solution:

- ignore trailing spaces in VARCHAR fields with binary collations when calculating hashes.
- return a proper error from create_myisam_from_heap() when conversion fails.


mysql-test/r/ctype_ucs2_def.result:
  Added a testcase for bug #27643.
mysql-test/r/heap_hash.result:
  Added a testcase for bug #27643.
mysql-test/t/ctype_ucs2_def.test:
  Added a testcase for bug #27643.
mysql-test/t/heap_hash.test:
  Added a testcase for bug #27643.
sql/sql_select.cc:
  Return an appropriate error instead of 'table is full' when conversion from MEMORY to MyISAM fails.
strings/ctype-bin.c:
  Added my_hash_sort_8bit_bin() which ignores trailing spaces when calculating hashes, and is now used for VARCHAR columns instead of my_hash_sort_bin().
strings/ctype-mb.c:
  Ignore trailing spaces when calculating a string hash in my_hash_sort_mb_bin().
strings/ctype-ucs2.c:
  Ignore trailing spaces when calculating a string hash in my_hash_sort_ucs2_bin().

mysqld crashed when a long-running explain query was killed from
another connection.

When the current thread caught a kill signal executing the function
best_extension_by_limited_search it just silently returned to  
the calling function greedy_search without initializing elements of
the join->best_positions array.
However, the greedy_search function ignored thd->killed status
after a calls to the best_extension_by_limited_search function, and
after several calls the greedy_search function used an uninitialized
data from the join->best_positions[idx] to search position in the
join->best_ref array. 
That search failed, and greedy_search tried to call swap_variables
function with NULL argument - that caused a crash.


sql/sql_select.cc:
  Fixed bug #28598.
  choose_plan(), greedy_search(), best_extension_by_limited_search()
  and find_best() functions have been changed to return TRUE in case
  of fatal error.
mysql-test/t/kill.test:
  Updated test case for bug #28598.
mysql-test/r/kill.result:
  Updated test case for bug #28598.

Integer values with 10 digits may or may not fit into an int column 
(e.g. 2147483647 vs 6147483647).
Thus when creating a temp table column for such an int we must
use bigint instead.
Fixed to use bigint.
Also subsituted a "magic number" with a named constant.


mysql-test/r/analyse.result:
  Bug #28492: Adjusted the results after having fixed the bug
mysql-test/r/metadata.result:
  Bug #28492: test case
mysql-test/r/olap.result:
  Bug #28492: Adjusted the results after having fixed the bug
mysql-test/r/sp.result:
  Bug #28492: Adjusted the results after having fixed the bug
mysql-test/r/view.result:
  Bug #28492: Adjusted the results after having fixed the bug
mysql-test/t/metadata.test:
  Bug #28492: test case
sql/field.h:
  Bug #28492: Replaced a magic number with a constant
sql/sql_select.cc:
  Bug #28492: Treat integers with 10 and more digits as 
  bigint.

- Renamed "Using join cache" to "Using join buffer".
- "Using join buffer" is now printed on the last
  table that "reads" from the join buffer cache.


mysql-test/r/archive_gis.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/compress.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/ctype_utf8.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/derived.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/distinct.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/func_group.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/func_group_innodb.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/gis.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/greedy_optimizer.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/group_by.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/group_min_max.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/index_merge_myisam.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/information_schema.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/innodb_gis.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/innodb_mysql.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/join.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/join_nested.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/key_diff.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/myisam.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/ndb_condition_pushdown.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/ndb_gis.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/range.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/row.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/select.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/ssl.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/ssl_compress.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/subselect.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/subselect3.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/union.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
mysql-test/r/view.result:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.
sql/sql_select.cc:
  Bug #27531: renamed "Using join cache" to "Using join buffer"
  and moved to the last table in the batch.

constant outer tables did not return null complemented
rows when conditions were evaluated to FALSE.
Wrong results were returned because the conditions over constant
outer tables, when being pushed down, were erroneously enclosed 
into the guard function used for WHERE conditions.


mysql-test/r/join_outer.result:
  Added a test case for bug #28571.
mysql-test/t/join_outer.test:
  Added a test case for bug #28571.
sql/sql_select.cc:
  Fixed bug #28571. Outer join queries with ON conditions over
  constant outer tables did not return null complemented
  rows when conditions were evaluated to FALSE.
  Wrong results were returned because the conditions over constant
  outer tables, when being pushed down, were erroneously enclosed 
  into the guard function used for WHERE conditions.
  The problem is fixed in the function make_join_select. Now the
  conditions over constant tables from ON expressions are pushed
  down after the conditions from WHERE has been pushed down.

CHECK OPTION and a subquery in WHERE condition.
The abort was triggered by setting the value of join->tables for
subqueries in the function JOIN::cleanup. This function was called
after an invocation of the JOIN::join_free method for subqueries
used in WHERE condition.


mysql-test/r/view.result:
  Added a test case for bug #28561.
mysql-test/t/view.test:
  Added a test case for bug #28561.
sql/sql_select.cc:
  Fixed bug #28561: assertion abort for update on multi-table view with
  CHECK OPTION and a subquery in WHERE condition.
  The abort was triggered by setting the value of join->tables for
  subqueries in the function JOIN::cleanup. This function was called
  after an invocation of the JOIN::join_free method for subqueries
  used in WHERE condition.
  Setting the value of join->tables to for a subquery created serious
  problems for checking WHERE condition after update of the multi-table
  view as this check is performed in the do_select function right
  after a call of the JOIN::join_free method.
  In fact setting join->tables to 0 in JOIN::cleanup is not needed 
  anywhere in the current code.

The problem reported is a compile bug,
reported by the development GCC team with GCC 4.2.

The original issue can no longer be reproduced in MySQL 5.1,
since the configure script no longer define HAVE_ATOMIC_ADD,
which caused the Linux atomic functions to be used (and cause a problem
with an invalid cast).

This patch implements some code cleanup for 5.1 only, which was identified
during the investigation of this issue.

With this patch, statistics maintained in THD::status_var are by definition
owned by the running thread, and do not need to be protected against race
conditions. These statistics are maintained by the status_var_* helpers,
which do not require any lock.


include/my_global.h:
  General cleanup of thread_safe_increment / statistic_increment
include/my_pthread.h:
  General cleanup of thread_safe_increment / statistic_increment
sql/filesort.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/handler.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_insert.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_parse.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_prepare.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_select.cc:
  General cleanup of thread_safe_increment / statistic_increment

When processing the USE/FORCE index hints
the optimizer was not checking if the indexes 
specified are enabled (see ALTER TABLE).
Fixed by:
 Backporting the fix for bug 20604 to 5.0


mysql-test/r/key.result:
  Test for BUG 20604.
  The important part of the test is the explain output that 
  tests what indexes are used.
mysql-test/r/myisam.result:
  Bug #28476: test cases
mysql-test/t/key.test:
  Bug 20604: 
  The minimal test case that reveals the bug. The optimizer for 
  aggregates relies on keys disabled with ALTER TABLE ... DISABLE KEYS
  not being in the set TABLE::keys_in_use_for_query.
  When the execution engine tries to use a disabled index, MyISAM
  returns an error.
mysql-test/t/myisam.test:
  Bug #28476: test cases
sql/sql_base.cc:
  Bug #28476: 
   - Ignore disabled indexes in USE/FORCE index
sql/sql_select.cc:
  Bug 20604 : The intersection operation between table->s->keys_in_use 
  and table->keys_in_use_for_query is no longer necessary.
  We can trust that the latter is a subset of the former.
sql/table.h:
  Bug 20604:
  Added comments to TABLE_SHARE::keys_in_use and
  TABLE::keys_in_use_for_query.

mysql-test/r/compress.result:
  result fix
mysql-test/r/information_schema.result:
  result fix
mysql-test/r/openssl_1.result:
  result fix
mysql-test/r/rpl_packet.result:
  result fix
mysql-test/r/status.result:
  result fix

for a query over an empty table right after its creation. 
The crash is the result of an attempt made by JOIN::optimize to evaluate
the WHERE condition when no records have been actually read.
The added test case can reproduce the crash only with InnoDB tables and
only with 5.0.x.
 


mysql-test/r/innodb_mysql.result:
  Added a test case for bug #28272.
mysql-test/t/innodb_mysql.test:
  Added a test case for bug #28272.
sql/sql_select.cc:
  Fixed bug #28272: crash that occurs when running an EXPLAIN command
  for a query over an empty table right after its creation. 
  The crash is the result of an attempt made by JOIN::optimize to evaluate
  the WHERE condition when no records have been actually read.
  Such attempts could be observed only with EXPLAIN commands.
  Now at the optimization phase the WHERE condition is never evaluated if 
  there is no record previously read from the table.