Select with a "NULL NOT IN" condition containing complex
subselect from the same table as in the outer select failed
with an assertion.


The failure was caused by a concatenation of circumstances:
1) an inner select was optimized by make_join_statistics to use
   the QUICK_RANGE_SELECT access method (that implies an index
   scan of the table);
2) a subselect was independent (constant) from the outer select;
3) a condition was pushed down into inner select.

During the evaluation of a constant IN expression an optimizer
temporary changed the access method from index scan to table
scan, but an engine handler was already initialized for index
access by make_join_statistics. That caused an assertion.


Unnecessary index initialization has been removed from
the QUICK_RANGE_SELECT::init method (QUICK_RANGE_SELECT::reset
reinvokes this initialization).


mysql-test/r/subselect3.result:
  Added test case for bug #37894.
mysql-test/t/subselect3.test:
  Added test case for bug #37894.
sql/opt_range.cc:
  Bug #37894: Assertion in init_read_record_seq in handler.h line 1444
  
  Unnecessary index initialization has been removed from
  the QUICK_RANGE_SELECT::init method (QUICK_RANGE_SELECT::reset
  reinvokes this initialization).

            with COALESCE and JOIN

The server returned to a client the VARBINARY column type
instead of the DATE type for a result of the COALESCE,
IFNULL, IF, CASE, GREATEST or LEAST functions if that result
was filesorted in an anonymous temporary table during
the query execution.

For example:
  SELECT COALESCE(t1.date1, t2.date2) AS result
    FROM t1 JOIN t2 ON t1.id = t2.id ORDER BY result;


To create a column of various date/time types in a
temporary table the create_tmp_field_from_item() function
uses the Item::tmp_table_field_from_field_type() method
call. However, fields of the MYSQL_TYPE_NEWDATE type were
missed there, and the VARBINARY columns were created
by default.
Necessary condition has been added.


mysql-test/r/metadata.result:
  Added test case for bug #39283.
mysql-test/t/metadata.test:
  Added test case for bug #39283.
sql/sql_select.cc:
  Bug #39283: Date returned as VARBINARY to client for queries
              with COALESCE and JOIN
  
  To create a column of various date/time types in a
  temporary table the create_tmp_field_from_item() function
  uses the Item::tmp_table_field_from_field_type() method
  call. However, fields of the MYSQL_TYPE_NEWDATE type were
  missed there, and the VARBINARY columns were created
  by default.
  Necessary condition has been added.

           derived table cause crash

When a multi-UPDATE command fails to lock some table, and
subsequently succeeds, the tables need to be reopened if
they were altered. But the reopening procedure failed for
derived tables.

Extra cleanup has been added.


mysql-test/r/lock_multi.result:
  Added test case for bug #38499.
mysql-test/t/lock_multi.test:
  Added test case for bug #38499.
sql/sql_union.cc:
  Bug#38499: flush tables and multitable table update with
             derived table cause crash
  
  Obsolete assertion has been removed.
sql/sql_update.cc:
  Bug#38499: flush tables and multitable table update with
             derived table cause crash
  
  Extra cleanup for derived tables has been added:
  1) unit.cleanup(),
  2) unit->reinit_exec_mechanism().

          ``FLUSH TABLES WITH READ LOCK''

Concurrent execution of 1) multitable update with a
NATURAL/USING join and 2) a such query as "FLUSH TABLES
WITH READ LOCK" or "ALTER TABLE" of updating table led
to a server crash.


The mysql_multi_update_prepare() function call is optimized
to lock updating tables only, so it postpones locking to
the last, and if locking fails, it does cleanup of modified
syntax structures and repeats a query analysis.  However,
that cleanup procedure was incomplete for NATURAL/USING join
syntax data: 1) some Field_item items pointed into freed
table structures, and 2) the TABLE_LIST::join_columns fields
was not reset.

Major change:
  short-living Field *Natural_join_column::table_field has
  been replaced with long-living Item*.


mysql-test/r/lock_multi.result:
  Added test case for bug #38691.
mysql-test/t/lock_multi.test:
  Added test case for bug #38691.
sql/item.cc:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  The Item_field constructor has been modified to allocate
  and copy original database/table/field names always (not
  during PS preparation/1st execution only), because
  an initialization of Item_field items with a pointer to
  short-living Field structures is a common practice.
sql/sql_base.cc:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  1) Type adjustment for Natural_join_column::table_field
     (Field to Item_field);
  2) The setup_natural_join_row_types function has been
     updated to take into account new
     first_natural_join_processing flag to skip unnecessary
     reinitialization of Natural_join_column::join_columns
     during table reopening after lock_tables() failure
     (like the 'first_execution' flag for PS).
sql/sql_lex.cc:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  Initialization of the new
  st_select_lex::first_natural_join_processing flag has
  been added.
sql/sql_lex.h:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  The st_select_lex::first_natural_join_processing flag
  has been added to skip unnecessary rebuilding of
  NATURAL/USING JOIN structures during table reopening
  after lock_tables failure.
sql/sql_update.cc:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  Extra cleanup calls have been added to reset
  Natural_join_column::table_field items.
sql/table.cc:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  Type adjustment for Natural_join_column::table_field
  (Field to Item_field).
sql/table.h:
  Bug #38691: segfault/abort in ``UPDATE ...JOIN'' while
            ``FLUSH TABLES WITH READ LOCK''
  
  Type of the Natural_join_column::table_field field has
  been changed from Field that points into short-living
  TABLE memory to long-living Item_field that can be
  linked to (fixed) reopened table.

so that if the substitution contains single-quotes, the program will fail.

(no conflicts)

The grep expression that finds a running "mysqld" program fails if the
"mysqld_safe" is running with the same PID.

Now, excise "ps" output that has the word " grep" or "mysqld_safe" in 
it, to be a little more certain that the matched process is not a false 
positive hit.  This will fail when the path to mysqld contains either
of those two names, which should be acceptable.

Additionally, some text to search could be truncated if very long.  
Expand the number of lines "ps" emits.

mysql-test-run.pl --start-and-exit starts but does not exit
Instead, it hangs with ActiveState perl. The error is
believed to be a bug in ActiveState implementation.
Workaround is using POSIX::_exit, as described here
http://www.perlmonks.org/?node_id=334610

Thanks to Philip Stoev for the idea of the patch.

The '@' symbol can not be used in the host name according to rfc952.
The fix:
added function check_host_name(LEX_STRING *str)
which checks that all symbols in host name string are valid and
host name length is not more than max host name length
(just moved check_string_length() function from the parser into check_host_name()).


mysql-test/r/create.result:
  test result
mysql-test/t/create.test:
  test case
sql/mysql_priv.h:
  added function check_host_name(LEX_STRING *str)
sql/sql_parse.cc:
  added function check_host_name(LEX_STRING *str)
  which checks that all symbols in host name string are valid and
  host name length is not more than max host name length(HOSTNAME_LENGTH).
sql/sql_yacc.yy:
  using newly added function check_host_name()

                  tests "<ENGINE>_func_view" fail

The problem:
I_S views table does not check the presence of SHOW_VIEW_ACL|SELECT_ACL
privileges for a view. It leads to discrepancy between SHOW CREATE VIEW
and I_S.VIEWS.
The fix:
added appropriate check.


mysql-test/r/information_schema_db.result:
  test result
mysql-test/t/information_schema_db.test:
  test case
sql/sql_show.cc:
  The problem:
  I_S views table does not check the presence of SHOW_VIEW_ACL|SELECT_ACL
  privileges for a view. It leads to discrepancy between SHOW CREATE VIEW
  and I_S.VIEWS.
  The fix:
  added appropriate check.

            
When analyzing the possible index use cases the server was re-using an internal structure.
This is wrong, as this internal structure gets updated during the analysis.
Fixed by making a copy of the internal structure for every place it needs to be used.
Also stopped the generation of empty SEL_TREE structures that unnecessary 
complicate the analysis.

mysql-test/r/index_merge.result:
  Bug#37943: test case
mysql-test/t/index_merge.test:
  Bug#37943: test case
sql/opt_range.cc:
  Bug#37943: 
   - Make copy constructors for SEL_TREE and sub-structures and use them when OR-ing trees.
   - don't generate empty SEL_TREEs. Return NULL instead.

from stored procedure. 

Problem: we replace all references to local variables in stored procedures     
with NAME_CONST(name, value) logging to the binary log. However, if the
value's collation differs we might get an 'illegal mix of collation'           
error as we don't pass the collation to the function.

Fix: pass the value's collation to NAME_CONST().

Note: actually we should pass to NAME_CONST() the value's derivation as well.
It's impossible without the parser modifying. Now we always set the 
derivation to DERIVATION_IMPLICIT, the same as local variables have.


mysql-test/r/binlog.result:
  Fix for bug#39182: Binary log producing incompatible character set query
  from stored procedure.
    - test result.
mysql-test/r/ctype_cp932_binlog.result:
  Fix for bug#39182: Binary log producing incompatible character set query
  from stored procedure.
    - results adjusted.
mysql-test/r/rpl_sp.result:
  Fix for bug#39182: Binary log producing incompatible character set query
  from stored procedure.
    - results adjusted.
mysql-test/t/binlog.test:
  Fix for bug#39182: Binary log producing incompatible character set query
  from stored procedure.
    - test case.
sql/item.cc:
  Fix for bug#39182: Binary log producing incompatible character set query
  from stored procedure.
    - allow NAME_CONST() to get _charset'foo' COLLATE 'bar' strings
  (see Item_func_set_collation).
sql/sp_head.cc:
  Fix for bug#39182: Binary log producing incompatible character set query
  from stored procedure. 
    - pass the value's collation to NAME_CONST().

Repush of change to fix tests on Pushbuild.

Server created "arc" directories inside database directories and
maintained there useless copies of .frm files.

Creation and renaming procedures of those copies as well as
creation of "arc" directories has been discontinued.
Removal procedure has been kept untouched to be able to
cleanup existent database directories by the DROP DATABASE
query. Also view renaming procedure has been updated to remove
these directories.


sql/parse_file.cc:
  Fixed bug #17823: 'arc' directories inside database directories.
  View/table creation and renaming procedures maintained
  backup copies of .frm files. Those copies are unused yet,
  so this feature was incomplete and unnecessary.
  
  1. Unwanted code has been hidden by FRM_ARCHIVE ifdefs
  (the FRM_ARCHIVE macro is not defined).
  
  2. Renaming procedure has been modified to remove obsolete
  "arc" directories.
sql/parse_file.h:
  Fixed bug #17823: 'arc' directories inside database directories.
  The "thd" parameter has been added to the rename_in_schema_file()
  function.
sql/sql_db.cc:
  Fixed bug #17823: 'arc' directories inside database directories.
  Scope of the mysql_rm_arc_files() function has been changed to
  global for use from the parse_file.cc file.
sql/sql_view.cc:
  Fixed bug #17823: 'arc' directories inside database directories.
  Added the "thd" argument to rename_in_schema_file() calls.

     JOIN for the subselect wasn't cleaned if we came upon an error
     during sub_select() execution. That leads to the assertion failure
     in close_thread_tables()

     part of the 6.0 code backported

per-file comments:
  mysql-test/r/sp-error.result
Bug#37949 Crash if argument to SP is a subquery that returns more than one row 
    test result

  mysql-test/t/sp-error.test
Bug#37949 Crash if argument to SP is a subquery that returns more than one row 
    test case

  sql/sp_head.cc
Bug#37949 Crash if argument to SP is a subquery that returns more than one row 
    lex->unit.cleanup() call added if not substatement

Machines with hostname set to "localhost" cause uniqueness errors in 
the SQL bootstrap data.

Now, insert zero lines for cases where the (lowercased) hostname is 
the same as an already-inserted 'localhost' name.  Also, fix a few tests 
that expect certain local accounts to have a certain host name.

Bug#38311 - Fix of some cruft from remove_files in ndb_autodiscover.test, clean up of distinct.test,
and replacing error numbers with error names.

            
A stored procedure involving substrings could crash the server on certain
platforms because of invalid memory reads.
          
During storing the new blob-field value, the cached value's address range
overlapped that of the new field value. This caused problems when the 
cached value storage was reallocated to provide access for a new 
characater set representation. The patch checks the address ranges, and if
they overlap, the new field value is copied to a new storage before it is
converted to the new character set.


mysql-test/r/sp.result:
  Added result set
mysql-test/t/sp.test:
  Added test case
sql/field.cc:
  The source and destination address ranges of a character conversion must not overlap or the 'from' address will be invalidated as the temporary value-
  object is re-allocated to fit the new character set.
sql/field.h:
  Added comments

Substituted use of MTR's remove_file function in the tests
Started with 5.0 tree and will clean up any offenders discovered during upmerge.