REORGANIZE PARTITION

There were several problems which lead to this this,
all related to bad error handling.

1) There was several bugs preventing the ddl-log to be used for
   cleaning up created files on error.

2) The error handling after the copy partition rows did not close
   and unlock the tables, resulting in deletion of partitions
   which were in use, which lead InnoDB to put the partition to
   drop in a background queue.

sql/ha_partition.cc:
  Bug#47343: InnoDB fails to clean-up after lock wait timeout on
             REORGANIZE PARTITION
  
  Better error handling, if partition has been created/opened/locked
  then make sure it is unlocked and closed before returning error.
  The delete of the newly created partition is handled by the ddl-log.
sql/sql_parse.cc:
  Bug#47343: InnoDB fails to clean-up after lock wait timeout on
             REORGANIZE PARTITION
  
  Fix a bug found when experimenting, thd could really be NULL here,
  as mentioned in the function header.
sql/sql_partition.cc:
  Bug#47343: InnoDB fails to clean-up after lock wait timeout on
             REORGANIZE PARTITION
  
  Used the correct .frm shadow name to put into the ddl-log.
  Really use the ddl-log to handle errors.
sql/sql_table.cc:
  Bug#47343: InnoDB fails to clean-up after lock wait timeout on
             REORGANIZE PARTITION
  
  Fixes of the ddl-log when used as error recovery (no crash).
  When executing an entry from memory (not read from disk)
  the name_len was not set correctly.

Added not_embedded to the new dbug_sync test file.

BUG#49481: RBR: MyISAM and bit fields may cause slave to stop on delete: 
cant find record
      
BUG#49482: RBR: Replication may break on deletes when MyISAM tables + 
char field are used

When using MyISAM tables, despite the fact that the null bit is
set for some fields, their old value is still in the row. This
can cause the comparison of records to fail when the slave is
doing an index or range scan.

We fix this by avoiding memcmp for MyISAM tables when comparing
records. Additionally, when comparing field by field, we first
check if both fields are not null and if so, then we compare
them. If just one field is null we return failure immediately. If
both fields are null, we move on to the next field.

Small fix in the test case. Changed the UNLOCK tables 
to happen after each insert.

In certain rare cases when a process was interrupted
during a FLUSH PRIVILEGES operation the diagnostic
area would be set to an error state but the function
responsible for the operation would still signal
success. This would lead to a debug assertion error
later on when the server would attempt to reset the
DA before sending the error message.

This patch fixes the issue by assuring that
reload_acl_and_cache() always fails if an error
condition is raised.

The second issue was that a KILL could cause
a console error message which referred to a DA
state without first making sure that such a
state existed.

This patch fixes this issue in two different
palces by first checking DA state before
fetching the error message.

 

sql/sql_acl.cc:
  * Make sure that there is an error to print before attempting to do so.
  * Minor style change: change 1 to TRUE for clarity.
sql/sql_parse.cc:
  * Always fail reload_acl_and_cache() if the query was killed.
sql/sql_servers.cc:
  * Make sure that there is an error to print before attempting to do so.

Problem: When RAND() is binlogged in statement mode, the seed is
binlogged too, so the replication slave generates the same
sequence of random numbers. This makes replication work in many
cases, but not in all cases: the order of rows is not guaranteed
for, e.g., UPDATE or INSERT...SELECT statements, so the row data
will be different if master and slave retrieve the rows in
different orders.
Fix: Mark RAND() as unsafe. It will generate a warning if
binlog_format=STATEMENT and switch to row-logging if
binlog_format=ROW.


mysql-test/extra/rpl_tests/rpl_row_func003.test:
  updated test case to ignore new warnings
mysql-test/suite/binlog/r/binlog_unsafe.result:
  updated result file
mysql-test/suite/binlog/t/binlog_unsafe.test:
  Added test for RAND().
  Also clarified some old comments.
mysql-test/suite/rpl/r/rpl_misc_functions.result:
  updated result file
mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result:
  updated test case to ignore new warnings
mysql-test/suite/rpl/r/rpl_optimize.result:
  updated result file
mysql-test/suite/rpl/r/rpl_row_func003.result:
  updated result file
mysql-test/suite/rpl/t/rpl_misc_functions.test:
  updated test case to ignore new warnings
mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test:
  updated test case to ignore new warnings
mysql-test/suite/rpl/t/rpl_optimize.test:
  updated test case to ignore new warnings
mysql-test/suite/rpl/t/rpl_trigger.test:
  updated test case to ignore new warnings
mysql-test/suite/rpl_ndb/r/rpl_ndb_func003.result:
  updated result file
sql/item_create.cc:
  Mark RAND() unsafe.

Problem: copying issuer's (or subject's) name tags into an internal
buffer from incoming stream we didn't check the buffer overflow. 
That may lead to memory overrun, crash etc.

Fix: ensure we don't overrun the buffer.

Note: there's no simple test case (exploit needed).


extra/yassl/taocrypt/include/asn.hpp:
  Fix for bug#50227: Pre-auth buffer-overflow in mySQL through yaSSL
    - CertDecoder::AddTag() introduced.
extra/yassl/taocrypt/src/asn.cpp:
  Fix for bug#50227: Pre-auth buffer-overflow in mySQL through yaSSL
    - copying data from incoming stream to the issuer_ or subject_
  buffers ensure we don't overrun them.
    - code cleanup.

Selecting of the CONCAT_WS(...<PS parameter>...) result into
a user variable may return wrong data.

Item_func_concat_ws::val_str contains a number of memory
allocation-saving optimization tricks. After the fix
for bug 46815 the control flow has been changed to a
branch that is commented as "This is quite uncommon!":
one of places where we are trying to concatenate
strings inplace. However, that "uncommon" place
didn't care about PS parameters, that have another
trick in Item_sp_variable::val_str(): they use the
intermediate Item_sp_variable::str_value field,
where they may store a reference to an external
argument's buffer.

The Item_func_concat_ws::val_str function has been
modified to take into account val_str functions
(such as Item_sp_variable::val_str) that return a
pointer to an internal Item member variable that
may reference to a buffer provided.


mysql-test/r/func_concat.result:
  Added test case for bug #50096.
mysql-test/t/func_concat.test:
  Added test case for bug #50096.
sql/item_strfunc.cc:
  Bug #50096: CONCAT_WS inside procedure returning wrong data
  
  The Item_func_concat_ws::val_str function has been
  modified to take into account val_str functions
  (such as Item_sp_variable::val_str) that return a
  pointer to an internal Item member variable that
  may reference to a buffer provided.

MySQL handles the join syntax "JOIN ... USING( field1,
... )" and natural joins by building the same parse tree as
a corresponding join with an "ON t1.field1 = t2.field1 ..."
expression would produce. This parse tree was not cleaned up
properly in the following scenario. If a thread tries to
lock some tables and finds that the tables were dropped and
re-created while waiting for the lock, it cleans up column
references in the statement by means a per-statement free
list. But if the statement was part of a stored procedure,
column references on the stored procedure's free list weren't
cleaned up and thus contained pointers to freed objects.

Fixed by adding a call to clean up the current prepared
statement's free list.


mysql-test/r/sp_sync.result:
  Bug#48157: Test case
mysql-test/t/sp_sync.test:
  Bug#48157: Test result
sql/item.h:
  Bug#48157: Commented field.
sql/sql_parse.cc:
  Bug#48157: Commented function.
sql/sql_update.cc:
  Bug#48157: fix

This includes "MYSQL_U_SCORE_VERSION" in "configure.in".

- "release" starts from 1
- "level" ("m2", "rc", ...) is included in the RPM version.

but don't take the "tree name" change.

- "release" starts from 1
- "level" ("m2", "rc", ...) is included in the RPM version.

32bit builds with the --enable-assembler flag (enabled by default)
fail with an error message: undefined reference to `strmov_overlapp'.

Since the fix for bug 48866 we use a home-grown strmov function
instead of the ctpcpy function, but the source file for this
function was missed in the Makefile.am.

The strings/Makefile.am file has been modified to include strmov.c
file into ASSEMBLER_x86 and ASSEMBLER_sparc32 sections.


strings/Makefile.am:
  Bug #49955: ld error message: undefined reference to `strmov_overlapp'
  
  The strings/Makefile.am file has been modified to include
  strmov.c file into ASSEMBLER_x86 and ASSEMBLER_sparc32 sections.

Recover the right contents of the index file at the end of the test case.

Manually deleteing one or more entries from 'master-bin.index', will
cause master infinitely loop to send one binlog file. 

When starting a dump session, master opens index file and search the binlog file
which is being requested by the slave. The position of the binlog file in the
index file is recorded. it will be used to find the next binlog file when current
binlog file has dumped completely. As only the position is used, it may
not get the correct file if some entries has been removed manually from the index file.
the master will reopen the current binlog file which has been dump completely
and redump it if it can not get the next binlog file's name from index file.
It obviously is a logical error.


Even though it is allowed to manually change index file,
but it is not recommended. so after this patch, master
sends a fatal error to slave and close the dump session if a new binlog file
has been generated and master can not get it from the index file.

Some improvements on the test case as suggested during review.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

For tables with metadata sizes ranging from 251 to 255 the size
of the event data (m_data_size) was being improperly calculated
in the Table_map_log_event constructor. This was due to the fact
that when writing the Table_map_log_event body (in
Table_map_log_event::write_data_body) a call to net_store_length
is made for packing the m_field_metadata_size. It happens that
net_store_length uses *one* byte for storing
m_field_metadata_size when it is smaller than 251 but *three*
bytes when it exceeds that value. BUG 42749 had already
pinpointed and fix this fact, but the fix was incomplete, as the
calculation in the Table_map_log_event constructor considers 255
instead of 251 as the threshold to increment m_data_size by
three. Thence, the window for having a mismatch between the
number of bytes written and the number of bytes accounted in the
event length (m_data_size) was left open for
m_field_metadata_size values between 251 and 255.

We fix this by changing the condition in the Table_map_log_event
constructor to match the one in the net_store_length, ie,
increment one byte if m_field_metadata_size < 251 and three if it
exceeds this value.

mysql-test/suite/rpl/r/rpl_row_tbl_metadata.result:
  Updated result file.
mysql-test/suite/rpl/t/rpl_row_tbl_metadata.test:
  Changes to the original test case: added slave and moved
  file into the rpl suite.
  
  New test case: replicates two tables one with 250 and 
  another with 252 metadata sizes. This exercises the usage
  of 1 or 3 bytes while packing the m_field_metadata_size.
sql/log_event.cc:
  Made the m_data_size calculation for the table map log event
  to match the number of bytes used while packing the 
  m_field_metadata_size value (according to net_store_length
  function in pack.c).

This bug is the same problem as Bug 49836 for 5.1 versions.

mysql-test/suite/rpl/r/rpl_geometry.result:
  Test case for bug 48776
mysql-test/suite/rpl/t/rpl_geometry.test:
  Test case for bug 48776
sql/rpl_utility.h:
  Add missing case MYSQL_TYPE_GEOMETRY

      
In statement-based or mixed-mode replication, use DROP TEMPORARY TABLE
to drop multiple tables causes different errors on master and slave, 
when one or more of these tables do not exist. Because when executed
on slave, it would automatically add IF EXISTS to the query to ignore
all ER_BAD_TABLE_ERROR errors.

To fix the problem, do not add IF EXISTS when executing DROP TEMPORARY
TABLE on the slave, and clear the ER_BAD_TABLE_ERROR error after
execution if the query does not expect any errors.

mysql-test/suite/rpl/r/rpl_drop_temp.result:
  Updated for the patch of bug#49137.
mysql-test/suite/rpl/t/rpl_drop_temp.test:
  Added the test file to verify if DROP MULTI TEMPORARY TABLE
  will cause different errors on master and slave, when one or
  more of these tables do not exist.
sql/log_event.cc:
  Added code to handle above cases which are 
  removed from sql_parse.cc
sql/sql_parse.cc:
  Remove the code to issue the 'Unknown table' error, 
  if the temporary table does not exist when dropping 
  it on slave. The above cases decribed in comments
  will be handled later in log_event.cc.

      
In statement-based or mixed-mode replication, use DROP TEMPORARY TABLE
to drop multiple tables causes different errors on master and slave, 
when one or more of these tables do not exist. Because when executed
on slave, it would automatically add IF EXISTS to the query to ignore
all ER_BAD_TABLE_ERROR errors.

To fix the problem, do not add IF EXISTS when executing DROP TEMPORARY
TABLE on the slave, and clear the ER_BAD_TABLE_ERROR error after
execution if the query does not expect any errors.


mysql-test/r/rpl_drop_temp.result:
  Updated for the patch of bug#49137.
mysql-test/t/rpl_drop_temp.test:
  Added the test file to verify if DROP MULTI TEMPORARY TABLE
  will cause different errors on master and slave, when one or
  more of these tables do not exist.
sql/log_event.cc:
  Added code to handle above cases which are
  removed from sql_parse.cc
sql/sql_parse.cc:
  Remove the code to issue the 'Unknown table' error,
  if the temporary table does not exist when dropping
  it on slave. The above cases decribed in comments
  will be handled later in log_event.cc.