GHz.

Some collations were causing IBMDB2I to report
inaccurate key range estimations to the optimizer
for LIKE clauses that select substrings. This can
be seen by running EXPLAIN. This problem primarily
affects multi-byte and unicode character sets.

This patch involves substantial changes to several
modules. There are a number of problems with the
character set and collation handling. These problems
have been or are being fixed,  and a comprehensive
test has been included which should provide much
better coverage than there was before. This test
is enabled only for IBM i 6.1, because that version
has support for the greatest number of collations.

mysql-test/suite/ibmdb2i/r/ibmdb2i_collations.result:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  result file for test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_collations.test:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  Tests for character sets and collations. This test
  is enabled only for IBM i 6.1, because that version
  has support for the greatest number of collations.
storage/ibmdb2i/db2i_conversion.cc:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  - Added support in convertFieldChars to enable records_in_range
    to determine how many substitute characters were inserted and
    to suppress conversion warnings.
  
  - Fixed bug which was causing all multi-byte and Unicode fields
    to be created as UTF16 (CCSID 1200) fields in DB2. The corrected
    code will now create UCS2 fields as UCS2 (CCSID 13488), UTF8
    fields (except for utf8_general_ci) as UTF8 (CCSID 1208), and
    all other multi-byte or Unicode fields as UTF16.  This will only
    affect tables that are newly created through the IBMDB2I storage
    engine. Existing IBMDB2I tables will retain the original CCSID
    until recreated. The existing behavior is believed to be
    functionally correct, but it may negatively impact performance
    by causing unnecessary character conversion. Additionally, users
    accessing IBMDB2I tables through DB2 should be aware that mixing 
    tables created before and after this change may require extra type
    casts or other workarounds.  For this reason, users who have
    existing IBMDB2I tables using a Unicode collation other than
    utf8_general_ci are encouraged to recreate their tables (e.g.
    ALTER TABLE t1 ENGINE=IBMDB2I) in order to get the updated CCSIDs
    associated with their DB2 tables.
  
  - Improved error reporting for unsupported character sets by forcing
    a check for the iconv conversion table at table creation time,
    rather than at data access time.
storage/ibmdb2i/db2i_myconv.h:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  Fix to set errno when iconv fails.
storage/ibmdb2i/db2i_rir.cc:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  Significant improvements were made to the records_in_range code
  that handles partial length string data in keys for optimizer plan
  estimation. Previously, to obtain an estimate for a partial key
  value, the implementation would perform any necessary character
  conversion and then attempt to determine the unpadded length of
  the partial key by searching for the minimum or maximum sort
  character. While this algorithm was sufficient for most single-byte
  character sets, it did not treat Unicode and multi-byte strings
  correctly. Furthermore, due to an operating system limitation,
  partial keys having UTF8 collations (ICU sort sequences in DB2)
  could not be estimated with this method.
  
  With this patch, the code no longer attempts to explicitly determine
  the unpadded length of the key. Instead, the entire key is converted
  (if necessary), including padding, and then passed to the operating
  system for estimation. Depending on the source and target character
  sets and collations, additional logic is required to correctly
  handle cases in which MySQL uses unconvertible or differently
  -weighted values to pad the key. The bulk of the patch exists
  to implement this additional logic.
storage/ibmdb2i/ha_ibmdb2i.h:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  The convertFieldChars declaration was updated to support additional 
  optional behaviors.

timeout
            
In STMT and MIXED modes, a statement that changes both non-transactional and
transactional tables must be written to the binary log whenever there are
changes to non-transactional tables. This means that the statement gets into the
binary log even when the changes to the transactional tables fail. In particular
, in the presence of a failure such statement is annotated with the error number
and wrapped in a begin/rollback. On the slave, while applying the statement, it
is expected the same failure and the rollback prevents the transactional changes
to be persisted.
            
Unfortunately, statements that fail due to concurrency issues (e.g. deadlocks,
timeouts) are logged in the same way causing the slave to stop as the statements
are applied sequentially by the SQL Thread. To fix this bug, we automatically
ignore concurrency failures on the slave. Specifically, the following failures
are ignored: ER_LOCK_WAIT_TIMEOUT, ER_LOCK_DEADLOCK and ER_XA_RBDEADLOCK.

          server

If the server connection was lost during repeated status commands,
the client would fail to detect this and the client output would be inconsistent.

This patch fixes this issue by making sure that the server is online
before the client attempts to execute the status command.


client/mysql.cc:
  * Replace variable "connected" with a call to mysql_real_query_for_lazy()
    will attempt to reconnect to server on if there is a failure.

The crash happend because for views which are joins
we have table_list->table == 0 and 
table_list->table->'any method' call leads to crash.
The fix is to perform table_list->table->file->extra()
method for all tables belonging to view.


mysql-test/r/view.result:
  test result
mysql-test/t/view.test:
  test case
sql/sql_insert.cc:
  added prepare_for_positional_update() function
  which updates extra info about primary key for
  tables belonging to view.

enabled message storing into error message list
for 'drop table' command


mysql-test/r/warnings.result:
  test result
mysql-test/t/warnings.test:
  test case
sql/sql_table.cc:
  We should skip error sending then we should return
  warnings to client as some functions may send its
  own errors, so we should set no_warnings_for_error= 0
  only in case of warning.
  The fix is to enable message storing into error message
  list for 'drop table' command(only for error case).
tests/mysql_client_test.c:
  test fix

 
Using DECIMAL constants with more than 65 digits in CREATE 
TABLE ... SELECT led to bogus errors in release builds or 
assertion failures in debug builds. 
 
The problem was in inconsistency in how DECIMAL constants and 
fields are handled internally. We allow arbitrarily long 
DECIMAL constants, whereas DECIMAL(M,D) columns are limited to 
M<=65 and D<=30. my_decimal_precision_to_length() was used in 
both Item and Field code and truncated precision to 
DECIMAL_MAX_PRECISION when calculating value length without 
adjusting precision and decimals. As a result, a DECIMAL 
constant with more than 65 digits ended up having length less 
than precision or decimals which led to assertion failures. 
 
Fixed by modifying my_decimal_precision_to_length() so that 
precision is truncated to DECIMAL_MAX_PRECISION only for Field 
object which is indicated by the new 'truncate' parameter. 
 
Another inconsistency fixed by this patch is how DECIMAL 
constants and expressions are handled for CREATE ... SELECT. 
create_tmp_field_from_item() (which is used for constants) was 
changed as a part of the bugfix for bug #24907 to handle long 
DECIMAL constants gracefully. Item_func::tmp_table_field() 
(which is used for expressions) on the other hand was still 
using a simplistic approach when creating a Field_new_decimal 
from a DECIMAL expression. 

mysql-test/r/type_newdecimal.result:
  Added a test case for bug #45262.
mysql-test/t/type_newdecimal.test:
  Added a test case for bug #45262.
sql/item.cc:
  Use the new 'truncate' parameter in 
  my_decimal_precision_to_length().
sql/item_cmpfunc.cc:
  Use the new 'truncate' parameter in 
  my_decimal_precision_to_length().
sql/item_func.cc:
  1. Use the new 'truncate' parameter in 
  my_decimal_precision_to_length().
  
  2. Do not truncate decimal precision to DECIMAL_MAX_PRECISION
  for additive expressions involving long DECIMAL constants.
  
  3. Fixed an incosistency in how DECIMAL constants and 
  expressions are handled for CREATE ... SELECT.
sql/item_func.h:
  Use the new 'truncate' parameter in 
  my_decimal_precision_to_length().
sql/item_sum.cc:
  Use the new 'truncate' parameter in 
  my_decimal_precision_to_length().
sql/my_decimal.h:
  Do not truncate precision to DECIMAL_MAX_PRECISION
  when calculating length in 
  my_decimal_precision_to_length() if 'truncate' parameter
  is FALSE.
sql/sql_select.cc:
  1. Use the new 'truncate' parameter in 
  my_decimal_precision_to_length().
  
  2. Use a more correct logic when adjusting value's length.

contains ONLY_FULL_GROUP_BY

The partitioning code needs to issue a Item::fix_fields()
on the partitioning expression in order to prepare 
it for being evaluated.
It does this by creating a special table and a table list 
for the scope of the partitioning expression.
But when checking ONLY_FULL_GROUP_BY the 
Item_field::fix_fields() was relying that there always be
cached_table set and was trying to use it to get the 
select_lex of the SELECT the field's table is in.
But the cached_table was not set by the partitioning code
that creates the artificial TABLE_LIST used to resolve the
partitioning expression and this resulted in a crash.
 
Fixed by rectifying the following errors :
1. Item_field::fix_fields() : the code that check for 
ONLY_FULL_GROUP_BY relies on having tables with 
cacheable_table set. This is mostly true, the only 
two exceptions being the partitioning context table
and the trigger context table.
Fixed by taking the current parsing context if no pointer
to the TABLE_LIST instance is present in the cached_table.

2. fix_fields_part_func() : 

2a. The code that adds the table being created to the 
scope for the partitioning expression is mostly a copy 
of the add_table_to_list and friends with one exception :
it was not marking the table as cacheable (something that
normal add_table_to_list is doing). This caused the 
problem in the check for ONLY_FULL_GROUP_BY in 
Item_field::fix_fields() to appear.
Fixed by setting the correct members to make the table
cacheable.
The ideal structural fix for this is to use a unified 
interface for adding a table to a table list 
(add_table_to_list?) : noted in a TODO comment

2b. The Item::fix_fields() was called with a NULL destination
pointer. This causes uninitalized memory reads in the 
overloaded ::fix_fields() function (namely 
Item_field::fix_fields()) as it expects a non-zero pointer 
there. Fixed by passing the source pointer similarly to how 
it's done in JOIN::prepare().

mysql-test/r/partition.result:
  Bug #45807: test case
mysql-test/t/partition.test:
  Bug #45807: test case
sql/item.cc:
  Bug #45807: fix the ONLY_FULL_GROUP_BY check code to 
  handle correctly non-cacheable tables.
sql/sql_partition.cc:
  Bug #45807: fix the Item::fix_fields() context
  initializatio for the partitioning expression in 
  CREATE TABLE.