being logged to slow query log

The problem is that the execution time for a multi-statement
stored procedure as a whole may not be accurate, and thus not
be entered into the slow query log even if the total time
exceeds long_query_time. The reason for this is that
THD::utime_after_lock used for time calculation may be reset
at the start of each new statement, possibly leaving the total
SP execution equal to the time spent executing the last
statement in the SP.

This patch stores the utime on start of SP execution, and
restores it on exit of SP execution. A test is added.


mysql-test/suite/sys_vars/r/slow_query_log_func.result:
  New test results for #47905.
mysql-test/suite/sys_vars/t/slow_query_log_func.test:
  New test case for #47905.
sql/sp_head.cc:
  Save and restore the THD::utime_after_lock on entry and 
  exit of execute_procedure().

The problem becomes apparent only if HAVE_purify is undefined.
It related to the part of code placed in open_table_from_share() fuction
where we initialize record buffer only if HAVE_purify is enabled.
So in case of HAVE_purify=OFF record buffer is not initialized
on open table stage.
Next we read key, find NULL value and update appropriate null bit
but do not update record buffer. After that the record is stored
in the join cache(store_record_in_cache). For CHAR fields we
strip trailing spaces and in our case this procedure uses
uninitialized record buffer.
The fix is to skip stripping space procedure in case of null values
for CHAR fields(partially based on 6.0 JOIN_CACHE implementation).


mysql-test/r/join.result:
  test case
mysql-test/t/join.test:
  test case
sql/field.cc:
  code updated according to new CACHE_FIELD struct
sql/sql_select.cc:
  code updated according to new CACHE_FIELD struct
sql/sql_select.h:
  CACHE_FIELD struct:
  added new fields: Field *field, uint type;
  removed fields: Field_blob *blob_field, bool strip;

error causes debug assertion

The IGNORE option of the multiple-table UPDATE command was
not intended to suppress errors caused by the
sql_safe_updates mode. This flag will raise an error if the
execution of UPDATE does not use a key for row retrieval,
and should continue do so regardless of the IGNORE option.

However the implementation of IGNORE does not support
exceptions to the rule; it always converts errors to
warnings and cannot be extended. The Internal_error_handler
interface offers the infrastructure to handle individual
errors, making sure that the error raised by
sql_safe_updates is not silenced.

Fixed by implementing an Internal_error_handler and using it
for UPDATE IGNORE commands.

          removed in MySQL 6.0

CREATE TABLE... TYPE= returns the warning "The syntax 
'TYPE=storage_engine' is deprecated and will be removed in 
MySQL 6.0. Please use 'ENGINE=storage_engine' instead" 

This syntax is deprecated already from version 5.4.4, so
the message has been changed.

In addition, the deprecation macro was changed to reflect
the ServerPT decision not to include version number in the
warning message.

A number of test result files have been changed as a
consequence of the change in the deprecation macro.

Queries optimized with GROUP_MIN_MAX didn't cleanup KEYREAD
optimization properly. As a result subsequent queries may
return incomplete rows (fields are initialized to default
values).

mysql-test/r/group_min_max.result:
  A test case for BUG#49902.
mysql-test/t/group_min_max.test:
  A test case for BUG#49902.
sql/opt_range.cc:
  Refactor of KEYREAD optimization switch so that KEYREAD
  handler state is in sync with st_table::key_read flag.
  
  All SQL code is supposed to switch KEYREAD optimization
  via st_table::set_keyread().
sql/opt_sum.cc:
  Refactor of KEYREAD optimization switch so that KEYREAD
  handler state is in sync with st_table::key_read flag.
  
  All SQL code is supposed to switch KEYREAD optimization
  via st_table::set_keyread().
sql/sql_select.cc:
  Refactor of KEYREAD optimization switch so that KEYREAD
  handler state is in sync with st_table::key_read flag.
  
  All SQL code is supposed to switch KEYREAD optimization
  via st_table::set_keyread().
sql/sql_update.cc:
  Refactor of KEYREAD optimization switch so that KEYREAD
  handler state is in sync with st_table::key_read flag.
  
  All SQL code is supposed to switch KEYREAD optimization
  via st_table::set_keyread().
sql/table.cc:
  Refactor of KEYREAD optimization switch so that KEYREAD
  handler state is in sync with st_table::key_read flag.
  
  All SQL code is supposed to switch KEYREAD optimization
  via st_table::set_keyread().
sql/table.h:
  Refactor of KEYREAD optimization switch so that KEYREAD
  handler state is in sync with st_table::key_read flag.
  
  All SQL code is supposed to switch KEYREAD optimization
  via st_table::set_keyread().

logging is disabled

Post-push fix: disabling test when running mysqld in 
               embedded mode.

Grouping by a subquery in a query with a distinct aggregate
function lead to a wrong result (wrong and unordered
grouping values).

There are two related problems:

1) The query like this:

   SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
   FROM t1 GROUP BY aa

returned wrong result, because the outer reference "t1.a"
in the subquery was substituted with the Item_ref item.

The Item_ref item obtains data from the result_field object
that refreshes once after the end of each group. This data
is not applicable to filesort since filesort() doesn't care
about groups (and doesn't update result_field objects with
copy_fields() and so on). Also that data is not applicable
to group separation algorithm: end_send_group() checks every
record with test_if_group_changed() that evaluates Item_ref
items, but it refreshes those Item_ref-s only after the end
of group, that is a vicious circle and the grouped column
values in the output are shifted.

Fix: if
       a) we grouping by a subquery and
       b) that subquery has outer references to FROM list
          of the grouping query,
     then we substitute these outer references with
     Item_direct_ref like references under aggregate
     functions: Item_direct_ref obtains data directly
     from the current record.

2) The query with a non-trivial grouping expression like:

   SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
   FROM t1 GROUP BY aa+0

also returned wrong result, since JOIN::exec() substitutes
references to top-level aliases in SELECT list with Item_copy
caching items. Item_copy items have same refreshing policy
as Item_ref items, so the whole groping expression with
Item_copy inside returns wrong result in filesort() and
end_send_group().

Fix: include aliased items into GROUP BY item tree instead
     of Item_ref references to them.



mysql-test/r/group_by.result:
  Test case for bug #45640
mysql-test/t/group_by.test:
  Test case for bug #45640
sql/item.cc:
  Bug #45640: optimizer bug produces wrong results
  
  Item_field::fix_fields() has been modified to resolve
  aliases in GROUP BY item trees into aliased items instead
  of Item_ref items.
sql/item.h:
  Bug #45640: optimizer bug produces wrong results
  
  - Item::find_item_processor() has been introduced.
  - Item_ref::walk() has been modified to apply processors
    to itself too (not only to referenced item).
sql/mysql_priv.h:
  Bug #45640: optimizer bug produces wrong results
  
  fix_inner_refs() has been modified to accept group_list
  parameter.
sql/sql_lex.cc:
  Bug #45640: optimizer bug produces wrong results
  
  Initialization of st_select_lex::group_fix_field has
  been added.
sql/sql_lex.h:
  Bug #45640: optimizer bug produces wrong results
  
  The st_select_lex::group_fix_field field has been introduced
  to control alias resolution in Itef_fied::fix_fields.
sql/sql_select.cc:
  Bug #45640: optimizer bug produces wrong results
  
  - The fix_inner_refs function has been modified to treat
    subquery outer references like outer fields under aggregate
    functions, if they are included in GROUP BY item tree.
  
  - The find_order_in_list function has been modified to
    fix Item_field alias fields included in the GROUP BY item
    trees in a special manner.

logging is disabled
      
The server would hit an assertion because of a DBUG violation.
There was a missing DBUG_RETURN and instead a plain return
was used.
      
This patch replaces the return with DBUG_RETURN.

into slow log
      
While processing a statement, down the mysql_parse execution
stack, the thd->enable_slow_log can be assigned to
opt_log_slow_admin_statements, depending whether one is executing
administrative statements, such as ALTER TABLE, OPTIMIZE,
ANALYZE, etc, or not. This can have an impact on slow logging for
statements that are executed after an administrative statement
execution is completed.
      
When executing statements directly from the user this is fine
because, the thd->enable_slow_log is reset right at the beginning
of the dispatch_command function, ie, everytime a new statement
is set is set to execute.
      
On the other hand, for slave SQL thread (sql_thd) the story is a
bit different. When in SBR the sql_thd applies statements by
calling mysql_parse. Right after, it calls log_slow_statement
function to log them if they take too long. Calling mysql_parse
directly is fine, but also means that dispatch_command function
is bypassed. As a consequence, thd->enable_slow_log does not get
a chance to be reset before the next statement to be executed by
the sql_thd. If the statement just executed by the sql_thd was an
administrative statement and logging of admin statements was
disabled, this means that sql_thd->enable_slow_log will be set to
0 (disabled) from that moment on. End result: sql_thd stops
logging slow statements.
      
We fix this by resetting the value of sql_thd->enable_slow_log to
the value of opt_log_slow_slave_statements right after
log_slow_stement is called by the sql_thd.

To 5.x Release
      
Notes
=====
      
This is a backport of BUG#23300 into 5.1 GA.
      
Original cset revid (in betony):
luis.soares@sun.com-20090929140901-s4kjtl3iiyy4ls2h

Description
===========
      
When using replication, the slave will not log any slow query
logs queries replicated from the master, even if the
option "--log-slow-slave-statements" is set and these take more
than "log_query_time" to execute.
                    
In order to log slow queries in replicated thread one needs to
set the --log-slow-slave-statements, so that the SQL thread is
initialized with the correct switch. Although setting this flag
correctly configures the slave thread option to log slow queries,
there is an issue with the condition that is used to check
whether to log the slow query or not. When replaying binlog
events the statement contains the SET TIMESTAMP clause which will
force the slow logging condition check to fail. Consequently, the
slow query logging will not take place.
                    
This patch addresses this issue by removing the second condition
from the log_slow_statements as it prevents slow queries to be
binlogged and seems to be deprecated.

When using MyIsam tables and processing concurrent DML
statements, the server may be sending back an OK to the client
before actually finishing the transaction commit procedure. This
has been reported before in BUG@37521 and BUG@29334.

This particular test case gets affected, because it performs the
following sequence:
  
  connect (conn2, ...)
  connection conn2;
  LOAD DATA CONCURRENT ...
  disconnect (conn2, ...)
  connection master;
  sync_slave_with_master
  diff_tables

At this point diff_tables may report difference in the table
content (the master seems to be missing the conn2 rows). 

To workaround this MyISAM concurrent DML statements issue and
make this test case deterministic, we wait on conn2 until the
rows inserted show up in the table. After this the test case
proceeds as normally would before this patch.

path names than 99 characters, using the USTAR format of the
resulting source TAR.

To be able to specify the use of USTAR when creating the source
TAR, we needed both to update the GNU autotools version requirements
slightly, and update the initiation of the tools to use more
modern constructs.

Performing fulltext prefix search (a word with truncation
operator) may cause a dead-loop. ft_min_word_len value
doesn't matter actually.

The problem was introduced along with "smarter index merge"
optimization.

mysql-test/r/fulltext.result:
  A test case for BUG#50351.
mysql-test/t/fulltext.test:
  A test case for BUG#50351.
storage/myisam/ft_boolean_search.c:
  When going up to first-level tree, we need to restore docid[0],
  so it informs fulltext index merge not to enter this second-level
  tree again (avoiding dead-loop).

Fixed 2 problems :
1. test_if_order_by_key() was continuing on the primary key
as if it has a primary key suffix (as the secondary keys do).
This leads to crashes in ORDER BY <pk>,<pk>.
Fixed by not treating the primary key as the secondary one
and not depending on it being clustered with a primary key.
2. The cost calculation was trying to read the records 
per key when operating on ORDER BYs that order on all of the 
secondary key + some of the primary key.
This leads to crashes because of out-of-bounds array access.
Fixed by assuming we'll find 1 record per key in such cases.

The problem was that the dbug facility was being used after the
per-thread dbug state had already been finalized. The was present
in a few functions which invoked decrement_handler_count, which
in turn invokes my_thread_end on Windows. In my_thread_end, the
per-thread dbug state is finalized. Any use after the state is
finalized ends up creating a new state.

The solution is to process the exit of a function before the
decrement_handler_count function is called.


sql/mysqld.cc:
  Process the function exit before decrement_handler_count is
  called, as it can end the per-thread dbug state on Windows.

merging patches prepared for 5.0 to 5.1-bt. That caused a few changes in the test file

Define env. vars for both timeout settings
Also incorporated 5.0 patch into mtr version 1

--extended-insert

Help message changed to the same as in the 5.1 online 
documentation.

  Define env. vars for both timeout settings
  This patch is for 5.0 (mtr v1) and should replaces for 5.1 up

improving comments

      
WL#5182 is a follow-up to WL#5154, deprecating a few more options
and system variables.


client/client_priv.h:
  The warning message has been changed to not include
  a specific version number in the text.
client/mysql.cc:
  --no-tee is deprecated
client/mysqldump.c:
  --all is deprecated
  -a now points to create-options
mysql-test/r/mysqlbinlog.result:
  Warning text changed
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
  Warning text changed
sql/mysql_priv.h:
  The warning message has been changed to not include
  a specific version number in the text.
sql/mysqld.cc:
  --use-symbolic-links is deprecated
  -s now points to --symbolic-links
  --warnings is deprecated
  -W now points to --log-warnings
          
  myisam_max_extra_sort_file_size is deprecated
  record_buffer is deprecated
          
  --log-update is deprecated
  --sql-bin-update-same is deprecated
  --skip-locking is deprecated
  --skip-symlink is deprecated
  --enable-locking is deprecated
  --delay-key-write-for-all-tables is deprecated

printstack() being present

When Bug#47391 was fixed, no assumption was made that support
for Solaris 8 was needed. Solaris 8 lacks printstack(), and 
the build breaks because of this.

This patch adds a test for the presence of printstack() to
configure.in for 5.0, and uses HAVE_PRINTSTACK to make
decisions rather than the __sun define.

The 'rpl_get_master_version_and_clock' test verifies if the slave I/O
thread tries to reconnect to master when it tries to get the values of
the UNIX_TIMESTAMP, SERVER_ID from master under network disconnection.
So the master server is restarted for making the transient network
disconnection, during the period the COM_REGISTER_SLAVE failures are
produced in server log file when the slave I/O thread tries to
register on master.

To fix the problem, suppress COM_REGISTER_SLAVE failures in server log
file by mtr suppression, because they are expected.


mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result:
  Removed mtr.add_suppression("Get master clock failed with error: ")
  and mtr.add_suppression("Get master SERVER_ID failed with error: ").
  Because they are suppressed globally.

MySQL's hash functions MD5 and SHA relied on the somewhat slow 
sprintf function to convert the digests to hex representations.
This patch replaces the sprintf with a specific and inline hex
conversion function.

Patch contributed by Jan Steemann.

sql/item_strfunc.cc:
  Add a hex conversion function.

NOTE: added TODO to the comments requested by reviewer during this
      merge.

should be exited before destroying the thread local storage.

When replicating from 4.1 master to 5.0 slave START SLAVE UNTIL can stop too late.
The necessary in calculating of the beginning of an event the event's length
did not correspond to the master's genuine information at the event's execution time.
That piece of info was changed at the event's relay-logging due to binlog_version<4 event
conversion by IO thread.

Fixed with storing the master genuine Query_log_event size into a new status
variable at relay-logging of the event. The stored info is extacted at the event
execution and participate further to caclulate the correct start position of the event
in the until-pos stopping routine.
The new status variable's algorithm will be only active when the event comes
from the master of version < 5.0 (binlog_version < 4).


mysql-test/r/rpl_until.result:
  results changed.
mysql-test/std_data/bug47142_master-bin.000001:
  a binlog from 4.1 master to replace one of the running 5.x master is added as 
  part of Bug #47142 regression test.
mysql-test/t/rpl_until.test:
  Regression test for Bug #47142 is added.
sql/log_event.cc:
  Storing and extracting the master's genuine size of the event from the status
  var of the event packet header.
  The binlog_version<4 query-log-event is 
  a. converted into the modern binlog_version==4 to store the original size of the event
     into a new status var; the converted representation goes into the relay log.
  b. the converted event is read out and the stored size is engaged in the start pos calculation.
  
  The new status is active only for events that IO thread instantiates for the sake of the conversion.
sql/log_event.h:
  Incrementing the max szie of MAX_SIZE_LOG_EVENT_STATUS because of the new status var;
  Defining the new status variable to hold the master's genuine event size;
  Augmenting the Query_log_event with a new member to hold a value to store/extact from the status
  var of the event packet header.

sql_udf.cc

Detailed revision comments:

r6471 | calvin | 2010-01-16 01:43:27 +0200 (Sat, 16 Jan 2010) | 4 lines
branches/5.1: fix bug#49396: main.innodb test fails in embedded mode

Change replace_result by using $MYSQLD_DATADIR. Tested in both embedded
mode and normal server mode.

removed wrongly introduced strlen calls


sql/events.cc:
  removed wrongly introduced strlen calls
sql/mysql_priv.h:
  removed wrongly introduced strlen calls
sql/repl_failsafe.cc:
  removed wrongly introduced strlen calls
sql/sql_db.cc:
  removed wrongly introduced strlen calls
sql/sql_parse.cc:
  removed wrongly introduced strlen calls
sql/sql_show.cc:
  removed wrongly introduced strlen calls