Fix Bug#39939 DROP TABLE/DISCARD TABLESPACE takes long time in
buf_LRU_invalidate_tablespace()

Improve implementation of buf_LRU_invalidate_tablespace by attempting
hash index drop in batches instead of doing it one by one.

Reviewed by: Heikki, Sunny, Marko
Approved by: Heikki

fix for bug# 38839 triggered this bug. When the offset and increment are > 1
we need to calculate the next value taking into consideration the two
variables. Previously we simply assumed they were 1 particularly offset was
never used. MySQL does its own calculation and that's probably why it seemed
to work in the past. We would return what we thought was the correct next
value and then MySQL would recalculate the actual value from that and return
it to the caller (e.g., handler::write_row()). Several new tests have been
added that try and catch some edge cases. The tests exposed a wrap around
error in MySQL next value calculation which was filed as bug#39828. The tests
will need to be updated once MySQL fix that bug.

One good side effect of this fix is that dict_table_t size has been
reduced by 8 bytes because we have moved the autoinc_increment field to
the row_prebuilt_t structure. See review-board for a detailed discussion.

rb://3

handler::get_auto_increment() doesn't allow a way to return the
specific error for why it failed.

rb://18

Print a warning if an attempt is made to get the free space for a table
whose .ibd file is missing or the tablespace has been discarded. This is a
followup to r2719.

Suggested by:	Inaam

Fix Bug#39438 Testcase for Bug#39436 crashes on 5.1 in fil_space_get_latch

In ha_innobase::info() - do not try to get the free space for a tablespace
which has been discarded with ALTER TABLE ... DISCARD TABLESPACE or if the
.ibd file is missing for some other reason.

ibd_file_missing and tablespace_discarded are manipulated only in
row_discard_tablespace_for_mysql() and in row_import_tablespace_for_mysql()
and the manipulation is protected/surrounded by
row_mysql_lock_data_dictionary()/row_mysql_unlock_data_dictionary() thus we
do the same in ha_innobase::info() when checking the values of those members
to avoid race conditions. I have tested the code-path with UNIV_DEBUG and
UNIV_SYNC_DEBUG.

Looks like it is not possible to avoid mysqld printing warnings in the
mysql-test case and thus this test innodb_bug39438 must be added to the
list of exceptional test cases that are allowed to print warnings. For this,
the following patch must be applied to the mysql source tree:

  --- cut ---
  === modified file 'mysql-test/lib/mtr_report.pl'
  --- mysql-test/lib/mtr_report.pl	2008-08-12 10:26:23 +0000
  +++ mysql-test/lib/mtr_report.pl	2008-10-01 11:57:41 +0000
  @@ -412,7 +412,10 @@
   
                   # When trying to set lower_case_table_names = 2
                   # on a case sensitive file system. Bug#37402.
  -                /lower_case_table_names was set to 2, even though your the file system '.*' is case sensitive.  Now setting lower_case_table_names to 0 to avoid future problems./
  +                /lower_case_table_names was set to 2, even though your the file system '.*' is case sensitive.  Now setting lower_case_table_names to 0 to avoid future problems./ or
  +
  +                # this test is expected to print warnings
  +                ($testname eq 'main.innodb_bug39438')
   		)
               {
                 next;                       # Skip these lines
  
  --- cut ---

The mysql-test is currently somewhat disabled (see inside
innodb_bug39438.test), after the above patch has been applied to the mysql
source tree, the test can be enabled.

rb://20

Reviewed by:	Inaam, Calvin
Approved by:	Heikki

Silence a compilation warning in UNIV_DEBUG.

Approved by:	Marko (via IM)

Fix Bug#38231 Innodb crash in lock_reset_all_on_table() on TRUNCATE + LOCK / UNLOCK

In TRUNCATE TABLE and discard tablespace: do not remove table-level S
and X locks and do not assert on such locks not being wait locks.
Leave such locks alone.

Approved by:	Heikki (rb://14)

to return the cause of failure we have to inform MySQL using the
sql_print_warning() function to return the cause for autoinc failure.
Previously we simply printed the error code, this patch prints the
text string representing the following two error codes:

DB_LOCK_WAIT_TIMEOUT
DB_DEADLOCK.

Bug#35498 Cannot get table test/table1 auto-inccounter value in ::info

Approved by Marko.

Instead of doing "svn log" for every revision between $START_REV and $END_REV,
lookup the revisions which actually contain changes to this branch and do
"svn log" and "svn diff" only for them.

This makes the script many times faster.

I have checked that both old and new variant create identical snapshots.

of order ::open() call by MySQL

Forward port of r2629

Under some conditions MySQL calls ::open with search_latch leading
to a deadlock as we try to acquire dict_sys->mutex inside ::open
breaking the latching order. The fix is to release search_latch.

Reviewed by: Heikki

Merge a change from MySQL (fix the failing innodb test):

  ------------------------------------------------------------
  revno: 2646.12.1
  committer: Mattias Jonsson <mattiasj@mysql.com>
  branch nick: wl4176_2-51-bugteam
  timestamp: Mon 2008-08-11 20:02:03 +0200
  message:
    Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
    partition is corrupt
    
    The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
    PARTITION took another code path (over mysql_alter_table instead of
    mysql_admin_table) which differs in two ways:
    1) alter table opens the tables in a different way than admin tables do
       resulting in returning with error before it tried the command
    2) alter table does not start to send any diagnostic rows to the client
       which the lower admin functions continue to use -> resulting in
       assertion crash
    
    The fix:
    Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
    the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
    Adding check in mysql_admin_table to setup the partition list for
    which partitions that should be used.
    
    
    Partitioned tables will still not work with
    REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
    to tables, REPAIR TABLE t USE_FRM, and check that the data still
    fulfills the partitioning function and then move the table back to
    being a partition.
    
    NOTE: I have removed the following functions from the handler
    interface:
    analyze_partitions, check_partitions, optimize_partitions,
    repair_partitions
    Since they are not longer needed.
    THIS ALTERS THE STORAGE ENGINE API

I have verified that OPTIMIZE TABLE actually rebuilds the table
and calls ANALYZE.

Approved by:	Heikki

value field in prebuilt. This field tracks the last value in an autoincrement
interval. We use this value to check whether we need to update a table's
AUTOINC counter, if the value written to a table is less than this value
then we avoid updating the table's AUTOINC value in order to reduce
mutex contention. If it's not reset (e.g., after a DELETE statement) then
there is the possibility of missing updates to the table's AUTOINC counter
resulting in a subsequent duplicate row error message under certain 
conditions (see the test case for details).

Bug #38839 - auto increment does not work properly with InnoDB after update

instead of parsing the query string.  This fixes MySQL Bug #37885 without
us having to implement lexical analysis of SQL comments in yet another place.

thd_is_select(): A new predicate.

row_search_for_mysql(): Use thd_is_select().

Approved by Heikki.

Fix Bug#38185 ha_innobase::info can hold locks even when called with HA_STATUS_NO_LOCK

The fix is to call fsp_get_available_space_in_free_extents() from
ha_innobase::info() only if HA_STATUS_NO_LOCK is not present in the flag
*AND*
change get_schema_tables_record() in MySQL's sql/sql_show.cc to call
::info() *without* HA_STATUS_NO_LOCK whenever a user issues SELECT FROM
information_schema.tables;

Without the change to sql/sql_show.cc this patch would lead to Bug#32440
resurfacing. I.e. delete_length would never be updated in ::info() and
will remain 0 forever, resulting in the free space not being shown
anywhere.

This is the change to sql/sql_show.cc for reference, it needs to be
committed to the MySQL repo before or at the same time with this change
to ha_innodb.cc:

 --- patch begins here ---
 --- sql/sql_show.cc.orig	2008-07-23 09:32:14.000000000 +0300
 +++ sql/sql_show.cc	2008-07-23 09:32:19.000000000 +0300
 @@ -3549,8 +3549,7 @@ static int get_schema_tables_record(THD 
  
      if(file)
      {
 -      file->info(HA_STATUS_VARIABLE | HA_STATUS_TIME | HA_STATUS_AUTO |
 -                 HA_STATUS_NO_LOCK);
 +      file->info(HA_STATUS_VARIABLE | HA_STATUS_TIME | HA_STATUS_AUTO);
        enum row_type row_type = file->get_row_type();
        switch (row_type) {
        case ROW_TYPE_NOT_USED:
 --- patch ends here ---

Approved by:	Heikki

Removed UNIV_INLINE qualifier from btr_search_info_get_ref_count().
Otherwise compilation failed on non-debug builds.

Pointed by: Vasil

Fixed a timing hole where a thread dropping an index can free the
in-memory index struct while another thread is still using
that structure to remove entries from adaptive hash index belonging
to one of the pages that belongs to the index being dropped.

The fix is to have a reference counter in the index struct and to
wait for this counter to drop to zero beforing freeing the struct.

Reviewed by: Heikki

Merge a change from MySQL (this fixes the failing innodb-replace test):

  revno: 2659
  committer: Mattias Jonsson <mattiasj@mysql.com>
  branch nick: b31210-51-bugteam
  timestamp: Tue 2008-06-03 13:25:41 +0200
  message:
    Bug#31210: INSERT DELAYED crashes server when used on partitioned tables
      
    Problem was an unclear error message since it could suggest that
    MyISAM did not support INSERT DELAYED.
    Changed the error message to say that DELAYED is not supported by the
    table, instead of the table's storage engine.
    The confusion is that a partitioned table is in somewhat sense using
    the partitioning storage engine, which in turn uses the ordinary
    storage engine. By saying that the table does not support DELAYED we
    do not give any extra informantion about the storage engine or if it
    is partitioned.
  modified:
    mysql-test/r/innodb-replace.result
    mysql-test/t/innodb-replace.test
    mysql-test/t/merge.test
    mysql-test/t/partition_hash.test
    sql/share/errmsg.txt
    sql/sql_insert.cc

 
Fix Bug#36942 Performance problem in lock_get_n_rec_locks (SHOW INNODB STATUS)
by not calling lock_get_n_rec_locks() from lock_print_info_summary() on
production builds.
 
Approved by:	Heikki (via IM)

 
Fix Bug#36941 Performance problem in ha_print_info (SHOW INNODB STATUS)
by disabling some of the code in ha_print_info() in production builds.
 
Approved by:	Heikki (via IM)

was not reset to 0 when the table was truncated.

Bug #37531 : After truncate, auto_increment behaves incorrectly for InnoDB

Merge change from MySQL (this fixes the failing innodb test):

  ChangeSet@1.2628  2008-05-13 17:01:02+05:00  gshchepa@host.loc
    Merge host.loc:/work/bugs/5.0-bugteam-30059
    into  host.loc:/work/bk/5.1-bugteam

NOTE: the diff for mysql-test/r/innodb.result in this ChangeSet looks
empty but the line:

Data truncated for column 'c' at row 1

is present in annotated mysql-test/r/innodb.result@1.215 [1]
and not present in annotated mysql-test/r/innodb.result@1.216 [2] so it
must have been deleted in the above ChangeSet.

[1] http://mysql.bkbits.net:8080/mysql-5.1/mysql-test/r/innodb.result?PAGE=anno&REV=4825e033JNz_463Ha9pwInSjqLvIQg
[2] http://mysql.bkbits.net:8080/mysql-5.1/mysql-test/r/innodb.result?PAGE=anno&REV=482982fcUoskWjhU2xMH2BZ4Gyukfg

Merge change from MySQL (this almost fixes the failing innodb test):

  ChangeSet@1.2613, 2008-05-07 09:58:21+04:00, sergefp@mysql.com +5 -0
    BUG#35850 "Performance regression in 5.1.23/5.1.24"
    - Disable the "prefer full scan on clustered primary key over full scan
      of any secondary key" rule introduced by BUG#35850.
    - Update test results accordingly
    (bk trigger: file this for BUG#35850)
  
    mysql-test/r/innodb.result@1.214, 2008-05-07 09:57:31+04:00, sergefp@mysql.com +15 -15
      BUG#35850 "Performance regression in 5.1.23/5.1.24"
      - Update test results

Fix Bug#36149 Read buffer overflow in srv0start.c found during "make test"

Use strncmp(3) instead of memcmp(3) to avoid reading past end of the string
if it is empty (*str == '\0'). This bug is _not_ a buffer overflow.

Discussed with:	Sunny (via IM)

storage engine". The test for REPLACE was an error of ommission since it's
classified as a simple INSERT. For REPLACE statements we don't acquire
the special AUTOINC lock for AUTOINC_NEW_STYLE_LOCKING with this fix.

Fix Bug#36819 ut_usectime does not handle errors from gettimeofday

by retrying gettimeofday() several times if it fails in ut_usectime().
If it fails on all calls then return error to the caller to be handled
at higher level.

Update the variable innodb_row_lock_time_max in SHOW STATUS output only
if ut_usectime() was successful.

code has been tested on a big-endian machine too.

Fix Bug#11894 innodb_file_per_table crashes w/ Windows .sym symbolic
link hack

The crash was due to un-handled error 3 (path not found). In the case
of file per table, change the call to os_file_handle_error_no_exit()
from os_file_handle_error(). Also, checks for full path pattern during
table create (Windows only), which is used in symbolic link and temp
table creation.

Approved by:	Heikki

Fix Bug#36600 SHOW STATUS takes a lot of CPU in buf_get_latched_pages_number

by removing the Innodb_buffer_pool_pages_latched variable from
SHOW STATUS output in non-UNIV_DEBUG compilation.
 
Approved by:	Heikki

Fix Bug#36434 ha_innodb.so is installed in the wrong directory

Change pkglib_LTLIBRARIES with pkgplugin_LTLIBRARIES which has been
forgotten in this commit: http://lists.mysql.com/commits/40206

Non-functional white space change in Makefile.am:
Use tabs for indentation and be consistent about spaces around the
equal sign.

Merge change from MySQL:

  ChangeSet@1.2563, 2008-03-18 19:42:04+04:00, gluh@mysql.com +1 -0
    Bug#35406 5.1-opt crashes on select from I_S.REFERENTIAL_CONSTRAINTS
    added intialization of f_key_info.referenced_key_name for the case when
    referenced table is dropped

Merge changes from MySQL:

  ChangeSet@1.2645, 2007-12-19 13:24:43+00:00, jperkin@chorlton.adsl.perkin.org.uk +14 -0
    Add new pkgplugindir handling to seperate plugins from libraries,
    and allow override for binary distributions.  Extend mysql_config
    to print compiled-in plugin location for third-party plugins to
    use.  Resolves bug#31736.

and

  ChangeSet@1.2646, 2008-01-02 13:00:46+00:00, jperkin@chorlton.adsl.perkin.org.uk +11 -0
    Use pkglibdir to simplify pkgplugindir, and fix the path in a
    couple of Makefiles.  Continuation of the fix for bug#31736.

Add vim modeline to hint it that plug.in is a config file
so it can be colorized.

Non-functional change: use tabs for indentation in plug.in.

Fix r2430, it should be CFLAGS="$CFLAGS ..." not CFLAGS="..."

Use CFLAGS instead of INNODB_CFLAGS to avoid having incompatible flags
inserted in CFLAGS and INNODB_CFLAGS and to avoid compiling mysql and
innodb with different flags.

Discussed with:	Sunny

* Use INNODB_CFLAGS insead of cluttering CFLAGS with InnoDB specific
  flags. CFLAGS are used to compile every file in the MySQL source tree.

* Add INNODB_DYNAMIC_CFLAGS to the flags of the dynamic plugin and use
  -prefer-non-pic to make the dynamic plugin faster on i386.

Approved by:	Sunny

Fix Bug#36169 create innodb compressed table with too large row size crashed

Sometimes it is possible that
row_drop_table_for_mysql(index->table_name, trx, FALSE); is invoked in
row_create_index_for_mysql() when the index object is freed so copy the
table name to a safe place beforehand and use the copy.

Approved by:	Sunny

and handler_delete

Add the calls to ha_statistic_increment() in ha_innobase::delete_row()
and ha_innobase::update_row().

In InnoDB, the row count is only a rough estimate used by SQL
optimization. InnoDB is now return row count 0 for TRUNCATE operation.