into  moksha.local:/Users/davi/mysql/push/mysql-5.0-runtime

This is a performance bug, affecting in particular the bison generated code
for the parser.

Prior to this fix, the grammar used a long chain of reduces to parse an
expression, like:
  bit_expr -> bit_term
  bit_term -> bit_factor
  bit_factor -> value_expr
  value_expr -> term
  term -> factor
etc

This chain of reduces cause the internal state automaton in the generated
parser to execute more state transitions and more reduces, so that the
generated MySQLParse() function would spend a lot of time looping to execute
all the grammar reductions.

With this patch, the grammar has been reorganized so that rules are more
"flat", limiting the depth of reduces needed to parse <expr>.

Tests have been written to enforce that relative priorities and properties
of operators have not changed while changing the grammar.

See the bug report for performance data.


mysql-test/r/parser_precedence.result:
  Improved test coverage for operator precedence
mysql-test/t/parser_precedence.test:
  Improved test coverage for operator precedence
sql/sql_yacc.yy:
  Simplified the grammar to improve performances

If, after the tables are locked, one of the conditions to read from a
HANDLER table is not met, the handler code wrongly jumps to a error path
that won't unlock the tables.

The user-visible effect is that after a error in a handler read command,
all subsequent handler operations on the same table will hang.

The fix is simply to correct the code to jump to the (same) error path that
unlocks the tables.


mysql-test/r/handler.result:
  Bug#30632 test case result
mysql-test/t/handler.test:
  Bug#30632 test case
sql/sql_handler.cc:
  Always unlock the internal and external table level locks if any of the conditions
  (including errors) to read from a HANDLER table are not met.

The problem from a user's perspective: user creates table A, and then tries
to CREATE TABLE a SELECT from A - and this causes a deadlock error, a hang,
or fails with a debug assert, but only if the storage engine is InnoDB.

The origin of the problem: InnoDB uses case-insensitive collation
(system_charset_info) when looking up the internal table share, thus returning
the same share for 'a' and 'A'.

Cause of the user-visible behavior: since the same share is returned to SQL
locking subsystem, it assumes that the same table is first locked (within the
same session) for WRITE, and then for READ, and returns a deadlock error.
However, the code is wrong in not properly cleaning up upon an error, leaving
external locks in place, which leads to assertion failures and hangs.

Fix that has been implemented: the SQL layer should properly propagate the
deadlock error, cleaning up and freeing all resources.

Further work towards a more complete solution: InnoDB should not use case
insensitive collation for table share hash if table names on disk honor the case.


mysql-test/r/innodb-deadlock.result:
  Bug#25164 test case result
mysql-test/t/innodb-deadlock.test:
  Bug#25164 test case. The CREATE TABLE may fail depending on the character set
  of the system and filesystem, but it should never hang.
sql/lock.cc:
  Unlock the storage engine "external" table level locks, if the MySQL thr_lock
  locking subsystem detects a deadlock error.

since this flag was explicitly removed in pushbuild for GCOV builds.

 BUILD_CMD => ['sh', '-c', 'perl -i.bak -pe "s/ \\\\\$static_link//" ' .
              'BUILD/compile-pentium-gcov; BUILD/compile-pentium-gcov'],

Moving $static_link to SETUP.sh broke this, and is now fixed.

Should this flag be needed on some platforms,
the proper location is compile-<platform>-gcov

Tested the amd64 and pentium64 build fine without it, and can run NDB tests.


BUILD/SETUP.sh:
  Removed $static_link from GCOV builds.

into  weblab.(none):/home/marcsql/TREE/mysql-5.0-30237

into  weblab.(none):/home/marcsql/TREE/mysql-5.0-23062

into  weblab.(none):/home/marcsql/TREE/mysql-5.0-30237


sql/sql_yacc.yy:
  Auto merged

This is a performance bug, related to the parsing or 'OR' and 'AND' boolean
expressions.

Let N be the number of expressions involved in a OR (respectively AND).

When N=1

For example, "select 1" involve only 1 term: there is no OR operator.

In 4.0 and 4.1, parsing expressions not involving OR had no overhead.
In 5.0, parsing adds some overhead, with Select->expr_list.

With this patch, the overhead introduced in 5.0 has been removed,
so that performances for N=1 should be identical to the 4.0 performances,
which are optimal (there is no code executed at all)

The overhead in 5.0 was in fact affecting significantly some operations.
For example, loading 1 Million rows into a table with INSERTs,
for a table that has 100 columns, leads to parsing 100 Millions of
expressions, which means that the overhead related to Select->expr_list
is executed 100 Million times ...

Considering that N=1 is by far the most probable expression,
this case should be optimal.

When N=2

For example, "select a OR b" involves 2 terms in the OR operator.

In 4.0 and 4.1, parsing expressions involving 2 terms created 1 Item_cond_or
node, which is the expected result.
In 5.0, parsing these expression also produced 1 node, but with some extra
overhead related to Select->expr_list : creating 1 list in Select->expr_list
and another in Item_cond::list is inefficient.

With this patch, the overhead introduced in 5.0 has been removed
so that performances for N=2 should be identical to the 4.0 performances.
Note that the memory allocation uses the new (thd->mem_root) syntax
directly.
The cost of "is_cond_or" is estimated to be neglectable: the real problem
of the performance degradation comes from unneeded memory allocations.

When N>=3

For example, "select a OR b OR c ...", which involves 3 or more terms.

In 4.0 and 4.1, the parser had no significant cost overhead, but produced
an Item tree which is difficult to evaluate / optimize during runtime.
In 5.0, the parser produces a better Item tree, using the Item_cond
constructor that accepts a list of children directly, but at an extra cost
related to Select->expr_list.

With this patch, the code is implemented to take the best of the two
implementations:
- there is no overhead with Select->expr_list
- the Item tree generated is optimized and flattened.

This is achieved by adding children nodes into the Item tree directly,
with Item_cond::add(), which avoids the need for temporary lists and memory
allocation

Note that this patch also provide an extra optimization, that the previous
code in 5.0 did not provide: expressions are flattened in the Item tree,
based on what the expression already parsed is, and not based on the order
in which rules are reduced.

For example : "(a OR b) OR c", "a OR (b OR c)" would both be represented
with 2 Item_cond_or nodes before this patch, and with 1 node only with this
patch. The logic used is based on the mathematical properties of the OR
operator (it's associative), and produces a simpler tree.


sql/item_cmpfunc.h:
  Improved performances for parsing boolean expressions
sql/sql_yacc.yy:
  Improved performances for parsing boolean expressions
mysql-test/r/parser_precedence.result:
  Added test cases to cover boolean operator precedence
mysql-test/t/parser_precedence.test:
  Added test cases to cover boolean operator precedence

into  adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime


mysql-test/r/query_cache.result:
  Auto merged
mysql-test/t/query_cache.test:
  Auto merged

Although the query cache doesn't support retrieval of statements containing
column level access control, it was still possible to cache such statements
thus wasting memory.
  
This patch extends the access control check on the target tables to avoid
caching a statement with column level restrictions. 

Views are excepted and can be cached but only retrieved by super user account.


mysql-test/t/query_cache_with_views.test:
  Rename: mysql-test/t/view_query_cache.test -> mysql-test/t/query_cache_with_views.test
mysql-test/r/query_cache_with_views.result:
  Rename: mysql-test/r/view_query_cache.result -> mysql-test/r/query_cache_with_views.result
mysql-test/r/query_cache.result:
  Modified test case to allow caching of views
mysql-test/t/query_cache.test:
  Modified test case to allow caching of views
sql/sql_cache.cc:
  Allow caching of views

into  weblab.(none):/home/marcsql/TREE/mysql-5.0-rt-merge


sql/sql_base.cc:
  Auto merged
sql/sql_handler.cc:
  Auto merged

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/50

into  adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime


mysql-test/r/query_cache.result:
  Auto merged
mysql-test/t/query_cache.test:
  Auto merged

Although the query cache doesn't support retrieval of statements containing
column level access control, it was still possible to cache such statements
thus wasting memory.

This patch extends the access control check on the target tables to avoid
caching a statement with column level restrictions. 


mysql-test/r/query_cache.result:
  Added test
mysql-test/t/query_cache.test:
  Added test
sql/sql_cache.cc:
  The function check_table_access leaves the artifact
  grant.want_privileges= 1, if a statement refers to tables with column level
  privileges. To avoid the statement from being stored into the query cache,
  it is enough to check this flag and set 'safe_to_cache_query' to zero.
sql/sql_cache.h:
  - Removed 'static' attribute or class methods
  - Added THD parameter to process_and_count_tables

This is a follow up for the patch for Bug#26162 "Trigger DML ignores low_priority_updates setting", where the stored procedure ignores the session setting of low_priority_updates.

For every table open operation with default write (TL_WRITE_DEFAULT) lock_type, downgrade the lock type to the session setting of low_priority_updates.


sql/lock.cc:
  Add late lock_type assertion.
sql/sql_base.cc:
  Possibly downgrade lock type to the the session setting of low_priority_updates and also remove early assertion.

Revert the fix for bug 21587.  That bug will be re-opened, and a new
fix must be created.

This patch provides compile helper scripts only,
no server logic is affected.

Before this patch, GCOV and GPROF build scripts were only provided for
pentium platforms.

With this patch, pentium, pentium64 and amd64 platforms have associated
helper build scripts.

The GCOV and GPROF specific compilation flags are set once in SETUP.sh,
to avoid code duplication.


BUILD/SETUP.sh:
  Moved GCOV and GPROF flags to compile-pentium-{gcov,gprof) to SETUP.sh
BUILD/compile-pentium-gcov:
  Moved GCOV and GPROF flags to compile-pentium-{gcov,gprof) to SETUP.sh
BUILD/compile-pentium-gprof:
  Moved GCOV and GPROF flags to compile-pentium-{gcov,gprof) to SETUP.sh
BUILD/compile-amd64-gcov:
  Added helper scripts for GCOV and GPROF builds.
BUILD/compile-amd64-gprof:
  Added helper scripts for GCOV and GPROF builds.
BUILD/compile-pentium64-gcov:
  Added helper scripts for GCOV and GPROF builds.
BUILD/compile-pentium64-gprof:
  Added helper scripts for GCOV and GPROF builds.

Apply innodb-5.0-ss1696 snapshot

Fixes:
- Bug#20090: InnoDB: Error: trying to declare trx to enter InnoDB
- Bug#23710: crash_commit_before fails if innodb_file_per_table=1
  At InnoDB startup consider the case where log scan went beyond
  checkpoint_lsn as a crash and initiate crash recovery code path.
- Bug#28781: InnoDB increments auto-increment value incorrectly with ON DUPLICATE KEY UPDATE
  We need to do some special AUTOINC handling for the following case:
  INSERT INTO t (c1,c2) VALUES(x,y) ON DUPLICATE KEY UPDATE ...
  We need to use the AUTOINC counter that was actually used by
  MySQL in the UPDATE statement, which can be different from the
  value used in the INSERT statement.
- Bug#29097: fsp_get_available_space_in_free_extents() is capped at 4TB
  Fix by typecasting the variables before multiplying them, so that the
  result of the multiplication is of type "unsigned long long".
- Bug#29155: Innodb "Parallel recovery" is not prevented
  Fix by enabling file locking on FreeBSD.  It has been disabled because
  InnoDB has refused to start on FreeBSD & LinuxThreads, but now it
  starts just fine.


innobase/fsp/fsp0fsp.c:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1614:
  branches/5.0:
  
  Merge r1605 from trunk:
  
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
innobase/include/fsp0fsp.h:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1614:
  branches/5.0:
  
  Merge r1605 from trunk:
  
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
innobase/include/univ.i:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1614:
  branches/5.0:
  
  Merge r1605 from trunk:
  
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
innobase/log/log0recv.c:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1608:
  branches/5.0: Bug#23710
  Back port of r1607 from trunk
  
  At InnoDB startup consider the case where log scan went beyond checkpoint_lsn as a crash and initiate crash recovery code path.
  
  reviewed by: Heikki
innobase/os/os0file.c:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1615:
  branches/5.0:
  
  Merge r1613 from trunk:
  
  Fix Bug#29155 by enabling file locking on FreeBSD.
  It has been disabled because InnoDB has refused to start on
  FreeBSD & LinuxThreads, but now it starts just fine.
  
  Approved by:	Heikki
innobase/srv/srv0srv.c:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1552:
  branches/5.0:
  Fix Bug#20090 as suggested in the bug followup by Heikki.
  
  Approved by:	Heikki
innobase/trx/trx0trx.c:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1596:
  branches/5.0: Merge r1595 from trunk:
  
  trx_commit_for_mysql(): Avoid acquiring and releasing kernel_mutex when
  trx->sess or trx_dummy_sess is non-NULL.
sql/ha_innodb.cc:
  Apply innodb-5.0-ss1696 snapshot
  
  Revision r1614:
  branches/5.0:
  
  Merge r1605 from trunk:
  
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
  
  
  Revision r1695:
  branches/5.0: Merge a change from MySQL AB:
  
  ChangeSet@1.2463.166.1  2007-06-20 19:22:27+03:00  monty@mysql.fi
  
  Allow multiple calls to mysql_server_end()
  (Part of fix for Bug 25621 Error in my_thread_global_end(): 1 threads
  didn't exit)
  Give correct error message if InnoDB table is not found
  (This allows us to drop a an innodb table that is not in the InnoDB registery)
  
  ha_innodb.cc:
    Give correct error message if InnoDB table is not found.
    (This allows us to drop a an innodb table that is not in the InnoDB
    registery)
  
  
  Revision r1606:
  branches/5.0: Formatting corrections.
  
  spotted by: Vasil
  
  
  Revision r1691:
  branches/5.0: Merge a change from MySQL AB:
  
  ChangeSet@1.2463.261.1  2007-07-20 14:17:15+03:00  gkodinov@magare.gmz
    Bug 29644: alter table hangs if records locked in share mode 
    by long running transaction
    
    On Windows opened files can't be deleted. There was a special
    upgraded lock mode (TL_WRITE instead of TL_WRITE_ALLOW_READ) 
    in ALTER TABLE to make sure nobody has the table opened
    when deleting the old table in ALTER TABLE. This special mode
    was causing ALTER TABLE to hang waiting on a lock inside InnoDB.
    This special lock is no longer necessary as the server is 
    closing the tables it needs to delete in ALTER TABLE.
    Fixed by removing the special lock.
    Note that this also reverses the fix for bug 17264 that deals with
    another consequence of this special lock mode being used.
  
  sql/ha_innodb.cc: Bug 29644: reverse the (now excessive) fix
  for bug 17264 (but leave the test case).
  
  
  Revision r1601:
  branches/5.0: Fix for bug#28781
  We need to do some special AUTOINC handling for the following case:
  
  INSERT INTO t (c1,c2) VALUES(x,y) ON DUPLICATE KEY UPDATE ...
  
  We need to use the AUTOINC counter that was actually used by
  MySQL in the UPDATE statement, which can be different from the
  value used in the INSERT statement.
  
  approved by: Sunny
  
  
  Revision r1692:
  branches/5.0: Merge a change from MySQL AB:
  
  ChangeSet@1.2463.267.1  2007-07-30 17:14:34+04:00  evgen@local
    Bug 24989: The DEADLOCK error is improperly handled by InnoDB.
    
    When innodb detects a deadlock it calls ha_rollback_trans() to rollback the 
    main transaction. But such action isn't allowed from inside of triggers and
    functions. When it happen the 'Explicit or implicit commit' error is thrown
    even if there is no commit/rollback statements in the trigger/function. This
    leads to the user confusion.
    
    Now the convert_error_code_to_mysql() function doesn't call the 
    ha_rollback_trans() function directly but rather calls the
    mark_transaction_to_rollback function and returns an error.
    The sp_rcontext::find_handler() now doesn't allow errors to be caught by the
    trigger/function error handlers when the thd->is_fatal_sub_stmt_error flag
    is set. Procedures are still allowed to catch such errors.
    The sp_rcontext::find_handler function now accepts a THD handle as a parameter.
    The transaction_rollback_request and the is_fatal_sub_stmt_error flags are 
    added to the THD class. The are initialized by the THD class constructor.
    Now the ha_autocommit_or_rollback function rolls back main transaction
    when not in a sub statement and the thd->transaction_rollback_request
    is set.
    The THD::restore_sub_statement_state function now resets the 
    thd->is_fatal_sub_stmt_error flag on exit from a sub-statement.
  
  innodb-big.test, innodb-big.result:
    Added a test case for the bug 24989: The DEADLOCK error is improperly
    handled by InnoDB.
  
  sql/ha_innodb.cc:
    Bug 24989: The DEADLOCK error is improperly handled by InnoDB.
    Now the convert_error_code_to_mysql() function doesn't call the
    ha_rollback_trans() function directly but rather calls the
    mark_transaction_to_rollback function and returns an error.
  
  
  Revision r1693:
  branches/5.0: Merge a change from MySQL AB:
  
  ChangeSet@1.2489.5.1  2007-07-31 17:42:48+04:00  evgen@local
    ha_innodb.cc:
      Warning fixed.

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/50

sql/sql_handler.cc:
  Add missing parameter to the @param tag and fix style.

into  mysql.com:/nfsdisk1/lars/MERGE/mysql-5.0-merge

into  mysql.com:/nfsdisk1/lars/MERGE/mysql-5.0-merge

into  mysql.com:/nfsdisk1/lars/MERGE/mysql-4.1-merge

mysql_ha_open calls mysql_ha_close on the error path (unsupported) to close the (opened) table before inserting it into the tables hash list handler_tables_hash) but mysql_ha_close only closes tables which are on the hash list, causing the table to be left open and locked.

This change moves the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close (mysql_ha_close).


mysql-test/r/handler.result:
  Bug#25856 test result
mysql-test/t/handler.test:
  Bug#25856 test case
sql/sql_handler.cc:
  Move the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close

into  gleb.loc:/home/uchum/work/bk/5.0-opt

into  synthia.local:/home/mydev/mysql-5.0-axmrg

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/50

into  synthia.local:/home/mydev/mysql-5.0-axmrg

into  synthia.local:/home/mydev/mysql-4.1-axmrg

into  gleb.loc:/home/uchum/work/bk/5.0-opt

into  bodhi.(none):/opt/local/work/mysql-5.0-runtime


mysql-test/r/federated.result:
  Auto merged
mysql-test/t/federated.test:
  Auto merged
sql/item.cc:
  Auto merged

under terms of bug#28875 for better performance.
The change appeared to require more changes in item_cmpfunc.cc,
which is dangerous in 5.0.
    
Conversion between a latin1 column and an ascii string constant
stopped to work.


mysql-test/r/ctype_recoding.result:
  Adding test case.
mysql-test/t/ctype_recoding.test:
  Adding test case.

into  zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-5.0-maint


strings/ctype-extra.c:
  Auto merged

into  zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-5.0-maint

Two character mappings were way off (backtick and tilde were "E"
and "Y"!), and three others were slightly rotated.  The first 
would cause collisions, and the latter was probably benign.

Now, assign the character mappings exactly to their normal values.


sql/share/charsets/ascii.xml:
  Change the character mapping for 
   "`" to "`" (was "E") and 
   "[" to "[" (was "\") and 
   "\" to "\" (was "]") and
   "]" to "]" (was "[") and 
   "~" to "~" (was "Y").
strings/ctype-extra.c:
  Generated from charsets directory.
mysql-test/r/ctype_ascii.result:
  Add new test file.
  
  Test all combinations of printable letter comparisons for 
  similarity.
mysql-test/t/ctype_ascii.test:
  Add new test file.
  
  Test all combinations of printable letter comparisons for 
  similarity.

into  magare.gmz:/home/kgeorge/mysql/autopush/B29536-5.0-opt

MySQL replicates the time zone only when operations that involve
it are performed. This is controlled by a flag. But this flag
is set only on successful operation.
The flag must be set also when there is an error that involves
a timezone (so the master would replicate the error to the slaves). 

Fixed by moving the setting of the flag before the operation
(so it apples to errors as well).


mysql-test/r/rpl_timezone.result:
  Bug #29536: test case
mysql-test/t/rpl_timezone.test:
  Bug #29536: test case
sql/field.cc:
  Bug #29536: move setting of the flag before the operation
  (so it apples to errors as well).
sql/time.cc:
  Bug #29536: move setting of the flag before the operation
  (so it apples to errors as well).

into  bodhi.(none):/opt/local/work/mysql-5.0-runtime

VIEW".

mysql_list_fields() C API function would incorrectly set MYSQL_FIELD::decimals
member for some view columns.

The problem was in an incomplete implementation of 
Item_ident_for_show::make_field(), which is responsible for view
columns metadata.


sql/item.cc:
  A fix for Bug#29306 -- properly initialize decimals in 
  Item_ident_for_show::make_field
tests/mysql_client_test.c:
  Add a test case forBug#29306. Fix warnings.