Some ut_a(!rec_offs_any_null_extern()) assertion failures are indicating
genuine BLOB bugs, others are bogus failures when rolling back incomplete
transactions at crash recovery. This needs more work, and until I get a
chance to work on it, other testing must not be disrupted by this.

revno 2995.37.209
revision id marko.makela@oracle.com-20110518120508-qhn7vz814vn77v5k
parent marko.makela@oracle.com-20110517121555-lmple24qzxqkzep4
timestamp: Wed 2011-05-18 15:05:08 +0300
message:
  Fix a bogus UNIV_SYNC_DEBUG failure in the fix of Bug #59641
  or Oracle Bug #11766513.

  trx_undo_free_prepared(): Do not acquire or release trx->rseg->mutex.
  This code is invoked in the single-threaded part of shutdown, therefore
  a mutex is not needed.

SYNTAX TRIGGERS IN ANY WAY

Table with triggers which were using deprecated (5.0-only) syntax became
unavailable for any DML and DDL after upgrade to 5.1 version of server.
Attempt to execute any statement on such a table resulted in parsing
error reported. Since this included DROP TRIGGER and DROP TABLE
statements (actually, the latter was allowed but was not functioning
properly for such tables) it was impossible to fix the problem without
manual operations on .TRG and .TRN files in data directory.

The problem was that failure to parse trigger body (due to 5.0-only
syntax) when opening trigger file for a table prevented the table
from being open. This made all operations on the table impossible
(except DROP TABLE which due to peculiarity in its implementation
dropped the table but left trigger files around).

This patch solves this problem by silencing error which occurs when
we parse trigger body during table open. Error message is preserved
for the future use and table is marked as having a broken trigger.
We also try to analyze parse tree to recover trigger name, which
will be needed in order to drop the broken trigger. DML statements
which invoke triggers on the table marked as having broken trigger
are prohibited and emit saved error message. The same happens for
DDL which change triggers except DROP TRIGGER and DROP TABLE which
try their best to do what was requested. Table becomes no longer
marked as having broken trigger when last such trigger is dropped.

row_build(): The record may contain null BLOB pointers when the server
is rolling back an insert that was interrupted by a server crash.

THE EVENT STATUS.

Any ALTER EVENT statement on a disabled event enabled it back
(unless this ALTER EVENT statement explicitly disabled the event).

The problem was that during processing of an ALTER EVENT statement
value of status field was overwritten unconditionally even if new
value was not specified explicitly. As a consequence this field
was set to default value for status which corresponds to ENABLE.

The solution is to check if status field was explicitly specified in
ALTER EVENT statement before assigning new value to status field.

If UNIV_DEBUG or UNIV_BLOB_LIGHT_DEBUG is enabled, add
!rec_offs_any_null_extern() assertions, ensuring that records do not
contain null pointers to externally stored columns in inappropriate
places.

btr_cur_optimistic_update(): Assert !rec_offs_any_null_extern().
Incomplete records must never be updated or deleted. This assertion
will cover also the pessimistic route.

row_build(): Assert !rec_offs_any_null_extern(). Search tuples must
never be built from incomplete index entries.

row_rec_to_index_entry(): Assert !rec_offs_any_null_extern() unless
ROW_COPY_DATA is requested. ROW_COPY_DATA is used for
multi-versioning, and therefore it might be valid to copy the most
recent (uncommitted) version while it contains a null pointer to
off-page columns.

row_vers_build_for_consistent_read(),
row_vers_build_for_semi_consistent_read(): Assert !rec_offs_any_null_extern()
on all versions except the most recent one.

trx_undo_prev_version_build(): Assert !rec_offs_any_null_extern() on
the previous version.

rb:682 approved by Sunny Bains

Problem: in case of wrong data insert into indexed GEOMETRY fields 
(e.g. NULL value for a not NULL field) MyISAM reported 
"ERROR 126 (HY000): Incorrect key file for table; try to repair it"
due to misuse of the key deletion function.

Fix: always use R-tree key functions for R-tree based indexes
and B-tree key functions for B-tree based indexes.

  FAIL IN EMBEDDED SERVER

FreeBSD 64 bit needs the FP_X_DNML to fpsetmask() to prevent exceptions from
propagating into mysql (as a threaded application).
However fpsetmask() itself is deprecated in favor of fedisableexcept().
1. Fixed the #ifdef to check for FP_X_DNML instead of i386.
2. Added a configure.in check for fedisableexcept() and, if present,
   this function is called insted of the fpsetmask().
No need for new tests, as the existing tests cover this already.
Removed the affected tests from the experimental list.

The problem is that clients implementing the 4.0 version of the
protocol (that is, mysql-4.0) do not null terminate a string
at the end of the authentication packet. These clients denote
the end of the string with the end of the packet.

Although this goes against the documented (see MySQL Internals
ClientServer Protocol wiki) description of the protocol, these
old clients still need to be supported.

The solution is to support the documented and actual behavior
of the clients. If a client is using the pre-4.1 version of
the protocol, the end of a string in the authentication packet
can either be denoted with a null character or by the end of
the packet. This restores backwards compatibility with old
clients implementing either the documented or actual behavior.

The problem is that although AIX implements bzero, its prototype
is not declared by default. Since AC_CHECK_FUNC(bzero) succeeds
even though a prototype is not declared, this breaks compilation
in C++ files where a prototype is required.

The solution is to only use bzero if a prototype is also declared.

will create multiple running events.

A CREATE IF NOT EXIST on an event that existed and was enabled caused
multiple instances of the event to run. Disabling the event didn't  help.
If the event was  dropped, the event stopped running, but when created
again, multiple instances of the event were still running. The only way
to get out of this situation was  to restart the server.

The problem was that Event_db_repository::create_event() didn't return
enough information to discriminate between situation when event didn't
exist and was created and when event did exist and was not created
(but a warning was emitted). As result in the latter case event
was added to in-memory queue of events second time. And this led to
unwarranted multiple executions of the same event.

The solution is to add out-parameter to Event_db_repository::create_event()
method which will signal that event was not created because it already
exists and so it should not be added to the in-memory queue.

HA_INNOBASE::UPDATE_ROW, TEMPORARY TABLE, TABLE LOCK".

Attempt to update an InnoDB temporary table under LOCK TABLES
led to assertion failure in both debug and production builds
if this temporary table was explicitly locked for READ. The 
same scenario works fine for MyISAM temporary tables.

The assertion failure was caused by discrepancy between lock 
that was requested on the rows of temporary table at LOCK TABLES
time and by update operation. Since SQL-layer requested a 
read-lock at LOCK TABLES time InnoDB engine assumed that upcoming
statements which are going to be executed under LOCK TABLES will 
only read table and therefore should acquire only S-lock.
An update operation broken this assumption by requesting X-lock.

Possible approaches to fixing this problem are:

1) Skip locking of temporary tables as locking doesn't make any
   sense for connection-local objects.
2) Prohibit changing of temporary table locked by LOCK TABLES ... 
   READ.

Unfortunately both of these approaches have drawbacks which make 
them unviable for stable versions of server.

So this patch takes another approach and changes code in such way
that LOCK TABLES for a temporary table will always request write
lock. In 5.1 version of this patch switch from read lock to write
lock is done inside of InnoDBs handler methods as doing it on 
SQL-layer causes compatibility troubles with FLUSH TABLES WITH
READ LOCK.

Problem: MYSQL_BIN_LOG::reset_logs acquires mutexes in wrong order.
The correct order is first LOCK_thread_count and then LOCK_log. This function
does it the other way around. This leads to deadlock when run in parallel
with a thread that takes the two locks in correct order. For example, a thread
that disconnects will take the locks in the correct order.
Fix: change order of the locks in MYSQL_BIN_LOG::reset_logs:
first LOCK_thread_count and then LOCK_log.

Assertion happens due to missing NULL value check in
Item_func_round::fix_length_and_dec() function.
The fix: added NULL value check for second parameter.

when selecting from I_S and views exist, in SP.

Symptoms: re-execution of prepared statement (or statement in a stored
routine) which read from one of I_S tables and which in order to fill
this I_S table had to open a view led to increasing memory consumption.

What happened in this situation was that during the process of view
opening for purpose of I_S filling view-related structures (like its
LEX) were allocated on persistent MEM_ROOT of prepared statement (or 
stored routine). Since this MEM_ROOT is not freed until prepared
statement deallocation (or expulsion of stored routine from the cache)
and code responsible for filling I_S is not able to re-use results of
view opening from previous executions this allocation ended up in
memory hogging.

This patch solves the problem by ensuring that when a view opened
for the purpose of I_S filling all its structures are allocated on
non-persistent runtime MEM_ROOT. This is achieved by activating a
temporary Query_arena bound to this MEM_ROOT.
Since this step makes impossible linking of view structures into
LEX of our prepared statement (or stored routine statement) this
patch also changes code filling I_S table to install a proxy LEX
before trying to open a view or a table. Consequently some code
which was responsible for backing-up/restoring parts of LEX when
view/table was opened during filling of I_S table became redundant
and was removed.

This patch doesn't contain test case for this bug as it is hard
to test memory hogging in our test suite.

Automerge mysql-5.1-gca into latest mysql-5.1.

              VM-WIN2003-32-A, SLES10-IA64-A 
      
The test case waits for master_pos_wait not to timeout, which
means that the deadlock between SQL and IO threads was 
succesfully and automatically dealt with.
      
However, very rarely, master_pos_wait reports a timeout. This
happens because the time set for master_pos_wait to wait was
too small (6 seconds). On slow test env this could be a 
problem.
      
We fix this by setting the timeout inline with the one used
in sync_slave_with_master (300 seconds). In addition we 
refactored the test case and refined some comments.

Issue:
While running embedded server, if client issues TEE command (\T foo/bar) and
"foo/bar" directory doesn't exist, it is suppose to give error. But it was
aborting.  This was happening because wrong error handler was being called.

Solution:
Modified calls to correct error handler. In embedded server case, there are 
two error handler (client and server) which are supposed to be called based 
on which context code is in. If it is in client context, client error handler
should be called otherwise server.

Test case:
Test case automation is not possible as current (following) code doesn't 
allow '\T' to be executed from command line (OR command read from a file):
[client/mysql.cc]
...
 static int
 com_tee(String *buffer __attribute__((unused)),
         char *line __attribute__((unused)))
 {
   char file_name[FN_REFLEN], *end, *param;

   if (status.batch) << THIS IS TRUE WHILE EXECUTING FROM COMMAND LINE.
      return 0;
 ...
So, not adding test case in GA. WIll add a test case in mysql-trunk after 
removing above code so that this could be properly tested before GA.

               LEAK WITH PARTITIONED ARCHIVE TABLES

CHECK TABLE against archive table, when file descriptors
are exhausted, caused server crash.

Archive didn't handle errors when opening data file for
CHECK TABLE.

There are two problems:
1. There is a missing check for 'year' parameter(year can not be greater than 9999) in
   makedate function. fix: added check that year can not be greater than 9999.
2. There is a missing check for zero date in from_days() function.
   fix: added zero date check into Item_func_from_days::get_date()
   function.

In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is
"Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld".
So 'ha_rows' was used as 'long'.
On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes.
So the printf-like code was reading only the first 4 bytes.
Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
so the first four bytes yield 0. So the warning message had "row 0" instead of
"row 1" in test outfile_loaddata.test:
-Warning	1366	Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+Warning	1366	Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0

All error-messaging functions which internally invoke some printf-life function
are potential candidate for such mistakes.
One apparently easy way to catch such mistakes is to use
ATTRIBUTE_FORMAT (from my_attribute.h).
But this works only when call site has both:
a) the format as a string literal
b) the types of arguments.
So:
  func(ER(ER_BLAH), 10);
will silently not be checked, because ER(ER_BLAH) is not known at
compile time (it is known at run-time, and depends on the chosen
language).
And
  func("%s", a va_list argument);
has the same problem, as the *real* type of arguments is not
known at this site at compile time (it's known in some caller).
Moreover,
  func(ER(ER_BLAH));
though possibly correct (if ER(ER_BLAH) has no '%' markers), will not
compile (gcc says "error: format not a string literal and no format
arguments").

Consequences:
1) ATTRIBUTE_FORMAT is here added only to functions which in practice
take "string literal" formats: "my_error_reporter" and "print_admin_msg".
2) it cannot be added to the other functions: my_error(),
push_warning_printf(), Table_check_intact::report_error(),
general_log_print().

To do a one-time check of functions listed in (2), the following
"static code analysis" has been done:
1) replace
  my_error(ER_xxx, arguments for substitution in format)
with the equivalent
  my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in
format),
so that we have ER(ER_xxx) and the arguments *in the same call site*
2) add ATTRIBUTE_FORMAT to push_warning_printf(),
Table_check_intact::report_error(), general_log_print()
3) replace ER(xxx) with the hard-coded English text found in
errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with
"Unknown error"), so that a call site has the format as string literal
4) this way, ATTRIBUTE_FORMAT can effectively do its job
5) compile, fix errors detected by ATTRIBUTE_FORMAT
6) revert steps 1-2-3.
The present patch has no compiler error when submitted again to the
static code analysis above.
It cannot catch all problems though: see Field::set_warning(), in
which a call to push_warning_printf() has a variable error
(thus, not replacable by a string literal); I checked set_warning() calls
by hand though.

See also WL 5883 for one proposal to avoid such bugs from appearing
again in the future.

The issues fixed in the patch are:
a) mismatch in types (like 'int' passed to '%ld')
b) more arguments passed than specified in the format.
This patch resolves mismatches by changing the type/number of arguments,
not by changing error messages of sql/share/errmsg.txt. The latter would be wrong,
per the following old rule: errmsg.txt must be as stable as possible; no insertions
or deletions of messages, no changes of type or number of printf-like format specifiers,
are allowed, as long as the change impacts a message already released in a GA version.
If this rule is not followed:
- Connectors, which use error message numbers, will be confused (by insertions/deletions
of messages)
- using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1)
could produce wrong messages or crash; such usage can easily happen if
installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx;
or if copying mysqld from 5.1.(n+1) into a 5.1.n installation.
When fixing b), I have verified that the superfluous arguments were not used in the format
in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm').
Had they been used, then passing them today, even if the message doesn't use them
anymore, would have been necessary, as explained above.

      
Impementing Test Review Comment.
      
Bug test scenario:      
SELECT is not returning result set for "equal" (=) and "NULL safe equal
operator"  (<=>) on BIT data type. Extending this scenario for all data types