losing it's .MYD

When OPTIMIZE TABLE is completed it attempts to rename temporary
file to original name. This step may fail on windows when a file
is opened. As a result data file might be deleted and optimized
copy of file (table_name.MYD) remains.

This situation is handled properly by my_delete_allow_opened, so
use it instead of my_delete when attempting to rename a file on
windows.

No suitable test case for this bug.


mysys/my_redel.c:
  Attempting to delete an opened file and to immediately create
  a new one with the same name may result in my_redel failure on
  windows. It may fail because file is not deleted until it is
  closed.
  
  This situation is handled properly by my_delete_allow_opened, so
  use it instead of my_delete.

into  chilla.local:/home/mydev/mysql-5.0-bug24985


sql/ha_myisam.cc:
  Auto merged
mysql-test/r/heap_btree.result:
  Bug#24985 - UTF8 ENUM primary key on MEMORY using BTREE
              causes incorrect duplicate entries
  Manual merge
mysql-test/t/heap_btree.test:
  Bug#24985 - UTF8 ENUM primary key on MEMORY using BTREE
              causes incorrect duplicate entries
  Manual merge

            causes incorrect duplicate entries
After merge fix

into  chilla.local:/home/mydev/mysql-5.0-bug24985


mysql-test/r/heap_btree.result:
  Auto merged
sql/ha_heap.cc:
  Auto merged
mysql-test/t/heap_btree.test:
  Bug#24985 - UTF8 ENUM primary key on MEMORY using BTREE
              causes incorrect duplicate entries
  Manual merge from 4.1

            causes incorrect duplicate entries

Keys for BTREE indexes on ENUM and SET columns of MEMORY tables
with character set UTF8 were computed incorrectly. Many
different column values got the same key value.

Apart of possible performance problems, it made unique indexes
of this type unusable because it rejected many different
values as duplicates.

The problem was that multibyte character detection was tried
on the internal numeric column value. Many values were not
identified as characters. Their key value became blank filled.

Thanks to Alexander Barkov and Ramil Kalimullin for the patch,
which sets the character set of ENUM and SET key segments to
the pseudo binary character set.


mysql-test/r/heap_btree.result:
  Bug#24985 - UTF8 ENUM primary key on MEMORY using BTREE
              causes incorrect duplicate entries
  Added test result.
mysql-test/t/heap_btree.test:
  Bug#24985 - UTF8 ENUM primary key on MEMORY using BTREE
              causes incorrect duplicate entries
  Added test.
sql/ha_heap.cc:
  Bug#24985 - UTF8 ENUM primary key on MEMORY using BTREE
              causes incorrect duplicate entries
  Set key segment charset to my_charset_bin for ENUM and SET
  columns.

into  chilla.local:/home/mydev/mysql-5.0--team


sql/ha_myisam.cc:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.0-bug26996


heap/hp_write.c:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.0-bug26996


heap/hp_write.c:
  Auto merged
mysql-test/r/heap_btree.result:
  Auto merged
mysql-test/t/heap_btree.test:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Manual merge from 4.1

Using a MEMORY table BTREE index for scanning for updatable rows
could lead to an infinite loop.

Everytime a key was inserted into a btree index, the position
in the index scan was cleared. The search started from the
beginning and found the same key again.

Now we do not clear the position on key insert an more.


heap/hp_write.c:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Removed the index-scan-breaking nulling of last_pos.
  The comment behind this line ("For heap_rnext/heap_rprev")
  was misleading. It should have been "Breaks heap_rnext/heap_rprev".
mysql-test/r/heap_btree.result:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Added test result.
mysql-test/t/heap_btree.test:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Added test.

into  blade08.mysql.com:/data0/istruewing/autopush/mysql-5.0-bug25289


sql/ha_myisam.cc:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.0-bug25289


sql/ha_myisam.cc:
  Bug#25289 - repair table causes "my_seek.c:56:
              my_seek: Assertion `fd != -1' failed"
  Manual merge from 4.1

into  mockturtle.local:/home/dlenev/src/mysql-5.0-merge

into  mockturtle.local:/home/dlenev/src/mysql-5.0-bg25966-2


sql/mysqld.cc:
  Auto merged

TABLE ... WRITE".

Memory and CPU hogging occured when connection which had to wait for table
lock was serviced by thread which previously serviced connection that was
killed (note that connections can reuse threads if thread cache is enabled).
One possible scenario which exposed this problem was when thread which
provided binlog dump to replication slave was implicitly/automatically
killed when the same slave reconnected and started pulling data through
different thread/connection.
The problem also occured when one killed particular query in connection
(using KILL QUERY) and later this connection had to wait for some table
lock.

This problem was caused by the fact that thread-specific mysys_var::abort
variable, which indicates that waiting operations on mysys layer should
be aborted (this includes waiting for table locks), was set by kill
operation but was never reset back. So this value was "inherited" by the
following statements or even other connections (which reused the same
physical thread). Such discrepancy between this variable and THD::killed
flag broke logic on SQL-layer and caused CPU and memory hogging.

This patch tries to fix this problem by properly resetting this member.

There is no test-case associated with this patch since it is hard to test
for memory/CPU hogging conditions in our test-suite.


sql/mysqld.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of other connections.
sql/sp_head.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of further statements.
sql/sql_parse.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of further statements.

TABLE ... WRITE".

CPU hogging occured when connection which had to wait for table lock was
serviced by thread which previously serviced connection that was killed
(note that connections can reuse threads if thread cache is enabled).
One possible scenario which exposed this problem was when thread which
provided binlog dump to replication slave was implicitly/automatically
killed when the same slave reconnected and started pulling data through
different thread/connection.
In 5.* versions memory hogging was added to CPU hogging. Moreover in
those versions the problem also occured when one killed particular query
in connection (using KILL QUERY) and later this connection had to wait for
some table lock.

This problem was caused by the fact that thread-specific mysys_var::abort
variable, which indicates that waiting operations on mysys layer should
be aborted (this includes waiting for table locks), was set by kill
operation but was never reset back. So this value was "inherited" by the
following statements or even other connections (which reused the same
physical thread). Such discrepancy between this variable and THD::killed
flag broke logic on SQL-layer and caused CPU and memory hogging.

This patch tries to fix this problem by properly resetting this member.

There is no test-case associated with this patch since it is hard to test
for memory/CPU hogging conditions in our test-suite.


sql/mysqld.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of other connections.

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build

  Updated to version 0.6 of the text


EXCEPTIONS-CLIENT:
  Updated to version 0.6 of the text

            my_seek: Assertion `fd != -1' failed"

In difficult optimize/repair situations the server could crash.
Under some circumstances the server retries an optimize/repair
with more elaborate options. But it did not check if the first
attempt failed so badly that a second one must not be tried.

This could happen when a new data file has been created
but it was not possible to open it. In this case the
repair leaves behind a table with closed data file.
This must not be used for another repair attempt.

We do now detect the closed data file and do not try
another repair attempt in this situation.

No test case. The required table corruption can not be
repeated easily. There is a test program attached to
bug 25433.


sql/ha_myisam.cc:
  Bug#25289 - repair table causes "my_seek.c:56:
              my_seek: Assertion `fd != -1' failed"
  Added code to detect a closed data file. It could be closed
  by a preceeding repair attempt. We must not try another
  repair then.

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build


configure.in:
  Auto merged

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build


configure.in:
  SCCS merged

  Added test for sched_yield() possibly in -lposix4 on Solaris


configure.in:
  Added test for sched_yield() possibly in -lposix4 on Solaris

into  mysql.com:/home/svoj/devel/mysql/BUG26881/mysql-5.0-engines


myisam/mi_create.c:
  Auto merged
mysql-test/t/merge.test:
  Auto merged
sql/ha_myisam.cc:
  Auto merged
sql/sql_parse.cc:
  Use local.
mysql-test/r/merge.result:
  SCCS merged

            differences in tables
Certain merge tables were wrongly reported as having incorrect definition:
- Some fields that are 1 byte long (e.g. TINYINT, CHAR(1)), might
  be internally casted (in certain cases) to a different type on a
  storage engine layer. (affects 4.1 and up)
- If tables in a merge (and a MERGE table itself) had short VARCHAR column (less
  than 4 bytes) and at least one (but not all) tables were ALTER'ed (even to an
  identical table: ALTER TABLE xxx ENGINE=yyy), table definitions went ouf of
  sync. (affects 4.1 only)

This is fixed by relaxing a check for underlying conformance and setting
field type to FIELD_TYPE_STRING in case varchar is shorter than 4
when a table is created.


myisam/mi_create.c:
  Added a comment.
mysql-test/r/merge.result:
  A test case for bug#26881.
mysql-test/t/merge.test:
  A test case for bug#26881.
sql/ha_myisam.cc:
  Relaxed some checks performed by check_definition():
  As comparing of fulltext keys (and key segments) is not yet implemented,
  only return an error in case one of keys is fulltext and other is not.
  Otherwise, if both keys are fulltext, accept them as is.
  
  As comparing of spatial keys (and key segments) is not yet implemented,
  only return an error in case one of keys is spatial and other is not.
  Otherwise, if both keys are spatial, accept them as is.
  
  A workaround to handle situation when field is casted from FIELD_SKIP_ZERO
  to FIELD_NORMAL. This could happen only in case field length is 1 and row
  format is fixed.
sql/sql_parse.cc:
  When a table that has varchar field shorter than 4 is created, field type is
  set to FIELD_TYPE_VAR_STRING. Later, when a table is modified using alter
  table, field type is changed to FIELD_TYPE_STRING (see Field_string::type).
  That means HA_OPTION_PACK_RECORD flag might be lost and thus null_bit might
  be shifted by alter table, in other words alter table doesn't create 100%
  equal table definition.
  
  This is usually not a problem, since when a table is created/altered,
  definition on a storage engine layer is based on one that is passed from
  sql layer. But it is a problem for merge engine - null_bit is shifted when
  a table (merge or underlying) is altered.
  
  Set field type to FIELD_TYPE_STRING in case FIELD_TYPE_VAR_STRING is shorter
  than 4 when a table is created as it is done in Field::type.

into  mysql.com:/home/svoj/devel/mysql/BUG26881/mysql-5.0-engines

into  mysql.com:/home/svoj/devel/mysql/BUG26881/mysql-5.0-engines


myisam/rt_index.c:
  Auto merged
sql/field.h:
  Auto merged

into  mysql.com:/home/svoj/devel/mysql/BUG26881/mysql-4.1-engines

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build


configure.in:
  SCCS merged

  Restored accidently removed line to check for zlib


configure.in:
  Restored accidently removed line to check for zlib

into  trift2.:/MySQL/M50/push-5.0

  Removed references to my_winsem.c


mysys/CMakeLists.txt:
  Removed references to my_winsem.c
mysys/Makefile.am:
  Removed references to my_winsem.c

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build


VC++Files/mysys/mysys.vcproj:
  Auto merged
VC++Files/mysys/mysys_ia64.dsp:
  Auto merged
mysys/Makefile.am:
  Auto merged
BitKeeper/deleted/.del-my_semaphore.c:
  Auto merged
VC++Files/mysys/mysys.dsp:
  SCCS merged

  Removed references to unused files


VC++Files/mysys/mysys.vcproj:
  Removed unused files
VC++Files/mysys/mysys_ia64.dsp:
  Removed unused files

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build


include/Makefile.am:
  Auto merged

  Removed unused files
.del-my_winsem.c:
  Delete: mysys/my_winsem.c
.del-my_semaphore.c:
  Delete: mysys/my_semaphore.c
.del-my_semaphore.h:
  Delete: include/my_semaphore.h


BitKeeper/deleted/.del-my_semaphore.c:
  Delete: mysys/my_semaphore.c
BitKeeper/deleted/.del-my_semaphore.h:
  Delete: include/my_semaphore.h
BitKeeper/deleted/.del-my_winsem.c:
  Delete: mysys/my_winsem.c
VC++Files/mysys/mysys.dsp:
  Removed unused files
configure.in:
  Removed unused files
include/Makefile.am:
  Removed unused files
mysys/Makefile.am:
  Removed unused files

into  ymer.(none):/usr/local/mysql/mysql-5.0-ndb