variable settings (rpl_sys)

Problem: under certain conditions (e.g. user variables usage in triggers)
accessing a user defined variable we may use a variables hash table that
belongs to already deleted thread. It happens if
thd= new THD;
has the same address as just deleted thd as we use
if (stored_thd == thd)
to check.
That may lead to unpredictable results, server crash etc.

Fix: use thread_id instead of thd address to distinguish threads.

Note: no simple and repeatable test case.


sql/item_func.cc:
  Fix for bug#40770: Server Crash when running with triggers including
  variable settings (rpl_sys)
    - store and use thd->thread_id to distinguish threads instead of
  thread address as it may be the same as just deleted thread had, 
  i.e. we may get (old_thd == new_thd) after
  delete old_thd;
  new_thd= new THD;
    - set entry_thread_id only when we get a real entry, clear it 
  if the hash search fails.
sql/item_func.h:
  Fix for bug#40770: Server Crash when running with triggers including
  variable settings (rpl_sys)
    - Item_func_set_user_var::entry_thread_id introduced.

Problem: memory leak occurs when we open a federated table
that has its share in the hash.

Fix: free not used memory.

Note: the fix should NOT be merged to 5.1 (the code changed).


sql/ha_federated.cc:
  Fix for bug#40875: Memory leak in FEDERATED handler
    - free memory (tmp_share.scheme) allocated in the parse_url()
  if it isn't used anymore.

set but is ignored".
                                  
This patch makes @@session.max_allowed_packed and
@@session.net_buffer_length read-only as suggested in the bug
report. The user will have to use SET GLOBAL (and reconnect)
to alter the session values of these variables.
                            
The error string ER_VARIABLE_IS_READONLY is introduced.
                            
Tests are modified accordingly.


sql/set_var.cc:
  The class sys_var_thd_ulong_session_readonly is introduced as
  a specialization of sys_var_thd_ulong implementing a read-only
  session variable. The class overrides check() and
  check_default() to achieve the read-only property for the
  session part of the variable.
sql/set_var.h:
  The class sys_var_thd_ulong_session_readonly is introduced as
  a specialization of sys_var_thd_ulong implementing a read-only
  session variable. The class overrides check() and
  check_default() to achieve the read-only property for the
  session part of the variable.
sql/share/errmsg.txt:
  New error ER_VARIABLE_IS_READONLY.

   additional fix - the 'debug' keyword removed

    That's a Win-specific error.
    When we create libmysqld.dll we have many libraries like mysys, dbug,
    strings, etc linked into that dll, so the application built upon
    this library shouldn't link these libraries to itself, rather use
    those inside the dll.

    Fixed by redirecting calls into the libmysqld.dll

per-file comments:
  dbug/dbug.c
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
    fake _db_something definitions added

  include/my_dbug.h
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
  fake _db_something declarations added

  libmysqld/examples/CMakeLists.txt
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
    superfluous libraries removed from linking

  libmysqld/libmysqld.def
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
    set of mysys functions added to the export section

		using crashes server

When the server is configured to use a RSA key, and when the client sends
a cipher-suite list that contains a non-RSA key as acceptable, the server 
would try to process that key even though it was impossible.

Now, yaSSL sets its own acceptable-cipher list according to what kind of
key the server is started with, and will never explore and try to pair 
impossible combinations.

This involves a partial import of the current YaSSL tree, not the whole
thing, so as to try to avoid introducing new bugs.

(Updated to avoid many whitespace changes and make diff smaller.)

      GLOBAL STATUS is calculated by studying the list of threads. In the
        embedded server threads were not linked to the internal list, so the
        calculation always returns 0. Fixed by 'linking' the embedded-server
        threads to the same list

per-file comments:
  libmysqld/lib_sql.cc
Bug#34517 SHOW GLOBAL STATUS does not work properly in embedded server.
    Add newly created 'threads' to the internal thread list.
    Remove them from the list as they're freed.

  mysql-test/r/information_schema.result
Bug#34517 SHOW GLOBAL STATUS does not work properly in embedded server.
    test result

  mysql-test/t/information_schema.test
Bug#34517 SHOW GLOBAL STATUS does not work properly in embedded server.
    test case added

Item_func_div didn't calculate the precision of the result properly.
  The result of 5/0.0001 is 5000 so we have to add decimals of the divisor
  to the planned precision.

per-file comments:
  mysql-test/r/type_newdecimal.result
Bug#31616 div_precision_increment description looks wrong
    test result fixed

  mysql-test/t/type_newdecimal.test
Bug#31616 div_precision_increment description looks wrong
    test case

  sql/item_func.cc
Bug#31616 div_precision_increment description looks wrong
    precision must be increased with args[1]->decimals parameter

                  
The function thd_security_context allocates memory on an unprotected MEM_ROOT if the
message length becomes longer than requested and the initial buffer memory needs to
be reallocated.
                  
This patch fixes the design error by copying parts of the reallocated buffer 
to the destination buffer. This works because the destination buffer isn't
owned by the String object and thus isn't freed when a new buffer is allocated.
Any new memory allocated by the String object is reclaimed when the object
is destroyed at the end of the function call.

with partitions

Pre push fix, optimized replace_regex, to cut 2 seconds
from test time.

sql_view.cc: required_view_parameters has been decreased by 2 
(not by 1) because its value was incorrect: 16 instead of 15
(minor performance issue).


sql/sql_view.cc:
  sql_view.cc: required_view_parameters has been decreased by 2 
  (not by 1) because its value was incorrect: 16 instead of 15
  (minor performance issue).

            missing after downgrade

Obsolete arc/ directory and view .frm file backup support
has been removed by the patch for bug 17823. However, that
bugfix caused a problem with "live downgrades" of the
server: if we rename some view 4 times under 5.1.29/5.0.72
and then try to rename it under 5.1.28/5.0.70 on the same
database, the server fails with a error:

  query 'RENAME TABLE ... TO ...' failed: 6: Error on
  delete of '....frm-0001' (Errcode: 2)

Also .frm file of that view may be lost (renamed to .frm~).

The server failed because it tried to rename latest 3
backup .frm files renaming the view: the server used an
integer value of the "revision" field of .frm file to
extract those file names. After the fix for bug 17823 those
files were not created/maintained any more, however the
"revision" field was incremented as usual. So, the server
failed renaming non existent files.

This fix solves the problem by removing the support for
"revision" .frm file field:
1. New server silently ignores existent "revision" fields
   in old .frm files and never write it down;
2. Old server assumes, that missing "revision" field in new
   .frm files means default value of 0.
3. Accordingly to the fix for bug 17823 the new server
   drops arc/ directory on alter/rename view, so after
   "live downgrade" old server begins maintenance of the
   arc/ directory from scratch without conflicts with .frm
   files.


sql/parse_file.cc:
  Bug #40021: Renaming view fails, archived .frm for view is
              missing after downgrade
  
  1. static write_parameter(): the old_version parameter
     and the section for FILE_OPTIONS_REV have been re moved.
  2. write_parameter(): the max_versions parameter has been
     removed;
  3. sql_create_definition_file(): removal of dead code;
  4. rename_in_schema_file(): revision and num_view_backups
     parameters and dead code have been removed;
  5. File_parser::parse(): FILE_OPTIONS_REV section has been
     removed.
sql/parse_file.h:
  Bug #40021: Renaming view fails, archived .frm for view is
              missing after downgrade
  
  1. The FILE_OPTIONS_REV constant has been removed.
  2. sql_create_definition_file and rename_in_schema_file
     functions: obsolete versions, revision and
     num_view_backups parameters have been removed.
sql/sql_db.cc:
  Bug #40021: Renaming view fails, archived .frm for view is
              missing after downgrade
  
  Commentary update.
sql/sql_trigger.cc:
  Bug #40021: Renaming view fails, archived .frm for view is
              missing after downgrade
  
  sql_create_definition_file() calls have been updates to
  new parameter lists.
sql/sql_view.cc:
  Bug #40021: Renaming view fails, archived .frm for view is
              missing after downgrade
  
  1. The mysql_create_view function code is used for both
     CREATE VIEW and ALTER queries, but query cache is
     necessary for ALTER command only. Check for a non first
     view revision has been replaced with a direct check for
     ALTER query.
  2. The num_view_backups global constant has been removed.
  3. view_parameters: the "revision" .frm field support has
     been removed.
  4. sql_create_definition_file and rename_in_schema_file
     function calls have been updates to new parameter lists.
sql/table.h:
  Bug #40021: Renaming view fails, archived .frm for view is
              missing after downgrade
  
  TABLE_LIST: the revision field has been removed.

tables can cause server to crash!

The bug will be fixed by patch for #34779: "crash in checksum table
on federated tables with blobs containing nulls"

Only a test case commited.


mysql-test/r/federated.result:
  Test case for bug #34774: key prefix on text field in federated 
  tables can cause server to crash!
    - test result.
mysql-test/t/federated.test:
  Test case for bug #34774: key prefix on text field in federated 
  tables can cause server to crash!
    - test case.

when InnoDB frm file corruption

Problem: mysqlcheck runs 'SHOW FULL TABLE' queries to get table lists.
The query may fail for some reasons (e.g. null .frm file) then
mysqlcheck doesn't process the database tables.

Fix: try to run 'SHOW TABLES' if 'SHOW FULL TABLES' failed.


client/mysqlcheck.c:
  Fix for bug#37527: mysqlcheck fails to report entire database 
  when InnoDB frm file corruption
    - run "SHOW TABLES" query if "SHOW /*!50002 FULL*/ TABLES" failed;
    - print error info if both failed.
mysql-test/r/mysqlcheck.result:
  Fix for bug#37527: mysqlcheck fails to report entire database 
  when InnoDB frm file corruption
    - test result.
mysql-test/t/mysqlcheck.test:
  Fix for bug#37527: mysqlcheck fails to report entire database 
  when InnoDB frm file corruption
    - test case.

Reordered include files so that no mess will be left if this test is run without InnoDB
Previously, this test would leave a database named 'federated' in such a case and would
cause tests that examined existing databases to fail.

issue 'The storage engine for the table doesn't support check' note for I_S tables


mysql-test/r/mysqlcheck.result:
  test result
mysql-test/t/mysqlcheck.test:
  test case
sql/sql_table.cc:
  issue 'The storage engine for the table doesn't support check' note for I_S tables

table->partition_info is unitialized when we open frm file only.
The fix is check share->partition_info_len which is set on
frm opening stage.


mysql-test/r/information_schema_part.result:
  test result
mysql-test/t/information_schema_part.test:
  test case
sql/sql_show.cc:
  table->partition_info is unitialized when we open frm file only.
  The fix is check share->partition_info_len which is set on
  frm opening stage.

VARIABLE_VALUE field is decreased to 1024 symbols.
(affected I_S tables: GLOBAL_VARIABLES, SESSION_VARIABLES,
 GLOBAL_STATUS, SESSION_STATUS).
The only variable which can be longer than 1024 is
init_connect. The variable will be truncated with warning.
Additional fix:
Added where condition filter which speed up queries which
have where condition with expressions which use VARIABLE_NAME
field.


mysql-test/r/information_schema.result:
  test result
mysql-test/r/show_check.result:
  result fix
mysql-test/suite/funcs_1/r/is_columns_is.result:
  result fix
mysql-test/t/information_schema.test:
  test case
sql/sql_show.cc:
  VARIABLE_VALUE field is decreased to 1024 symbols.
  (affected I_S tables: GLOBAL_VARIABLES, SESSION_VARIABLES,
   GLOBAL_STATUS, SESSION_STATUS).
  The only variable which can be longer than 1024 is
  init_connect. The variable will be truncated with warning.
  Additional fix:
  Added where condition filter which speed up queries which
  have where condition with expressions which use VARIABLE_NAME
  field.

changed 'charset', 'collation' field length from 64 to MY_CS_NAME_SIZE(32)
in tables:
SCHEMATA, TABLES, COLUMNS, CHARACTER_SETS,
COLLATIONS, COLLATION_CHARACTER_SET_APPLICABILITY


mysql-test/r/create.result:
  result fix
mysql-test/r/information_schema.result:
  result fix
mysql-test/r/show_check.result:
  result fix
mysql-test/suite/funcs_1/r/is_character_sets.result:
  result fix
mysql-test/suite/funcs_1/r/is_collation_character_set_applicability.result:
  result fix
mysql-test/suite/funcs_1/r/is_collations.result:
  result fix
mysql-test/suite/funcs_1/r/is_columns.result:
  result fix
mysql-test/suite/funcs_1/r/is_columns_is.result:
  result fix
mysql-test/suite/funcs_1/r/is_routines.result:
  result fix
mysql-test/suite/funcs_1/r/is_schemata.result:
  result fix
mysql-test/suite/funcs_1/r/is_tables.result:
  result fix
sql/sql_show.cc:
  changed 'charset', 'collation' field length from 64 to MY_CS_NAME_SIZE(32)
  in tables:
  SCHEMATA, TABLES, COLUMNS, CHARACTER_SETS,
  COLLATIONS, COLLATION_CHARACTER_SET_APPLICABILITY
  additional fix: ROUTINES.SQL_MODE is changed to be VARCHAR(addon for Bug#29153)

added FLUSH TABLES after INSERT because UPDATE_TIME is updated with delay
on Win.


mysql-test/suite/funcs_1/datadict/is_tables.inc:
  test fix
mysql-test/suite/funcs_1/r/is_tables.result:
  result fix

With fix for bug 25951 index hints are ignored for fulltext
searches, as handling of fulltext indexes is different from
handling regular indexes. Meaning it is not possible to
implement true index hints support for fulltext indexes within
the scope of current fulltext architecture.

The problem is that prior to fix for bug 25951, some useful
index hints still could be given for boolean mode searches.

This patch implements special index hints support for fulltext
indexes with the following characteristics:
- all index hints are still ignored for NLQ mode searches -
  it cannot work without an index;
- for 5.1 and up index hints FOR ORDER BY and FOR GROUP BY are
  still ignored for fulltext indexes;
- boolean mode searches honor USE/FORCE/IGNORE INDEX hints;
- as opposed to index hints for regular indexes, index hints
  for fulltext BOOLEAN mode searches affect the usage of the
  index for the whole query.

mysql-test/r/fulltext.result:
  A test case for BUG#38842.
mysql-test/t/fulltext.test:
  A test case for BUG#38842.
sql/item_func.cc:
  For boolean mode searches, which can work without fulltext
  index, use keys_in_use_for_query bitmap instead of keys_in_use.
  
  The effect is that boolean mode searches now honor index hints.

mysql_config did not output -ldl (or equivalent) when needed for --libmysqld-libs,
so its output could be insufficient to build an application using the embedded
server.

LIBDL was already set in configure; it's now propagated all the way into the
relevant mysql_config scripts.

scripts/Makefile.am:
  We "manually" replace configure's constants in .sh scripts using sed,
  so AC_SUBST() alone is not good enough. Add LIBDL to the substitution
  list; we'll be needing it for mysql_config --libmysqld-libs
scripts/mysql_config.pl.in:
  Add LIBDL (usually -ldl) to output of mysql_config --libmysqld-libs (perl version)
scripts/mysql_config.sh:
  Add LIBDL (usually -ldl) to output of mysql_config --libmysqld-libs (shell version)

japanese characters.

Fix - removed obsolvete setlocale from my_init.c . In MBCS 
environments it caused unwanted character-to-byte translations
in fputc()  in client code and wrong output as result.