1. 05 Feb, 2014 1 commit
    • Michael Widenius's avatar
      Replication changes for CREATE OR REPLACE TABLE · 5426facd
      Michael Widenius authored
      - CREATE TABLE is by default executed on the slave as CREATE OR REPLACE
      - DROP TABLE is by default executed on the slave as DROP TABLE IF NOT EXISTS
      
      This means that a slave will by default continue even if we try to create
      a table that existed on the slave (the table will be deleted and re-created) or
      if we try to drop a table that didn't exist on the slave.
      This should be safe as instead of having the slave stop because of an inconsistency between
      master and slave, it will fix the inconsistency.
      Those that would prefer to get a stopped slave instead for the above cases can set slave_ddl_exec_mode to STRICT. 
      
      - Ensure that a CREATE OR REPLACE TABLE which dropped a table is replicated
      - DROP TABLE that generated an error on master is handled as an identical DROP TABLE on the slave (IF NOT EXISTS is not added in this case)
      - Added slave_ddl_exec_mode variable to decide how DDL's are replicated
      
      New logic for handling BEGIN GTID ... COMMIT from the binary log:
      
      - When we find a BEGIN GTID, we start a transaction and set OPTION_GTID_BEGIN
      - When we find COMMIT, we reset OPTION_GTID_BEGIN and execute the normal COMMIT code.
      - While OPTION_GTID_BEGIN is set:
        - We don't generate implict commits before or after statements
        - All tables are regarded as transactional tables in the binary log (to ensure things are executed exactly as on the master)
      - We reset OPTION_GTID_BEGIN also on rollback
      
      This will help ensuring that we don't get any sporadic commits (and thus new GTID's) on the slave and will help keep the GTID's between master and slave in sync.
      
      
      mysql-test/extra/rpl_tests/rpl_log.test:
        Added testing of mode slave_ddl_exec_mode=STRICT
      mysql-test/r/mysqld--help.result:
        New help messages
      mysql-test/suite/rpl/r/create_or_replace_mix.result:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/r/create_or_replace_row.result:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/r/create_or_replace_statement.result:
        Testing replication of create or replace
      mysql-test/suite/rpl/r/rpl_gtid_startpos.result:
        Test must be run in slave_ddl_exec_mode=STRICT as part of the test depends on that DROP TABLE should fail on slave.
      mysql-test/suite/rpl/r/rpl_row_log.result:
        Updated result
      mysql-test/suite/rpl/r/rpl_row_log_innodb.result:
        Updated result
      mysql-test/suite/rpl/r/rpl_row_show_relaylog_events.result:
        Updated result
      mysql-test/suite/rpl/r/rpl_stm_log.result:
        Updated result
      mysql-test/suite/rpl/r/rpl_stm_mix_show_relaylog_events.result:
        Updated result
      mysql-test/suite/rpl/r/rpl_temp_table_mix_row.result:
        Updated result
      mysql-test/suite/rpl/t/create_or_replace.inc:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/create_or_replace_mix.cnf:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/create_or_replace_mix.test:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/create_or_replace_row.cnf:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/create_or_replace_row.test:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/create_or_replace_statement.cnf:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/create_or_replace_statement.test:
        Testing of CREATE OR REPLACE TABLE with replication
      mysql-test/suite/rpl/t/rpl_gtid_startpos.test:
        Test must be run in slave_ddl_exec_mode=STRICT as part of the test depends on that DROP TABLE should fail on slave.
      mysql-test/suite/rpl/t/rpl_stm_log.test:
        Removed some lines
      mysql-test/suite/sys_vars/r/slave_ddl_exec_mode_basic.result:
        Testing of slave_ddl_exec_mode
      mysql-test/suite/sys_vars/t/slave_ddl_exec_mode_basic.test:
        Testing of slave_ddl_exec_mode
      sql/handler.cc:
        Regard all tables as transactional in commit if OPTION_GTID_BEGIN is set.
        This is to ensure that statments are not commited too early if non transactional tables are used.
      sql/log.cc:
        Regard all tables as transactional in commit if OPTION_GTID_BEGIN is set.
        Also treat 'direct' log events as transactional (to get them logged as they where on the master)
      sql/log_event.cc:
        Ensure that the new error from DROP TABLE when trying to drop a view is treated same as the old one.
        Store error code that slave expects in THD.
        Set OPTION_GTID_BEGIN if we find a BEGIN.
        Reset OPTION_GTID_BEGIN if we find a COMMIT.
      sql/mysqld.cc:
        Added slave_ddl_exec_mode_options
      sql/mysqld.h:
        Added slave_ddl_exec_mode_options
      sql/rpl_gtid.cc:
        Reset OPTION_GTID_BEGIN if we record a gtid (safety)
      sql/sql_class.cc:
        Regard all tables as transactional in commit if OPTION_GTID_BEGIN is set.
      sql/sql_class.h:
        Added to THD: log_current_statement and slave_expected_error
      sql/sql_insert.cc:
        Ensure that CREATE OR REPLACE is logged if table was deleted.
        Don't do implicit commit for CREATE if we are under OPTION_GTID_BEGIN
      sql/sql_parse.cc:
        Change CREATE TABLE -> CREATE OR REPLACE TABLE for slaves
        Change DROP TABLE -> DROP TABLE IF EXISTS for slaves
        CREATE TABLE doesn't force implicit commit in case of OPTION_GTID_BEGIN
        Don't do commits before or after any statement if OPTION_GTID_BEGIN was set.
      sql/sql_priv.h:
        Added OPTION_GTID_BEGIN
      sql/sql_show.cc:
        Enhanced store_create_info() to also be able to handle CREATE OR REPLACE
      sql/sql_show.h:
        Updated prototype
      sql/sql_table.cc:
        Ensure that CREATE OR REPLACE is logged if table was deleted.
      sql/sys_vars.cc:
        Added slave_ddl_exec_mode
      sql/transaction.cc:
        Added warning if we got a GTID under OPTION_GTID_BEGIN
      5426facd
  2. 31 Jan, 2014 1 commit
    • Michael Widenius's avatar
      Fixes for CREATE_OR_REPLACE · 43f6e118
      Michael Widenius authored
      - MDEV-5587 Server crashes in Locked_tables_list::restore_lock on CREATE OR REPLACE .. SELECT under LOCK
      - MDEV-5586 Assertion `share->tdc.all_tables.is_empty() || remove_type != TDC_RT_REMOVE_ALL' fails in tdc_remove_table
      - MDEV-5588 Strange error on CREATE OR REPLACE table over an existing view
      
      mysql-test/r/create_or_replace.result:
        Added test cases
      mysql-test/r/lowercase_view.result:
        New error message
      mysql-test/r/merge.result:
        New error message
      mysql-test/r/multi_update.result:
        New error message
      mysql-test/r/subselect.result:
        New error message
      mysql-test/r/subselect_exists_to_in.result:
        New error message
      mysql-test/r/subselect_no_mat.result:
        New error message
      mysql-test/r/subselect_no_opts.result:
        New error message
      mysql-test/r/subselect_no_scache.result:
        New error message
      mysql-test/r/subselect_no_semijoin.result:
        New error message
      mysql-test/r/view.result:
        New error message
      mysql-test/suite/funcs_1/r/myisam_views-big.result:
        New error message
      mysql-test/t/create_or_replace.test:
        New tests
      mysql-test/t/view.test:
        New error message
      sql/share/errmsg-utf8.txt:
        Added new error message
      sql/sql_base.cc:
        Updated error message
        Do an automatic UNLOCK TABLES if we don't have any locked tables (safety fix)
      sql/sql_db.cc:
        Updated arguments
      sql/sql_load.cc:
        New error message
      sql/sql_parse.cc:
        Check that we are not using a table we are dropping and re-creating
      sql/sql_table.cc:
        Added parameter to mysql_rm_table_no_locks() to not automaticly do UNLOCK TABLES
        Added better error message if trying to drop a view with DROP TABLE
        Don't try to create something we select from
      sql/sql_table.h:
        Updated prototypes
      43f6e118
  3. 29 Jan, 2014 2 commits
    • Michael Widenius's avatar
      Fixed compiler warnings · 2410ecac
      Michael Widenius authored
      Made stopping of slave more robust
      Fixed tokudb test cases that gave different results between runs
      Speed up some slow tokudb tests by adding begin ... commit
      
      
      mysql-test/extra/rpl_tests/rpl_stop_slave.test:
        Ensure that slaves are properly synced before they are stopped.
        (Otherwise some tests results will be different between runs)
      storage/innobase/buf/buf0buf.cc:
        Fixed compiler warning
      storage/tokudb/mysql-test/tokudb/r/cluster_filter_unpack_varchar_and_int_hidden.result:
        Test case could be solved with index or range scan.
      storage/tokudb/mysql-test/tokudb/t/cluster_filter_unpack_varchar_and_int_hidden.test:
        Test case could be solved with index or range scan.
      storage/tokudb/mysql-test/tokudb_bugs/r/5733_innodb.result:
        Speed up test by adding begin...commit
      storage/tokudb/mysql-test/tokudb_bugs/r/5733_tokudb.result:
        Speed up test by adding begin...commit
      storage/tokudb/mysql-test/tokudb_bugs/t/5733_innodb.test:
        Speed up test by adding begin...commit
      storage/tokudb/mysql-test/tokudb_bugs/t/5733_tokudb.test:
        Speed up test by adding begin...commit
      storage/tokudb/mysql-test/tokudb_mariadb/r/compression.result:
        Added drop table (safety)
      storage/tokudb/mysql-test/tokudb_mariadb/t/compression.test:
        Added drop table (safety)
      2410ecac
    • Michael Widenius's avatar
      Implementation of MDEV-5491: CREATE OR REPLACE TABLE · 7ffc9da0
      Michael Widenius authored
      Using CREATE OR REPLACE TABLE is be identical to
      
      DROP TABLE IF EXISTS table_name;
      CREATE TABLE ...;
      
      Except that:
      
      * CREATE OR REPLACE is be atomic (now one can create the same table between drop and create).
      * Temporary tables will not shadow the table name for the DROP as the CREATE TABLE tells us already if we are using a temporary table or not.
      * If the table was locked with LOCK TABLES, the new table will be locked with the same lock after it's created.
      
      Implementation details:
      - We don't anymore open the to-be-created table during CREATE TABLE, which the original code did.
        - There is no need to open a table we are planning to create. It's enough to check if the table exists or not.
      - Removed some of duplicated code for CREATE IF NOT EXISTS.
      - Give an error when using CREATE OR REPLACE with IF NOT EXISTS (conflicting options).
      - As a side effect of the code changes, we don't anymore have to internally re-prepare prepared statements with CREATE TABLE if the table exists.
      - Made one code path for all testing if log table are in use.
      - Better error message if one tries to create/drop/alter a log table in use
      - Added back disabled rpl_row_create_table test as it now seams to work and includes a lot of interesting tests.
      - Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
      - Aligned CREATE OR REPLACE parsing code in sql_yacc.yy for TABLE and VIEW
      - Changed interface for drop_temporary_table() to make it more reusable
      - Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
      - Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument. Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
      - Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
      - Added restart_trans_for_tables() to be able to restart a transaction.
      - DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
      - Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
      - Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
      
      
      
      
      mysql-test/include/commit.inc:
        With new code we get fewer status increments
      mysql-test/r/commit_1innodb.result:
        With new code we get fewer status increments
      mysql-test/r/create.result:
        Added testing of create or replace with timeout
      mysql-test/r/create_or_replace.result:
        Basic testing of CREATE OR REPLACE TABLE
      mysql-test/r/partition_exchange.result:
        New error message
      mysql-test/r/ps_ddl.result:
        Fewer reprepares with new code
      mysql-test/suite/archive/discover.result:
        Don't rediscover archive tables if the .frm file exists
        (Sergei will look at this if there is a better way...)
      mysql-test/suite/archive/discover.test:
        Don't rediscover archive tables if the .frm file exists
        (Sergei will look at this if there is a better way...)
      mysql-test/suite/funcs_1/r/innodb_views.result:
        New error message
      mysql-test/suite/funcs_1/r/memory_views.result:
        New error message
      mysql-test/suite/rpl/disabled.def:
        rpl_row_create_table should now be safe to use
      mysql-test/suite/rpl/r/rpl_row_create_table.result:
        Updated results after adding back disabled test
      mysql-test/suite/rpl/t/rpl_create_if_not_exists.test:
        Added comment
      mysql-test/suite/rpl/t/rpl_row_create_table.test:
        Added CREATE OR REPLACE TABLE test
      mysql-test/t/create.test:
        Added CREATE OR REPLACE TABLE test
      mysql-test/t/create_or_replace-master.opt:
        Create logs
      mysql-test/t/create_or_replace.test:
        Basic testing of CREATE OR REPLACE TABLE
      mysql-test/t/partition_exchange.test:
        Error number changed as we are now using same code for all log table change issues
      mysql-test/t/ps_ddl.test:
        Fewer reprepares with new code
      sql/handler.h:
        Moved things around a bit in a structure to get better alignment.
        Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
        Added 3 elements to end of HA_CREATE_INFO to be able to store state to add backs locks in case of LOCK TABLES.
      sql/log.cc:
        Reimplemented check_if_log_table():
        - Simpler and faster usage
        - Can give error messages
        
        This gives us one code path for allmost all error messages if log tables are in use
      sql/log.h:
        New interface for check_if_log_table()
      sql/slave.cc:
        More logging
      sql/sql_alter.cc:
        New interface for check_if_log_table()
      sql/sql_base.cc:
        More documentation
        Changed interface for drop_temporary_table() to make it more reusable
        Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
        Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument.  Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
        Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
        Check for command number instead of open_strategy of CREATE TABLE was used.
        Added restart_trans_for_tables() to be able to restart a transaction.  This was needed in "create or replace ... select" between the drop table and the select.
      sql/sql_base.h:
        Added and updated function prototypes
      sql/sql_class.h:
        Added new prototypes to Locked_tables_list class
        Added extra argument to select_create to avoid double call to eof() or send_error()
        - I needed this in some edge case where the table was not created against expections.
      sql/sql_db.cc:
        New interface for check_if_log_table()
      sql/sql_insert.cc:
        Remember position to lock information so that we can reaquire table lock for LOCK TABLES + CREATE OR REPLACE TABLE SELECT. Later add back the lock by calling restore_lock().
        Removed one not needed indentation level in create_table_from_items()
        Ensure we don't call send_eof() or abort_result_set() twice.
      sql/sql_lex.h:
        Removed variable that I temporarly added in an earlier changeset
      sql/sql_parse.cc:
        Removed old test code (marked with QQ)
        Ensure that we have open_strategy set as TABLE_LIST::OPEN_STUB in CREATE TABLE
        Removed some IF NOT EXISTS code as this is now handled in create_table_table_impl().
        Set OPTION_KEEP_LOGS later. This code had to be moved as the test for IF EXISTS has changed place.
        DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
      sql/sql_partition_admin.cc:
        New interface for check_if_log_table()
      sql/sql_rename.cc:
        New interface for check_if_log_table()
      sql/sql_table.cc:
        New interface for check_if_log_table()
        Moved some code in mysql_rm_table() under a common test.
        - Safe as temporary tables doesn't have statistics.
        - !is_temporary_table(table) test was moved out from drop_temporary_table() and merged with upper level code.
        - Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
        - Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
        - In mysql_create_like_table(), restore table->open_strategy() if it was changed.
        - Re-test if table was a view after opening it.
      sql/sql_table.h:
        New prototype for mysql_create_table_no_lock()
      sql/sql_yacc.yy:
        Added syntax for CREATE OR REPLACE TABLE
        Reuse new code for CREATE OR REPLACE VIEW
      sql/table.h:
        Added name for enum type
      sql/table_cache.cc:
        More DBUG
      7ffc9da0
  4. 09 Jan, 2014 1 commit
    • Michael Widenius's avatar
      Cleanups: · 659304d4
      Michael Widenius authored
      - Updated help for mysql-test-run
      - Added OQGraph to all cmake error output regarding OQGraph to make it easier to spot problems
      - Suppressed warning messages from OQGraph
      - Added test for version_malloc_library variable
      
      mysql-test/mysql-test-run.pl:
        Updated help
      mysql-test/suite/sys_vars/r/version_malloc_library_basic.result:
        Added test for version_malloc_library variable
      mysql-test/suite/sys_vars/t/version_malloc_library_basic.test:
        Added test for version_malloc_library variable
      storage/oqgraph/CMakeLists.txt:
        Added OQGraph to all cmake error output regarding OQGraph to make it easier to spot problems
      storage/oqgraph/mysql-test/oqgraph/boundary_conditions.result:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/boundary_conditions.test:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/invalid_operations.result:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/invalid_operations.test:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/isnull.result:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/isnull.test:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/regression_1233113.result:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/regression_1233113.test:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/regression_drop_after.result:
        Suppressed warning messages
      storage/oqgraph/mysql-test/oqgraph/regression_drop_after.test:
        Suppressed warning messages
      659304d4
  5. 02 Jan, 2014 1 commit
    • Michael Widenius's avatar
      Fixes to get valgrind to work with jemalloc · 273078c5
      Michael Widenius authored
      - Added MALLOC_LIBRARY variable to hold name of malloc library
      - Back ported valgrind related fixes from jemalloc 3.4.1 to the included jemalloc 3.3.1
      - Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free() to avoid clash with jemalloc 3.4.1
      - Use option --soname-synonyms=somalloc=NON to valgrind when using jemalloc
      - Show version related variables in mysqld --help
        -- Added SHOW_VALUE_IN_HELP marker
      
      Increased back_log to 150 as the original value was a bit too small
      
      
      CMakeLists.txt:
        Added MALLOC_LIBRARY variable to hold name of malloc library
      cmake/jemalloc.cmake:
        Added MALLOC_LIBRARY variable to hold name of malloc library
      config.h.cmake:
        Added MALLOC_LIBRARY variable to hold name of malloc library
      extra/jemalloc/ChangeLog:
        Updates changelog
      extra/jemalloc/include/jemalloc/internal/arena.h:
        Backported valgrind fixes from jemalloc 3.4.1
      extra/jemalloc/include/jemalloc/internal/jemalloc_internal.h.in:
        Backported valgrind fixes from jemalloc 3.4.1
      extra/jemalloc/include/jemalloc/internal/private_namespace.h:
        Backported valgrind fixes from jemalloc 3.4.1
      extra/jemalloc/include/jemalloc/internal/tcache.h:
        Backported valgrind fixes from jemalloc 3.4.1
      extra/jemalloc/src/arena.c:
        Backported valgrind fixes from jemalloc 3.4.1
      include/my_bitmap.h:
        Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free() to avoid clash with jemalloc 3.4.1
      mysql-test/mysql-test-run.pl:
        Use option --soname-synonyms=somalloc=NON to valgrind when using jemalloc
      mysql-test/valgrind.supp:
        Supression of memory leak in OpenSuse 12.3
      mysys/my_bitmap.c:
        Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free()
      sql/ha_ndbcluster_binlog.cc:
        Renames
      sql/ha_ndbcluster_cond.h:
        Renames
      sql/ha_partition.cc:
        Renames
      sql/handler.cc:
        Renames
      sql/item_subselect.cc:
        Renames
      sql/log_event.cc:
        Renames
      sql/log_event_old.cc:
        Renames
      sql/mysqld.cc:
        Renames
        Show version related variables in mysqld --help
      sql/opt_range.cc:
        Renames
      sql/opt_table_elimination.cc:
        Renames
      sql/partition_info.cc:
        Renames
      sql/rpl_injector.h:
        Renames
      sql/set_var.h:
        Renames
      sql/slave.cc:
        Renames
      sql/sql_bitmap.h:
        Renames
      sql/sql_insert.cc:
        Renames
      sql/sql_lex.h:
        Renames
      sql/sql_parse.cc:
        Renames
      sql/sql_partition.cc:
        Renames
      sql/sql_select.cc:
        Renames
      sql/sql_show.cc:
        Renames
      sql/sql_update.cc:
        Renames
      sql/sys_vars.cc:
        Show version related variables in mysqld --help
      sql/sys_vars.h:
        Added SHOW_VALUE_IN_HELP marker for variables that should be shown in --help
      sql/table.cc:
        Renames
      sql/table.h:
        Removed not used bitmap_init_value
      storage/connect/ha_connect.cc:
        Removed compiler warning
      storage/maria/ma_open.c:
        Renames
      unittest/mysys/bitmap-t.c:
        Renames
      273078c5
  6. 27 Dec, 2013 1 commit
    • Michael Widenius's avatar
      Increased back_log to 150, but not more than max_connections. · e41788d2
      Michael Widenius authored
      - This was done to get better performance when doing a lot of connections.
      Ensure that thread_cache_size is not larger than max_connections (trivial optimizations).
      Fixed that the --host_cache_size=# startup option works
      
      mysql-test/r/variables.result:
        Increase back_log to 150
      sql/hostname.cc:
        Fixed that the --host_cache_size=# startup option works
      sql/mysqld.cc:
        Ensure that back_log and thread_cache_size is not set higher than max_connections (as this would not make any sense).
      sql/sys_vars.cc:
        Increased back_log to 150
      e41788d2
  7. 23 Dec, 2013 1 commit
  8. 22 Dec, 2013 14 commits
  9. 20 Dec, 2013 3 commits
  10. 19 Dec, 2013 4 commits
  11. 17 Dec, 2013 10 commits
  12. 16 Dec, 2013 1 commit