Merged Facebook commit 617aef9f911d825e9053f3d611d0389e02031225
authored by Inaam Rana to InnoDB storage engine (not XtraDB)
from https://github.com/facebook/mysql-5.6

WL#7047 - Optimize buffer pool list scans and related batch processing

Reduce excessive scanning of pages when doing flush list batches. The
fix is to introduce the concept of "Hazard Pointer", this reduces the
time complexity of the scan from O(n*n) to O.

The concept of hazard pointer is reversed in this work. Academically
hazard pointer is a pointer that the thread working on it will declar
such and as long as that thread is not done no other thread is allowe
do anything with it.

In this WL we declare the pointer as a hazard pointer and then if any
thread attempts to work on it, it is allowed to do so but it has to a
the hazard pointer to the next valid value. We use hazard pointer sol
reverse traversal of lists within a buffer pool instance.

Add an event to control the background flush thread. The background f
thread wait has been converted to an os event timed wait so that it c
signalled by threads that want to kick start a background flush when
buffer pool is running low on free/dirty pages.

(forgot to do "git add" for two files in the previous commit for MDEV-5528)

we do not return simulated out of file space on read operation,
that would cause crash.

Merging from 10.0 (pre-requisite for  MDEV-5528)

Merge Facebook commit 154c579b828a60722a7d9477fc61868c07453d08
and e8f0052f9b112dc786bf9b957ed5b16a5749f7fd authored
by Steaphan Greene from https://github.com/facebook/mysql-5.6

Optimize prefix index queries to skip cluster index lookup when possible.

Currently InnoDB will always fetch the clustered index (primary key
index) for all prefix columns in an index, even when the value of a
particular record is smaller than the prefix length. This change
optimizes that case to use the record from the secondary index and avoid
the extra lookup.

Also adds two status vars that track how effective this is:

innodb_secondary_index_triggered_cluster_reads:
Times secondary index lookup triggered cluster lookup.

innodb_secondary_index_triggered_cluster_reads_avoided:
Times prefix optimization avoided triggering cluster lookup.

Use fn_ext2() to get the file extension from last occurrence
of FN_EXTCHAR ('.') instead.

Merge Facebook commit 4f3e0343fd2ac3fc7311d0ec9739a8f668274f0d
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6

Adds innodb_idle_flush_pct to enable tuning of the page flushing rate
when the system is relatively idle. We care about this, since doing
extra unnecessary flash writes shortens the lifespan of the flash.

Merge Facebook commit ca40b4417fd224a68de6636b58c92f133703fc68
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6
Change the default value for innodb_log_compressed_pages to false

Logging these pages is a waste. We don't want this to be enabled.

One caution here: If the zlib version used by innodb is changed, but
the running version is still the previous version, and the running
version crashes, it is possible crash recovery could fail.

When crash recovery uses a zlib version at all different than the
version used by the crashed instance, it is possible that a redone
compression could fail, where the original did not, because the new
zlib version compresses the same data to a slightly larger size.

Because of the nature of compression, this is even possible when
upgrading to a version of zlib which actually peforms overall better
compression than the previous version.

If this happens, mysql will fail to recover, since a page split can
not be safely triggered during crash recovery.

So, either the exact zlib version must be controlled between builds,
or these rare recovery failures must be accepted. The cost of
logging these pages is quite high, so we consider this limitation to
be worthwhile.

This failure scenario can not happen if there was a clean shutdown.
This is only relevant to restarting crashed instances, or starting an
instance built via a hot backup too (XtraBackup).

Analysis: fil_extend_space_to_desired_size() does not provide file
node to os_aio(). This failed on Windows only because on Windows
we do not use posix_fallocate() to extend file space.

Fix: Add file node to os_aio() function call and make sure that
we do not use NULL pointer at os_aio_array_reserve_slot(). Additionally,
make sure that we do not use 0 as file_block_size (512 is the minimum).

New generation hard drives, SSDs and NVM devices support 4K
sector size. Supported sector size can be found using fstatvfs()
or GetDiskFreeSpace() functions.

Fix failure seen on dict_foreign_remove_partial.

GEOMETRY_COLUMNS and SPATIAL_REF_SYS tables added to the INFORMATION_SCHEMA.

type and that we decompress fist page if it is page compressed
before really accessing it.

The method subselect_union_engine::no_rows() must take
into account the fact that now unit->fake_select_lex is
NULL for for select_union_direct objects.

Add testcase.

Make log_slow_verbosity=explain actually print ANALYZE (that
is, EXPLAIN otuput with two extra columns).

Make ANALYZE work for
- ANALYZE SELECT ... INTO @var
- ANALYZE INSERT SELECT ...;
- ANALYZE SELECT .. INTO OUTFILE

Although the original code of mysql-5.7 was adjusted
to the current MariaDB code the main ideas of the optimization
were preserved.

Added API for atomic operations with explicit memory barrier. Only gcc
atomic builtins are currently supported. If they're unavailable, fall
back to atomic operations with implicit full memory barrier.

Add ANALYZE STMT handling into INSERT command.