mariadb: build RocksDB without fallocate

https://github.com/facebook/rocksdb/commit/7291a3f813e563efbd6870465b1063a115480373 is not enough. Anway, given how SST files are written, fallocate is unlikely to be useful, even less on SSD. RocksDB seems to have options to disable at runtime (see env/io_posix.cc) but I see nothing for that in MariaDB configuration. Steps to reproduce: With a Btrfs partition (no specific mkfs option and for example 2GB size) and following MariaDB configuration: [mysqld] datadir = /tmp/datadir disable-log-bin plugin-load = ha_rocksdb rocksdb_default_cf_options = compression=kNoCompression;bottommost_compression=kZSTD execute the following queries with `mysql -uroot`: > USE test; > CREATE TABLE t(x MEDIUMBLOB) ENGINE=RocksDB; > INSERT INTO t VALUES(repeat('.',16777215)); > INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t; > INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t; > INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t; Disk usage can be checked with: > SHOW ENGINE RocksDB STATUS \G Level Files Size ... ------------------------- ... L0 1/0 144.00 MB ... L2 1/0 288.00 MB ... Sum 2/0 432.00 MB ... and this can be compacted as follows: > SET GLOBAL rocksdb_compact_cf='default'; > SHOW ENGINE RocksDB STATUS \G Level Files Size ... ------------------------- ... L0 0/0 0.00 KB ... L1 0/0 0.00 KB ... L2 1/0 16.28 KB ... Sum 1/0 16.28 KB ... Now using `compsize` (on Debian, it's provided by btrfs-compsize package): # with -DWITH_FALLOCATE=NO $ compsize /tmp/datadir/#rocksdb/000029.sst Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 20K 20K 20K none 100% 20K 20K 20K # without -DWITH_FALLOCATE=NO $ compsize /tmp/datadir/#rocksdb/000029.sst Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 35M 35M 20K none 100% 35M 35M 20K

mariadb: build RocksDB without fallocate
https://github.com/facebook/rocksdb/commit/7291a3f813e563efbd6870465b1063a115480373 is not enough. Anway, given how SST files are written, fallocate is unlikely to be useful, even less on SSD. RocksDB seems to have options to disable at runtime (see env/io_posix.cc) but I see nothing for that in MariaDB configuration. Steps to reproduce: With a Btrfs partition (no specific mkfs option and for example 2GB size) and following MariaDB configuration: [mysqld] datadir = /tmp/datadir disable-log-bin plugin-load = ha_rocksdb rocksdb_default_cf_options = compression=kNoCompression;bottommost_compression=kZSTD execute the following queries with `mysql -uroot`: > USE test; > CREATE TABLE t(x MEDIUMBLOB) ENGINE=RocksDB; > INSERT INTO t VALUES(repeat('.',16777215)); > INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t; > INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t; > INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t; Disk usage can be checked with: > SHOW ENGINE RocksDB STATUS \G Level Files Size ... ------------------------- ... L0 1/0 144.00 MB ... L2 1/0 288.00 MB ... Sum 2/0 432.00 MB ... and this can be compacted as follows: > SET GLOBAL rocksdb_compact_cf='default'; > SHOW ENGINE RocksDB STATUS \G Level Files Size ... ------------------------- ... L0 0/0 0.00 KB ... L1 0/0 0.00 KB ... L2 1/0 16.28 KB ... Sum 1/0 16.28 KB ... Now using `compsize` (on Debian, it's provided by btrfs-compsize package): # with -DWITH_FALLOCATE=NO $ compsize /tmp/datadir/#rocksdb/000029.sst Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 20K 20K 20K none 100% 20K 20K 20K # without -DWITH_FALLOCATE=NO $ compsize /tmp/datadir/#rocksdb/000029.sst Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 35M 35M 20K none 100% 35M 35M 20K
e6c1bb74 · Julien Muchembled · d5b4701e · e6c1bb74
Commit e6c1bb74 authored Nov 26, 2021 by Julien Muchembled
Hide whitespace changes
Inline Side-by-side

Showing with 4 additions and 0 deletions

component/mariadb/buildout.cfg component/mariadb/buildout.cfg +4 -0

No files found.
--- a/component/mariadb/buildout.cfg
+++ b/component/mariadb/buildout.cfg
@@ -67,6 +67,10 @@ configure-options =
  -DCMAKE_INSTALL_RPATH=${:CMAKE_LIBRARY_PATH}
  -DCMAKE_INCLUDE_PATH=${unixodbc:location}/include
  -DCMAKE_LIBRARY_PATH=${unixodbc:location}/lib
+# for RocksDB - see
+#  https://lore.kernel.org/linux-btrfs/ed3642c2-682e-08a1-f18d-2d63409b7631@nexedi.com/T/
+  -DWITH_FALLOCATE=NO
+##
 CMAKE_CFLAGS = -I${bzip2:location}/include -I${jemalloc:location}/include -I${libaio:location}/include -I${libxml2:location}/include -I${ncurses:location}/include -I${openssl:location}/include -I${pcre:location}/include -I${readline5:location}/include -I${xz-utils:location}/include -I${zlib:location}/include -I${unixodbc:location}/include -I${lz4:location}/include -I${snappy:location}/include -I${zstd:location}/include
 CMAKE_LIBRARY_PATH = ${bzip2:location}/lib:${jemalloc:location}/lib:${libaio:location}/lib:${libxml2:location}/lib:${ncurses:location}/lib:${openssl:location}/lib:${pcre:location}/lib:${readline5:location}/lib:${xz-utils:location}/lib:${zlib:location}/lib:${unixodbc:location}/lib:${lz4:location}/lib:${snappy:location}/lib:${zstd:location}/lib
 environment =