• Jérome Perrin's avatar
    software/erp5/test: fix a race in test_caucase_ipv4_bootstrap_from_mariabackup_incremental · d018c3f7
    Jérome Perrin authored
    This test runs our mariadb-backup script 3 times in a row, but this
    script fails when runnning twice for the same second, as we can see at
    the end of the output:
    
        srv/backup/mariabackup/20260224162438.inc.xb.zstd already exists; stdin is an input - not proceeding.
    
    full output for reference:
    
        2026-02-24 16:24:39,371 - erp5-py3 - slapos.testing.testcase - ERROR - runBackup failed with output:
        [00] 2026-02-24 16:24:38 Connecting to MariaDB server host: localhost, user: root, password: not set, port: not set, socket: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/var/run/mariadb.sock
        [00] 2026-02-24 16:24:39 Using server version 11.8.6-MariaDB-log
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup based on MariaDB server 11.8.6-MariaDB Linux (x86_64)
        [00] 2026-02-24 16:24:39 incremental backup from 45482 is enabled.
        [00] 2026-02-24 16:24:39 uses posix_fadvise().
        [00] 2026-02-24 16:24:39 cd to /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/mariadb/
        [00] 2026-02-24 16:24:39 open files limit requested 0, set to 1048576
        [00] 2026-02-24 16:24:39 mariabackup: using the following InnoDB configuration:
        [00] 2026-02-24 16:24:39 innodb_data_home_dir =
        [00] 2026-02-24 16:24:39 innodb_data_file_path = ibdata1:12M:autoextend
        [00] 2026-02-24 16:24:39 innodb_log_group_home_dir = ./
        2026-02-24 16:24:39 0 [Note] InnoDB: Number of transaction pools: 1
        2026-02-24 16:24:39 0 [Note] InnoDB: Using Linux native AIO
        2026-02-24 16:24:39 0 [Note] InnoDB: Buffered log writes (block size=512 bytes)
        [00] 2026-02-24 16:24:39 mariabackup: Generating a list of tablespaces
        [00] 2026-02-24 16:24:39 Connecting to MariaDB server host: localhost, user: root, password: not set, port: not set, socket: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/var/run/mariadb.sock
        [00] 2026-02-24 16:24:39 Connecting to MariaDB server host: localhost, user: root, password: not set, port: not set, socket: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/var/run/mariadb.sock
        [00] 2026-02-24 16:24:39 Starting BACKUP STAGE START
        [00] 2026-02-24 16:24:39 Acquiring BACKUP LOCKS...
        [00] 2026-02-24 16:24:39 Acquired locks for BACKUP STAGE START
        [00] 2026-02-24 16:24:39 Streaming /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/mariadb//aria_log_control to <STDOUT>
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Loading aria_log_control.
        [00] 2026-02-24 16:24:39 aria_log_control: last_log_number: 1
        [00] 2026-02-24 16:24:39 Start scanning aria tables.
        [00] 2026-02-24 16:24:39 Start scanning aria log files.
        [00] 2026-02-24 16:24:39 Found 1 aria log files, minimum log number 1, maximum log number 1
        [00] 2026-02-24 16:24:39 Stop scanning aria tables.
        [00] 2026-02-24 16:24:39 Streaming .//undo001
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming .//undo002
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming .//undo003
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming ./mysql/transaction_registry.ibd
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming ./mysql/gtid_slave_pos.ibd
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming ./mysql/innodb_table_stats.ibd
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming ./mysql/innodb_index_stats.ibd
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 Streaming ibdata1
        [00] 2026-02-24 16:24:39         ...done
        [00] 2026-02-24 16:24:39 aria table file ./mysql/func.MAI is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/func.MAD is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/global_priv.MAI is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/global_priv.MAD is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/roles_mapping.MAI is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/roles_mapping.MAD is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_name.MAI is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_name.MAD is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone.MAI is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone.MAD is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_transition.MAI is copied successfully.
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_transition.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_leap_second.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_leap_second.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/plugin.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/plugin.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/proxies_priv.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/proxies_priv.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/columns_priv.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/columns_priv.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/procs_priv.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/procs_priv.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/tables_priv.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/tables_priv.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/servers.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/servers.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/event.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/event.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_transition_type.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/time_zone_transition_type.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/db.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/db.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/proc.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/proc.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./sys/sys_config.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./sys/sys_config.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 Start copying aria log file tail: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/mariadb//aria_log.00000001
        [00] 2026-02-24 16:24:39 Stop copying aria log file tail: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/mariadb//aria_log.00000001, copied 6176768 bytes
        [00] 2026-02-24 16:24:39 Finished BACKUP STAGE START
        [00] 2026-02-24 16:24:39 Starting BACKUP STAGE FLUSH
        [00] 2026-02-24 16:24:39 Acquired locks for BACKUP STAGE FLUSH
        [00] 2026-02-24 16:24:39 Start scanning common engine tables, need backup locks: 0, collect log and stat tables: 1
        [00] 2026-02-24 16:24:39 Log table found: mysql.general_log
        [00] 2026-02-24 16:24:39 Collect log table file: ./mysql/general_log.CSV
        [00] 2026-02-24 16:24:39 Log table found: mysql.slow_log
        [00] 2026-02-24 16:24:39 Collect log table file: ./mysql/slow_log.CSV
        [00] 2026-02-24 16:24:39 Collect log table file: ./mysql/slow_log.CSM
        [00] 2026-02-24 16:24:39 Collect log table file: ./mysql/general_log.CSM
        [00] 2026-02-24 16:24:39 Stop scanning common engine tables
        [00] 2026-02-24 16:24:39 Start copying aria log file tail: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/mariadb//aria_log.00000001
        [00] 2026-02-24 16:24:39 Stop copying aria log file tail: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/mariadb//aria_log.00000001, copied 0 bytes
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_relation.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_relation.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_category.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_category.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_keyword.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_keyword.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_topic.MAI is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 aria table file ./mysql/help_topic.MAD is copied successfully.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 Finished BACKUP STAGE FLUSH
        [00] 2026-02-24 16:24:39 Started BACKUP STAGE BLOCK_DDL
        [00] 2026-02-24 16:24:39 Acquired locks for BACKUP STAGE BLOCK_DDL
        [00] 2026-02-24 16:24:39 Getting InnoDB LSN
        [00] 2026-02-24 16:24:39 InnoDB LSN: 45498, Flushing Logs
        [00] 2026-02-24 16:24:39 Flushed Logs
        [00] 2026-02-24 16:24:39 Start scanning common engine tables, need backup locks: 1, collect log and stat tables: 0
        [00] 2026-02-24 16:24:39 Stop scanning common engine tables
        [00] 2026-02-24 16:24:39 Starting to backup non-InnoDB tables and files
        [01] 2026-02-24 16:24:39 Streaming ./erp5/db.opt to <STDOUT>
        [01] 2026-02-24 16:24:39         ...done
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        [00] 2026-02-24 16:24:39 Copied file ./mysql/slow_log.CSV for log table `mysql`.`slow_log`, 0 bytes
        [01] 2026-02-24 16:24:39 Error: copy_file() failed.
        [00] 2026-02-24 16:24:39 Failed to copy file ./erp5/db.opt
        [00] 2026-02-24 16:24:39 Backup files error
        [00] 2026-02-24 16:24:39 Copied file ./mysql/general_log.CSV for log table `mysql`.`general_log`, 0 bytes
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        mariabackup: Stopping log copying thread.
        /srv/slapgrid/slappart15/srv/runner/shared/mariadb/89a353a5e657629be0c64f7fd582c293/bin/mariadb-backup: Error writing file 'UNKNOWN' (errno: 32 "Broken pipe")
        Checking backup...
        Doing incremental backup...
    
        zstd: /srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/m0/srv/backup/mariabackup/20260224162438.inc.xb.zstd already exists; stdin is an input - not proceeding.
    d018c3f7