The test would take really long if it was run without --mem.
Fixed by adding --innodb-flush-log-at-trx-commit=2