erp5: new `test-runner` instance parameter

We have new parameters "how many parallel tests this instance can run"
( test-runner.node-count ) and "how many supplementary databases each
test needs" ( test-runner.extra-database-count ).

Instead of configuring "I need a total of 15 databases", we configure "I
need to be able to run 3 tests in parallel, each test requiring 4 extra
databases" which generates 3 * (4 +1) databases.

software/erp5/README.rst

@@ -107,5 +107,27 @@ should be enough room for evolution (as between smtp and mariadb types). It is
 important to not allocate any port after 2200 as user may have assigned ports
 to his zope processes.
 
+
+Running test suite
+==================
+
+All zope partitions will contain ``runTestSuite`` and ``runUnitTest`` script, unless the
+instance was requested with ``test-runner.enabled`` set to false.
+
+The following scripts will be created:
+
+* ``runUnitTest`` runs one test in a temporary instance. Developers can use this script
+to easily run one individual test.
+* ``runTestSuite`` runs a full test suite, optionally by running several tests in parallel.
+This will be used by the test infrastructure to run all ERP5 tests.
+The instance needs to set a value for ``test-runner.node-count`` large enough for
+the test node.
+
+While most of ERP5 tests use the default mariadb database, some tests needs more databases
+connections, for example to test the use of an external datawarehouse or test
+`hot reindexing` features of ERP5. When requesting, set ``extra-database-count`` to the
+number of extra database connections needed for the tests.
+
+
 .. _RewriteRules: http://httpd.apache.org/docs/current/en/mod/mod_rewrite.html#rewriterule
-.. _VirtualHostMonster: http://docs.zope.org/zope2/zope2book/VirtualHosting.html
\ No newline at end of file
+.. _VirtualHostMonster: http://docs.zope.org/zope2/zope2book/VirtualHosting.html

software/erp5/instance-erp5-input-schema.json

@@ -345,5 +345,26 @@
       },
       "type": "object"
     }
+  },
+  "test-runner": {
+    "description": "Test runner parameters.",
+    "properties": {
+      "enabled": {
+        "description": "Generate helper scripts to run test suite.",
+        "default": true,
+        "type": "boolean"
+      },
+      "node-count": {
+        "description": "Number of tests this instance can execute in parrallel.",
+        "default": 3,
+        "type": "integer"
+      },
+      "extra-database-count": {
+        "description": "Number of extra databases this instance tests will need.",
+        "default": 3,
+        "type": "integer"
+      }
+    },
+    "type": "object"
   }
 }

software/erp5/instance-mariadb-schema.json

@@ -49,7 +49,7 @@
       "type": "array"
     },
     "test-database-amount": {
-      "description": "The number of test databases to create, adding auto-generated entries to database-list",
+      "description": "The number of test databases to create, adding auto-generated entries to database-list. Deprecated, use testrunner.node-count and testrunner.extra-database-count",
       "default": 1,
       "minimum": 0,
       "type": "integer"

stack/erp5/buildout.hash.cfg

@@ -74,7 +74,7 @@ md5sum = d41d8cd98f00b204e9800998ecf8427e
 
 [template-erp5]
 filename = instance-erp5.cfg.in
-md5sum = 83e0f627633ec3e394dc27d06c627b3b
+md5sum = 7b1dbfe4f2929294e5bcfa1333348825
 
 [template-zeo]
 filename = instance-zeo.cfg.in
@@ -82,7 +82,7 @@ md5sum = 3e650915959ff31c9c13c84069bbcd35
 
 [template-zope]
 filename = instance-zope.cfg.in
-md5sum = e73678921067506e710ae11e41f0a9a8
+md5sum = 76266ced0d35d30c26ef2f563c2256c3
 
 [template-balancer]
 filename = instance-balancer.cfg.in

stack/erp5/instance-erp5.cfg.in

@@ -8,6 +8,16 @@
 {% set jupyter_dict = slapparameter_dict.get('jupyter', {}) -%}
 {% set has_jupyter = jupyter_dict.get('enable', jupyter_enable_default.lower() in ('true', 'yes')) -%}
 {% set jupyter_zope_family = jupyter_dict.get('zope-family', '') -%}
+{% set test_runner_enabled = slapparameter_dict.get('test-runner', {}).get('enabled', True) -%}
+{% set test_runner_node_count = slapparameter_dict.get('test-runner', {}).get('node-count', 3) -%}
+{% set test_runner_extra_database_count = slapparameter_dict.get('test-runner', {}).get('extra-database-count', 3) -%}
+{% set test_runner_total_database_count = test_runner_node_count * (1 + test_runner_extra_database_count) -%}
+{# Backward compatibility for mariadb.test-database-amount #}
+{% set mariadb_test_database_amount = slapparameter_dict.get('mariadb', {}).get('test-database-amount') -%}
+{% if mariadb_test_database_amount is not none -%}
+{%   set test_runner_total_database_count = mariadb_test_database_amount %}
+{%   set test_runner_enabled = mariadb_test_database_amount > 0 %}
+{% endif -%}
 {% set monitor_base_url_dict = {} -%}
 {% set caucase_url = slapparameter_dict.get('caucase', {}).pop('url', '') -%}
 {% set monitor_dict = slapparameter_dict.get('monitor', {}) %}
@@ -44,7 +54,7 @@ config-name = {{ name }}
 
 {{ request('memcached-persistent', 'kumofs', 'kumofs', {'tcpv4-port': 2000}, {'url': True, 'monitor-base-url': False}, key_config={'monitor-passwd': 'monitor-htpasswd:passwd'}) }}
 {{ request('memcached-volatile', 'kumofs', 'memcached', {'tcpv4-port': 2010, 'ram-storage-size': 64}, {'url': True, 'monitor-base-url': False}, key_config={'monitor-passwd': 'monitor-htpasswd:passwd'}) }}
-{{ request('mariadb', 'mariadb', 'mariadb', {'tcpv4-port': 2099, 'max-slowqueries-threshold': monitor_dict.get('max-slowqueries-threshold', 1000), 'slowest-query-threshold': monitor_dict.get('slowest-query-threshold', '') }, {'database-list': True, 'test-database-list': True, 'monitor-base-url': False}, key_config={'monitor-passwd': 'monitor-htpasswd:passwd'}) }}
+{{ request('mariadb', 'mariadb', 'mariadb', {'tcpv4-port': 2099, 'max-slowqueries-threshold': monitor_dict.get('max-slowqueries-threshold', 1000), 'slowest-query-threshold': monitor_dict.get('slowest-query-threshold', ''), 'test-database-amount': test_runner_total_database_count}, {'database-list': True, 'test-database-list': True, 'monitor-base-url': False}, key_config={'monitor-passwd': 'monitor-htpasswd:passwd'}) }}
 {% if has_posftix -%}
 {{   request('smtp', 'postfix', 'smtp', {'tcpv4-port': 2025, 'smtpd-sasl-user': 'erp5@nowhere'}, key_config={'smtpd-sasl-password': 'publish-early:smtpd-sasl-password'}) }}
 {%- else %}
@@ -169,6 +179,7 @@ config-timezone = {{ dumps(slapparameter_dict.get('timezone', 'UTC')) }}
 config-cloudooo-retry-count = {{ slapparameter_dict.get('cloudooo-retry-count', 2) }}
 config-wendelin-core-zblk-fmt = {{ dumps(slapparameter_dict.get('wendelin-core-zblk-fmt', '')) }}
 config-zodb-dict = {{ dumps(zodb_dict) }}
+config-test-runner-enabled = {{ dumps(test_runner_enabled) }}
 {% for server_type, server_dict in storage_dict.iteritems() -%}
 {%   if server_type == 'neo' -%}
 config-neo-cluster = ${publish-early:neo-cluster}

stack/erp5/instance-zope.cfg.in

@@ -8,6 +8,7 @@
 {% set node_id_index_format = '-%%0%ii' % (len(str(instance_index_list[-1])), ) -%}
 {% set part_list = [] -%}
 {% set publish_list = [] -%}
+{% set test_runner_enabled = slapparameter_dict['test-runner-enabled'] -%}
 {% set longrequest_logger_base_path = buildout_directory ~ '/var/log/longrequest_logger_' -%}
 {% macro section(name) %}{% do part_list.append(name) %}{{ name }}{% endmacro -%}
 {% set bin_directory = parameter_dict['buildout-bin-directory'] -%}
@@ -87,7 +88,7 @@ context =
     key password test-zelenium-runner-parameter:password
     key bin_path test-zelenium-runner-parameter:bin-path
 {% else -%}
-{% if slapparameter_dict['mysql-test-url-list'] -%}
+{% if test_runner_enabled -%}
 [{{ section('run-unit-test-userhosts-wrapper') }}]
 <= userhosts-wrapper-base
 wrapped-command-line = ${runUnitTest:wrapper-path}

[Vincent Pelletier]

I'm not a fan of this syntax to describe a parameter.

Also, I do not see what this README section brings that couldn't be brought by the schema. This file is not intended to repeat everything there is in the schema, only for things which do not fit in the schema.

[Jérome Perrin]

I'm not a fan of this syntax to describe a parameter.

you mean this syntax of using . to represent the hierarchy of the json ?

I agree that this documentation is not really needed. I'll drop it and add a bit more directly in schema.

[Jérome Perrin]

I'll describe here that this must >= of what's configured on the test node.

[Vincent Pelletier]

I would rather temove this from the template entirely, but implement the fallback.

[Jérome Perrin]

Good, I'll do this.