Before this patch all ERP5 SlapOS Integration tests only run with ZEO storage. We should also run them against NEO, because we are using ERP5 with NEO in SlapOS.

For tests which shouldn't be split into NEO/ZEO flavoured test cases, this patch adds the '__parameterized__' class attribute, which can be set to False (default to True).

This is a follow-up MR coming from !1283 (closed).

It's an implementation of the draft solution proposed in note !1283 (comment 172107).

Tests are pending. I'll post results once they are done.

I added some explanations regarding some technical details there:

Implementation without parameterized

In my proposed solution I decided to not use parameterized, but to implement the needed functionality by myself. I have the following reasons for this decision:

1. parameterized doesn't hide inherited tests

In order to hide tests of the original class which becomes parameterized, the parameterized package uses the delattr function. This won't remove tests which are inherited from a parent. This could be problematic in case the user applies a Mixin logic - as we do it. For instance consider the following code:

import unittest
from parameterized import parameterized_class
                                          
class TestMixin(object):                                
    def test_a(self):                                   
        self.assertEqual(self.x, self.x)

@parameterized_class(("x",), [(1,), ("a",)])
class ParameterizedTest(unittest.TestCase, TestMixin): ...  

Running this code will raise an error:

bash-5.1$ python -m unittest discover -v
test_a (tests.ParameterizedTest) ... ERROR
test_a (tests.ParameterizedTest_0) ... ok
test_a (tests.ParameterizedTest_1_a) ... ok

======================================================================
ERROR: test_a (tests.ParameterizedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/levinericzimmermann/Projects/Nexedi/patchtests/parameterized-tests/tests/__init__.py", line 8, in test_a
    self.assertEqual(self.x, self.x)
AttributeError: 'ParameterizedTest' object has no attribute 'x'

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (errors=1)

Although the above example doesn't happen with our current code base, it could happen. For now, we have tests which would run multiple times with the same parameters.

2. The parameterized package may not be actively maintained anymore.

Please see the following issue.

3. Implementing the provided functionality ourselves is small and avoids an additional dependency.

It more or less only encompasses the following lines:

1. https://lab.nexedi.com/levin.zimmermann/slapos/blob/09794d1dff92ee7e79c5e05c2cf3a7a80e8fa7c0/software/erp5/test/test/__init__.py#L65
2. https://lab.nexedi.com/levin.zimmermann/slapos/blob/09794d1dff92ee7e79c5e05c2cf3a7a80e8fa7c0/software/erp5/test/test/__init__.py#L69-70
3. https://lab.nexedi.com/levin.zimmermann/slapos/blob/09794d1dff92ee7e79c5e05c2cf3a7a80e8fa7c0/software/erp5/test/test/__init__.py#L80-81
4. https://lab.nexedi.com/levin.zimmermann/slapos/blob/09794d1dff92ee7e79c5e05c2cf3a7a80e8fa7c0/software/erp5/test/test/__init__.py#L88-91
5. https://lab.nexedi.com/levin.zimmermann/slapos/blob/09794d1dff92ee7e79c5e05c2cf3a7a80e8fa7c0/software/erp5/test/test/__init__.py#L93-94

---

The disadvantage in not using parameterized may be that we actually need tests to verify that our metaclass works as expected.

\cc @kirr \cc @jerome