WIP: run tests wit zope.testrunner (!1695) · Merge requests · nexedi / slapos

WIP: run tests wit zope.testrunner

testing with to list tests

  cd software/erp5/test

  env PYTHONNOUSERSITE=1 \
      MAKEFLAGS=-j128 \
      SLAPOS_SR_TEST_NAME=erp5-py3 \
      SLAPOS_TEST_SKIP_SOFTWARE_REBUILD=1 \
      SLAPOS_TEST_VERBOSE=0 \
      SLAPOS_TEST_SKIP_SOFTWARE_CHECK=1 \
      SLAPOS_TEST_DEBUG=0 \
      PYTHONWARNINGS=ignore \
      zope-testrunner  --color -vD --path . --tests-pattern test --list-tests

and to run :

  env PYTHONNOUSERSITE=1 \
      MAKEFLAGS=-j128 \
      SLAPOS_SR_TEST_NAME=erp5-py3 \
      SLAPOS_TEST_SKIP_SOFTWARE_REBUILD=1 \
      SLAPOS_TEST_VERBOSE=0 \
      SLAPOS_TEST_SKIP_SOFTWARE_CHECK=1 \
      SLAPOS_TEST_DEBUG=0 \
      PYTHONWARNINGS=ignore \
      zope-testrunner  --color -vD --path . --tests-pattern test

problems:

zope.testrunner could support setupModule / setupClass directly (or at least the workaround could be put directly in slapos.core testing on the default SlapOSInstanceTestCase)
zope.testrunner -D trigger debuggers on expected failures (probably similar to https://github.com/zopefoundation/zope.testrunner/issues/141 )
hitting Ctrl+C seems "even worse" than default unit test -> we can probably register a signal handler in setupClass to stop slapos standalone subsystem ourselves.
nxdtest probably needs support for good reporting on test_result_module

zope.testrunner