See merge request nexedi/erp5!1445

See merge request nexedi/erp5!1447

See merge request nexedi/erp5!1437

 * expose firefox console and error logs in `geckodriver.log`
 * expose `geckodriver.log` in the folder testnodes expose on http
 * fix some cases where test were detected as "did not run" and cases where "did not run" tests were not reported as failures

See merge request nexedi/erp5!1444

We are already closing Storage on shutdown (see nearby Storage.close call in
the patch), but the DB handle was not closed. With classic ZODB it does not
really matter in practice, because not closing DB is only leaking RAM resources
and the program is anyway terminated soon.

However with wendelin.core 2 things are different: in addition to ZODB storage
server, there is also synthetic WCFS filesystem from which files are opened
and memory-mmaped. In runUnitTest we start both ZODB and WCFS servers and we
also shut down them both in the end. The filesystem server can be cleanly
unmounted and shutdown only when there are no opened files left on it.

Wendelin.core 2 client works by complementing each ZODB connection (zconn) with
WCFS-level connection (wconn) to WCFS server. Those two zconn and wconn are
kept in sync by wendelin.core 2 client logic: whenever zconn adjusts its view
of the database, so does wconn. And whenever zconn is garbage collected, so
does wconn is closed to release resources and close corresponding files opened
on WCFS. In addition to garbage-collection, wconn is also closed when zconn.db
- the ZODB DB handle via which zconn was created - is closed. This is needed to
be able to reliably trigger freeing WCFS resources, because even after DB is
closed, zconn can still stay alive forever being referenced from some python
object - e.g. a frame or traceback or something else.

The latter scenario actually happens during runUnitTest run. As the result it
leads to inability to unmount and stop WCFS server cleanly:

    $ ./runUnitTest --load --save --with_wendelin_core -v erp5_core_test:testWendelinCore
    ...
    test (erp5.component.test.erp5_version.testWendelinCore.TestWendelinCoreBasic) ... ok

    ----------------------------------------------------------------------
    Ran 1 test in 0.105s

    OK
    F0618 19:05:46.359140   35468 wcfs/client/wcfs.cpp:486] CRITICAL: pinner: wcfs /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: wlink25: recvReq: link is down
    F0618 19:05:46.359173   35468 wcfs/client/wcfs.cpp:487] CRITICAL: wcfs server will likely kill us soon.
    CRITICAL: pinner: wcfs /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: wlink25: recvReq: link is down
    CRITICAL: wcfs server will likely kill us soon.
    Traceback (most recent call last):
      File ".../bin/runUnitTest", line 312, in <module>
        sys.exit(runUnitTest.main())
      File ".../parts/erp5/Products/ERP5Type/tests/runUnitTest.py", line 926, in main
        run_only=run_only,
      File ".../parts/erp5/Products/ERP5Type/tests/runUnitTest.py", line 709, in runUnitTestList
        wcfs_server.stop()
      ...
      File ".../parts/wendelin.core/wcfs/__init__.py", line 543, in _fuse_unmount
        raise RuntimeError("%s\n(more details logged)" % emsg)
    RuntimeError: fuse_unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: failed: fusermount: failed to unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: Device or resource busy
    (more details logged)

    # logs
    2021-06-18 19:05:45.978 INFO root wcfs: unmount/stop wcfs pid32981 @ /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2
    2021-06-18 19:05:46.068 WARNING root fuse_unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: failed: fusermount: failed to unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: Device or resource busy
    2021-06-18 19:05:46.068 WARNING root # lsof /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2
    2021-06-18 19:05:46.357 WARNING root COMMAND     PID       USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
    runUnitTe 32175 slapuser34   24r   REG   0,48      111    4 /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2/.wcfs/zurl
    runUnitTe 32175 slapuser34   25u   REG   0,48        0    7 /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2/head/watch
    runUnitTe 32175 slapuser34   26r   REG   0,48  2097152    9 /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2/head/bigfile/00000000000078b4

    2021-06-18 19:05:46.358 WARNING root -> kill -TERM wcfs.go ...
    2021-06-18 19:05:46.358 WARNING root -> abort FUSE connection ...

I've debugged things a bit and even with
kirr/ZODB@bbd03b3a ZODB connection stays alive
being referenced from some frame objects.

-> Fix this problem by explicitly closing ZODB DB nearby tests shutdown before call to wcfs_server.stop

This is not really for errors during the test itself, but for errors when
ERP5TypeFunctionalTestCase manipulates browser to execute selenium test.

This depends on the support in slapos nexedi/slapos!999

It can happen that a test running selenium fail in python, but not in selenium,
like for example test_result_module/20210615-CDADEC14/183
To prevent such tests from being reported as PASS, we make the total number of
failures being the sum of the python unittest failures and the selenium failures.

It happens, for example with erp5_officejs_ui_test:testFunctionalOfficeJSPyodideNotebook
on test_result_module/20210615-CDADEC14/183 that the test is considered as
not running, because the test result table was not populated in time.

Add an explicit wait to prevent such problems.

This fixes commit ea4cb55b.

Our 'rmtree' helper has never been supposed to be called on a
non-existing folder: the ENOENT handling is an unrelated
implementation detail for Python 2.

Also drop erp5.util's rmtree and use the one from slapos.core,
which is a [testnode] dependency.

This front page gadget allows user to:
* create any kind of document
* launch any kind of report
* contribute a new documents

This gadget can be used to replicate the xhtml access page.

This is follow-up to 5796a17a (core_test: Add test to make sure that
wendelin.core basically works; nexedi/erp5!1429).

In that commit it was said that testWendelinCore

    "currently passes with wendelin.core 1, which is the default.
     It also passes as live test with wendelin.core 2.
     However with wendelin.core 2 it currently fails when run on testnodes
     ...
     because we need to amend ERP5 test driver

     1. to run tests on a real storage instead of in-RAM Mapping Storage(*), and
     2. to spawn WCFS server for each such storage."

This patch addresses that latter problem to run testWendelinCore under
testnode infrastructure.

@rafael and @jerome suggested that we can force a test to be run on a
real storage via `runUnitTest --load --save` or via `--activity_node=n`.

@rafael also suggested not to generally change the testing driver, but instead
make step-by-step progress and first tag each test that uses wendelin.core with an
option. Let's go this way now: runUnitTest/custom_zodb are taught to launch
WCFS server if wendelin.core usage is requested and software is built with
wendelin.core 2.

With both changes combined testWendelinCore should now pass OK when run
on a testnode with both wendelin.core 1 and wendelin.core 2.

This patch is based on a draft patch by @rafael: https://lab.nexedi.com/rafael/erp5/commit/14e3a777.

This patch also relies on recent wendelin.core 2 wcfs.py rework which
exposed functionality to start WCFS server and to further control it:
kirr/wendelin.core@5bfa8cf8.

/cc @tomo, @romain, @jerome, @seb

Eventhough _getBudgetDict was cached, getAvailableBudgetDict also
calculates getCurrentBudget for each cell, which is also heavy.

Duplicate actions are actions from the same category with the same name.

See merge request !1442

With software release test suites, there are probably
files/folders with permission issues.

See commit 380037f8.

so that we can access this log directly via http

This firefox setting cause console.log messages or javascript errors to be
printed on stdout, which should help diagnosing failures sometimes.

See merge request !1439

By setting XORG_LOCK_DIR, we can isolate processes when running on slapos test
nodes, so that instead of having all test runners used the shared /tmp, which
eventually fail with

    EnvironmentError: All displays locked : [':123', ':124', ':125', ...

errors when too many testnodes are currently running or too many testnodes have
left some stale lock files. Eventhough since fbdb4ec9 (test: Do not check
presence of lockfile in /tmp/ to assume Xvfb is running for this display, 2017-06-07)
things have improved, it happens that the lock files are owned by a different
unix user (because it was left by another testnode), then Xvfb is not able to
remove the stale lock file.

This is backward compatibility code for previous commit.

actions in erp5_jio_view category does not seem to be directly presented
to user, so rename them so that they don't use the same name "View" for
coding style test_DuplicateActions

Duplicate actions are actions from the same category with the same name.

`inst/test0-0`  ->  `i/0`

See merge request nexedi/erp5!1441