* switch software/slapos-testing (`SlapOS.Eggs.UnitTest-*`) to use nxdtest and update README to describe how to run tests locally with nxdtest's runTestSuite. Switch tests to `python -m unittest` (or slight variations) instead of the deprecated `python setup.py`, except a few tests which only pass when invoked with `setup.py`.
 * switch software/slapos-sr-testing (`SlapOS.SoftwareReleases.IntegrationTest-*`) to use nxdtest and update README to describe how to run tests locally with nxdtest's runTestSuite. Switch all tests to `python -m unittest discover` intead of the deprecated `python setup.py` - which was especially problematic in these tests, because it sets `$PYTHONPATH` which affect subprocesses ran by the tests. In follow up commits we'll drop the workarounds for `$PYTHONPATH`.

`EggTestSuite` becomes unused and will be removed in follow up changes.

See merge request nexedi/slapos!862

 * use python3 (newest builds requires python3)
 * correctly exclude release directories during rsync so that resiliency
works
 * use correct names for project/software/instances

We also try to switch the tests to use python -m unittest as a way of
invoking tests, instead of the deprecated python setup.py test

When use this interpreter to run buildout, buildout will pre-compile eggs with
different level of optimisations, for this it spawns sys.executable with -O
option (see [1]). Since pymain does not support -O argument and python only
support changing optimization level at startup, we can not easily handle -O
argument, execpt by re-executing ourselves with PYTHONOPTIMIZE environment
variable, which has same effect than setting -O flag

1: https://lab.nexedi.com/nexedi/slapos.buildout/blob/488b2dfd3088cba7b322827c904fe1f93fba73e6/src/zc/buildout/easy_install.py#L1716-1722

Change to run all tests with python -m unittest, which will allow us to
remove workaround for the $PYTHONPATH set by setup.py test.

For this we use `[python-interpreter]` which uses pygolang.main to behave
like a "normal" python interpreter with eggs available, but without the
usual problems of interpreters generated by zc.recipe.eggs - it supports
all command line flags and have current directory in sys.path.

Tests now have an explicit name, that we set to the software name as before,
and the special test dynamically checking all profiles json schemas (part
of slapos.cookbook egg, at the root of slapos repository),is named
json-schemas. It is also changed to run as default tests in python2 and
python3, not only in extras for python2.

fixes "component/trafficserver: Version up to 8.1.0"

Jérome reports that Pygolang build on Python3 fails because geventmp turned out to be not pinned:

nexedi/slapos!862 (comment 121359)

Geventmp is indeed used by pygolang only on Python3

nexedi/pygolang@cd67996e

and with Python2-only testing I missed to handle it.

-> Fix it and add test for pygolang/py3.

/reviewed-on nexedi/slapos!864

See merge request !856

Globbing can be unreliable after incremental profile update and rebuild because
buildout does not remove old eggs. Also it is much more clean to be precise.

-> Use [python-interpreter] to build bin/python with sys.path to wendelin.core
and other needed eggs.

Without referencing component/numpy/ numpy will be installed from just PyPI.

Go1.15 is incremental improvement over Go1.14 with better compiler and runtime:

https://blog.golang.org/go1.15
https://golang.org/doc/go1.15

Don't drop support for Go1.13 yet, as that (no longer supported) Go
release is still being used by replication-manager and restic
components.

Remain default at Go1.14 yet.

Switch helloworld to Go1.15 and test this patch on that
software-release.

Going Go1.14.10 -> Go1.14.12 brings in runtime and security fixes:

https://golang.org/doc/devel/release.html#go1.14

Tested on helloworld SR.

Tests added:
- connexion parameters
- monitor url
- slaves instanciation

Adds connexion parameters:
- ipv6
- port
- ipv4

Upgrade dnspython from 1.15.0 to 1.16.0

Removes unused parameters in instance-powerdns-replicate.cfg.jinja:
- private-ipv4
- public-ipv4
- domain

See merge request nexedi/slapos!860

When invoking tests with setup.py test, missing eggs are installed before
running tests, using their latest available version, but we want to always
install eggs using buildout and using pinned versions.

The section template-slapuser-script creates the "bin/slapos" wrapper
which is needed inside runner-import.sh script.

Otherwise it's almost impossible to check for which node statistics are shown.

Fixes 8d5910dc

Limit stats to frontend entries only, as they are the most important for the
frontend operator, and also having thousands of entries makes the stats page
unusable.

fixes 8d5910dc

Despite the introspection has replaced old style Caddy-based log-access, some
bits were not removed, so remove then now.

fixes 0c830c4c

It was done in frontend partition, but missed in kedifa.

fixes 3469e864

This reverts commit 5d46a499.

Changes to "fix argument parsing" (to allow passing argument containing space)
had a regression that backslash characters are escaped.

Let's go back to slapos.toolbox 0.112 for now.

When updating existing instance, running slapos node instance failed with:

    ln: failed to create symbolic link '/srv/slapgrid/slappart9/srv/frontend-static/logo.png': File exists

This fixes by doing nothing if logo.png already exists

In nexedi/slapos@74d18b9d I switched ERP5 to use caucase certificate, but caucase is not yet usable for this and this caused issues with upgrading existing instances.

We'll maybe try caucase again here, in that case 620c9332 can be reverted if we want to reuse some parts of what was made.

See merge request nexedi/slapos!857

Revert "software/erp5: use a caucase managed certificate for balancer"

This reverts commit 74d18b9d and also follow
up fixup ( 555b26a2 ).

We are not ready to use caucase here, there are still too many problems with
caucase (keys are lost at each SR update etc) and design might still evolve,
so let's go back to self signed certificate for now.

Also remove the promise and the updater, since they are also not working and
causing problems on instances that have been updated (and where the key no
longer match the certificate)

fix balancer CSR generation:

Caucase rerequest uses a CSR *template* and use it to generate
a new CSR with a new key, so we should not use the actual key to
generate this CSR, because it is caucase rerequest job to generate
the key.
Also, we should be careful not to generate a new CSR every time this
command run, otherwise a new key will be generated and a new CSR will
be sent to caucase, but caucase will not sign it automatically (since
we configure it to sign only one certificate).

This means that the case of IP address changes is currently not
supported automatically. To support it we would need to:
  - force generation of a new CSR template
  - force caucase rerequester to request a new certificate (by removing
  existing certificate)
  - force caucased to sign the new certificate

This commit also fix indentation and remove simplefile macro that is no longer used

See merge request nexedi/slapos!837

Just running test.py with ip and port allows to expose the internal testing
backend. IPv4 and IPv6 are supported.

By default there is no sense to play with software type, since it's fixed in
slapos.testing.testcase.

slapos.cookbook:softwaretype tends to lowercase keys in each section, which
has been undetected due to using lowercase references of slaves in the tests.

By restructuring information in the sections, and putting slave references
inside of dumped part of information, now the slave reference case is kept.

Also real care was taken to stabilise published lists by sorting them, as it
also slipped, that they could be unstable.

Tests has been updated to catch this issue, also other tests were fixed, as
they had wrong assertions.