Having only one section [wendelin.core] instead of [wendelin.core] and
[wendelin.core-dev] is easier to handle in "we want to use such and such
particular version" scenarious without deciding in advance whether an SR needs
to inherit from wendelin/software.cfg or wendelin/software-dev.cfg

/reviewed-on !874

we no longer run tests with `python setup.py test`, so this hack is no
longer necessary.

in ceb063a4 (software/repman: tests python3 compliance, 2020-10-23), test
profile was changed to run on python3 only but this was accidentally
reverted in 3a1e5628 (software/slapos-sr-testing: use nxdtest, 2020-10-28),
by running repman also on python2.

Restore the situation where we run repman only on python3

The initial intent was to use an interpreter named `python_for_test` and not
`python`, so that software do not accidentally pick up this python during
compilation.
6033e4fa (software/slapos-sr-testing: fix python3 profile, 2020-12-01)
was a quick fix for some python3 compatibility issues discovered after merge,
but it was not correct, because by using ${python-interpreter:eggs}, it was
installing the default python-interpreter section, which has an interpreter
named `python`.

This caused issues while building proxysql, which assumes /usr/bin/env python
is python2.

The fix is to override python-interpreter directly, we don't need another
section eggs here.

It got broken in 3a1e5628 (software/slapos-sr-testing: use nxdtest, 2020-10-28)

This is to match Theia file hierarchy

https://github.com/eclipse-theia/theia/blob/v1.8.0/CHANGELOG.md#v180---26112020

Until now, standalone subsystem was started as a daemon first time a terminal
was openned and since it was running as daemon, stopping the theia instance
did not stop any of the services running in the embedded slapos.

Before slapos.core!265 there was
two supervisor running as daemon:
 - ~/srv/slapos/etc/supervisord.conf which runs slapos proxy etc
 - ~/srv/slapos/inst/etc/supervisord.conf with runs instances in the embedded
   slapos
After, the second one runs as a service in the first one, but the first one
was still running as daemon.

This changes so that the first supervisor runs as a service managed by the
Theia instance, so stopping Theia instance will effectively stop the services.

When upgrading, running instances should continue to run as detached. To
attach them to the new service, procedure could be something like this

Stop supervisors inside Theia instance:

  supervisorctl -c ~/srv/slapos/etc/supervisord.conf shutdown
  supervisorctl -c ~/srv/slapos/inst/etc/supervisord.conf shutdown

Restart slappartX:slapos-standalone-instance-XXX-on-watch from host slapos

We are observing some segmentation fault with python curses applications
that might be because the TERMINFO are different from the system one.
I did not debugged, but since I set this environment variable I did not
observe any segmentation fault.

By comparing strace, the invocations seems same with or without $TERMINFO.
In both cases the termcaps from the correct ncurses are selected, this
just seem to workaround for some reason.

 * use python3 (newest builds requires python3)
 * correctly exclude release directories during rsync so that resiliency
works
 * use correct names for project/software/instances

We also try to switch the tests to use python -m unittest as a way of
invoking tests, instead of the deprecated python setup.py test

Change to run all tests with python -m unittest, which will allow us to
remove workaround for the $PYTHONPATH set by setup.py test.

For this we use `[python-interpreter]` which uses pygolang.main to behave
like a "normal" python interpreter with eggs available, but without the
usual problems of interpreters generated by zc.recipe.eggs - it supports
all command line flags and have current directory in sys.path.

Tests now have an explicit name, that we set to the software name as before,
and the special test dynamically checking all profiles json schemas (part
of slapos.cookbook egg, at the root of slapos repository),is named
json-schemas. It is also changed to run as default tests in python2 and
python3, not only in extras for python2.

Globbing can be unreliable after incremental profile update and rebuild because
buildout does not remove old eggs. Also it is much more clean to be precise.

-> Use [python-interpreter] to build bin/python with sys.path to wendelin.core
and other needed eggs.

Tests added:
- connexion parameters
- monitor url
- slaves instanciation

Adds connexion parameters:
- ipv6
- port
- ipv4

Upgrade dnspython from 1.15.0 to 1.16.0

Removes unused parameters in instance-powerdns-replicate.cfg.jinja:
- private-ipv4
- public-ipv4
- domain

When invoking tests with setup.py test, missing eggs are installed before
running tests, using their latest available version, but we want to always
install eggs using buildout and using pinned versions.

The section template-slapuser-script creates the "bin/slapos" wrapper
which is needed inside runner-import.sh script.

Otherwise it's almost impossible to check for which node statistics are shown.

Fixes 8d5910dc

Limit stats to frontend entries only, as they are the most important for the
frontend operator, and also having thousands of entries makes the stats page
unusable.

fixes 8d5910dc

Despite the introspection has replaced old style Caddy-based log-access, some
bits were not removed, so remove then now.

fixes 0c830c4c

It was done in frontend partition, but missed in kedifa.

fixes 3469e864

When updating existing instance, running slapos node instance failed with:

    ln: failed to create symbolic link '/srv/slapgrid/slappart9/srv/frontend-static/logo.png': File exists

This fixes by doing nothing if logo.png already exists

Revert "software/erp5: use a caucase managed certificate for balancer"

This reverts commit 74d18b9d and also follow
up fixup ( 555b26a2 ).

We are not ready to use caucase here, there are still too many problems with
caucase (keys are lost at each SR update etc) and design might still evolve,
so let's go back to self signed certificate for now.

Also remove the promise and the updater, since they are also not working and
causing problems on instances that have been updated (and where the key no
longer match the certificate)

Just running test.py with ip and port allows to expose the internal testing
backend. IPv4 and IPv6 are supported.

By default there is no sense to play with software type, since it's fixed in
slapos.testing.testcase.

slapos.cookbook:softwaretype tends to lowercase keys in each section, which
has been undetected due to using lowercase references of slaves in the tests.

By restructuring information in the sections, and putting slave references
inside of dumped part of information, now the slave reference case is kept.

Also real care was taken to stabilise published lists by sorting them, as it
also slipped, that they could be unstable.

Tests has been updated to catch this issue, also other tests were fixed, as
they had wrong assertions.

See commit e83edfc9.

This reverts commit cc1713c3.

Unfortunately a lot of installation OS ISOs (like Debian 9 and 10) do not
support such device during installation process, because of missing drivers
on the OS CD.

Also note the ISOs limitation in the advanced field.