use "latest" in generate_download_plugins_cfg.py, so that re-running this
script updates plugins to their latest versions.

slapos.recipe.build verifies signatures of downloaded files and can also use
buildout cache or download from shacache, which

This introduces a new download-plugins.cfg, containing the URLs of
extensions and their md5sum signatures. This profile is extended by
software.cfg and there's a section using slapos.recipe.build to actually
download extensions.

A generate_download_plugins_cfg.py python script is also included, to be
run by software release maintainers to update download-plugins.cfg using the
list of plugins included in generate_download_plugins_cfg.py

Update extensions to newer version and to the URLs from openvsx instead of
github, since openvsx prevent maintainers to modify releases.

add vscode-references-view (
https://open-vsx.org/extension/ms-vscode/references-view ) it's installed
in default Theia and looks good.

Some java extensions were not on openvsx, they were removed or updated.
Newer java extension seem to associate .cfg as "properties" files, so force a
mapping to make them open as zc-buildout.

python was also updated, but not to latest version which does not work on
Theia.

Fixes compatibility with glibc >= 2.30 .
Update users to new usage pattern.

It's much easier to find the slave by it's reference than by it's title for
the cluster administrator, so use the reference in this case.

Fixes "caddy-frontend: Add promise for rejected slaves", even if title seemed
more human readable, but it's unusable.

Since haproxy does non-HTTP style checks to the python provided backend it
started to emit a lot of logs about broken pipe, which are not important
in context of testing and shall be silenced.

See https://en.wiktionary.org/wiki/instanciated :

> Misspelling of instantiated.

See merge request nexedi/slapos!858

See merge request nexedi/slapos!863

Allow each instance to support a zone list instead of a single zone.
Each slave is set for only one defined zone.

While caucase-updater aibcc was added, it was copy & pasted from aikc, and
by mistake it shared data_dir, which results with not working update of
certificates.

This fixes this issue introduced in "caddy-frontend: Setup backend client auth"

We are starting to use wendelin.core not only in Wendelin context.
So it makes sense to have it installed in base ERP5, like we already do for
example for NumPy and SciPy.

/reviewed-by @rafael, @jp
/reviewed-on nexedi/slapos!874

Having only one section [wendelin.core] instead of [wendelin.core] and
[wendelin.core-dev] is easier to handle in "we want to use such and such
particular version" scenarious without deciding in advance whether an SR needs
to inherit from wendelin/software.cfg or wendelin/software-dev.cfg

/reviewed-on nexedi/slapos!874

haproxy can be controlled with this socket, so it might be useful
to "expose" it - it's not really expose because we only use a
UNIX socket.

The apachedex reports when produced on backend will be wrong, because haproxy
logs timings in milliseconds and apachedex parses as microsecond, but as far as
I know we produce reports from frontend logs, so it should not really affect
our operations.

Two main differences of haproxy are file format for certificates and logs.

HAProxy also uses certificates in PEM format, but it expect its own server
certificate and the key to be in the same file (although recent version seems
to accept separate files, we don't use this now) and the CRL and CA certificates
also all together in the same file.
We change to use the same file for certificate and key and for CA and CRL, in
the updater script we we build PEM files by containing all CA certificates and
all CRL together.
Also, since haproxy needs to be reloaded when certificate change, we run it in
master-worker mode, with a pid file so that we can signal it to reload.

For the logs, since haproxy does not log to file, we introduce a rsyslogd to
log to a file. The log format is same as with httpd, except that timing are not
in microseconds but in milliseconds - this did not seem to be configurable.
This is a problem for apachedex reports on log, for that we plan to use an
updated version of apachedex with support for `%{ms}T` for durations.

HAProxy is configured with same timeouts, except:
 - "connect" timeout has been increased a bit (from 5 to 10s), because the
   comment "The connection should be immediate on LAN" was no longer true, now
   that haproxy is accessed from frontend.
 - the server entries for testrunner are a very long timeout (8h) because some
   ERP5 functional tests exceeed the 305s timeout.

The SSL configuration is with current "modern" config from https://ssl-config.mozilla.org/

Tests have been modified a bit, because haproxy uses HTTP/2.0 and not 1.1
like httpd was doing several haproxy features (keep alive and gzip
compression) are only available when backend uses HTTP/1.1, so we adjusted
tests to use a 1.1 backend.

There was also differences with logs, because of the time being in milliseconds.

TestPublishedURLIsReachableMixin._checkERP5IsReachable was also updated, it
was working by chance because when accessed behind httpd->haproxy->zope, zope
was producing a redirect URL that was the URL of haproxy, which could be
resolved by chance. This test was updated to access zope with a path that
contains VirtualHostMonster magic, as the shared frontend ( with "zope" software
type) is supposed to set.

This should hopefuly solve the "502 Proxy Error" that we are observing with httpd.

we no longer run tests with `python setup.py test`, so this hack is no
longer necessary.

See merge request !871

This profile will change in stack/erp5 and in erp5 haproxy will also play the
role of apache, but in slapos-master we will keep both apache and haproxy for
a while.

There can be a short delay after the request until the request is logged, so
wait a bit more before inspecting the log.

There can be lines from requests made in previous tests, so make sure to only
read the last line.

Since haproxy checks backend is up at an inteval or 3 seconds with a timeout
of 3 seconds, it can happen that the server reply in a bit more of 3 seconds
and haproxy mark it as down.

By using a shorter response time in test, we can achieve the same test coverage
without this race condition and test runs a bit faster.

Since we don't use caucase yet, it's necessary to make sure this feature is
not lost.

This resource is just a certificate, not necessary a certificate to use as
client.

The test expecting that X-Forwarded-For is empty can also accept the
case where X-Forwarded-For header is not present.

Further work by Vincent Pelletier:
- do not enable openldap dependency, we do not need such advanced features
  (yet ?)
- refresh noroot.patch
- if we this component is to reference m4, then it should extend it

Fixes FTBFS on gcc 10.

See merge request nexedi/slapos!872

This upgrade mainly fixes gpython and pymain to be more compatible with
CPython when handling command line. In particular it moves
sys.executable setup and -O handling into pymain:

nexedi/pygolang!10
nexedi/pygolang@21756bd3
nexedi/pygolang@11b367c6
nexedi/pygolang@8564dfdd

we no longer run tests with `python setup.py test`, so this hack is no
longer necessary.

in ceb063a4 (software/repman: tests python3 compliance, 2020-10-23), test
profile was changed to run on python3 only but this was accidentally
reverted in 3a1e5628 (software/slapos-sr-testing: use nxdtest, 2020-10-28),
by running repman also on python2.

Restore the situation where we run repman only on python3

The initial intent was to use an interpreter named `python_for_test` and not
`python`, so that software do not accidentally pick up this python during
compilation.
6033e4fa (software/slapos-sr-testing: fix python3 profile, 2020-12-01)
was a quick fix for some python3 compatibility issues discovered after merge,
but it was not correct, because by using ${python-interpreter:eggs}, it was
installing the default python-interpreter section, which has an interpreter
named `python`.

This caused issues while building proxysql, which assumes /usr/bin/env python
is python2.

The fix is to override python-interpreter directly, we don't need another
section eggs here.

This code seems not used, before nexedi/slapos@51c3773e and nexedi/slapos@b77d7bae it was not possible to import.

See merge request nexedi/slapos!869