See merge request nexedi/slapos!880

This also pulls in crawshaw.io/sqlite because it is used by lonet.

Due to unknown yet reason ATS and Caddy are loosing connections to haproxy,
which results with increased amount of 5xx (about 0.5% more cases).

No known solution was found yet on haproxy side, but this kind of workaround
stabilises frontend and definitely drops amount of in-frontend 5xx. The
observed situation with this workaround is no more 5xx sourcing in frontend
itself.

See merge request !878

See https://galene.org
Use first official version (0.1)
Co-authored-by: Thomas Gambier <thomas.gambier@nexedi.com>

See merge request nexedi/slapos!868

Introduce a new "software release upgrade test", which tests a scenario of keating
an ERP5 instance with an old version of software release and then update it to
current version.

To support this test and because we are using this technique for most of slapos
softwares nowadays, configure zopes to restart automatically when their
configuration change.

See merge request nexedi/slapos!875

Confirms SlapOS ERP5 software release can be updated from
a reference version to current version

When root instance is updated to a new software release URL, it will re-request
all the instances with the new software release URL.
To make sure the new root instance does not appear has ready when it is
re-requested with new software release URL, introduce a promise that will check
that the instances requested by the root instance have the same software
release URL. For now we do this only for Zope instances, because they are
stateless and restart automatically on configuration changes, unlike stateful
instances like mariadb or ZEO that we don't restart automatically (yet ?).

We are using this pattern for most of our services since several
months without any issue, so let's also use it for zopes. This
makes automatic upgrade possible.

Also remove "zope running current products" promise, since we
restart we no longer need to check this.

moves python2 release to component/jupyter-py2 because it is required by ERP5
but not maintained anymore

moves to python 3 :
- test/test.py
- jupyter_notebook_config.py.jinja
- ERP5kernel.py

modifies custom.js to create a workaround for events issue
see (https://github.com/jupyter/notebook/issues/2499)

upgrades slapos.cookbook version to 1.0.171 to use zero_knowledge recipe
with python 3

adds jupyter partition check in software/erp5/test/test_erp5.py

Fixes compatibility with glibc >= 2.30 .
Update users to new usage pattern.

It's much easier to find the slave by it's reference than by it's title for
the cluster administrator, so use the reference in this case.

Fixes "caddy-frontend: Add promise for rejected slaves", even if title seemed
more human readable, but it's unusable.

Since haproxy does non-HTTP style checks to the python provided backend it
started to emit a lot of logs about broken pipe, which are not important
in context of testing and shall be silenced.

See https://en.wiktionary.org/wiki/instanciated :

> Misspelling of instantiated.

See merge request nexedi/slapos!858

See merge request nexedi/slapos!863

Allow each instance to support a zone list instead of a single zone.
Each slave is set for only one defined zone.

While caucase-updater aibcc was added, it was copy & pasted from aikc, and
by mistake it shared data_dir, which results with not working update of
certificates.

This fixes this issue introduced in "caddy-frontend: Setup backend client auth"

We are starting to use wendelin.core not only in Wendelin context.
So it makes sense to have it installed in base ERP5, like we already do for
example for NumPy and SciPy.

/reviewed-by @rafael, @jp
/reviewed-on nexedi/slapos!874

Having only one section [wendelin.core] instead of [wendelin.core] and
[wendelin.core-dev] is easier to handle in "we want to use such and such
particular version" scenarious without deciding in advance whether an SR needs
to inherit from wendelin/software.cfg or wendelin/software-dev.cfg

/reviewed-on nexedi/slapos!874

haproxy can be controlled with this socket, so it might be useful
to "expose" it - it's not really expose because we only use a
UNIX socket.

The apachedex reports when produced on backend will be wrong, because haproxy
logs timings in milliseconds and apachedex parses as microsecond, but as far as
I know we produce reports from frontend logs, so it should not really affect
our operations.

Two main differences of haproxy are file format for certificates and logs.

HAProxy also uses certificates in PEM format, but it expect its own server
certificate and the key to be in the same file (although recent version seems
to accept separate files, we don't use this now) and the CRL and CA certificates
also all together in the same file.
We change to use the same file for certificate and key and for CA and CRL, in
the updater script we we build PEM files by containing all CA certificates and
all CRL together.
Also, since haproxy needs to be reloaded when certificate change, we run it in
master-worker mode, with a pid file so that we can signal it to reload.

For the logs, since haproxy does not log to file, we introduce a rsyslogd to
log to a file. The log format is same as with httpd, except that timing are not
in microseconds but in milliseconds - this did not seem to be configurable.
This is a problem for apachedex reports on log, for that we plan to use an
updated version of apachedex with support for `%{ms}T` for durations.

HAProxy is configured with same timeouts, except:
 - "connect" timeout has been increased a bit (from 5 to 10s), because the
   comment "The connection should be immediate on LAN" was no longer true, now
   that haproxy is accessed from frontend.
 - the server entries for testrunner are a very long timeout (8h) because some
   ERP5 functional tests exceeed the 305s timeout.

The SSL configuration is with current "modern" config from https://ssl-config.mozilla.org/

Tests have been modified a bit, because haproxy uses HTTP/2.0 and not 1.1
like httpd was doing several haproxy features (keep alive and gzip
compression) are only available when backend uses HTTP/1.1, so we adjusted
tests to use a 1.1 backend.

There was also differences with logs, because of the time being in milliseconds.

TestPublishedURLIsReachableMixin._checkERP5IsReachable was also updated, it
was working by chance because when accessed behind httpd->haproxy->zope, zope
was producing a redirect URL that was the URL of haproxy, which could be
resolved by chance. This test was updated to access zope with a path that
contains VirtualHostMonster magic, as the shared frontend ( with "zope" software
type) is supposed to set.

This should hopefuly solve the "502 Proxy Error" that we are observing with httpd.

we no longer run tests with `python setup.py test`, so this hack is no
longer necessary.

See merge request nexedi/slapos!871

This profile will change in stack/erp5 and in erp5 haproxy will also play the
role of apache, but in slapos-master we will keep both apache and haproxy for
a while.

There can be a short delay after the request until the request is logged, so
wait a bit more before inspecting the log.

There can be lines from requests made in previous tests, so make sure to only
read the last line.

Since haproxy checks backend is up at an inteval or 3 seconds with a timeout
of 3 seconds, it can happen that the server reply in a bit more of 3 seconds
and haproxy mark it as down.

By using a shorter response time in test, we can achieve the same test coverage
without this race condition and test runs a bit faster.