Confirms SlapOS ERP5 software release can be updated from
a reference version to current version

haproxy can be controlled with this socket, so it might be useful
to "expose" it - it's not really expose because we only use a
UNIX socket.

The apachedex reports when produced on backend will be wrong, because haproxy
logs timings in milliseconds and apachedex parses as microsecond, but as far as
I know we produce reports from frontend logs, so it should not really affect
our operations.

Two main differences of haproxy are file format for certificates and logs.

HAProxy also uses certificates in PEM format, but it expect its own server
certificate and the key to be in the same file (although recent version seems
to accept separate files, we don't use this now) and the CRL and CA certificates
also all together in the same file.
We change to use the same file for certificate and key and for CA and CRL, in
the updater script we we build PEM files by containing all CA certificates and
all CRL together.
Also, since haproxy needs to be reloaded when certificate change, we run it in
master-worker mode, with a pid file so that we can signal it to reload.

For the logs, since haproxy does not log to file, we introduce a rsyslogd to
log to a file. The log format is same as with httpd, except that timing are not
in microseconds but in milliseconds - this did not seem to be configurable.
This is a problem for apachedex reports on log, for that we plan to use an
updated version of apachedex with support for `%{ms}T` for durations.

HAProxy is configured with same timeouts, except:
 - "connect" timeout has been increased a bit (from 5 to 10s), because the
   comment "The connection should be immediate on LAN" was no longer true, now
   that haproxy is accessed from frontend.
 - the server entries for testrunner are a very long timeout (8h) because some
   ERP5 functional tests exceeed the 305s timeout.

The SSL configuration is with current "modern" config from https://ssl-config.mozilla.org/

Tests have been modified a bit, because haproxy uses HTTP/2.0 and not 1.1
like httpd was doing several haproxy features (keep alive and gzip
compression) are only available when backend uses HTTP/1.1, so we adjusted
tests to use a 1.1 backend.

There was also differences with logs, because of the time being in milliseconds.

TestPublishedURLIsReachableMixin._checkERP5IsReachable was also updated, it
was working by chance because when accessed behind httpd->haproxy->zope, zope
was producing a redirect URL that was the URL of haproxy, which could be
resolved by chance. This test was updated to access zope with a path that
contains VirtualHostMonster magic, as the shared frontend ( with "zope" software
type) is supposed to set.

This should hopefuly solve the "502 Proxy Error" that we are observing with httpd.

This profile will change in stack/erp5 and in erp5 haproxy will also play the
role of apache, but in slapos-master we will keep both apache and haproxy for
a while.

There can be a short delay after the request until the request is logged, so
wait a bit more before inspecting the log.

There can be lines from requests made in previous tests, so make sure to only
read the last line.

Since haproxy checks backend is up at an inteval or 3 seconds with a timeout
of 3 seconds, it can happen that the server reply in a bit more of 3 seconds
and haproxy mark it as down.

By using a shorter response time in test, we can achieve the same test coverage
without this race condition and test runs a bit faster.

Since we don't use caucase yet, it's necessary to make sure this feature is
not lost.

This resource is just a certificate, not necessary a certificate to use as
client.

The test expecting that X-Forwarded-For is empty can also accept the
case where X-Forwarded-For header is not present.

This reverts commit 5d46a499.

Changes to "fix argument parsing" (to allow passing argument containing space)
had a regression that backslash characters are escaped.

Let's go back to slapos.toolbox 0.112 for now.

When updating existing instance, running slapos node instance failed with:

    ln: failed to create symbolic link '/srv/slapgrid/slappart9/srv/frontend-static/logo.png': File exists

This fixes by doing nothing if logo.png already exists

In nexedi/slapos@74d18b9d I switched ERP5 to use caucase certificate, but caucase is not yet usable for this and this caused issues with upgrading existing instances.

We'll maybe try caucase again here, in that case 620c9332 can be reverted if we want to reuse some parts of what was made.

See merge request nexedi/slapos!857

Revert "software/erp5: use a caucase managed certificate for balancer"

This reverts commit 74d18b9d and also follow
up fixup ( 555b26a2 ).

We are not ready to use caucase here, there are still too many problems with
caucase (keys are lost at each SR update etc) and design might still evolve,
so let's go back to self signed certificate for now.

Also remove the promise and the updater, since they are also not working and
causing problems on instances that have been updated (and where the key no
longer match the certificate)

fix balancer CSR generation:

Caucase rerequest uses a CSR *template* and use it to generate
a new CSR with a new key, so we should not use the actual key to
generate this CSR, because it is caucase rerequest job to generate
the key.
Also, we should be careful not to generate a new CSR every time this
command run, otherwise a new key will be generated and a new CSR will
be sent to caucase, but caucase will not sign it automatically (since
we configure it to sign only one certificate).

This means that the case of IP address changes is currently not
supported automatically. To support it we would need to:
  - force generation of a new CSR template
  - force caucase rerequester to request a new certificate (by removing
  existing certificate)
  - force caucased to sign the new certificate

This commit also fix indentation and remove simplefile macro that is no longer used

See merge request nexedi/slapos!837

Just running test.py with ip and port allows to expose the internal testing
backend. IPv4 and IPv6 are supported.

By default there is no sense to play with software type, since it's fixed in
slapos.testing.testcase.

slapos.cookbook:softwaretype tends to lowercase keys in each section, which
has been undetected due to using lowercase references of slaves in the tests.

By restructuring information in the sections, and putting slave references
inside of dumped part of information, now the slave reference case is kept.

Also real care was taken to stabilise published lists by sorting them, as it
also slipped, that they could be unstable.

Tests has been updated to catch this issue, also other tests were fixed, as
they had wrong assertions.

See commit e83edfc9.

See merge request nexedi/slapos!850

This reverts commit cc1713c3.

Unfortunately a lot of installation OS ISOs (like Debian 9 and 10) do not
support such device during installation process, because of missing drivers
on the OS CD.

Also note the ISOs limitation in the advanced field.

Make the value and its changes easier to read.

Revert f8f72a17 ([erp5] don't use caucase generated certificate for now, 2019-03-12) since nothing prevents us drom using caucase certificate now.
 
Use [managed resources](nexedi/slapos.core!259) to simplify existing tests and introduce tests for:

## Access Log

 - [x] balancer partition should produce logs in apache "combined" log format with microsecond timing of requests.
 - [x] these logs should be rotated daily
 - [x] an [apachedex](https://lab.nexedi.com/nexedi/apachedex) report is ran on these logs daily.

## Balancing

 - [x] requests are balanced to multiple backends using round-robin algorithm
 - [x] if backend is down it is excluded
 - [x] a "sticky cookie" is used so that clients are associated to the same backend
    - [x] the cookie is set by balancer
    - [x] when client comes with a cookie it "sticks" on the associated backend
    - [x] if "sticked" backend is down, another backend will be used

## Content-Encoding

 - [x] balancer encodes responses in gzip for some configured content types.

## HTTP

 - [x] Server uses HTTP/1.1 or more and keep connection with clients

## TLS (server certificate)

In this MR we also change apache to use a caucase managed certificate and add test coverage for:

 - [x] balancer listen on https with a certificate that can be verified using the CA from caucase.
 - [x] balancer uses the new certificate when its own certificate is renewed.

But we don't add support for:
 -  ~~balancer can be instantiated with a certificate and key passed as SlapOS request parameters (code [here](https://lab.nexedi.com/nexedi/slapos/blob/757c1a4ddee93659d5e2649e4252d87bf9494566/stack/erp5/instance-balancer.cfg.in#L208-213))~~ this use case is the job of caucase, so we no longer support this.

## TLS (client certificate)
 - [x] balancer verifies frontend certificates from frontend caucases ( also tested in "Forwarded-For" section )
 - [x] if frontend provided a verified certificate, balancer set `remote-user` header
 - [x] balancer updates CRL from caucases ( `caucase-updater-housekeeper` )
 - (NOT TESTED) balancer updates CA certificate from caucase ( `caucase-updater-housekeeper` ). Since this is would be complex to test and basic functionality of `caucase-updater-housekeeper` for frontend caucases is covered by CRL test, we don't test this for simplicity.

## "Forwarded-For" header

This was also covered by existing tests:  

 - [x] balancer set `X-Forwarded-For` header when frontend certificate can be verified
 - [x] balancer strips existing `X-Forwarded-For`

## Integration with the rest of ERP5 software release

This was also covered by existing tests:  

- [x] The https URL of each Zope family is published and replies properly
- [x] Some https URLs are generated for `runUnitTest`, so that test run with an https certificate. This is also covered by regular ERP5 functional tests.

See merge request nexedi/slapos!840

See merge request nexedi/slapos!839

https://github.com/eclipse-theia/theia/blob/v1.7.0/CHANGELOG.md#v170---29102020

The only change to support python3 in software/plantuml is the move
from hexagonit to slapos download recipe in component/tomcat.
We also move the test to python3 by using plantuml egg v0.3.0. As this
egg doesn't support python2, the test is not done anymore in Python2.

Future Cython+ work will use it.

Fix for #20200514-218C705 - \[testnode\] frontend for log access

Depends on nexedi/erp5!1304

See merge request nexedi/slapos!848