See merge request nexedi/slapos!820

This addresses both :
 - some random failures we sometimes have when installing `python-language-server` fail
 - caddy no longer being installed here since nexedi/slapos!821

See merge request nexedi/slapos!828

It has to be ensured, that parameters are correctly passed to the partitions.

Each node allows for global statistic access for full backend-haproxy, which
is exposed using special frontend, and then transferred back to the master
partition, so that the administrator can access it.

Prevent this kind of errors when first installation fail:

    Error: [Errno 13] Permission denied: 'parts/python-language-server/bin/activate'

by using --clear the virtualenv is recreated from scratch if the command failed
for some reason.

We are observing mariadb crashes in ERP5 with 10.4, so revert back to 10.3

Crashes errors are:

    InnoDB: Assertion failure in file .../mariadb-10.4.14/storage/innobase/lock/lock0lock.cc line 6900

in non debug mode and:

    mysqld: .../mariadb-10.4.14/storage/innobase/row/row0sel.cc:4480: dberr_t row_search_mvcc(byte*, page_cur_mode_t, row_prebuilt_t*, ulint, ulint): Assertion `prebuilt->sql_stat_start || trx->state == TRX_STATE_ACTIVE || (prebuilt->table->no_rollback() && trx->state == TRX_STATE_NOT_STARTED)' failed.

in debug mode.

This is an incompatible change for ERP5 instances which might have
already been running mariadb 10.4, because their database would have
been updated to 10.4, but mariadb does not support "downgrading".
Hopefully we don't have such cases, but if that's the case one
solution seems to dump databases with 10.4 and load in 10.3 and
I guess in this process the system databases such as "mysql" should
be excluded from this process.

See merge request !823

Changes:

 * traffic_cop was removed, so use traffic_manager directly
 * logging.config was changed to logging.yaml
 * made records.config and storage.config similar to original files
 * proxy.config.admin.synthetic_port option was removed
 * proxy.config.process_manager.mgmt_port option was removed
 * test: ignore traffic.out in logs
 * test: update ATS version
 * pqsn field was removed and replaced with shn, so follow upstream:
   https://github.com/apache/trafficserver/commit/b0969c91ebc52b37f4c3195ec17d4d0c1c18650c
 * add a test to prove squid.log working, as upgrade resulted with not
   created file

Require minially gcc 8 for C++17 support

Copied part of gowork.install:command to activate GO111MODULES and also the
[caddy] section is minimised, as the whole build happnes in proper place.

defines 'configure-command = true' because configure script does not exist

Should be used soon.

There was missing promise from other work.

See merge request nexedi/slapos!814

This is critical feature of SlapOS Master instantiation, and very easy to
damage, thus testing it is important.

It's useful for some tests to use.

Some profiles needs to extends the path with binaries location of provided
software.

This reverts commit 440b88c5

As there might be some reasons why the instance is not found explain what to
in each case.

See merge request nexedi/slapos!810

Thanks to using check_execute_command with logrotate -d one can assure, that
logrotate is for sure correctly configured.

By disabling delaycompress filenames are going to be stable, on delaying the
compression is not needed.

In some cases there is not need to have delayed compression of the logs.

By using regular expressions, matching exact host names up to the optional
port and putting wildcard matches in the end, the Haproxy acl rules will allow
to direct request to correct backend.

See merge request nexedi/slapos!815

Because of escaping of slapos.cookbook:wrapper kedifa is never reloaded, so
use instead jinja2 with template_wrapper for it.

Also adapt to kill from dash (-HUP).

This partially reverts commit 20fb7c58.

Unfortunately having trustable X-Forwarded for in case of SlapOS Master
is not so simple, and not needed for now, as the approach is a bit different
on that apache, thus let's revert for now.

The update of AddOutputFilterByType is kept.

And not globally inside slapuser $HOME (which is go builtin default).

`go env` output: before:

    (go.work) slapuser34@vifibcloud-rapidspace-hosting-007:~/srv/runner$ go env
    GO111MODULE=""
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/srv/slapgrid/slappart34/.cache/go-build"
    GOENV="/srv/slapgrid/slappart34/.config/go/env"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOINSECURE=""
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="linux"
    GOPATH="/srv/slapgrid/slappart34/srv/runner/software/09b17891535725b7c44e64fb2e856ebe/go.work"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/srv/slapgrid/slappart34/srv/runner/shared/golang1.14/da4c5c8222d5573da4d0e547cfc1793e"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/srv/slapgrid/slappart34/srv/runner/shared/golang1.14/da4c5c8222d5573da4d0e547cfc1793e/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    AR="ar"
    CC="gcc"
    CXX="g++"
    CGO_ENABLED="1"
    GOMOD=""
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build196479275=/tmp/go-build -gno-record-gcc-switches"

after:

    (go.work) slapuser34@vifibcloud-rapidspace-hosting-007:~/srv/runner$ go env
    GO111MODULE=""
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/srv/slapgrid/slappart34/srv/runner/software/09b17891535725b7c44e64fb2e856ebe/go.work/cache"
    GOENV="/srv/slapgrid/slappart34/srv/runner/software/09b17891535725b7c44e64fb2e856ebe/go.work/goenv"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOINSECURE=""
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="linux"
    GOPATH="/srv/slapgrid/slappart34/srv/runner/software/09b17891535725b7c44e64fb2e856ebe/go.work"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/srv/slapgrid/slappart34/srv/runner/shared/golang1.14/da4c5c8222d5573da4d0e547cfc1793e"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/srv/slapgrid/slappart34/srv/runner/shared/golang1.14/da4c5c8222d5573da4d0e547cfc1793e/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    AR="ar"
    CC="gcc"
    CXX="g++"
    CGO_ENABLED="1"
    GOMOD=""
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build233719895=/tmp/go-build -gno-record-gcc-switches"

i.e. we change $GOCACHE and $GOENV to be per-workspace:

    --- a/goenv.before
    +++ b/goenv.after
    @@ -1,8 +1,8 @@
     GO111MODULE=""
     GOARCH="amd64"
     GOBIN=""
    -GOCACHE="/srv/slapgrid/slappart34/.cache/go-build"
    -GOENV="/srv/slapgrid/slappart34/.config/go/env"
    +GOCACHE="/srv/slapgrid/slappart34/srv/runner/software/09b17891535725b7c44e64fb2e856ebe/go.work/cache"
    +GOENV="/srv/slapgrid/slappart34/srv/runner/software/09b17891535725b7c44e64fb2e856ebe/go.work/goenv"
     GOEXE=""
     GOFLAGS=""
     GOHOSTARCH="amd64"

Explanation:

- Starting from Go1.10 the go command maintains cache for built packages, test results, etc:

  https://golang.org/doc/go1.10#build

  while this cache tries to remember by which go version and in which
  environment a package was built, and reusing the cached entry only when it is
  indeed correct to do so, by keeping this cache in a shared place - not per go
  workspace - we expose it to interactions with other go installments, which,
  due to e.g. software bugs might misbehave.

  -> keeping the build cache inside go workspace makes things more safe.

- Starting from Go1.13 the go command allows to keep its configuration and environment inside a file:

  https://golang.org/doc/go1.13#go-env-w
  https://go.googlesource.com/proposal/+/master/design/30411-env.md

  The default location for such file is per-user, which can bring
  misinteraction among several go workspace installments and wrt system go
  installment.

  -> move this configuration file into inside go workspace.

/reviewed-by: TrustMe

These are old Go releases that were stopped being updated in 2018 - mid
2019:

https://golang.org/doc/devel/release.html#go1.9
https://golang.org/doc/devel/release.html#go1.10
https://golang.org/doc/devel/release.html#go1.11

and that are not used by any in-tree SR or component.

Don't drop Go1.12 and Go1.13 yet: they are too outdated and no longer
supported, but those Go releases are still in use by the following
slapos software:

Go1.12: Caddy, GitLab, Grafana
Go1.13: RepMan, Restic

The maintainers of those software are advised to upgrade to Go1.14 due
to at least recent security fixes in net/http package.

Go1.10 is outdated by today as it was last updated in Jan 2019:

	https://golang.org/doc/devel/release.html#go1.10

Go1.14 is current oldstable - i.e. it is still supported and mature.

As currently no in-tree SR or component use gowork's default, the switch
should not introduce any breakage.

Going Go1.14.3 -> Go1.14.9 brings in many compiler and runtime fixes,
including security fixes to package net/http:

https://golang.org/doc/devel/release.html#go1.14

Switch helloweb from go1.13 -> go1.14 to test.

Tested on helloworld SR.