Xavier Thompson reports that when Theia is compiled inside Theia, Go
fails to install with the error showing that golang14's make.bash calls
`go install` to install cmd/go and other commands to the outer
go.work/bin instead of inside $GOROOT_FINAL set by the build recipe.

Similarly to nexedi/slapos!928 this
might be to the fact that internal `go install` prioritizes $GOBIN even
in the presence of set $GOROOT_FINAL.

-> Don't try to fix Go itself, and make the compilation more robust by
unsetting $GOBIN, so that even if that variable is set in the
environment where buildout runs, it does not affect the compilation
procedure.

Fixes: 8eac67a5 (golang: Prepare for future GOPATH removal)
Reported-on: https://www.erp5.com/group_section/forum/Go-compilation-error-P6f5WisQkc
Tested-by: @xavier_thompson
Reviewed-on: nexedi/slapos!932

   Partially updated from original instance-erp5.cfg.in file.

Introduces devperm manager with lsblk.

Use json to generate .slapos-disk-permission in a way, that devperm manager
will be able to consume it.

   Partially updated from original instance-erp5.cfg.in file

Use-case: when an host enry is used to define the outgoing binding IP of
zopes in a partition, this IP is host-dependent.

So that it proagates root instance's "balancer" parameter, as other
request-* sections do. No functional change is expected, this is rather
for code simplification and factorisation.

use this new mechanism, which avoids this parameter being misused
(inconsistent between partitions of the same family).
Preserve backward compatibility, giving precedence to the new mechanism.

Call them deadlines, as this is what they really are, but the "wrong" name
stuck on ERP5 side (for now).
Also, introduce a per-family-override mechanism, to avoid having per-
option equivalents for such mechanism.

Also, stop poping it: nothing else seems to be accessing the containing
dict, so mutating it does not seem necessary.

gitaly's Makefile forcibly sets $GOPATH to install in _build
folder. Once gitaly is built, it is expected to be in _build/bin/gitaly
and copied from there.

With 8eac67a5 (golang: Prepare for future GOPATH removal, 2021-02-26) the
env.sh sets GOBIN, to its own $GOPATH/bin

Since gitaly's build does not set $GOBIN, even if it overrides $GOPATH, the
final bin/gitaly is not in $GOPATH/bin/gitaly (with $GOPATH from Makefile),
but in $GOBIN/gitaly (with $GOBIN from env.sh)

To prevent this, unset $GOBIN before running make, which keeps the old
behavior of only using the $GOPATH from the Makefile

Update grafana, to use more recent golang. This also brings support of multi line logs in loki.

See merge request !921

See merge request nexedi/slapos!925

When the outside world path does not match the Zope path (typically: Web
Site).

By adding failover url the user is able to configure special backend to use
in case if the real backend is down.

Original PoC was done by Kazuhiko SHIOZAKI <kazuhiko@nexedi.com>.

Both loki and promtail were not configured properly and listening on all IPs.
This was partially because doc was wrong at the time, but also this was not
done properly.

update softwares to:
  - grafana 7.4.3
  - telegrah 1.17.3
  - influxdb 1.8.4
  - loki current master ( 2.1.0 )

and components to:
  - golang 1.16
  - nodejs 14.16.0
  - yarn 1.22.10
  - python 3.7

This new loki version supports parsing multi-line logs, as demonstrated in
the updated test.

Since this version we also remove cache at the end of build, which
makes the software directory smaller and accessorily prevent test failures
during the ldd checks from checkSoftware with some binaries in the cache.

https://github.com/zopefoundation/ZEO/commit/503dccb1

We upgraded ZEO5 to 5.2.2 in commit 34ebf8b5 (ZEO5: v↑ (5.2.0 -> 5.2.2)), and
so now it works with msgpack 0.6.2 provided by stack/slapos.cfg.

The component remains on wendelin.core 1, but the build environment is
adjusted so that both (any of) wendelin.core 1 or wendelin.core 2 could
be built:

- Add zodbtools that wendelin.core 2 requires,
- Add Go and zlib to gowork that are neede to build WCFS.

GOPATH is going to go away in Go 1.17 and most of in-tree SlapOS things are
already built using modules - including upcoming wendelin.core 2 that also
builds/uses NEO/go. This way maintaining GOPATH-based approach becomes just
unneccesary burden (modulo development, where it is still needed sometime
unfortunately https://github.com/golang/go/issues/37755#issuecomment-771927771)

What was "latest" in 2018 is outdated by now.
-> Just use the versions that is provided by component/wendelin.core that neotest extends from.

Add support for using Go modules to golang/gowork infrastructure:

- Users can now request to install a module via gowork:install as. e.g.
  in the following example:

        [gowork]
        install =
            lab.nexedi.com/kirr/neo/go/...@v0.0.0-20210103165133-f3effa6c535f
            golang.org/x/tools/gopls@v0.4.3
            ${helloweb:location}/go:./...

  The first two request to install programs from an external module at particular
  revision/version. The latter requests to install programs from locally
  cloned/checked-out module source.

  The documentation now talks only about programs, because "package
  installation" became unnecessary long time ago as Go toolchain uses
  right packages and recompiles things as needed automatically since
  introduction of the Go build cache in go 1.10.

- The change comes accompanied by corresponding helloweb change that
  reworks it to a) become a module itself, and b) to use other modules -
  that are not explicitly cloned by buildout - so that we can be sure
  that module way of fetching/building things actually works.

  kirr/helloweb@a7c788ae

- Non-module way - e.g. build via GOPATH - is still supported (because
  e.g. software/gitlab still uses it), but not explicitly documented and 
  scheduled to be deprecated and removed. The reason for this is that
  upstream Go is going to remove support for GOPATH and leave only
  module-based approach in Go1.17

  https://github.com/golang/go/issues/37755#issuecomment-771879911

/cc @jerome, @luke, @tomo, @alain.takoudjou
/reviewed-on nexedi/slapos!924

Add support for using Go modules to golang/gowork infrastructure:

- Users can now request to install a module via gowork:install as. e.g.
  in the following example:

	[gowork]
	install =
	    lab.nexedi.com/kirr/neo/go/...@v0.0.0-20210103165133-f3effa6c535f
	    golang.org/x/tools/gopls@v0.4.3
	    ${helloweb:location}/go:./...

  The first two request to install programs from an external module at particular
  revision/version. The latter requests to install programs from locally
  cloned/checked-out module source.

  The documentation now talks only about programs, because "package
  installation" became unnecessary long time ago as Go toolchain uses
  right packages and recompiles things as needed automatically since
  introduction of the Go build cache in go 1.10.

- The change comes accompanied by corresponding helloweb change that
  reworks it to a) become a module itself, and b) to use other modules -
  that are not explicitly cloned by buildout - so that we can be sure
  that module way of fetching/building things actually works.

  kirr/helloweb@a7c788ae

- Non-module way - e.g. build via GOPATH - is still supported (because
  e.g. software/gitlab still uses it), but not explicitly documented and
  scheduled to be deprecated and removed. The reason for this is that
  upstream Go is going to remove support for GOPATH and leave only
  module-based approach in Go1.17

  https://github.com/golang/go/issues/37755#issuecomment-771879911

Some software releases - e.g. wendelin.core - only use ${go:exe} and
does not put anything into ${gowork:install}.

Put emphasis on that gowork defines Go workspace and explain first
settings that are related to that definition. Only after that say how to
use gowork.install. The reason for this is that gowork.install will
become optional in the next patch.

gowork.cfg idea is to specify a GOPATH snapshot with a list of
repositories and their revisions. GOPATH way of building things is going
to go away and we'll soon switch default mode to build go things to do
so via Go modules.

Refactor helloweb a bit before doing that as a preparatory step.

This allows factorising its version pins.

Update PEM to the latest available version.

test we can actually log in to repman and that hte monitoring API report
a cluster as up

no need to stick to an old EOL version

Instead of having softwares install yarn, unify this in nodejs stack.

Yarn usage is similar to nodejs usage, if a specific version is needed,
software should use macro to expose which yarn version to use, example:

    [yarn]
    <= yarn-1.17.3

Then sections can use yarn by having ${yarn:location}/bin/ in their path.

yarn will use the default [nodejs], so to another nodejs version, the same
pattern can be used:

    [nodejs]
    <= nodejs-10.6.0

Since gmp is used by nft and nft is used by firewalld which is included
in slapos-node package, we need to have support for all CPU (disable-assembly).
Otherwise, you can see this kind of errors in dmesg:

firewalld[16932] trap invalid opcode ip:7f5f26ad61fb sp:7fff95cf7480 error:0 in libgmp.so.10.3.1[7f5f26ac7000+51000]

Note that --host=none-pc-linux-gnu is needed to enable the shared libraries.