Otherwise it fails with the following exception because of non-ASCII characters
(`[A-Z&é@{]{3,7})`) (see unconvert()) below):

  => erp5_core/PathTemplateItem/portal_preferences/default_site_preference.xml
     File "zodbpickle-2.0.0-py3.7-linux-x86_64.egg/zodbpickle/pickle_3.py", line 844, in load
        dispatch[key[0]](self)
     File "zodbpickle-2.0.0-py3.7-linux-x86_64.egg/zodbpickle/pickle_3.py", line 1035, in load_short_binstring
        self.append(self.decode_string(data))
     File "zodbpickle-2.0.0-py3.7-linux-x86_64.egg/zodbpickle/pickle_3.py", line 989, in decode_string
        return value.decode(self.encoding, self.errors)
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 19: ordinal not in range(128)

This reverts commit 6c399134.

The strategy for compatibility is that:
 - haproxy still listen on the same port as before, without rewrite rule.
   This is called "legacy" port.
 - for each frontend from request parameters, we introduce an haproxy
   frontend with a rewrite for the corresponding `internal-path`
   parameter.
 - the shared frontend instance is updated to use this new frontend
   entry from haproxy. This will cause a small downtime until the shared
   frontend is updated to the new URL on ERP5, but since this feature
   was not used, it's OK.

Technical details are that we:
 - split haproxy config to have frontends and backends.
 - introduce one frontend in haproxy for each frontend from request
   parameters.
 - routing-rule-list argument is still honored the same way, globally
   and after path from frontend.
 - change the shared frontend requests to use "" type, no longer "zope"
   type.
 - we don't do automatic detection of /VirtualHostRoot in URL but always
   add it, because it could be used to trick zope into thinking it
   serves requests for an arbitrary host and do open redirects
 - before using the request's host header in virtualhost path, we check
   that it does not contain /, to prevent injection of virutalhost path
   elements through the host header.
 - we don't use the "path" parameter from shared frontend, because we
   want the frontend to be simple, so we don't want it to rewrite the
   request path (which is also the reason why we deprecated "zope" type)
 - the tests have changed a lot, because they were using what's now the
   "legacy" URL types, so we updated it to use the new URL types with
   all the /VirtualHostRoot/../ in path and also because they use IPv6
   URL, no longer IPv4

and save the already allocated ports in a state file, so that requesting
new families does not change already allocated ports.

This reverts commit 620c9332 (stack/erp5: stop using caucase managed
certificate for balancer, 2020-11-10) with an updated design. We add a
caucase service for balancer in the balancer partition. The caucase
service from the root partition (that was not used) is removed.

The underlying idea is that the default configuration should use multiple
caucases with limited scope, here we have one caucase to manage the
certificate used by haproxy server in the balancer partition, so we put
one caucase to manage this certificate and the caucase is configured to
auto-accept one certificate only. The plan is that when we will add a
certificate for mariadb server, we'll add another caucase inside this
mariadb server.

For more advanced usage and also to support the cases where a new
certificate needs to be re-emitted for some reason, users can request
with an existing caucase URL. In that case, they will have to accept
the certificate requests.

Notable changes:

balancer/ssl/caucase-url is no longer documented in parameters, this is
an internal parameter, users can pass one global caucase service to
manage all partition

CAUCASE environment variable is no longer set when running zope. There
was no identified use case and with this new approach of multiple
caucases, the term "caucase" alone became ambiguous.

This is not documented in schema and has no effect in erp5 (but this is
still used for slapos-master)

This change the format or the (mostly) unused frontend parameter to
support requesting more than one frontend and also enable the request of
a frontend by default, so that requesting a frontend separately is no
longer needed.

The `frontend` parameter now also supports requesting frontends for
specific paths on the ERP5 backend, the example below requests a
frontend serving directly a web site, with the necessary rewrite rules:

```js
{
  "frontend": {
    "default": {
      "internal-path": "/erp5/web_site_module/renderjs_runner/"
    }
  }
}
```

The example below requests a default frontend to the erp5 root, to
access the ZMI or erp5_xhtml_style interface and two web sites:

```js
{
  "frontend": {
    "default": {},
    "erp5js": {
      "internal-path": "/erp5/web_site_module/renderjs_runner/"
    },
    "crm": {
      "internal-path": "/erp5/web_site_module/erp5_officejs_support_request_ui/"
    }
  }
}
```

The example below has an explicit definition of the zope families using
`zope-partition-dict` parameter, because there is more than one zope
family, no frontend is requested by default:

```js
{
  "zope-partition-dict": {
    "backoffice": {
      "family": "backoffice"
    },
    "web": {
      "family": "web"
    },
    "activities": {
      "family": "activities"
    }
  }
}
```

Continuing this example, to have frontends for backoffice and web
families, the frontend request can specify the families, like it is
demonstrated in the example below. In this example, we don't specify an
entry for "activities" family, so no frontend will be requested for
this family.

```js
{
  "frontend": {
    "backoffice": {
      "zope-family": "backoffice"
    },
    "web": {
      "zope-family": "web",
      "internal-path": "/erp5/web_site_module/web_site/"
    }
  }
  "zope-partition-dict": {
    "backoffice": {
      "family": "backoffice"
    },
    "web": {
      "family": "web"
    },
    "activities": {
      "family": "activities"
    }
  }
}
```

This makes urljoin works as expected and generally makes sense because
this is a collection.

See merge request nexedi/slapos!1500

Directly expose all passlib.hash supported hashes, using a `passwd-`
prefix. For example, to access `sha256_crypt`, use `passwd-sha256-crypt`
option name.

  [secret]
  recipe = slapos.cookbook:generate.password

  [config-file]
  hashed-password = ${secret:passwd-sha256-crypt}

This changes the format of storage-path, it used to be the password in
plain text, it is now a mapping also containing hashed passwords, to
have the same hashed passwords for each buildout run.

This needs collaboration from publish_early recipe, because .pop(k) does
raised a KeyError with the dict.__missing__  approach.

In MultiRU there will be only one ENB which supports all TDD, FDD, LTE and NR
cells and different types of radio units - all at the same time. This patch is
preparatory step for that: it merges gnb configuration template into enb
configuration template, so that enb.jinja2.cfg now serves both enb and gnb
instances.

In this patch for now we only move code from gnb.jinja2.cfg without changing
it(*) and wrap parts with `if do_lte` and `if do_nr` correspondingly.

The end result of rendered enb.cfg and gnb.cfg stays the same modulo space
changes and added innoccent `#define TDD` as Appendix shows.

(*) the only exception is set up of gtp_addr which has practically the same
code, was wrapped with `if mme_list` in enb and `if amf_list` in gnb, and is now
wrapped with `if mme_list or amf_list`.

/cc @lu.xu, @tomo, @xavier_thompson, @Daetalus
/reviewed-by @jhuge
/reviewed-on !1512

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch.

```
$ ./pythonwitheggs slapos-render-config.py && (git diff -w --no-index config/enb.cfg.old config/enb.cfg ; git diff -w --no-index config/gnb.cfg.old config/gnb.cfg)
```

```diff
diff --git a/config/enb.cfg.old b/config/enb.cfg
index fdf3ab24d..cb46697ea 100644
--- a/config/enb.cfg.old
+++ b/config/enb.cfg
@@ -1,11 +1,15 @@

+
 #define TDD                 1

 #define N_RB_DL             50
+
 #define N_ANTENNA_DL        2
+
 #define N_ANTENNA_UL        2

+
 {

   log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,phy.level=info,file.rota>
@@ -24,7 +28,6 @@
   rx_gain: 43,

   com_addr: "127.0.1.2:9001",
-
   mme_list: [

     {
@@ -33,9 +36,9 @@

   ],

+
   gtp_addr: "127.0.1.1",

-
   enb_id: 0x1A2D0,

   cell_list: [{
@@ -49,7 +52,6 @@
       ],
     }
   ],
-
   cell_default: {
     plmn_list: [
       "00101",
@@ -226,4 +228,6 @@
   meas_gap_config: "gp0",
   ho_from_meas: true,
   },
+
+
 }
\ No newline at end of file
diff --git a/config/gnb.cfg.old b/config/gnb.cfg
index e3d671e09..4e47a2094 100644
--- a/config/gnb.cfg.old
+++ b/config/gnb.cfg
@@ -1,15 +1,21 @@

+
+#define TDD                 1
+
+
 #define N_ANTENNA_DL        2

 #define N_ANTENNA_UL        2

+
 {

   log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,rrc.level=debug,rrc.max_size=1,phy.level=info,file.rota>

   log_filename: "log/gnb.log",

+
   rf_driver: {
       name: "sdr",
       args: "dev0=/dev/sdr0",
@@ -30,15 +36,17 @@
   ],

-
-
   gtp_addr: "127.0.1.1",

   gnb_id_bits: 28,
   gnb_id: 0x12345,
   en_dc_support: true,
-  cell_list: [],
+
+  cell_list: [
+  ],
+
+
   nr_cell_list: [
   {
     rf_port: 0,
```

It stopped to be used after 49ce8ef5 (software/ors-amarisoft: Provide dedicated
TAP interface for each Radio Unit).

/cc @lu.xu, @tomo, @xavier_thompson, @Daetalus
/reviewed-by @jhuge
/reviewed-on nexedi/slapos!1512

Because

1) those services are needed and used only by ru/ promises like check_cpri_lock
   and check_rx_saturated.
2) in general we will need to initialize and setup radio units not only in eNB -
   for example UEsim will use the same code library to initialize radio units.
   Thus the proper place to keep everything required for RU to be operational have to
   be located inside ru/ and activated by that radio-units library.

Push corresponding code from instance-enb to ru/ and do only minor adjustments
to instance-gnb trying not to break it, since gnb does not currently use rulib,
and because in the future gnb will be replaced by enb which will be serving
both lte and nr cells in the same service.

/cc @lu.xu, @tomo, @xavier_thompson, @Daetalus
/reviewed-by @jhuge
/reviewed-on nexedi/slapos!1511

Because:

- ssh server is needed for and used by ru/lopcomm/ only
- in general we will need to initialize and setup radio units not only in eNB -
  for example UEsim will use the same code library to initialize radio units.
  Thus the proper place to keep everything required for RU to be operational have to
  be located inside ru/ and activated by that radio-units library.

/cc @lu.xu, @tomo, @xavier_thompson, @Daetalus
/reviewed-by @jhuge
/reviewed-on nexedi/slapos!1510

Dnsmasq insists on dhcp-range's prefixlen to be at most 64, which
triggers the following error if original slaptap is wider than that:

    dnsmasq: prefix length must be at least 64 at line 5 of /srv/slapgrid/slappart6/etc/dnsmasq.cfg

-> Fix it by capping provided range to /64

/cc @tomo, @xavier_thompson, @Daetalus
/reported-by @lu.xu
/reviewed-by @jhuge
/reviewed-on nexedi/slapos!1509

our new nexedi.org currently no longer serve static pages with links,
using absolute links should allow us to have test passing until this is
fixed.

also expose MB_UNAGGREGATED_QUERY_ROW_LIMIT and MB_AGGREGATED_QUERY_ROW_LIMIT

These partition references should be kept short, they are a mechanism
to use a short path for unix sockets, because unix socket paths can not
exceed 108 characters.

When running the test in theia, this was causing errors:

   # [ALERT]    (100453) : config : parsing [/srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/with-max-rlimit-nofile6/etc/haproxy.cfg:37] : log : socket path '/srv/slapgrid/slappart15/srv/runner/instance/slappart7/tmp/inst/with-max-rlimit-nofile6/var/run/log.sock' too long (max 97