Server returns slave list with request and publish keys, but only request keys
are important.

In order to avoid needless updates and nonsense data remove those polluted
keys before publishing information about each slave.

Caddy and ATS shall have same timeouts for the requests.

It defaults to 600s, which is good reasonable chosen before.

ATS by default tries 4 times (1 non configurable, 3 retries) a given query to
the backend. Despite of "connection" in the name, ATS waits for first byte
from the backend, and it means that backend might already processing the
request, just no bytes are sent. In order to avoid this allow ATS to try only
once.

  The Remote User is managed on the backend apache which will unset it anyway.

Added part for logrotation into part_list, resulting with installing it.

/reviewed-on nexedi/slapos!606

This allows much better understanding for long running Kedifa processes.

/reviewed-on nexedi/slapos!597

Due to missing monitor frontend in tests this change is not covered by tests.

/reviewed-on nexedi/slapos!592

As caddy_configuration_state provides information with exit code use it
directly in the if instead of running it with $(), which stops execution when
configuration is not changed.

Improve checking for too old result - do not remove the file, instead check
for too old result and then recalculate.

/reviewed-on nexedi/slapos!589

As Kedifa speaks HTTP check that it can reply with HTTP, not that just
it is possible to connect to.

/reviewed-on nexedi/slapos!591

In some cases domain can come from "outside" of the profile, and be filled
with "garbage", so if custom_domain is set, do not overwrite it.

…ration state calculation""

The states has been correctly propagated and used. Original author had no
idea what he was doing, possibly because of some incorrect assumptions while
developing this fix.

/reviewed-on nexedi/slapos!587

The last state value has been reverted by mistake, put back correct one.

Frontend operator shall have easy access to information about rejected
slaves, possibly the best in the JSON file.

Also the keys for the human readable information are slave's titles, not
references.

The information is published via hand crafted HTTPS endpoint.

Note: The SSL certificate is generated manually. Existing caucase is special
      for KeDiFa, this is another step to move all generated certificates (or
      otherwise self-signed) to internal, full automatic caucase.

Rename was missing in buildout.hash.cfg

/reviewed-on nexedi/slapos!575

Before normalize_ae_gzip was 0, and better keep it this way as it is
suspected root problem of incorrectly varianted cache depending on
client Accept-Encoding.

Instead of expensive and long checking of the configuration during promise run
read last stored state.

This decouples configuration validation calculation from promise check. The
validation information is updated often (on each configuration change, on each
reload, etc) and every 2 hours.

Validation happens on each configuration change, but for sure it is checked
each 2 hours.

State of configuration is calculated in separate script.

Move files from var/log/trafficserver/*old to
srv/backup/logrotate/trafficserver, xz them and clean files older than a
year.

As some tools are not configurable with logrotate provide a script to
"rotate" files generated by those tools.
This script more or less follows logrotate behaviour:

 * moves rotated files to logrotate directory
 * compresses them with xz
 * have to be configured to keep only files for some days

Roll logs each 24h, as it would be expected.

Stderr to null redirection happened in the wrong place.

Promises will fail with stderr filled, and it can be such if sha256sum emits
error messages, so just discard them.

Some arguments needs Caddy process restart, so implement it with
hash-files and also inform the master partition requester about parameters
which will result with process restart.

Kedifa partition was missing monitoring at all, so add it and monitor kedifa
and exposer ip and port.

Partition running caddy was missing monitoring for exposer, so add it.

Use unreal address to avoid any tries for network connectivity.

Kedifa requires some time to process new slave, and during that time the key
download URL is not available, but as it is required for proper mapping file,
use some url to mimic it.

During buildout run no network communication is required in order to prepare
fallback certificates, so call kedifa-updater with --prepare-only

It is needed by users to check certificate of KeDiFa while uploading
certificates.

Each time new slave appears the kedifa-updater has to be run immediately, in
order for certificates to be properly setup.

Otherwise caddy can be left in non-runnable state until next kedifa-updater
would run again.

caddy-frontend master partition does not implement any promise in etc/promise,
all is migrated to etc/plugin

caddy-frontend-is-running-actual-software-release promise is not needed
anymore, as hash-files are used.

By default whole slave makes websocket connection to the backend.
With websocket-path, only the path has websocket style connections,
the rest is standard HTTP.