See merge request nexedi/slapos.core!351

SlapOS ticket RSS have one entry for each ticket, with the content of
the latest event as description, but when using a feed reader application,
we'd like to have a new entry for each message posted to the ticket, to
be notified that a new message was posted.

Feed reader applications fetch the RSS and have to determine if each
entry is already in their database or not. In a scenario like below, we
observe different behaviors for different applications:

  1. a user open a ticket "help" with a first message "I need help !".
    The RSS entry for this ticket is:

    <item>
      <link>https://slapos.example.com/#/support_request_module/1</link>
      <title>Help</title>
      <pubDate>Mon, 01 Jan 2001 10:57:10 +0100</pubDate>
      <description>I need help !</description>
    </item>

   => feed reader fetches the RSS and add a new entry with title "Help"

  2. support operator reply with a second message "how can I help you ?"
    The RSS entry for the ticket becomes:

    <item>
      <link>https://slapos.example.com/#/support_request_module/1</link>
      <title>Help</title>
      <pubDate>Mon, 02 Jan 2001 08:32:12 +0100</pubDate>      <== this is different
      <description>How can I help you ?</description>           <== this is different
    </item>

now depending on the implementation of the feed reader, it will either:

 - add a new entry, this is the best behavior. In the RSS readers I tested,
  liferea and newsboat do this. Probably they consider that the entry is
   different because it has a new pubDate and/or description.
 - update the existing entry "in place" - this is not so good, because
  this does not appear as a new entry and the original message is no
  longer in the RSS reader. TinyTinyRSS behaves like this (at least in
  version 17.4, which is a bit old)
 - consider they already have the entry and don't do anything - this is
  even worse, because user can not see there was a reply. FreshRSS
  behaves like this.

A RSS for tickets will always have the problem that the feed reader must
refresh often enough if we want to have a copy of all messages in the
feed reader (and that's why ERP5 CRM's uses a RSS of Events and not
Tickets), but we can make this situation better by using guid on messages.

By constructing a guid [1] that will become different every time a new
message was posted, feed readers consider each item as a a new, different
entry. At least this is the case for all the feed reader I tested, except
ERP5 builtin feed reader which only consider the `link` attribute.

Strictly speaking this implementation has a "problem" that these guids
are not permalinks, but I believe that's something we should address in
erp5_rss_style (if guid does not look like an URL, set isPermalink="false"),
but in practice it does not seem to be a problem.

1: https://validator.w3.org/feed/docs/rss2.html#ltguidgtSubelementOfLtitemgt

It can happen that slapos can be terminated without removing the pid
file, and another long running process using the same pid from the one
in the pid file is running, which cause slapos node to never run.

This makes setRunning a little bit clever, by ignoring the process
unless it also has `slapos` in its name.

When using HTTPS, slapos.libnetworkcache uses exclusively the client certificate
(which must be configured) so https://*.nxdcdn.com can't work from slapos.libnetworkcache

And without download-dir-url, download-cache-url is quite useless.

Reorder options to clarify the difference between upload & https.

See merge request nexedi/slapos.core!345

   We just leave as it is, and let the user decide for himself the value.

  Same as Subscription Requests

On python3 slapos.core for python2 software release, the output was bytes:

```
2021-11-05 17:28:39 slapos[12654] INFO Error with promises for the following partitions:
2021-11-05 17:28:39 slapos[12654] INFO   slappart7[balancer]: b'Promise \'check-apachedex-result.py\' failed with output: ERROR \'"/srv/slapgrid/slappart3/srv/runner/software/cc0326f0dcb093f56c01291c300c8481/bin/check-apachedex-result" --apachedex_path "/srv/slapgrid/slappart3/srv/runner/instance/slappart7/srv/monitor/private/apachedex" --status_file /srv/slapgrid/slappart3/srv/runner/instance/slappart7/srv/monitor/private/apachedex.report.json --threshold "70"\' run with failure, output: \'Score too low: 0% - Threshold: 70.0%\\n\''
```

See merge request !344

    It add a script to be custom on project site, if you have short term users, that requires short password expiration timing.

   Included for the changes on the configured chinese taxation

1) when ansible is still python2 and the callback "json" was introduced
(e.g. in Ubuntu 18.04) we need to use "strict_import" to be sure to use
the json python module and not the json callback

2) when ansible is python3 the syntax must be python3 compatible

3) also remove an uneeded data.copy()

As reported on https://github.com/xolox/python-coloredlogs/issues/107
logging objects with a __str__ method returning non-ascii characters
raises UnicodeDecodeError.

We have vendored coloredlogs version 0.5 long time ago, so just
apply the suggested fix here for now.

Because we were reading subprocess output as bytes, the output in case
of promise errors was something like b'ERROR ...'

There was another problem writing string to an binary file in error path.

When using virtualenv, the module paths are not simple list, but
instances of importlib._bootstrap_external._NamespacePath.

According to docs [1] module's __path__ does is not a list, but an
iterator producing strings.

Cast to a list to consume these iterators.

[1]
https://docs.python.org/3/reference/import.html?highlight=__path__#path__

We fork a subprocess to serve requests, but the listening socket is bound
in server constructor and inherited by subprocess. With this pattern, we
have to explicitly close the socket from parent process to prevent the
leak.

Allow to have missing bin/buildout of the software release, especially to cover
the case, when SR is not installed for partition being processed, eg in order
to destroy it.

  This work is unfinished, so it will be reworked on a branch.

See merge request nexedi/slapos.core!329

    For the time, Taxes were 19.6% on quite old systems.

  Remove this feature in favor of use projects/organisation to
  share servers, since it gives you a proper view of the allocation.

When --only-cp is provided, iterate over a filtered partition list
instead of repeatedly checking at each iteration if the current
partition is in the --only-cp list.

    We index now for very few portal types the bool(checkConsistency()) in order to search inconsistent documents.

    The indexation is limited to the scope we must monitor rather them all documents for 2 major reasons:

        - Too much useless constrants on erp5 code base, not applicable for our project (ie.: On person)
        - Performance on indexation is penalized from this extra cost, specially when index large trees (Deliveries or open orders)**

    ** Not so much, however, on a scale of millions of documents it can make a major difference, so this commits keeps up to the minimal implementation.

   If partition is None, it means the computer was reformated and there is garbage on it. It is meaningless to process this.

The `local-software-release-url` option allows migrating the software
URLs which are local paths by rebasing them on the path provided by
the option.

Do not migrate software release URLs if the old root path and the
new root path are subpaths or superpaths one of the other.

In addition, do not migrate an URL if the old one refers to an
existing file and the new one doesn't.

Also, create a backup of the database before migrating.

See merge request nexedi/slapos.core!338