group_id is used as key of OOBtree and as documented, it's not
possible to mix keys that can not be compared, so we can not have a mix
of string and bytes, for consistency with other BTrees, such as the
ones used for OFS.
group_id is also used in a SQL column which is BINARY, this is
problematic on py3 because the selected values will be returned as bytes,
but we expect str here. Because we don't want to run a data migration,
we adjust the select methods to convert to str while selecting.

Since years there was a warning that id_group must be a string, now we
make it a bit stricter, we also enforce that the id_group is valid UTF-8.

A few more tests and assertions were also added.

Reviewed-on: nexedi/erp5!1980

Since Zope commit 020685087 (`Allow ZPublisher to handle a query string together
with a request body (#1124)`, 2023-05-15) Zope tries to process all XML HTTP
requests as XML-RPC and we need to tell that these SOAP requests are not XML-RPC.

Co-authored-by: Arnaud Fontaine <arnaud.fontaine@nexedi.com>

Followup of 94739085.

key()-based sorting is now used instead, available since Python 2.4.

Jupyter integration is more or less abandoned. Just do the minimum to make tests pass.

Also, remove `if True/1/False/0` statement raising pylint warnings.
Co-authored-by: Arnaud Fontaine <arnaud.fontaine@nexedi.com>

* Zope API changes:
  + publish():
    - stdin is now BytesIO rather than StringIO.
    - Returned value of a script is passed to str() in python2, not in python3 anymore.
  + HTTPResponse `body` property is now bytes().
  + OFS.Image.File file parameter is bytes().
  + zope.interface implements() is now @implementer decorator.
  + Python standard logging module recommended instead of zLOG.
* Python3 API changes:
  + builtin reduce() was removed.
  + urlnorm is now available.
  + Use BytesIO rather than StringIO to follow py3 API.
  + hmac.new() requires digestmod argument from Python 3.8.
  + Use six.moves library to handle moved objects from py2 to py3.
  + `modernize -f xrange_six` then slightly adjusted manually to just use range
     where it does not make a significant difference (for example in test).
  + base64.b64encode() now expects bytes().
  + UserDict() interface changed:
    - New parameter in update() and pop().
    - `failobj` setdefault parameter renamed to `default`.
  + ensure_list() on dict.{values,items}() and list(dict) for dict.keys() when
    we really need a list and not an iterable (Python3).
  + Make dict iteration works on both version of Python.
    - Use six.iter{items,values,keys}().
    - has_key() has been removed.
    - Make sure that dict.{items,values,keys}() returns a real list when
    modified (ensure_list()).
  + Comparisons between int and NoneType raises TypeError.
    + BTrees key must be str() not int() (_getOb()).
  + No more unbound methods in python3 so use six.get_unbound_function().
  + Exceptions:
    - No longer behave as sequences. Using `args` attribute instead.
    - When an exception has been assigned using `as target`, it is cleared at
      the end of the except clause.
  + file: py2 was returning `str` upon reading, now it returns text strings.
    Also, opening mode is text strings by default.
  + Data strings are bytes().
    - Replace str() by bytes().
  + iterators no longer have next() method, instead there is next() builtin.
  + New ConnectionError exception so rename existing one to not clash.
  + Integer division is now with //.
  + __nonzero__ is now __bool__.
  + apply() does not exist anymore.
  + Deprecated threading.Thread isAlive() has been removed.
  + im_func replaced by __func__.
  + Use six.with_metaclass() to define metaclass in a cross-compatible way with py2 and py3.
  + Only test method can be marked as expectedFailure(), not assert statement anymore.
  + os.path.walk() removed.
  + HTMLParser never fails: no strict mode nor HTMLParseError anymore (Python #15114).
  + Unpickler.find_global() is now Unpickler.find_class().
Co-Authored-by: Kazuhiko SHIOZAKI <kazuhiko@nexedi.com>
Co-Authored-by: Arnaud Fontaine <arnaud.fontaine@nexedi.com>
Co-Authored-by: Carlos Ramos Carreño <carlos.ramos@nexedi.com>
Co-Authored-by: Emmeline Vouriot <emmeline.vouriot@nexedi.com>

some conversion (TextDocument) depend on the content type, so it's
necessary to first update the content type before trying to convert to
base format

zope5: Since 5.8.1 missing Content-Type HTTP header is interpreted as application/x-www-form-urlencoded.

Zope.git:
  commit 5b324f6c461f5ea1cc069739b6c32a1a5ff59df9
  Date:   Thu Jan 19 07:15:18 2023 +0100
    replace `cgi.FieldStorage` by `multipart` (#1094)
    * interpret a missing `CONTENT_TYPE` as `application/x-www-form-urlencoded`

With cgi.FieldStorage, it was interpretated as text/plain so Content-Type has to
be provided now (even for WebDAV despite RFC 4918 stating that it *SHOULD* be
provided).

Some Unit Tests did not provide such header at all but this was wrong (such as
erp5_stripe:testStripe where the real request has application/json as
Content-Type and not text/plain).

The workflow method transitions were configured to use a different
guard from the user action transitions.

"Manager bypass" was not set

The default character_set_system is utf8mb3 (>= MariaDB 10.6), utf8 (<= MariaDB 10.5).

See merge request nexedi/erp5!1977

- app does not manipulate data structure anymore (jio does)
- use union storage for multiple masters
- remote slapos masters fetch is done in replicateopml storage repair method
- sync repair method recreates the storage on new master urls, and updates the stored objects accordingly
- add erp5monitor layer in monitoring jio storage

- add notify parameters on controller
- fix form_view custom submit handling

erp5_core: reindex recursively only if indexable children type exists in Base_reindexObjectSecurity.

This is important especially for SyncML Subscription having tons of SyncML Signatures that is non-indexable.

Note: first I tried the change in the restricted python only like :

--- product/ERP5/bootstrap/erp5_core/SkinTemplateItem/portal_skins/erp5_core/Base_reindexObjectSecurity.py
+++ product/ERP5/bootstrap/erp5_core/SkinTemplateItem/portal_skins/erp5_core/Base_reindexObjectSecurity.py
@@ -4,7 +4,8 @@
 # with lots of content could mean hours of non-usable overloaded system.
 type_tool = context.getPortalObject().portal_types
 for portal_type_name in context.getTypeInfo().getTypeAllowedContentTypeList():
-  if getattr(type_tool, portal_type_name).getTypeAcquireLocalRole():
+  if getattr(type_tool, portal_type_name).getTypeAcquireLocalRole() and \
+      type_tool.getPortalTypeClass(portal_type_name).isIndexable():
     reindex = context.recursiveReindexObject
     break
 else:

but I got the following exception :

  File "Script (Python)", line 8, in Base_reindexObjectSecurity
    type_tool.getPortalTypeClass(portal_type_name).isIndexable():
  File "/(SR)/eggs/AccessControl-4.4-py2.7-linux-x86_64.egg/AccessControl/users.py", line 179, in allowed
    if self._check_context(object):
  File "/(SR)/parts/erp5/product/ERP5Type/patches/AccessControl_patch.py", line 44, in _check_context
    return aq_inContextOf(getattr(object, '__self__', object), context, 1)
TypeError: unbound method _aq_dynamic() must be called with Address instance as first argument (got str instance instead)

Plus, remove the duplicate of 'text-align'

This is follow up of commit 0000dee0 (erp5_core: update
mimetypes_registry with shared-mime-info_2.4-5 and media-types_10.1.0.,
2024-09-17), after this change, Products.MimetypesRegistry build regular
expressions with fnmatch.translate, which on python2 produces regular
expressions like:

     >>> fnmatch.translate('*.aaa')
     '.*\\.aaa\\Z(?ms)

such expressions cause a warning in python3:

    >>> re.compile('.*\\.aaa\\Z(?ms)')
    ...
    re.error: global flags not at the start of the expression at position 9

Running this on python3 would be incompatible with python2, because
`fnmatch.translate` on python3 generates regular expressions that are not
valid on python2.

To solve this mimetypes_registry.xml was edited by hand to move the
flags at the beginning, replacing for example '.*\\.aaa\\Z(?ms)' by
'(?ms).*\\.aaa\\Z'

The ssl/x509 were renamed after we decided to change the name of the
holding property sheet during the code review. The code was updated,
but not this form

  Otherwise it gets other unrelated objects from a full text search

This is tested in test_14_multiple_workflow_different_permission_roles

See merge request nexedi/erp5!1982

Otherwise
  https://example.com/foo
would match
  https://example.com/
but not the language-selector
  https://example.com/en/

and not as manager, as some scripts in the workflow have proxy roles, so
it would be nice to test them too.

The CA certificate can be stored on the connector (it is obtained from
the clammit instance).

To store properties related to x509 certificates, used for exemple in
TLS connections