This is what happens with PYTHONHASHSEED=0, but when set to something
random, the order was not guaranteed.

Revealed by erp5_accounting_ui_test:testERP5CurrencyExchangeLine
PYTHONHASHSEED: 438 on py2
Co-authored-by: Kazuhiko SHIOZAKI <kazuhiko@nexedi.com>

When searching for something like for example "A AND B", make sure that
the AND complex query contains query for ["A", "B"], not ["B", "A"].
This is probably always equivalent in generated SQL, but this makes
testing easier.

Revealed by testSQLCatalog.TestSQLCatalog with PYTHONHASHSEED 279 on py2

When cutting with bytes, we might cut in the middle of a multi-bytes
character and later decoding it would cause an error. We could switch
to sending bytes encoded to base64, because the limit would be more
natural to be number of bytes, but just decoding and cutting the string
is easier and in practice not so different.

This bug was revealed by testERP5SyncML.TestERP5SyncML.test_28_PartialData
with PYTHONHASHSEED 872 and test_28_PartialData with PYTHONHASHSEED 238
( on python2 )

The order is used during syncml and if "title" comes before "first_name"
and "last_name", synchronizing persons causes conflicts (because title
of persons is computed from first_name and last_name).
Luckily, sorting properties gives an order that is OK for syncml tests,
so we don't need to invent a new mechanism to control the order of
properties in asXML.

if their content type ended in xml ( such as image/svg+xml ),
TextDocument._convertToBaseFormat was resetting the content type to
application/xml.

When doing something like:

  wp = web_page_module.newContent(
    portal_type='Web Page',
    content_type='image/svg+xml',
    text_content=svg
  )

wp.getContentType() was sometimes 'image/svg+xml' sometimes
'application/xml', depending on the order of edit.

for erp5_web_renderjs_ui_test:testFunctionalRJSPage failures with
PYTHONHASHSEED 347

This produces stable output that we can assert more easily in the test

This solves a failure with testBudget, it should only impact test, but
it's probably better to have a deterministic ordering here.

When a business template has multiple dependencies, these dependencies
were installed in an order defined by iteration in dictionaries, which
is not deterministic when using a random PYTHONHASHSEED on python2.

As an example, when we have business template "b" depending on "b1" and
"b2", the only guarantee was that "b" would be installed after "b1" and
"b2", but which one of "b1" and "b2" is installed first was undefined,
now it is stable, "b1" should be installed before "b2", because we added
a sort. In practice it should not matter, because if in that example
"b2" really needs to be installed after "b1", then "b2" should declare a
dependency on "b1".

test_resolveBusinessTemplateListDependency was failing on python3
because the order on python3 was different from the one using
PYTHONHASHSEED=0 on python2.

In that case, the order of actions rendering is undefined, we want to
have unique priorities to have deterministic order of actions.

This probably happened only in the test, but it's safer not to retry
for ever.

With PYTHONHASHSEED=0, editing a document like this:

    foo.edit(
      title="live_test_%s" % new_id,
      reference="live_test_%s" % new_id
    )

only sets the reference, because it sets the reference first and then
when setting title it already has the "live_test_%s" % new_id value (
because title is acquired from reference).

With PYTHONHASHSEED set, the order is not guaranteed and sometimes
title is edited before reference. This is revealed by
testHalJsonStyle.TestERP5Document_getHateoas_mode_search with
PYTHONHASHSEED 465.

Co-authored-by: Kazuhiko SHIOZAKI <kazuhiko@nexedi.com>

otherwise edit is called with both (new) resource_uid and (old) resource_value.

revealed by erp5_web_renderjs_ui_test:testFunctionalRJSDeveloperMode
with PYTHONHASHSEED: 977 on py2

erp5_workflow_test:testWorkflowAndDCWorkflow PYTHONHASHSEED=872 fails
with:

    ValueError: UID mismatch for <Workflow Script at /erp5_portal_7faf002ae74ecbab0dce45c9fca0f5e1/portal_workflow/testing_erp5_interaction_workflow/script_afterValidate>

Using portal_transforms is not deterministic, because we don't have
direct chain from ods to text, so the conversion is two steps, either
ods=>html=>text or ods=>pdf=>text.

This caused problem with hash randomization, revealed by
testDeferredStyle.TestODSDeferredStyle.test_report_method_access_request
with PYTHONHASHSEED=538 on python2. When using PYTHONHASHSEED=0 , this
was using ods=>html=>text and the assertion from the test was OK, but
when using ods=>pdf=>text the test fails because the words are cut at
end of line.

use a valid image in testERP5ImageProcessor, this test evolved and
became a test for ERP5 behavior with broken images.

This indirectly solves a failure with testERP5ImageProcessor
and PYTHONHASHSEED: 805

  Remove forbidden properties when retrieve the properties from the schema.

      - template and options isn't  part of json schema spec, so it isn't possible to use this feature globally.
      - template also could be used to call callbacks, so despite we block unsafe-eval, it still better remove it.
      - both were removed because it can lead to parameter injection, where by saving the form w/o editing anything, it changes the parameters, it adds non-visible values, which can up to some extend be a security risk.

   Update the description to display the "default" value as a hint, if it was provided into the schema.

constructs like `portal_catalog(title=d.keys())` were allowed on
python2, we can allow them on python3 as well.

With some combinations of worklists using two additional security_uid
columns coming from local role groups, some documents might be excluded
from worklists, without the fix, when running with a random
PYTHONHASHSEED, the test can fail with:

    FAIL: test_worklist_exclusionlist_collision (erp5.component.test.erp5_version.testERP5CatalogSecurityUidOptimization.TestSecurityUidOptimizationWorklist)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "<portal_components/test.erp5.testERP5CatalogSecurityUidOptimization>", line 370, in test_worklist_exclusionlist_collision
        'security_uid_or_alternate_security_uid_draft': 1,
      File "<portal_components/test.erp5.testERP5CatalogSecurityUidOptimization>", line 213, in assertWorklistCount
        expected_count_by_worklist_id,
    AssertionError: {'security_uid_or_alternate_security_uid_draft': 1} != {'collision_worklist': 1, 'security_uid_or_alternate_security_uid_draft': 1}
    - {'security_uid_or_alternate_security_uid_draft': 1}
    + {'collision_worklist': 1, 'security_uid_or_alternate_security_uid_draft': 1}
    ?  +++++++++++++++++++++++++

What happens in sumCatalogResultByWorklist is something like this:

    (Pdb) pp criterion_dict
    {'alternate_security_uid': <ExclusionList [11, 14]>,
     'other_security_uid': frozenset([13L]),
     'portal_type': frozenset(['Person']),
     'validation_state': frozenset(['draft'])}

    (Pdb) pp catalog_result.dictionaries()
    [{'alternate_security_uid': 12,
      'count': Decimal('1'),
      'other_security_uid': 13,
      'portal_type': 'Person',
      'validation_state': 'draft'}]

Depending on the worklists grouped together in grouped_worklist_dict and
the order this dict is iterated, we may reach a situation where
criterion_id_list contains 'alternate_security_uid', because for another
worklist it was not an ExclusionList but a list to be applied, in this
case, this if condition is false:

        for criterion_id in criterion_id_list:
          criterion_value_set = criterion_dict[criterion_id]
          if result_line[criterion_id] not in criterion_value_set:
            is_candidate = False

and the row is not counted in collision_worklist worklist.
Co-authored-by: Jérome Perrin <jerome@nexedi.com>

These are printed like a normal list, so it's a bit hard to figure out
if they are supposed to be excluded when debugging

This refactors security_uid_innodb_catalog test to have the config in a
test business template and include a workflow with some worklists.

This also depends on erp5_worklist_sql, to test worklists when a
combination of erp5_security_uid_innodb_catalog and erp5_worklist_sql
are enabled, which a common situation.

Otherwise we get an error when just adding a worklist