See merge request nexedi/erp5!1486

See merge request nexedi/erp5!1498

Double translation happened when:

- For sql_catalog.isValidColumn(name)
- When search_columns is empty
- When selection_sort_order and sort_columns are empty

So delaying translation on column titles until *_column_list are finalised.

Before this fix, double translation on Sale Order title was happened every time
displaying a Sale Order Line, for example.

Category spreadsheet import did not delete categories if some documents
where related to these categories. While this may make sense when using
this import in site with already some data, in many cases, we want to
be able to delete the categories, so that the result is that the category
tree match the imported spreadsheet.

This extend the dialog to support a new "force delete" mode that delete
categories even if they are use, leaving the documents using categories
with broken relations.

Spotted when comparing this code with ZPublisher.Publish.

Once a response has been written to, the status and headers are supposed
to have been sent over the network. So any future change to them is moot.
Likewise, setBody does nothing: only write can continue appending to the
existing body.
As in the case of WSGI the response does not actually get sent just yet:
- lock the body
- store finalize() result
Also, make the order in which body and stdout are returned consistent with
HTTPResponse: body is returned first (as part of HTTPResponse.write's call
to self.outputBody, which calls self.__str__, which emits body), and then
stdout content.
Also, add support for cases whre the existing body is a file or an
IUnboundStreamIterator.

err_hook may re-raise when it is specifically told not to render an error
page. Such exception does not mean err_hook failed, so it should not be
re-raised by WSGIPublisher: re-raising in WSGIPublisher means handing the
exception over to waitress, which will have no other choice than producing a
generic error page that we have no control over.

Transaction must be aborted even if the state is getting so bad that the
exception is being re-raised. Leaving an open-ended transaction is never
acceptable.

MariaDB seems to be using inconsistent lock acquisition order when
executing the activity reservation queries. As a consequence, it produces
internal deadlocks, which it detects. Upon detection, it kills one of the
involved query, which causes message reservation to fail, despite the
presence of executable activities.
To avoid depending on MariaDB internal lock acquisition order, acquire an
explicit table-scoped lock before running the activity reservation queries.

On an otherwise-idle 31 processing node cluster with the following
activities spawned, designed to stress activity reservation queries
(many ultra-short activities being executed one at a time):
  active_getTitle = context.getPortalObject().portal_catalog.activate(
    activity='SQLQueue',
    priority=5,
    tag='foo',
  ).getTitle
  for _ in xrange(40000):
    active_getTitle()
the results are:
- a 26% shorter activity execution time: from 206s with the original code
  to 152s
- a 100% reduction in reported deadlocks from 300 with the original code
  to 0

There is room for further improvements at a later time:
- tweaking the amount of time spent waiting for this new lock to be
  available, set for now at 1s.
- possibly bypassing this lock altogether when there are too few processing
  nodes simultaneously enabled, or even in an adaptive reaction to
  deadlock errors actually happening.
- cover more write accesses to these tables with the same lock

From a production environment, it appears that the getReservedMessageList
method alone is involved in 95% of these deadlocks, so for now this change
only targets this method.

testRestrictedPythonSecurity imports *_black_list variables.

- start supporting newer astroid (untested)
- lxml so files can be installed with names like
  'builder.x86_64-linux-gnu.so'

This fixes commit ea53f23e.
See also commit d4f6a2cb.

This fixes commit ea53f23e:

  ERROR Application Couldn't install ERP5Type
  Traceback (most recent call last):
    File "OFS/Application.py", line 691, in install_product
      initmethod(context)
    File "Products/ERP5Type/__init__.py", line 141, in initialize
      initializeProductDocumentRegistry()
    File "Products/ERP5Type/InitGenerator.py", line 62, in initializeProductDocumentRegistry
      importLocalDocument(class_id, class_path=class_path)
    File "Products/ERP5Type/Utils.py", line 994, in importLocalDocument
      module = __import__(module_path, {}, {}, (module_path,))
    File "Products/ERP5/Document/ExternalMethod.py", line 38, in <module>
      class ExternalMethod(XMLObject, ZopeExternalMethod, ExpressionMixin):
  TypeError: Error when calling the metaclass bases
      metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

What happened is that the API of ExpressionMixin changed in commit
df85ef46 and the backward compatibility
code (product/ERP5/mixin/expression.py) was broken.

Also fix 'filter' tab on in the catalog.
See also commit 6ad56d89.

Quantity unit was not defined in default supply property sheet, so the
fields were created local properties, because the accessors were not
existant. This change make the accessors for quantity unit on default
supply lines generated.

The implementation details is that it replaces priced_quantity, which was
defined twice

See nexedi/erp5@09e0bbcf

erp5_smart_assistant and erp5_travel_expense have cross looping dependencies.
Install both bt5 for the tests, but the real fix would be to clean up those bt5.

See merge request nexedi/erp5!1494

Update local frontend slave with backend balanver's IPv6 address if Test Suite is configured to use a local frontend slave. This allows a develop to use a local frontend (possible in same machine) and avoid network effect from generated test results.

See 1b5e3c72

See nexedi/erp5@1a807abb which removes one translated string

Stabilize random failures