-  [product] allow instrospecion of External Methods
-  [hal json] introspect form dialog methods for `uids` parameter to inject list of selected UIDS from the previous view
-  [hal_json] introspect listbox's list_method for `uid` parameter to inject list of selected UIDS from the previous view
-  [hal_json] add meta field `extra_param_json` into every dialog form to hold script's constant elsewhere than in fields
-  [hal_json] add dynamic field's definition into form definition so they do not need to be hardcoded in javascript
-  [product] add ability of Form to hash its data using `hash_validated_data`
-  [renderjs_ui] move form submit&response handling into one place - Page Form
-  [hal_json] Warn user about performing an action on unrestricted set of documents
-  [hal_json] Form Action failure returns full form with a message instead of just a message
-  [hal_json] Supports keep_items in Base_renderForm for Form Dialogs (where it matters)

By using `os.path.normpath` we also support paths with a trailing slash.

Also, change the JSON format and update tests according to these changes

The script though that user was scrolling the test output while in reality we just
add a tiny difference of 0.5 pixel.

/reviewed-on nexedi/erp5!632

Fixes dropbox app key in officejs media player.

/reviewed-on nexedi/erp5!628

This first reverts the following 2 commits:
  566c0c5f
  3a08c758

to at least fix the issue that the context method takes 2 arguments that aren't
available from FormBoxEditor.edit() without slowing things even more.

About deferred style, widget editor objects can be ignored for the moment. If
we ever need to restore fully functional objects, the regression in commit
  a5a2f1cd
could be solved with the following patch:

--- a/product/ERP5Form/FormBox.py
+++ b/product/ERP5Form/FormBox.py
@@ -131,22 +131,32 @@ def render(self, field, key, value, REQUEST, render_prefix=None):
 class FormBoxEditor:
   """An editor returned from FormBox validation able to `edit` document."""
 
+  path = None
+
   def __init__(self, result, context):
     """Initialize with all necessary information for editing.
 
     Keep a reference to the correct context and don't expect the caller to provide it
     during the edit phase because they don't have access to the widget anymore.
     """
     self.attr_dict, self.editor_list = result
+    self.context = context
     self.edit = lambda _: self._edit(context)
 
   def __getstate__(self):
-    pass
+    return (self.attr_dict, self.editor_list,
+            self.path or self.context.getPhysicalPath())
+
+  def __setstate__(self, state):
+    self.attr_dict, self.editor_list, self.path = state
 
   def __call__(self, REQUEST):
     # Called by Base_edit in case of FormValidationError
     pass
 
+  def edit(self, context):
+    return self._edit(context.unrestrictedTraverse(self.path))
+
   def _edit(self, context):
     """Edit inside correct context."""
     context.edit(**self.attr_dict)

This commit also removes the unused view() method on editors.

/reviewed-on nexedi/erp5!622

/reviewed-on nexedi/erp5!626

/reviewed-on nexedi/erp5!625

erp5_l10n_fa is needed to test the RTL direction.

same as b6eec549

Since 76cc938e we have diffs each time we edit ZSQL Methods. This is a "big commit" to change all remaining ZSQL methods. I checked diffs one by one, but I would appreciate others check this as well and confirm that it's OK to do so.

From the root of repository, I did :

```
grep -rl Products.ZSQLMethods.SQL  . | grep '\.xml$' | xargs ~/bin/zsql_backslash_n.py
```

With a script containing
```python

from lxml import etree
import sys

for filename in sys.argv[1:]:
  with open(filename) as f:
    tree = etree.parse(f)
    root = tree.getroot()
    for el in tree.xpath('//item/key/string[text() = "arguments_src"]/../../value/string'):
      if el.text:
        el.text = el.text.replace(r'\r\n', r'\n')

    # force <string> element to have a text, so that they export as <string></string> and not <string/>
    for el in tree.xpath('//string[not(text())]'):
      el.text = ''

  with open(filename, 'w') as f:
    f.write(
        '<?xml version="1.0"?>\n'
        + etree.tostring(root, pretty_print=True, encoding="utf-8"))

  print filename
```
( script snippet: nexedi/erp5$277 )

/cc @jm @georgios.dagkakis @Nicolas

/reviewed-on nexedi/erp5!496

Conflicts:
	product/ERP5/bootstrap/erp5_mysql_innodb_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/z_getitem_by_path.xml
	product/ERP5/bootstrap/erp5_mysql_innodb_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/z_getitem_by_uid.xml
	product/ERP5/bootstrap/erp5_mysql_innodb_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/z_produce_reserved_uid_list.xml

When document has pending activities, we refuse changing ID ( because there might be pending `updateRelatedContent` activities if I remember correctly ), but it's done in a way that breaks the "atomic" aspect of the transaction a bit, because we

As a result, this happens sometimes that not all properties user changed are modified. In the example below, the change to *Include Documents in Site Map* is not saved (and also change to *ID*):

![erp5-sorryPendingActivitiesSavePartially](/uploads/ff4bfd6ad0e8a42ba3684cccdc450e21/erp5-sorryPendingActivitiesSavePartially.gif)

( screencast of editing a document to change ids and several other
properties - after clicking save, we can see that changing id is refused
because there is pending activities. Other properties that where changes
at the same times are not all modified, which breaks the
transactionality we can usally expect when editing documents in ERP5 )

The changed here is to use a field validator that refused editing when there are pending activities, so that user gets a:

![erp5-pending-activiities](/uploads/bfe825560bdee34f0443e8e36884f21c/erp5-pending-activiities.png)

( screenshot of the change: now edition is rejected )

and the result is either all changes are applied or no change is applied at all.

This is done by:
 * introducing a new `my_view_mode_id` field in `erp5_core`'s `Base_viewFieldLibrary`
 * using this field as proxy field of all editable `my_id` fields. Maybe I forgot some business templates, I changed only the most common ones. I intentionally did not change all fields of `erp5_ui_test` because I think they are used to compare speed of proxy fields vs traditional fields.

0352f50fd543fda2712bb8ca93d8a8814f975a26 introduces a Zelenium test exercising this new behavior.

/reviewed-on !504

Conflicts:
	product/ERP5/bootstrap/erp5_core/SkinTemplateItem/portal_skins/erp5_core/Base_viewFieldLibrary.xml

This is a fix for this issue:

Create a payment transaction group, a source reference will be generated (something like `PTG0123`. The field is read only on the form, so it's clearly not something we expect user to edit, but a kind of unique generated reference.

Clone that payment transaction group, on the cloned payment transaction group, source reference is empty.

The fix here is to generate a new source reference.

A side effect of these changes is that destination reference is no longer reset after clone, because before this `Delivery_afterClone` was used and this script reset both source and destination reference.

/reviewed-on !535

Numbering tests can be used to control test execution order, but developers should write independent tests that should not rely on test execution order.

`getTitle` is useless. Test runner will print the class name.

/reviewed-on !534

The fact that ERP5 does not remember the selected language has been reported as a bug to us.

Instead of using a session cookie, this change make the cookie persists. For one year. This is a bit arbitrary, I copied that behavior from the "remember my username" [cookie](https://lab.nexedi.com/nexedi/erp5/blob/557c20bdf0e/product/ERP5Type/patches/CookieCrumbler.py#L138) that we use to have on the login page.

/reviewed-on nexedi/erp5!506

In internal transactions, we can have a payment line where source is using a bank account and destination is using another bank account.

Bank Reconciliation did not support these lines properly, because it simply assumed that *"if there's a bank reconciliation on this line, it means the line is reconciled"*, but this was incorrect, because source and destination needs to be able to reconcile this line independently.

The fixes is to change the rule from *"if there's a bank reconciliation on this line, it means the line is reconciled"* to *"if there's a bank reconciliation with the same bank account as this line's source payment, it means the line is reconciled for source"* and *"if there's a bank reconciliation with the same bank account as this line's destination payment, it means the line is reconciled for destination"*.

The related  key used to select the already reconciled lines or lines to reconcile becomes a bit more complex, it joins with category table to ensure the bank reconciliation's bank account  (which is `source_payment` on the bank reconciliation) is the same as the `section_uid` of the line in stock table that we are considering. This also has a side effect that this related key only work with queries on stock table, but it was never intended to be used elsewhere.

Another problem is that queries made by bank reconciliation module become a bit more slower. For sites where this is a problem, the recommended approach is to replace the related keys by columns on stock table. In cfec8d2b (which is part of this MR) we add indexation methods. In 5afb0020 (which is not part of this MR and not planned for inclusion in "default" ERP5) we show an example of how to enable these methods to actually add the columns in stock table.

/reviewed-on !495

We realized that the references of notification messages used in credential request management had a typo (`crendential` vs `credential`) and that the message used for a new credential request was not using the same `credential_request` prefix as others.

This MR changes the messages as follow:

| Wrong Reference | Correct Reference |
| --- | --- |
| erp5-subscription.notification | credential_request-subscription |
| crendential_request-confirmation-without-password | credential_request-confirmation-without-password |
| crendential_request-confirmation-with-password | credential_request-confirmation-with-password |
| crendential_request-confirmation-without-password | credential_request-confirmation-without-password |
| crendential_recovery-reset-link | credential_recover-reset-link |
| crendential_recovery-username | credential_recovery-username |

This is an incompatible change that can affect projects that have defined some custom notification messages without explicitly setting the references on the system preferences (ie. getting the default value from the property definition).
I found one project using custom notification messages, but preference was defined.
Other projects I checked did not override these notification messages.

In our projects, we mostly use messages for credential requests, but the reference for the notification message is usually defined as a property of the web section.

So I'm confident this should not affect projects and we can proceed with this clean up without causing too much troubles. To check if you need to adjust notification messages, check if you have customized notification message with reference *Wrong Reference*.

/reviewed-on !492

The background:

Following up !393 , we updated security configuration used in Nexedi ERP5 , we wanted to add a security rule so that users uploading a `Personal/Private` document are allowed to "share" that document (technically, it's sharing with themselves only because it's a private document, but it's useful to distinguish from drafts).

Then we discovered that in the current workflow configuration, only `Assignor` role is allowed to *share*, but `Assignor` is also allowed to *publish*, but we did not want users to publish their personal documents..

---
Original commit message:

Previously, only Assignor was able to publish, share and release, this
make it impossible to have security configuration where some user can
only share and not publish documents.

To address this issue in the more backward compatible way possible, we
enable these transitions for Associate and keep them enabled for
Assignor role.

/reviewed-on !497