git-svn-id: https://svn.erp5.org/repos/neo/trunk@2113 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2112 71dcc9de-d417-0410-9af5-da40c76e7ee4

This fix an issue where a client was notified down, dropped from node
manager then detected as disconnected: the node was not found.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2111 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2110 71dcc9de-d417-0410-9af5-da40c76e7ee4

Store deadlocks might happen when the order in which storage nodes receive
object data (and hence, grant write lock for those objects) is not
consistent among clients. So when a store timeouts, it might be because
another transaction got the store lock on this storage and object, while
it might wait for locks we got on other storage nodes or objects.
In doubt, report a conflict to abort current transaction in hope for the
other transaction to succeed.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2109 71dcc9de-d417-0410-9af5-da40c76e7ee4

As we don't (and must not, otherwise it would deadlock) take a write lock
on a storage node when detecting a conflict, it is possible that multiple
conflicts get successively reported for a single object in a single
transaction. To solve this, client must tolerate multiple conflict for a
single object. The last resolved conflict being sent to all storage nodes,
transaction data will be consistent at tpc_finish.

- conflict_serial_dict and resolved_conflict_serial_dict become
  multivalued (sets instead of scalar)
- each resolution only resolve conflict with highest serial if greater
  than previous resolutions for the same object
  NOTE: this assumes that ("r" being conflict resolution method, "Tn"
  being the TID on which object is based, and "Cn" being a conflict with
  transaction n) when r(T0, C1) generates T1 but triggers C2,
    r(T0, C1) + r(T1, C2) == r(T0, C2)

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2108 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2107 71dcc9de-d417-0410-9af5-da40c76e7ee4

Such function is provided when message is queued for send when an answer
is expected, and is called when the answer is not arrived after expiration
of the timeout delay. Depending on the return value of this callback, the
timeout is ignored (True) or passed through (False).

Also, move code to refresh next timeout value to a separate function, for
reusability.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2106 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2105 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2104 71dcc9de-d417-0410-9af5-da40c76e7ee4

This allows always checking whether a storage accepted the object for a
conflicting serial value, so do it.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2103 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2102 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2101 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2100 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2099 71dcc9de-d417-0410-9af5-da40c76e7ee4

Between closure request and effect, a client node can be registered to the
node manager without a connection. As there is no needs to use an
unconnected client node, always set only_identified parameter to True.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2098 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2097 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2096 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2095 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2094 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2093 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2092 71dcc9de-d417-0410-9af5-da40c76e7ee4

A master load a partition table from the network only during recovery, this
operation requires a different behaviour than the load() for other nodes.
Check incoming data and raise before any change on the partition table to
avoid inconsistency between ptid and partition table data.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2091 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2090 71dcc9de-d417-0410-9af5-da40c76e7ee4

Add some XXX where the cluster is stopped without wait for the storages
to process the pending requests (eg. outdate cells of a just-down node).

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2089 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2088 71dcc9de-d417-0410-9af5-da40c76e7ee4

There was an inconsistency between the ptid and the partition table stored
in the manager between the processing of the AnswerLastIDs and the
AnswerPartitionTable packets. A storage could disconnect leaving the
master with a wrong partition table.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2087 71dcc9de-d417-0410-9af5-da40c76e7ee4

Allow know the primary master as RUNNING state instead of UNKNOWN.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2086 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2085 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2084 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2083 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2082 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2081 71dcc9de-d417-0410-9af5-da40c76e7ee4

Also, extend logging.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2080 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2079 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2078 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2077 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2076 71dcc9de-d417-0410-9af5-da40c76e7ee4

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2075 71dcc9de-d417-0410-9af5-da40c76e7ee4

This avoid get zombie client nodes in manager when the cluster fallback to
verification state.

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2074 71dcc9de-d417-0410-9af5-da40c76e7ee4