Move TODO/XXX to TODO-list, tasks created.

- Delayed node acceptation
- Verification process review
- Partition table tweaking improvement required
- Node/Connection state review
- Error packet review

git-svn-id: https://svn.erp5.org/repos/neo/trunk@1442 71dcc9de-d417-0410-9af5-da40c76e7ee4

TODO

@@ -6,6 +6,10 @@ RC = Release Critical (for next release)
         TEMPORARILY_DOWN becomes UNAVAILABLE
         DOWN becomes UNKNOWN
         BROKEN is removed ?
+    - Clarify the use of each error codes:
+      - NO_ERROR error packet vs ACK special packet
+      - NOT_READY removed (connection kept opened until ready)
+      - Split PROTOCOL_ERROR (BAD IDENTIFICATION, ...)
 RC  - Clarify cell state signification
     - Add docstrings (think of doctests)
 RC  - Update README (TODOs should be dropped/moved here)
@@ -30,6 +34,14 @@ RC  - Review output of pylint (CODE)
       a node, and as all nods can be reperesented as Node class instances, 
       such instance should directly contain associated connection for code 
       simplicity.
+      Link node state and connection state, a node cannot be seens as running 
+      and unconnected at the same time from a node point of view. 
+      Consider something like:
+        - UNAVAILABLE implies UNCONNECTED
+        - UNKNOWN implies UNCONNECTED
+        - RUNNING implies CONNECTED
+        - IDENTIFIED implies RUNNING
+        - DOWN implies that the node is dropped from manager/db/config
     - Rework indexes in NodeManager class (CODE)
       NodeManager should provide indexes to quickly find nodes by type, UUID, 
       (ip, port), connection type (listening...) and state (identified...)
@@ -56,6 +68,16 @@ RC  - Review output of pylint (CODE)
       (don't wait at all).
       There is such delay somewhere in master node startup (near the end of 
       the election phase).
+    - Implements delayed connection acceptation.
+      Currently, any node that connects to early to another that is busy for 
+      some reasons is immediately rejected with the 'not ready' error code. This
+      should be replaced by a queue in the listening node that keep a pool a
+      nodes that will be accepted late, when the conditions will be satisfied.
+      This is mainly the case for :
+        - Client rejected before the cluster is operational
+        - Empty storages rejected during recovery process
+      Masters implies in the election process should still reject any connection
+      as the primary master is still unknown.
     - Connections must support 2 simultaneous handlers (CODE)
       Connections currently define only one handler, which is enough for 
       monothreaded code. But when using multithreaded code, there are 2 
@@ -196,6 +218,8 @@ RC  - Review output of pylint (CODE)
       when moving on to next phase.
     - Optimize operational status check by recording which rows are ready
       instead of parsing the whole partition table. (SPEED)
+    - Improve partition table tweaking algorithm to reduce differences between
+      frequently and rarely used nodes (SCALABILITY)
 
     Client
     - Client should prefer storage nodes it's already connected to when 

neo/client/app.py

@@ -456,15 +456,6 @@ class Application(object):
                     break
                 if conn.getUUID() is None:
                     # Node identification was refused by master.
-                    # Sleep a bit an retry.
-                    # XXX: This should be replaced by:
-                    # - queuing RequestIdentification at master side
-                    # - sending the acceptance from master when it becomes
-                    #   ready
-                    # Thus removing the need to:
-                    # - needlessly bother the primary master every 5 seconds
-                    #   (...per client)
-                    # - have a sleep in the code (yuck !)
                     sleep(1)
             if self.uuid is not None:
                 conn.lock()

neo/master/app.py

@@ -488,21 +488,6 @@ class Application(object):
         logging.info('start to verify data')
 
         # Gather all unfinished transactions.
-        #
-        # FIXME this part requires more brainstorming. Currently, this deals with
-        # only unfinished transactions. But how about finished transactions?
-        # Suppose that A and B have an unfinished transaction. First, A and B are
-        # asked to commit the transaction. Then, A succeeds. B gets down. Now,
-        # A believes that the transaction has been committed, while B still believes
-        # that the transaction is unfinished. Next, if B goes back and A is working,
-        # no problem; because B's unfinished transaction will be committed correctly.
-        # However, when B goes back, if A is down, what happens? If the state is
-        # not very good, B may be asked to abort the transaction!
-        #
-        # This situation won't happen frequently, and B shouldn't be asked to drop
-        # the transaction, if the cluster is not ready. However, there might be
-        # some corner cases where this may happen. That's why more brainstorming
-        # is required.
         self.asking_uuid_dict = {}
         self.unfinished_tid_set = set()
         for conn in em.getConnectionList():

neo/master/pt.py

@@ -242,10 +242,6 @@ class PartitionTable(neo.pt.PartitionTable):
                 self.count_dict[node] += 1
                 num_cells += 1
 
-        # FIXME still not enough. It is necessary to check if it is possible
-        # to reduce differences between frequently used nodes and rarely used
-        # nodes by replacing cells.
-
         self.log()
         return changed_cell_list
 

neo/node.py

@@ -83,19 +83,15 @@ class Node(object):
         return False
 
     def isIdentified(self):
-        # XXX: knowing the node's UUID is sufficient ?
         return self._uuid is not Node
 
     def isRunning(self):
-        # FIXME: is it like 'connected' ?
         return self._state == NodeStates.RUNNING
 
     def isTemporarilyDown(self):
-        # FIXME: is it like 'unconnected' or UNKNOWN state ?
         return self._state == NodeStates.TEMPORARILY_DOWN
 
     def isDown(self):
-        # FIXME: is it like 'unconnected' or 'forgotten' ?
         return self._state == NodeStates.DOWN
 
     def isBroken(self):

neo/protocol.py

@@ -29,7 +29,6 @@ MAX_PACKET_SIZE = 0x4000000
 PACKET_HEADER_SIZE = 10
 
 class ErrorCodes(Enum):
-    # TODO: clarify the use of each error code
     NO_ERROR = Enum.Item(0)
     NOT_READY = Enum.Item(1)
     OID_NOT_FOUND = Enum.Item(2)