1. 30 Nov, 2015 1 commit
    • Minimize the amount of work during tpc_finish · 7eb7cf1b
      NEO did not ensure that all data and metadata are written on disk before
      tpc_finish, and it was for example vulnerable to ENOSPC errors.
      In other words, some work had to be moved to tpc_vote:
      
      - In tpc_vote, all involved storage nodes are now asked to write all metadata
        to ttrans/tobj and _commit_. Because the final tid is not known yet, the tid
        column of ttrans and tobj now contains NULL and the ttid respectively.
      
      - In tpc_finish, AskLockInformation is still required for read locking,
        ttrans.tid is updated with the final value and this change is _committed_.
      
      - The verification phase is greatly simplified, more reliable and faster. For
        all voted transactions, we can know if a tpc_finish was started by getting
        the final tid from the ttid, either from ttrans or from trans. And we know
        that such transactions can't be partial so we don't need to check oids.
      
      So in addition to minimizing the risk of failures during tpc_finish, we also
      fix a bug causing the verification phase to discard transactions with objects
      for which readCurrent was called.
      
      On performance side:
      
      - Although tpc_vote now asks all involved storages, instead of only those
        storing the transaction metadata, the client has been improved to do this
        in parallel. The additional commits are also all done in parallel.
      
      - A possible improvement to compensate the additional commits is to delay the
        commit done by the unlock.
      
      - By minimizing the time to lock transactions, objects are read-locked for a
        much shorter period. This is even more important that locked transactions
        must be unlocked in the same order.
      
      Transactions with too many modified objects will now timeout inside tpc_vote
      instead of tpc_finish. Of course, such transactions may still cause other
      transaction to timeout in tpc_finish.
      Julien Muchembled committed
  2. 29 Oct, 2015 1 commit
  3. 21 Oct, 2015 1 commit
    • Do not send invalidations for objects on which only readCurrent was called · 9682722b
      This fixes invalid next_serial entries in cache,
      and the following error for values not in cache:
      
        Traceback (most recent call last):
          File "ZODB/Connection.py", line 856, in setstate
            self._setstate(obj)
          File "ZODB/Connection.py", line 894, in _setstate
            self._load_before_or_conflict(obj)
          File "ZODB/Connection.py", line 922, in _load_before_or_conflict
            if not self._setstate_noncurrent(obj):
          File "ZODB/Connection.py", line 945, in _setstate_noncurrent
            assert end is not None
        AssertionError
      Julien Muchembled committed
  4. 24 Sep, 2015 2 commits
  5. 15 Sep, 2015 2 commits
  6. 14 Aug, 2015 1 commit
    • Do not reconnect too quickly to a node after an error · d898a83d
      For example, a backup storage node that was rejected because the upstream
      cluster was not ready could reconnect in loop without delay, using 100% CPU
      and flooding logs.
      
      A new 'setReconnectionNoDelay' method on Connection can be used for cases where
      it's legitimate to quickly reconnect.
      
      With this new delayed reconnection, it's possible to remove the remaining
      time.sleep().
      Julien Muchembled committed
  7. 12 Aug, 2015 2 commits
  8. 09 Jul, 2015 1 commit
  9. 03 Jul, 2015 2 commits
  10. 24 Jun, 2015 1 commit
  11. 21 May, 2015 1 commit
  12. 07 Nov, 2014 1 commit
  13. 20 Jun, 2014 1 commit
    • client: clean up import/export code · d562bf8f
      Export:
      - Remove leftover warning about a bug that was fixed in
        commit e76af297
      - In neomigrate script, open NEO storage read-only.
      - IStorageIteration is already implemented.
      
      Import:
      - Review comments.
      - In neomigrate script, warn that IStorageRestoreable is not implemented.
      - Do not call 'close' method on source iterator. BaseStorage does not do it and
        this is not part of ZODB API. In the case of FileStorage, resource are freed
        automatically during garbage collection.
      Julien Muchembled committed
  14. 19 Jun, 2014 2 commits
    • client: in iterator records, export data serial as stored by NEO · e76af297
      There is simply no way to guess data serials and instead of producing random
      values, the only solution is to retrieve the values from storages.
      
      There are still differences for data serials between FileStorage and NEO:
      - NEO always resolves to original serial, to avoid any indirection
        (which slightly speeds up undo at the expense of a more complex pack code)
      - NEO does not make any difference between object deletion and creation undone
        (data serial always null in storage)
      It has to be decided whether NEO implementation should be changed about this.
      
      Apart from that, conversion database back from NEO should be fixed.
      testExportFileStorageBug passes and there was in fact no FileStorage bug.
      
      Another change is that iterator does not trash the client cache anymore.
      Julien Muchembled committed
    • client: simplify iterator · 441145e5
      Julien Muchembled committed
  15. 04 Jun, 2014 1 commit
  16. 07 Jan, 2014 6 commits
  17. 04 Jan, 2014 3 commits
  18. 17 Dec, 2013 1 commit
  19. 13 Dec, 2013 1 commit
  20. 08 Nov, 2013 2 commits
  21. 28 Oct, 2013 1 commit
  22. 03 Jan, 2013 1 commit
  23. 26 Aug, 2012 1 commit
  24. 23 Aug, 2012 1 commit
  25. 20 Aug, 2012 2 commits
  26. 23 Jul, 2012 1 commit