1. 30 Nov, 2015 1 commit
    • Julien Muchembled's avatar
      Minimize the amount of work during tpc_finish · 7eb7cf1b
      Julien Muchembled authored
      NEO did not ensure that all data and metadata are written on disk before
      tpc_finish, and it was for example vulnerable to ENOSPC errors.
      In other words, some work had to be moved to tpc_vote:
      
      - In tpc_vote, all involved storage nodes are now asked to write all metadata
        to ttrans/tobj and _commit_. Because the final tid is not known yet, the tid
        column of ttrans and tobj now contains NULL and the ttid respectively.
      
      - In tpc_finish, AskLockInformation is still required for read locking,
        ttrans.tid is updated with the final value and this change is _committed_.
      
      - The verification phase is greatly simplified, more reliable and faster. For
        all voted transactions, we can know if a tpc_finish was started by getting
        the final tid from the ttid, either from ttrans or from trans. And we know
        that such transactions can't be partial so we don't need to check oids.
      
      So in addition to minimizing the risk of failures during tpc_finish, we also
      fix a bug causing the verification phase to discard transactions with objects
      for which readCurrent was called.
      
      On performance side:
      
      - Although tpc_vote now asks all involved storages, instead of only those
        storing the transaction metadata, the client has been improved to do this
        in parallel. The additional commits are also all done in parallel.
      
      - A possible improvement to compensate the additional commits is to delay the
        commit done by the unlock.
      
      - By minimizing the time to lock transactions, objects are read-locked for a
        much shorter period. This is even more important that locked transactions
        must be unlocked in the same order.
      
      Transactions with too many modified objects will now timeout inside tpc_vote
      instead of tpc_finish. Of course, such transactions may still cause other
      transaction to timeout in tpc_finish.
      7eb7cf1b
  2. 25 Nov, 2015 2 commits
  3. 05 Oct, 2015 2 commits
  4. 24 Sep, 2015 2 commits
  5. 12 Aug, 2015 1 commit
    • Julien Muchembled's avatar
      Remove useless testEvent · 71e30fb9
      Julien Muchembled authored
      Such kind of test has never helped to detect regressions and any bug in
      EpollEventManager would be quickly reported by other tests.
      
      testConnection may go the same way if it keeps annoying me too much.
      71e30fb9
  6. 30 Jun, 2015 1 commit
  7. 21 May, 2015 1 commit
  8. 05 May, 2015 1 commit
  9. 09 Dec, 2014 1 commit
  10. 05 Dec, 2014 2 commits
  11. 30 Jul, 2014 1 commit
  12. 24 Jul, 2014 2 commits
  13. 20 Jun, 2014 1 commit
    • Julien Muchembled's avatar
      client: clean up import/export code · d562bf8f
      Julien Muchembled authored
      Export:
      - Remove leftover warning about a bug that was fixed in
        commit e76af297
      - In neomigrate script, open NEO storage read-only.
      - IStorageIteration is already implemented.
      
      Import:
      - Review comments.
      - In neomigrate script, warn that IStorageRestoreable is not implemented.
      - Do not call 'close' method on source iterator. BaseStorage does not do it and
        this is not part of ZODB API. In the case of FileStorage, resource are freed
        automatically during garbage collection.
      d562bf8f
  14. 03 Jun, 2014 2 commits
  15. 07 Jan, 2014 1 commit
  16. 22 Aug, 2012 1 commit
  17. 21 Aug, 2012 1 commit
  18. 20 Aug, 2012 3 commits
  19. 20 Jul, 2012 1 commit
  20. 17 Jul, 2012 2 commits
  21. 16 Jul, 2012 3 commits
  22. 13 Jul, 2012 1 commit
  23. 22 Mar, 2012 1 commit
    • Vincent Pelletier's avatar
      Drop testProtocol. · 0dd0a6cb
      Vincent Pelletier authored
      Since lib.protocol has been rewritten to be semantic-oriented and
      duplicates a lot less code, this test is just bloated (testing the same
      code over and over) and a maintenance burden.
      0dd0a6cb
  24. 21 Mar, 2012 1 commit
  25. 20 Mar, 2012 1 commit
    • Julien Muchembled's avatar
      Review logging to keep all debugging information in RAM and flush only if useful · 1fce5cc4
      Julien Muchembled authored
      The main goal of this patch is to keep all DEBUG logs and packet logger enabled
      without exploding disk usage.
      This is done by keeping the last 16 MB (by default) of debugging information in
      a RAM buffer, and to emit it to an SQLite DB upon RTMIN signal or in case of
      warning and more severe log.
      
      Implementation is also cleaned up for better integration within a framework
      or if run standalone. NEO logger is now a direct child of root handler.
      Only warnings and more severe logs are forwarded to root handler.
      
      A new script 'neolog' is added to pretty-print the contents of the SQLite log.
      
      In unit tests, logging events are not buffered but emitted immediately.
      When a test passes, payloads of all exchanged packets are discarded to reduce
      disk usage on test bots.
      
      This slows down performance tests by about 15 % because even if nothing is
      written to disk, debug and packet log records are now always rendered.
      1fce5cc4
  26. 13 Mar, 2012 1 commit
  27. 24 Feb, 2012 1 commit
    • Julien Muchembled's avatar
      Implements backup using specialised storage nodes and relying on replication · 8e3c7b01
      Julien Muchembled authored
      Replication is also fully reimplemented:
      - It is not done anymore on whole partitions.
      - It runs at lowest priority not to degrades performance for client nodes.
      
      Schema of MySQL table is changed to optimize storage layout: rows are now
      grouped by age, for good partial replication performance.
      This certainly also speeds up simple loads/stores.
      8e3c7b01
  28. 23 Feb, 2012 2 commits