1. 04 Mar, 2021 1 commit
  2. 14 Oct, 2019 1 commit
  3. 26 Apr, 2019 1 commit
  4. 11 Mar, 2019 1 commit
  5. 26 Feb, 2019 1 commit
    • Julien Muchembled's avatar
      qa: new tool to stress-test NEO · 38e98a12
      Julien Muchembled authored
      Example output:
      
          stress: yes (toggle with F1)
          cluster state: RUNNING
          last oid: 0x44c0
          last tid: 0x3cdee272ef19355 (2019-02-26 15:35:11.002419)
          clients: 2308, 2311, 2302, 2173, 2226, 2215, 2306, 2255, 2314, 2356 (+48)
                  8m53.988s (42.633861/s)
          pt id: 4107
              RRRDDRRR
           0: OU......
           1: ..UO....
           2: ....OU..
           3: ......UU
           4: OU......
           5: ..UO....
           6: ....OU..
           7: ......UU
           8: OU......
           9: ..UO....
          10: ....OU..
          11: ......UU
          12: OU......
          13: ..UO....
          14: ....OU..
          15: ......UU
          16: OU......
          17: ..UO....
          18: ....OU..
          19: ......UU
          20: OU......
          21: ..UO....
          22: ....OU..
          23: ......UU
      38e98a12
  6. 15 Nov, 2018 1 commit
    • Julien Muchembled's avatar
      qa: new expectedFailure testcase method · 4150ffb1
      Julien Muchembled authored
      The idea is to write:
      
        with self.expectedFailure(...): \
      
      just before the statement that is expected to fail. Contrary to the existing
      decorator, we want to:
      - be sure that the test fails at the expected line;
      - be able to remove an expectedFailure without touching the code around.
      4150ffb1
  7. 07 Aug, 2018 1 commit
    • Julien Muchembled's avatar
      Use argparse instead of optparse · 9f1e4eef
      Julien Muchembled authored
      Besides the use of another module for option parsing, the main change is that
      there's no more Config class that mixes configuration for different components.
      Application classes now takes a simple 'dict' with parsed values.
      
      The changes in 'neoctl' are somewhat ugly, because command-line options are not
      defined on the command-line class, but this component is likely to disappear
      in the future.
      
      It remains possible to pass options via a configuration file. The code is a bit
      complex but isolated in neo.lib.config
      
      For SSL, the code may be simpler if we change for a single --ssl option that
      takes 3 paths. Not done to not break compatibility. Hence, the hack with
      an extra OptionList class in neo.lib.app
      
      A new functional test tests the 'neomigrate' script, instead of just the
      internal API to migrate data.
      9f1e4eef
  8. 19 Jun, 2018 1 commit
  9. 24 May, 2018 1 commit
  10. 15 May, 2018 1 commit
    • Julien Muchembled's avatar
      importer: new option to write back new transactions to the source database · 30a02bdc
      Julien Muchembled authored
      By doing the work with secondary connections to the underlying databases,
      asynchronously and in a separate process, this should have minimal impact on
      the performance of the storage node. Extra complexity comes from backends that
      may lose connection to the database (here MySQL): this commit fully implements
      reconnection.
      30a02bdc
  11. 16 Apr, 2018 1 commit
    • Julien Muchembled's avatar
      Fix a few issues with ZODB5 · 1316c225
      Julien Muchembled authored
      In the Importer storage backend, the repickler code never really worked with
      ZODB 5 (use of protocol > 1), and now the test does not pass anymore.
      
      The other issues caused by ZODB commit 12ee41c47310156027a674932df34b60de86ba36
      are fixed:
      
        TypeError: list indices must be integers, not binary
      
        ValueError: unsupported pickle protocol: 3
      
      Although not necessary as long as we don't support Python 3,
      this commit also replaces `str` by `bytes` in a few places.
      1316c225
  12. 12 Jun, 2017 1 commit
    • Julien Muchembled's avatar
      master: improve algorithm to tweak the partition table · 2ca7c335
      Julien Muchembled authored
      The most important change is that it does not discard readable cells too
      quickly anymore. A partition can now have multiple FEEDING cells, to avoid
      going below the wanted level of replication.
      
      The new algorithm is also better at minimizing the amount replication.
      2ca7c335
  13. 12 May, 2017 1 commit
  14. 28 Apr, 2017 1 commit
    • Julien Muchembled's avatar
      client: fix possible data corruption after conflict resolutions with replicas · 46c36465
      Julien Muchembled authored
      This really fixes the bug described in
      commit 40bac312,
      which could probably be reverted. It only reduced the probability of failure.
      
      What happened is that the second conflict on 'a' for t3 what first reported by
      an answer to first store with:
      - a base serial at which a=0
      - a conflict serial at which a=7
      However, the cached data is not 8 anymore but 12, since a second store already
      occurred after the first conflict (reported by the other storage node).
      
      When this conflict was resolved before receiving the conflict for second store,
      it gave:
      
        resolve(old=0, saved=7, new=12) -> 19
      
      instead of:
      
        resolve(old=4, saved=7, new=12) -> 15
      
      (if we still had the data of the first store, we could also do
        resolve(old=0, saved=7, new=8)
       but that would be inefficient from a memory point of view)
      
      The bug was difficult to reproduce. testNotifyReplicated had to be run many
      many times before that race conditions trigger it. The test was changed to
      enforce some of them, and the above scenario now happens almost always.
      46c36465
  15. 27 Apr, 2017 1 commit
  16. 24 Apr, 2017 1 commit
    • Julien Muchembled's avatar
      Reimplement election (of the primary master) · 23b6a66a
      Julien Muchembled authored
      The election is not a separate process anymore.
      It happens during the RECOVERING phase, and there's no use of timeouts anymore.
      
      Each master node keeps a timestamp of when it started to play the primary role,
      and the node with the smallest timestamp is elected. The election stops when
      the cluster is started: as long as it is operational, the primary master can't
      be deposed.
      
      An election must happen whenever the cluster is not operational anymore, to
      handle the case of a network cut between a primary master and all other nodes:
      then another master node (secondary) takes over and when the initial primary
      master is back, it loses against the new primary master if the cluster is
      already started.
      23b6a66a
  17. 18 Apr, 2017 1 commit
  18. 31 Mar, 2017 2 commits
  19. 23 Mar, 2017 1 commit
  20. 14 Feb, 2017 1 commit
  21. 02 Feb, 2017 1 commit
  22. 19 Jan, 2017 1 commit
  23. 18 Jan, 2017 1 commit
  24. 13 Jan, 2017 1 commit
  25. 28 Dec, 2016 2 commits
  26. 27 Nov, 2016 2 commits
  27. 26 Feb, 2016 1 commit
  28. 25 Jan, 2016 1 commit
  29. 21 Jan, 2016 1 commit
  30. 30 Nov, 2015 1 commit
    • Julien Muchembled's avatar
      Minimize the amount of work during tpc_finish · 7eb7cf1b
      Julien Muchembled authored
      NEO did not ensure that all data and metadata are written on disk before
      tpc_finish, and it was for example vulnerable to ENOSPC errors.
      In other words, some work had to be moved to tpc_vote:
      
      - In tpc_vote, all involved storage nodes are now asked to write all metadata
        to ttrans/tobj and _commit_. Because the final tid is not known yet, the tid
        column of ttrans and tobj now contains NULL and the ttid respectively.
      
      - In tpc_finish, AskLockInformation is still required for read locking,
        ttrans.tid is updated with the final value and this change is _committed_.
      
      - The verification phase is greatly simplified, more reliable and faster. For
        all voted transactions, we can know if a tpc_finish was started by getting
        the final tid from the ttid, either from ttrans or from trans. And we know
        that such transactions can't be partial so we don't need to check oids.
      
      So in addition to minimizing the risk of failures during tpc_finish, we also
      fix a bug causing the verification phase to discard transactions with objects
      for which readCurrent was called.
      
      On performance side:
      
      - Although tpc_vote now asks all involved storages, instead of only those
        storing the transaction metadata, the client has been improved to do this
        in parallel. The additional commits are also all done in parallel.
      
      - A possible improvement to compensate the additional commits is to delay the
        commit done by the unlock.
      
      - By minimizing the time to lock transactions, objects are read-locked for a
        much shorter period. This is even more important that locked transactions
        must be unlocked in the same order.
      
      Transactions with too many modified objects will now timeout inside tpc_vote
      instead of tpc_finish. Of course, such transactions may still cause other
      transaction to timeout in tpc_finish.
      7eb7cf1b
  31. 05 Oct, 2015 1 commit
  32. 30 Sep, 2015 1 commit
  33. 24 Sep, 2015 1 commit
  34. 23 Sep, 2015 1 commit
  35. 15 Sep, 2015 1 commit
  36. 12 Aug, 2015 2 commits