Skip to content

N
neoppod
Switch branch/tag
  1. 29 May, 2020 2 commits
  3. 22 Oct, 2019 1 commit
  5. 14 Oct, 2019 3 commits
  7. 16 Aug, 2019 1 commit
    • Julien Muchembled's avatar
      New feature: monitoring · e434c253
      Julien Muchembled authored 
      This task is done by the admin node, in 2 possible ways:
- email notifications, as soon as some state change;
- new 'neoctl print summary' command that can be used periodically
  to check the health of the database.
They report the same information.

About backup clusters:

The admin of the main cluster also monitors selected backup clusters,
with the help of their admin nodes.

Internally, when a backup master node connects to the upstream master node,
it receives the address of the upstream admin node and forwards it to its
admin node, which is therefore able to connect to the upstream admin node.
So the 2 admin nodes remain connected and communicate in 2 ways:
- the backup node notifies upstream about the health of the backup cluster;
- the upstream node queries the backup node periodically to check whether
  replication is not too late.

TODO:

A few things are hard-coded and we may want to configure them:
- backup lateness is checked every 10 min;
- backup is expected to never be late.

There's also no delay to prevent 2 consecutive emails from having the same
Date: (unfortunately, the RFC 5322 does not allow sub-second precision),
in which case the MUA can display them in random order. This is mostly
confusing when one notification is OK and the other is not, because one
may wonder if there's a new problem.
      e434c253
  9. 26 Apr, 2019 2 commits
  11. 26 Feb, 2019 1 commit
    • Julien Muchembled's avatar
      qa: new tool to stress-test NEO · 38e98a12
      Julien Muchembled authored 
      Example output:

    stress: yes (toggle with F1)
    cluster state: RUNNING
    last oid: 0x44c0
    last tid: 0x3cdee272ef19355 (2019-02-26 15:35:11.002419)
    clients: 2308, 2311, 2302, 2173, 2226, 2215, 2306, 2255, 2314, 2356 (+48)
            8m53.988s (42.633861/s)
    pt id: 4107
        RRRDDRRR
     0: OU......
     1: ..UO....
     2: ....OU..
     3: ......UU
     4: OU......
     5: ..UO....
     6: ....OU..
     7: ......UU
     8: OU......
     9: ..UO....
    10: ....OU..
    11: ......UU
    12: OU......
    13: ..UO....
    14: ....OU..
    15: ......UU
    16: OU......
    17: ..UO....
    18: ....OU..
    19: ......UU
    20: OU......
    21: ..UO....
    22: ....OU..
    23: ......UU
      38e98a12
  13. 07 Aug, 2018 1 commit
    • Julien Muchembled's avatar
      Use argparse instead of optparse · 9f1e4eef
      Julien Muchembled authored 
      Besides the use of another module for option parsing, the main change is that
there's no more Config class that mixes configuration for different components.
Application classes now takes a simple 'dict' with parsed values.

The changes in 'neoctl' are somewhat ugly, because command-line options are not
defined on the command-line class, but this component is likely to disappear
in the future.

It remains possible to pass options via a configuration file. The code is a bit
complex but isolated in neo.lib.config

For SSL, the code may be simpler if we change for a single --ssl option that
takes 3 paths. Not done to not break compatibility. Hence, the hack with
an extra OptionList class in neo.lib.app

A new functional test tests the 'neomigrate' script, instead of just the
internal API to migrate data.
      9f1e4eef
  15. 20 Mar, 2018 2 commits
  17. 28 Dec, 2016 1 commit
  19. 27 Nov, 2016 1 commit
  21. 25 Nov, 2016 1 commit
  23. 03 Jun, 2014 1 commit
  25. 17 Dec, 2013 1 commit
  27. 20 Mar, 2012 1 commit
    • Julien Muchembled's avatar
      Review logging to keep all debugging information in RAM and flush only if useful · 1fce5cc4
      Julien Muchembled authored 
      The main goal of this patch is to keep all DEBUG logs and packet logger enabled
without exploding disk usage.
This is done by keeping the last 16 MB (by default) of debugging information in
a RAM buffer, and to emit it to an SQLite DB upon RTMIN signal or in case of
warning and more severe log.

Implementation is also cleaned up for better integration within a framework
or if run standalone. NEO logger is now a direct child of root handler.
Only warnings and more severe logs are forwarded to root handler.

A new script 'neolog' is added to pretty-print the contents of the SQLite log.

In unit tests, logging events are not buffered but emitted immediately.
When a test passes, payloads of all exchanged packets are discarded to reduce
disk usage on test bots.

This slows down performance tests by about 15 % because even if nothing is
written to disk, debug and packet log records are now always rendered.
      1fce5cc4
  29. 13 Mar, 2012 1 commit
  31. 24 Feb, 2012 1 commit
    • Julien Muchembled's avatar
      Implements backup using specialised storage nodes and relying on replication · 8e3c7b01
      Julien Muchembled authored 
      Replication is also fully reimplemented:
- It is not done anymore on whole partitions.
- It runs at lowest priority not to degrades performance for client nodes.

Schema of MySQL table is changed to optimize storage layout: rows are now
grouped by age, for good partial replication performance.
This certainly also speeds up simple loads/stores.
      8e3c7b01
  33. 09 Nov, 2011 1 commit
  35. 26 Oct, 2011 1 commit
  37. 09 Sep, 2011 2 commits
  39. 05 Sep, 2011 1 commit
  41. 15 Jun, 2011 6 commits
  43. 14 Jun, 2011 3 commits
  45. 24 Mar, 2011 1 commit
  47. 23 Mar, 2011 2 commits
  49. 08 Feb, 2011 2 commits
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备14008524号