neoppod:6275f7c6c51115c62b55038af6926cabdb5bd7de commitshttps://lab.nexedi.com/nexedi/neoppod/-/commits/6275f7c6c51115c62b55038af6926cabdb5bd7de2015-10-26T19:05:32+01:00https://lab.nexedi.com/nexedi/neoppod/-/commit/6275f7c6c51115c62b55038af6926cabdb5bd7deRelease version 1.5.12015-10-26T19:05:32+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/7469e55bba9c51705f2ce7451c2cebc69538a9e6storage: faster resumption when many transactions have already been imported ...2015-10-26T13:25:29+01:00Julien Muchembledjm@nexedi.com
The previous SQL query caused a full table scan of the 'trans' table at startup.https://lab.nexedi.com/nexedi/neoppod/-/commit/0f0700a8a2cca9402bc0cfb3671977fc73be5987tests: regenerate patch to ZODB3 using git2015-10-21T20:25:20+02:00Julien Muchembledjm@nexedi.com
I used git-diff for each file and concatenated the result to preverse the order.https://lab.nexedi.com/nexedi/neoppod/-/commit/badd9de3af0b2e0ced77296be9dc5cbb14891f08client: add assertion in cache to detect wrong invalidation2015-10-21T11:41:45+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/9682722b1c12b63e520d396d13abb609399eb65bDo not send invalidations for objects on which only readCurrent was called2015-10-21T02:49:02+02:00Julien Muchembledjm@nexedi.com
This fixes invalid next_serial entries in cache,
and the following error for values not in cache:
Traceback (most recent call last):
File "ZODB/Connection.py", line 856, in setstate
self._setstate(obj)
File "ZODB/Connection.py", line 894, in _setstate
self._load_before_or_conflict(obj)
File "ZODB/Connection.py", line 922, in _load_before_or_conflict
if not self._setstate_noncurrent(obj):
File "ZODB/Connection.py", line 945, in _setstate_noncurrent
assert end is not None
AssertionErrorhttps://lab.nexedi.com/nexedi/neoppod/-/commit/b3522b1b63a8c311b4f0e9ef13b1fb01a1a5a45eimporter: fix crash when aborting transaction2015-10-20T18:37:00+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/68401e7031b3711dfd1c85df04c87b9783d47858mysql: use fewer queries to fill (t)obj when storing a transaction2015-10-19T16:25:22+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/b70b4689869300e48c6a696ed1b32722ef20387amysql: refuse to start if max_allowed_packet is too small2015-10-19T16:25:22+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/f4a66782565ba5423fd1bc11cd680b49f670431cDo not flood logs when a client node sends a big packet in threaded tests2015-10-19T16:25:22+02:00Julien Muchembledjm@nexedi.com
When run with MySQL, testBasicStore (neo.tests.threaded.test.Test) was slow
and generated log exceeded 29MB.https://lab.nexedi.com/nexedi/neoppod/-/commit/c9658ff32a439317a536f7ee56090b16a6613ed4Importer: fix retrieval of an object from ZODB when next serial in NEO2015-10-19T16:25:22+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/f1bc3c32c5cfa7f5a1df90461c673d3e9cb375e9storage: speed up checking of replicas2015-10-16T20:29:39+02:00Julien Muchembledjm@nexedi.com
This increases the number of rows to check per AskCheck*Range packets.https://lab.nexedi.com/nexedi/neoppod/-/commit/7af9d2d34fdd2b3b80040221bff653637bd98396storage: partially fix a potential crash during replication2015-10-13T20:44:07+02:00Julien Muchembledjm@nexedi.com
And document 3 bugs found by running many times testBackupNodeLost. About the
tic() issue, I had a case where the test exited instead of looping forever after
the storage crash.https://lab.nexedi.com/nexedi/neoppod/-/commit/6da59ae833db36463ad8c10badc40dc8186b1363storage: fix crash when a corruption is found while checking TIDs2015-10-12T15:37:28+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/9bd14bf28e0a99b6366f2668d310bf30af745a8cRelease version 1.52015-10-05T19:43:23+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/bff5c82f4bc3215e60e6f3efe0ce5b52bb578222Add SSL support2015-10-05T17:09:59+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/107ca7dfb6981f8c963a6c9d5fcacaa5605b6d9fneoctl: make -l option log everything on disk automatically2015-10-05T17:01:48+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/9d26bb513f294ef4b77bccf590e2ebef40d44acdIn importer.conf example, explain why the source DB can't be open read-only2015-10-02T20:01:33+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/f2babb121fc098033364c1c9253d31215adf049eExpand ~(user) construction for all paths in configuration2015-10-02T18:40:58+02:00Julien Muchembledjm@nexedi.com
Before, it was only done for 'logfile'.https://lab.nexedi.com/nexedi/neoppod/-/commit/57481c35685240af62660959e92688ce8436f96aReview API betweeen connections and connectors2015-10-01T16:20:46+02:00Julien Muchembledjm@nexedi.com
- Review error handling. Only 2 exceptions remain in connector.py:
- Drop useless exception handling for EAGAIN since it should not happen
if the kernel says the socket is ready.
- Do not distinguish other socket errors. Just close and log in a generic way.
- No need to raise a specific exception for EOF.
- Make 'connect' return a boolean instead of raising an exception.
- Raise appropriate exception when answer/ask/notify is called on a closed
non-MT connection.
- Add support for more complex connectors, which may need to write for a read
operation, or to read when there's pending data to send. This will be
required for SSL support (more exactly, the handshake will be done in
a transparent way):
- Move write buffer to connector.
- Make 'receive' fill the read buffer, instead of returning the read data.
- Make 'receive' & 'send' return a boolean to switch polling for writing.
- Tolerate that sockets return 0 as number of bytes sent.
- In testConnection, simply delete all failing tests, as announced
in commit <a href="/sfermigier/neo/-/commit/71e30fb9b8941b200dc1768d59a58e73fe5b354f" data-original="71e30fb9b8941b200dc1768d59a58e73fe5b354f" data-link="false" data-link-reference="false" data-project="778" data-commit="71e30fb9b8941b200dc1768d59a58e73fe5b354f" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="Remove useless testEvent" class="gfm gfm-commit has-tooltip">71e30fb9</a>.https://lab.nexedi.com/nexedi/neoppod/-/commit/36a32f23904d5594387c02b0e79de53f03a72448tests: add "last" symlink to last temporary directory2015-09-30T19:28:40+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/518c7588fd6daf13c087978fc4af58d30a40b62dAllow to specify log file in configuration file, and expand ~(user) construction2015-09-24T20:22:17+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/32b2d173d920797bc7b9fdf8dd9fbd2ab17ab190Move common command-line options to neo.lib.config2015-09-24T20:22:17+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/727899e2cab3460f8ddb1bac026ed8e5f549d33aPass app as first parameter of (*Client|Listening)Connection2015-09-24T20:22:17+02:00Julien Muchembledjm@nexedi.com
Application will hold SSL parameters.https://lab.nexedi.com/nexedi/neoppod/-/commit/7d5b155980afbc07eed092acc92f4d841ca7265bFix remaining memory leaks and make handler instances become singletons2015-09-24T20:22:16+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/9fdd750fbaa5510042094eb7a6bdec3cf4974ef0Simplify cleanup of HandlerSwitcher when closing a connection2015-09-23T19:24:09+02:00Julien Muchembledjm@nexedi.com
This frees a reference to the last handler and there's no need to make the
instance reusable.https://lab.nexedi.com/nexedi/neoppod/-/commit/aaf2251e72d272f391cb21b2a05876afc9e254declient: do nothing (instead of raising) if a closed Storage is closed again2015-09-23T19:24:02+02:00Julien Muchembledjm@nexedi.com
This follows the behaviour of FileStorage.https://lab.nexedi.com/nexedi/neoppod/-/commit/d75fcc597fbfca501e0104fb9fa497809a3346f3Fix leak of file descriptors in unit tests2015-09-23T18:57:07+02:00Julien Muchembledjm@nexedi.com
There remain only one leak in ClientApplicationTests.test_connectToPrimaryNode
because of Mock objects.https://lab.nexedi.com/nexedi/neoppod/-/commit/c88c6ac5a3412f09664f2dc9d16c8301c1cbe11aTODO: document which mock library we should use2015-09-15T18:14:35+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/a72ddfb30d0cc9e5a3cc95ad6fc5ac517798cd4cadmin: do not reset the list of known masters from configuration (or command...2015-09-15T17:03:08+02:00Julien Muchembledjm@nexedi.comadmin: do not reset the list of known masters from configuration (or command line) when reconnecting
This is questionable but a lot of NodeManager must be reviewed if we want to do
differently. At least, admin nodes now behave like clients.
https://lab.nexedi.com/nexedi/neoppod/-/commit/6f6d071daba4d9f7f3989af044651c8c658cc8c9Simplify setup of monkey-patches in threaded tests2015-09-15T16:58:37+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/3e1ed6a43b18fed8f2b4c5be2f0a6de9ea1ae42aSimplify polling thread in threaded apps2015-09-15T16:53:46+02:00Julien Muchembledjm@nexedi.com
It's been a long time that the polling thread never ends and don't need to be
restarted. On the other side, there will be a need for the admin to define a
different polling loop, hence the move from threaded_poll to threaded_app.https://lab.nexedi.com/nexedi/neoppod/-/commit/f5f42522a9b4ceb54bd80d34e345df2b9993001aMove code from neo.client to neo.lib, since admins will be also multi-threaded2015-09-15T16:48:55+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/50d25d007dc49edc9d2c11d71e9c2f9d4c7eda9aDrop 'background' mode completely in threaded tests2015-09-15T16:41:09+02:00Julien Muchembledjm@nexedi.com
It was still used to stop a cluster.https://lab.nexedi.com/nexedi/neoppod/-/commit/4253d24fe0c01b5ae66c87b6dcf709cbe1506573Stop using 'background' mode in threaded tests2015-09-15T16:37:44+02:00Julien Muchembledjm@nexedi.com
This makes tests easier to write, with more determinism.
If only I had the idea to monkey-patch SimpleQueue several years ago.https://lab.nexedi.com/nexedi/neoppod/-/commit/7025db52513639f881e5996c8a87850cdc4c3fa5Rewrite of scheduler for threaded tests2015-09-15T15:53:38+02:00Julien Muchembledjm@nexedi.com
The previous implementation was built around a 'pending' global variable that
was set by a few monkey-patches when some network activity was pending between
nodes. All this is replaced by an extra epoll object is used to wait for nodes
that have pending network events: this is simpler, and faster since it
significantly reduces the number of context switches.https://lab.nexedi.com/nexedi/neoppod/-/commit/610093411e2c24d68c40f5b2696660000754f407Thread.isAlive is deprecated2015-09-14T18:04:47+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/af06676a2404e688f7e4f30f3b593d083aab6057Fix potential deadlock when connecting to primary master2015-09-07T11:30:05+02:00Julien Muchembledjm@nexedi.com
This is a regression caused by commit <a href="/kirr/neo/-/commit/eef52c27bc9955f8e68f0442089afb8fc03987f7" data-original="eef52c27bc9955f8e68f0442089afb8fc03987f7" data-link="false" data-link-reference="false" data-project="73" data-commit="eef52c27bc9955f8e68f0442089afb8fc03987f7" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="Tickless poll loop, for lowest latency and cpu usage" class="gfm gfm-commit has-tooltip">eef52c27</a>
("Tickless poll loop, for lowest latency and cpu usage"), affecting:
- admins
- storages
- primary masters of backup clustershttps://lab.nexedi.com/nexedi/neoppod/-/commit/9531c9cb70e5f86747fd980822995bb3a2a526ddclient: drop now useless wrapper to log safely in poll thread during shutdown2015-08-28T20:52:09+02:00Julien Muchembledjm@nexedi.com
Recent Python already catches exceptions due to garbage collection on exit.https://lab.nexedi.com/nexedi/neoppod/-/commit/e27358d130e1222d5244d9bb686a769ed22a979dstorage: fix history() not waiting oid to be unlocked2015-08-28T20:52:09+02:00Julien Muchembledjm@nexedi.com
This fixes a random failure in testClientReconnection:
Traceback (most recent call last):
File "neo/tests/threaded/test.py", line 754, in testClientReconnection
self.assertTrue(cluster.client.history(x1._p_oid))
failureException: None is not truehttps://lab.nexedi.com/nexedi/neoppod/-/commit/79be7787add43ed9cd2d100cf87586466cb0cf6eFix random failure in testRecycledClientUUID2015-08-28T20:52:09+02:00Julien Muchembledjm@nexedi.com
Traceback (most recent call last):
File "neo/tests/threaded/test.py", line 838, in testRecycledClientUUID
x = client.load(ZERO_TID)
[...]
File "neo/tests/threaded/test.py", line 822, in notReady
m2s.remove(delayNotifyInformation)
File "neo/tests/threaded/__init__.py", line 482, in remove
del self.filter_dict[filter]
KeyError: <function delayNotifyInformation at 0x7f511063a578>