neoppod:v1.8.1 commitshttps://lab.nexedi.com/nexedi/neoppod/-/commits/v1.8.12017-11-07T10:45:47+01:00https://lab.nexedi.com/nexedi/neoppod/-/commit/03b5b47eaecad8dca4d4deac422020598e79f8aaRelease version 1.8.12017-11-07T10:45:47+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/4feea25f68d06df7ac289a416a88e22cb681dd72neomigrate: fix typo in a warning message2017-10-27T15:58:31+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/65bc5c118fbc5692e71c76b206ac897b0677ada6fixup! storage: fix possible crash when delaying replication requests2017-09-29T20:14:40+02:00Julien Muchembledjm@nexedi.com
This reverts commit <a href="/nexedi/neoppod/-/commit/d3c22487c0fc207b543816ddc6170944ce4b4d3f" data-original="d3c22487c0fc207b543816ddc6170944ce4b4d3f" data-link="false" data-link-reference="false" data-project="72" data-commit="d3c22487c0fc207b543816ddc6170944ce4b4d3f" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="storage: fix possible crash when delaying replication requests" class="gfm gfm-commit has-tooltip">d3c22487</a> partially
and fixes the bug in a much simpler way.https://lab.nexedi.com/nexedi/neoppod/-/commit/7e186442e561d70afdc61a02ab1a6a6d341f3e9aneoctl: make cell padding consistent when displaying the partition table2017-09-29T18:22:48+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/d3c22487c0fc207b543816ddc6170944ce4b4d3fstorage: fix possible crash when delaying replication requests2017-09-29T16:10:27+02:00Julien Muchembledjm@nexedi.com
Traceback (most recent call last):
[...]
File "neo/storage/handlers/client.py", line 115, in askStoreObject
*e.args)
File "neo/lib/handler.py", line 333, in queueEvent
self.sortQueuedEvents()
File "neo/lib/handler.py", line 326, in <lambda>
self._event_queue.sort(key=key))()
File "neo/storage/transactions.py", line 67, in __lt__
return self.locking_tid < other.locking_tid
AttributeError: 'NoneType' object has no attribute 'locking_tid'
Pending events:
(None, <askFetchTransactions: ...>)
(<Transaction(C13, locking_tid=03c266508a058388, tid=None, age=0.21s) at 0x7f086bbc3d50>, <_askStoreObject: ...>)https://lab.nexedi.com/nexedi/neoppod/-/commit/49631a9f23492f886ec50cf1a8bfd68ee4295bc5qa: bug found in assignment of storage node ids, add test2017-09-11T19:44:00+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/aeeaef8942ac58c31b2af72f0c0ec08cbca23dafUpdate comment of RECOVERING state2017-09-05T17:35:30+02:00Julien Muchembledjm@nexedi.com
It was out-of-date since commit <a href="/kirr/neo/-/commit/23b6a66a1e83084b3c825189fdeaf15e6a41d00c" data-original="23b6a66a1e83084b3c825189fdeaf15e6a41d00c" data-link="false" data-link-reference="false" data-project="73" data-commit="23b6a66a1e83084b3c825189fdeaf15e6a41d00c" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="Reimplement election (of the primary master)" class="gfm gfm-commit has-tooltip">23b6a66a</a>.https://lab.nexedi.com/nexedi/neoppod/-/commit/524ec26973a7867771e9145cd6f1263b1f7fd8f4Add support for OpenSSL >= 1.12017-08-29T01:36:58+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/0868de70ab20bd1d2de8a8704e5ba19ebe55c5c8mysql: more index hints2017-07-11T16:27:16+02:00Julien Muchembledjm@jmuchemb.eu
This time we had issues with _getObject and getReplicationObjectList
on production.
It seems related to the upgrade of MariaDB, from 10.0.21 to 10.1.25.
See also commits <a href="/kirr/neo/-/commit/eaa00a88416c10928f82ed37356756a64298d848" data-original="eaa00a88416c10928f82ed37356756a64298d848" data-link="false" data-link-reference="false" data-project="73" data-commit="eaa00a88416c10928f82ed37356756a64298d848" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="mysql: force _getNextTID() to use appropriate/whole index" class="gfm gfm-commit has-tooltip">eaa00a88</a>
and <a href="/kirr/neo/-/commit/13911ca3d871a427fdf098c00558023ab76683a8" data-original="13911ca3d871a427fdf098c00558023ab76683a8" data-link="false" data-link-reference="false" data-project="73" data-commit="13911ca3d871a427fdf098c00558023ab76683a8" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="mysql: fix use of wrong SQL index when checking for dropped partitions" class="gfm gfm-commit has-tooltip">13911ca3</a>https://lab.nexedi.com/nexedi/neoppod/-/commit/166825536677282a7f4a3f69229e9d7997b33c04Release version 1.82017-07-04T16:04:33+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/adb2fbeca462f9d1cefc2d3106e5fc22fb70a6feREADME: update URLs2017-07-04T16:02:59+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/dfa8440a4ff087f8f20db0cb30ffa31c177cf18bREADME: update wrt added support for RocksDB and recent ZODB2017-06-30T18:14:13+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/3771fe3bb3ea6baa148a2043230b72fab445cbe2storage: update DatabaseManager.getLastTID docstring2017-06-29T12:03:02+02:00Julien Muchembledjm@nexedi.com
The explanation became wrong during a git-rebase, when it was decided
to keep the old code that drop partitions. The new one needs more work
and it is kept in a branch.https://lab.nexedi.com/nexedi/neoppod/-/commit/66743daa76f669c86292e1fbfbdef8cfb01c0fcfneolog: new --decompress option2017-06-16T13:01:01+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/4b34d91057ece5536341a3a470852258430b317ddoc: update TODO about missing invalidations in read-only mode2017-06-15T21:19:04+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/3bd2fbc973de057a701e4a4b7d322f57293779dbmysql: remove obsolete comment about broken PARTITIONing support2017-06-15T21:19:04+02:00Julien Muchembledjm@nexedi.com
Using NEO 0.9.1 and partitioning enabled, I could reproduce the issue with
MariaDB 5.3.5, but not with MariaDB 5.3.12 and 5.5.23. I suppose it was fixed.
In testOudatedCellsOnDownStorage,
'select count(*) from obj' returned a wrong value (always 1).
Strangely, 'select count(*) from test_neo0.obj' was always correct (102).https://lab.nexedi.com/nexedi/neoppod/-/commit/d75e7b89d81fcf95d8db6b96ce4aaea155cda617qa: make ClusterPdb compatible with the simple pdb of neo.tests2017-06-14T12:27:48+02:00Julien Muchembledjm@nexedi.com
When 'neo.tests.cluster' is loaded (usually when functional tests are run),
__builtin__.pdb is replaced by an extended pdb, which should behave the same
way if it is used like the former.
winpdb is so slow that a console pdb is often preferred.https://lab.nexedi.com/nexedi/neoppod/-/commit/71421744bf71dba69c9e6717a252921211446204client: fix NameError when a secondary master reports that it's not the primary2017-06-13T20:05:49+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/c995200a045eed183ab8e29fadeff26a0aa247d3storage: new --disable-drop-partitions option2017-06-12T13:13:32+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/6ee3504167fdf7efee077d97e178dc5f4c4f7dacqa: add testDropPartitions2017-06-12T13:13:32+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/32988e0459b3b6754c834126c1520f764d6ffebbBetter use of __import__2017-06-12T11:49:08+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/48e29a3f2012cbe7c9f845426c892d75f59423cdqa: update list of excluded tests in testSSL2017-06-12T11:48:57+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/2ca7c3354232bbb56c41ae99537ed904cb4e24e1master: improve algorithm to tweak the partition table2017-06-12T11:48:50+02:00Julien Muchembledjm@nexedi.com
The most important change is that it does not discard readable cells too
quickly anymore. A partition can now have multiple FEEDING cells, to avoid
going below the wanted level of replication.
The new algorithm is also better at minimizing the amount replication.https://lab.nexedi.com/nexedi/neoppod/-/commit/ca58ccd7fe93522aa440cbabad4d1d1ba2a47b7cstorage: ignore unassigned partitions when looking for last oids/tids2017-06-12T11:48:38+02:00Julien Muchembledjm@nexedi.com
The MySQL implementation is written to work around the issue reported at
<a href="https://jira.mariadb.org/browse/MDEV-12867" rel="nofollow noreferrer noopener" target="_blank">https://jira.mariadb.org/browse/MDEV-12867</a>https://lab.nexedi.com/nexedi/neoppod/-/commit/3600e59c4f2088dece00ad7e05450bd4a0dda30bneolog: new option to hide the node column2017-06-12T11:48:28+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/f6eb02b44b54f55fa92bed0b9d4eeeaf9b29268fRemove packet timeouts2017-05-12T15:59:11+02:00Julien Muchembledjm@nexedi.com
Since it's not worth anymore to keep track of the last connection activity
(which, btw, ignored TCP ACKs, i.e. timeouts could theorically be triggered
before all the data were actually sent), the semantics of closeClient has also
changed. Before this commit, the 1-minute timeout was reset whenever there was
activity (connection still used as server). Now, it happens exactly 100 seconds
after the connection is not used anymore as client.https://lab.nexedi.com/nexedi/neoppod/-/commit/9b70f88f4d1a6fead6d5cca8af7772ddcb56fe41Use TCP keepalives instead of applicative pings2017-05-12T15:59:11+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/d51920c0a188ebb796cd121cd1d82b385d4834c0Remove unused 'on_timeout' feature on connections2017-05-12T15:59:11+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/735fb9d180129f7c5f595f38d2c0aea74c1bb7f0setup.py: use SHA256 instead of MD5 to check the integrity of mock.py2017-05-12T15:59:11+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/86b7ebbd48314bcde38b6ebdc3903cad98fde1e3storage: prevent 2 nodes from working with the same database2017-05-12T15:29:03+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/8d42a2e602b2b3d1efb03b23434f98505005344aclient: remove useless debug log2017-05-11T17:06:05+02:00Julien Muchembledjm@nexedi.com
The next line (MTClientConnection) already logs new connections
and the storage node is necessarily in RUNNING state.https://lab.nexedi.com/nexedi/neoppod/-/commit/09bc404f7b816c02eeced463899ffa36e266fd45Make the identification of the primary master easier with 'neoctl print node'2017-05-10T14:18:57+02:00Julien Muchembledjm@nexedi.com
Now, the primary master is the running master with None displayed in the
last column. Before, it could be the id timestamp of when it was secondary,
which was obsolete information.https://lab.nexedi.com/nexedi/neoppod/-/commit/27c27f0b8e6989b6a6dbdbdcd96e251bc58c8a7fclient: fix AttributeError when connected to a master that happens to be seco...2017-05-10T13:55:17+02:00Julien Muchembledjm@nexedi.com
This fixes up commit <a href="/nexedi/neoppod/-/commit/23b6a66a1e83084b3c825189fdeaf15e6a41d00c" data-original="23b6a66a1e83084b3c825189fdeaf15e6a41d00c" data-link="false" data-link-reference="false" data-project="72" data-commit="23b6a66a1e83084b3c825189fdeaf15e6a41d00c" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="Reimplement election (of the primary master)" class="gfm gfm-commit has-tooltip">23b6a66a</a>,
which reimplements election.
poll raised, retrying
Traceback (most recent call last):
...
File "neo/client/handlers/master.py", line 41, in notPrimaryMaster
super(PrimaryNotificationsHandler, self).notPrimaryMaster(*args)
File "neo/lib/handler.py", line 157, in notPrimaryMaster
assert primary != self.app.server
File "neo/client/app.py", line 109, in __getattr__
return self.__getattribute__(attr)
AttributeError: 'Application' object has no attribute 'server'https://lab.nexedi.com/nexedi/neoppod/-/commit/1a093d76bf2de66ace1abc86b6cc46a98272ba96qa: speed up SQlite tests by accessing DBs in unsafe mode (e.g. no sync)2017-05-04T12:41:44+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/fbcf9c506bc01bf20f3d6ff330907499096e8fa1master: fix identification of unknown masters2017-05-02T14:49:24+02:00Julien Muchembledjm@nexedi.com
This fixes the following crash:
Traceback (most recent call last):
...
File "neo/master/handlers/identification.py", line 94, in requestIdentification
uuid = app.getNewUUID(uuid, address, node_type)
File "neo/master/app.py", line 449, in getNewUUID
assert uuid != self.uuid
AssertionErrorhttps://lab.nexedi.com/nexedi/neoppod/-/commit/29e8323c770689b17bb88fb46ba4d4adbc13213aBetter logging of connector errors2017-04-28T22:40:09+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/d33cedc021c76add8c3dc06a860be04df70f3ccbqa: relax assertion in testDeadlockAvoidanceBeforeInvolvingAnotherNode2017-04-28T15:42:01+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/46c3646530e0e8fe6d6615d1f1aaa6ddeeefbc13client: fix possible data corruption after conflict resolutions with replicas2017-04-28T15:26:08+02:00Julien Muchembledjm@nexedi.com
This really fixes the bug described in
commit <a href="/nexedi/neoppod/-/commit/40bac312c3bf45681ca53bb7453c8b39e2f6b28f" data-original="40bac312c3bf45681ca53bb7453c8b39e2f6b28f" data-link="false" data-link-reference="false" data-project="72" data-commit="40bac312c3bf45681ca53bb7453c8b39e2f6b28f" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="Fix sorting of delayed events" class="gfm gfm-commit has-tooltip">40bac312</a>,
which could probably be reverted. It only reduced the probability of failure.
What happened is that the second conflict on 'a' for t3 what first reported by
an answer to first store with:
- a base serial at which a=0
- a conflict serial at which a=7
However, the cached data is not 8 anymore but 12, since a second store already
occurred after the first conflict (reported by the other storage node).
When this conflict was resolved before receiving the conflict for second store,
it gave:
resolve(old=0, saved=7, new=12) -> 19
instead of:
resolve(old=4, saved=7, new=12) -> 15
(if we still had the data of the first store, we could also do
resolve(old=0, saved=7, new=8)
but that would be inefficient from a memory point of view)
The bug was difficult to reproduce. testNotifyReplicated had to be run many
many times before that race conditions trigger it. The test was changed to
enforce some of them, and the above scenario now happens almost always.https://lab.nexedi.com/nexedi/neoppod/-/commit/3dabba3a3eb36a776c97630948d93e48d5db1cbeqa: new --readable-tid runner option2017-04-27T20:42:29+02:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/nexedi/neoppod/-/commit/2b65e3ae6d230ad3a236c61d09e21082f36c58a4qa: make Patch work on functions2017-04-27T20:42:29+02:00Julien Muchembledjm@nexedi.com