- 18 Dec, 2023 4 commits
-
-
Kirill Smelkov authored
Julien notes this is very likely unneeded: nexedi/neoppod!21 (diffs, comment 195929) We had it like this since 01a01c8c (client: Add support for zodburi), but I rechecked zodburi codebase now and it does not do any similar lowering anywhere. So drop support for case normalization in zurl options. /cc @levin.zimmermann /reviewed-by @jm /reviewed-on nexedi/neoppod!21 (cherry-picked from commit 798c9f25)
-
Kirill Smelkov authored
Unfortunately after creating SSL context it is not possible, or at least I could not find how, to retrieve original credentials with which the context was created. However wendelin.core needs to be able to take a client storage, reconstruct zurl to refer to that particular storage, and pass that zurl to wcfs, so that wcfs, in turn, could access the same ZODB database. Given a NEO client instance, it is already possible to retrieve master_nodes, cluster name, and detect whether SSL is being in use. However without being able to retrieve original SSL credentials, reconstructed zurl will not be full and wcfs won't be able to use exactly the same secrets as python part does. -> Help wendelin.core by remembering which ca/cert/key were used to build SSL context. This information is used by zstor_2zurl in wendelin.core here: https://lab.nexedi.com/nexedi/wendelin.core/blob/885b3556/lib/zodb.py#L390-418 /cc @levin.zimmermann /reviewed-by @jm /reviewed-on !21 (cherry-picked from commit 17af7f27)
-
Kirill Smelkov authored
Similarly to how it is done with e.g. http:// and https:// - if neos:// is given TLS usage is forced and ca/cert/key must be there either in the URI itself, or in $NEO_CA, $NEO_CERT and $NEO_KEY environment variables mimicking the way how e.g. for https:// TLS credentials are taken from host environment, not from the uri. The latter might be usability convenience, but is also useful for WCFS which needs to be able to remove secrets from uri on zurl normalization. Please see discussion at !18 (comment 184439) for details. /cc @levin.zimmermann /reviewed-by @jm /reviewed-on !21 (cherry-picked from commit bc3e38ea)
-
Kirill Smelkov authored
Because list of masters and cluster name must be already present in netloc and path. Previously e.g. neo://db@α,β,γ?master_nodes=a,b,c" would mean to use master nodes {a,b,c} not {α,β,γ}. Now it is treated as invalid URI to remove ambiguity. Same for cluster name. /cc @levin.zimmermann /reviewed-by @jm /reviewed-on !21 (cherry-picked from commit 22ccebd6)
-
- 11 May, 2021 2 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
(cherry picked from commit ba0bc779)
-
- 22 Mar, 2021 1 commit
-
-
Julien Muchembled authored
(cherry picked from commit fa581be5)
-
- 19 Aug, 2020 3 commits
-
-
Julien Muchembled authored
====================================================================== FAIL: check_tid_ordering_w_commit (neo.tests.zodb.testBasic.BasicTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "ZODB/tests/BasicStorage.py", line 397, in check_tid_ordering_w_commit self.assertEqual(results.pop('lastTransaction'), tids[1]) File "neo/tests/__init__.py", line 301, in assertEqual return super(NeoTestBase, self).assertEqual(first, second, msg=msg) failureException: '\x03\xd8\x85H\xbffp\xbb' != '\x03\xd8\x85H\xbfs\x0b\xdd' (cherry picked from commit f4cb59d2)
-
Julien Muchembled authored
This requires ZODB >= 5.6.0 (cherry picked from commit a7d101ec)
-
Julien Muchembled authored
(cherry picked from commit 43029be2)
-
- 22 May, 2020 2 commits
-
-
Julien Muchembled authored
This fixes the following assertion: Traceback (most recent call last): File "neo/master/app.py", line 172, in run self._run() File "neo/master/app.py", line 182, in _run self.playPrimaryRole() File "neo/master/app.py", line 302, in playPrimaryRole self.backup_app.provideService()) File "neo/master/backup_app.py", line 114, in provideService node, conn = bootstrap.getPrimaryConnection() File "neo/lib/bootstrap.py", line 74, in getPrimaryConnection poll(1) File "neo/lib/event.py", line 160, in poll to_process.process() File "neo/lib/connection.py", line 504, in process self._handlers.handle(self, self._queue.pop(0)) File "neo/lib/connection.py", line 92, in handle self._handle(connection, packet) File "neo/lib/connection.py", line 107, in _handle pending[0][1].packetReceived(connection, packet) File "neo/lib/handler.py", line 125, in packetReceived self.dispatch(*args) File "neo/lib/handler.py", line 75, in dispatch method(conn, *args, **kw) File "neo/lib/handler.py", line 159, in notPrimaryMaster assert primary != self.app.server AttributeError: 'BackupApplication' object has no attribute 'server' (cherry picked from commit dba07e72)
-
Julien Muchembled authored
-
- 07 Jan, 2020 1 commit
-
-
Julien Muchembled authored
-
- 28 Apr, 2019 1 commit
-
-
Julien Muchembled authored
-
- 27 Apr, 2019 12 commits
-
-
Julien Muchembled authored
The following 2 operations can be onerous and they should not be directly usable without some kind of confirmation by the user: - Dropping a node now requires to first stop it. - Tweaking does not exclude anymore automatically DOWN nodes, because a node could go DOWN between the moment the user sends the command to tweak and the actual tweak by the master.
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
Initially, I wanted to do the simulation inside neoctl but it has no knowledge of the topology (the master don't send devpath values of storage nodes). Therefore, the work is delegated to the master node, which implies a change of the protocol.
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
This stops abusing ProtocolError, which disconnects the admin node needlessly. The many 'if ... raise RuntimeError' in neo/neoctl/neoctl.py could be turned into assertions.
-
Julien Muchembled authored
-
Julien Muchembled authored
neoctl gets a new command to change the number of replicas. The number of replicas becomes a new partition table attribute and like the PT id, it is stored in the config table. On the other side, the configuration value for the number of partitions is dropped, since it can be computed from the partition table, which is always stored in full. The -p/-r master options now only apply at database creation. Some implementation notes: - The protocol is slightly optimized in that the master now sends automatically the whole partition tables to the admin & client nodes upon connection, like for storage nodes. This makes the protocol more consistent, and the master is the only remaining node requesting partition tables, during recovery. - Some parts become tricky because app.pt can be None in more cases. For example, the extra condition in NodeManager.update (before app.pt.dropNode) was added for this is the reason. Or the 'loadPartitionTable' method (storage) that is not inlined because of unit tests. Overall, this commit simplifies more than it complicates. - In the master handlers, we stop hijacking the 'connectionCompleted' method for tasks to be performed (often send the full partition table) on handler switches. - The admin's 'bootstrapped' flag could have been removed earlier: race conditions can't happen since the AskNodeInformation packet was removed (commit d048a52d).
-
Julien Muchembled authored
It is often faster to set up replicas by stopping a node (and any underlying database server like MariaDB) and do a raw copy of the database (e.g. with rsync). So far, it required to stop the whole cluster and use tools like 'mysql' or sqlite3' to edit: - the 'pt' table in databases, - the 'config.nid' values of the new nodes. With this new option, if you already have 1 replica, you can set up new replicas with such fast raw copy, and without interruption of service. Obviously, this implies less redundancy during the operation.
-
Julien Muchembled authored
-
- 26 Apr, 2019 4 commits
-
-
Julien Muchembled authored
--kill-mysqld should be combined with something like -f .3 -r .1 to give storage nodes enough time to recover. And also -D 0 to focus testing on the storage backend rather than NEO.
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
-
- 16 Apr, 2019 5 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
-
Julien Muchembled authored
This also reverts commit 442bb43a.
-
- 05 Apr, 2019 3 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
This fixes up commit be839e92.
-
Julien Muchembled authored
-
- 01 Apr, 2019 1 commit
-
-
Julien Muchembled authored
-
- 21 Mar, 2019 1 commit
-
-
Julien Muchembled authored
This is not used currently.
-