1. 21 Dec, 2023 4 commits
    • Levin Zimmermann's avatar
      Revert "Y client: Adjust URI scheme to move client-specific options to fragment" · 736b4686
      Levin Zimmermann authored
      This reverts commit kirr/neo@4c9414ea.
      This patch was added at a time when nexedi/neoppod!18 wasn't
      resolved yet, but we already wanted to proceed with WCFS. Now the NEO MR
      is resolved and we decided to mostly leave the NEO zurl as it was
      originally implemented in nexedi/neoppod!6.
      This means we don't need this patch anymore which changed the NEO
      zurl format.
      736b4686
    • Levin Zimmermann's avatar
      client.go: Fix URI client option parsing for supported + unsupported options · f8531ca5
      Levin Zimmermann authored
      Before this patch, the parser ignored options which were already supported
      by the client (for instance 'read-only') and even raised an error. But the
      client can already use this option: as a9246333 describes this should happen in the
      local storage URL parser.
      
      Furthermore not-yet-supported client options (for instance compress) broke
      the NEO client before this patch. Now these options only raise a warning which
      informs the user that they are ignored. Why? We want to use pre-complete NEO
      in real-world projects together with NEO/py clusters. Those real-world projects
      may already specify options which aren't supported by our NEO/go client yet.
      But it doesn't matters so much, because those options are mostly
      relevant for other NEO/py cluster clients (e.g. zope nodes). Instead of
      filtering those parameters before parsing them to NEO/go in a higher
      level (e.g. SlapOS), NEO/go should already support any valid NEO URL
      and raise warnings for not yet implemented features.
      f8531ca5
    • Levin Zimmermann's avatar
      client_test: Add tests for NEO URI parser · 1c724899
      Levin Zimmermann authored
      This test was missing so far. Particularly recent changes of the
      NEO URI scheme [1], but also problems with valid old URI [2] stressed
      out the necessity for comprehensive NEO URI parser tests.
      
      [1] kirr/neo@4c9414ea
      [2] 573514c6 (comment 184417)
      1c724899
    • Levin Zimmermann's avatar
      client: Refactor openClientByURL for easier testing · 7904f767
      Levin Zimmermann authored
      With all the recent changes of the NEO URI scheme we need to reliably test
      the function which parses the URI and convert it into the different
      parameter. Testing is much simpler if we can only analyse how the URI
      parsing works. Therefore this patch moves NEO URI parsing to an
      external function.
      7904f767
  2. 02 Jun, 2023 1 commit
    • Levin Zimmermann's avatar
      zodb/storage: Delegate zurl opts extract to drivers · b6222954
      Levin Zimmermann authored
      We can't extract storage client options in general-purpose zodburi
      opener, because zodburi aren't standardized. Each storage may define its
      own syntax, therefore it is better if this is done decentralized and
      independent in each local storage driver.
      b6222954
  3. 24 May, 2023 2 commits
    • Kirill Smelkov's avatar
      Y client: Adjust URI scheme to move client-specific options to fragment · 4c9414ea
      Kirill Smelkov authored
      For example option `compress` controls kind of compression that _client_
      performs when saving data to server. Similarly cache-size, logfile and
      read-only adjust on-client behaviour, not server.
      
      From nexedi/neoppod!18 (comment 124725) :
      
          In general the most correct thing to do is:
      
          - use host part for where to connect (host:port, list of host ports, UNIX socket, etc)
          - use path part to identify a database or other on-server resource
          - use query part for parameters that are passed to remote server (e.g. `storage` in case of ZEO)
          - use fragment part for local parameters that are not passed to remote server (e.g. local `logfile`)
          - use credentials part for things required to authenticate/encrypt.
      
          To normalize an URL wcfs client would drop credentials and fragment, but keep host, path and query.
      
          Fragments are documented not to be sent to remote side and to be evaluated by local side only.
      
      -> Move options that control client behaviour to fragment.
      4c9414ea
    • Kirill Smelkov's avatar
      Y client: Don't allow master_nodes and name to be present in options · 6047f893
      Kirill Smelkov authored
      Because list of masters and cluster name must be already present in
      netloc and path. Previously e.g.
      
      	neo://α,β,γ/db?master_nodes=a,b,c"
      
      would mean to use master nodes {a,b,c} not {α,β,γ}. Now it is treated as
      invalid URI to remove ambiguity. Same for cluster name.
      6047f893
  4. 18 Jan, 2023 3 commits
  5. 04 Nov, 2022 1 commit
  6. 03 Nov, 2022 1 commit
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · a875f56c
      Kirill Smelkov authored
      * master:
        go/neo/t/neotest: Use python -c 'print ...' in a way that works on both py2 and py3
        go/neo/t/tzodb.py: Fix zhash for Python3
      a875f56c
  7. 18 May, 2022 2 commits
    • Kirill Smelkov's avatar
      go/neo/t/neotest: Use python -c 'print ...' in a way that works on both py2 and py3 · 789c9ed9
      Kirill Smelkov authored
      Without parenthesis it was failing on py3:
      
      	(neo) (py3.venv) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t$ ./neotest info-local
      	date:   Wed, 18 May 2022 11:05:50 +0300
      	xnode:  kirr@deca.navytux.spb.ru (2401:5180:0:af::1 192.168.0.3 (+ 1·ipv4))
      	uname:  Linux deca 5.10.0-13-amd64 #1 SMP Debian 5.10.106-1 (2022-03-17) x86_64 GNU/Linux
      	cpu:    Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
      	  File "<string>", line 1
      	    print '%.2fGHz' % (400000 / 1E6)
      	          ^
      	SyntaxError: invalid syntax
      789c9ed9
    • Kirill Smelkov's avatar
      go/neo/t/tzodb.py: Fix zhash for Python3 · 30329f5a
      Kirill Smelkov authored
      On py3 dict.keys() returns iterator instead of list:
      
          $ ./tzodb.py zhash
          Traceback (most recent call last):
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t/./tzodb.py", line 141, in <module>
              main()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t/./tzodb.py", line 138, in main
              zhash()
            File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/neo/t/./tzodb.py", line 59, in zhash
              optv, argv = getopt(sys.argv[2:], "h", ["help", "check=", "bench="] + hashRegistry.keys())
          TypeError: can only concatenate list (not "dict_keys") to list
      30329f5a
  8. 25 Nov, 2021 2 commits
  9. 24 Nov, 2021 1 commit
    • Kirill Smelkov's avatar
      X go.mod: v↑ go123 · dfe935b7
      Kirill Smelkov authored
      Instead of requiring users to explicitly mark their tests as finished in
      particular subject-to-deadlock places, tracetest was fixed to
      unconditionally detect "extra sends" deadlocks automatically:
      
      go123@3b19f68c
      dfe935b7
  10. 19 Nov, 2021 2 commits
  11. 12 Nov, 2021 1 commit
    • Kirill Smelkov's avatar
      X go.mod: v↑ * · ea538368
      Kirill Smelkov authored
      go get: upgraded github.com/fsnotify/fsnotify v1.4.10-0.20200417215612-7f4cf4dd2b52 => v1.5.1
      go get: upgraded github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b => v1.0.0
      go get: upgraded github.com/gwenn/gosqlite v0.0.0-20201008200117-82e079acf5b6 => v0.0.0-20211101095637-b18efb2e44c8
      go get: upgraded github.com/gwenn/yacr v0.0.0-20200112083327-bbe82c1f4d60 => v0.0.0-20211101095056-492fb0c571bc
      go get: upgraded github.com/kisielk/og-rek v1.1.0 => v1.2.0
      go get: upgraded github.com/soheilhy/cmux v0.1.5-0.20210205191134-5ec6847320e5 => v0.1.5
      go get: upgraded github.com/tinylib/msgp v1.1.5 => v1.1.6
      go get: upgraded golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 => v0.0.0-20211111160137-58aab5ef257a
      go get: upgraded golang.org/x/sys v0.0.0-20210301091718-77cc2087c03b => v0.0.0-20211111213525-f221eed1c01e
      go get: upgraded golang.org/x/text v0.3.5 => v0.3.7
      go get: upgraded lab.nexedi.com/kirr/go123 v0.0.0-20210128150852-c20e95f0f789 => v0.0.0-20210906140734-c9eb28d9e408
      
      Highlights:
      
      - og-rek brings support for pickle protocol 5
      - go123  brings in support for Go1.17
      ea538368
  12. 04 Oct, 2021 13 commits
  13. 01 Oct, 2021 2 commits
  14. 08 Sep, 2021 5 commits
    • Kirill Smelkov's avatar
      X go/neo: Fix credentials parsing with go1.17 · 87199da2
      Kirill Smelkov authored
      Due to security concerns, go1.17, even though very reluctantly,
      similarly to Python broke backward compatibility and stopped to treat ';'
      as separator when parsing URL queries.
      
      We still want to accept ';' as separator in credential part.
      -> Work it around by replacing ';' -> to '&'.
      
      See https://golang.org/doc/go1.17#semicolons and https://github.com/golang/go/issues/25192 for context.
      
      See also: cf685fb5 (fixup! Y client: Fix URI scheme to move credentials out of query)
      87199da2
    • Kirill Smelkov's avatar
      fixup! Y client: Fix URI scheme to move credentials out of query · cf685fb5
      Kirill Smelkov authored
      parse_qsl no longer treats ';' as valid query separator for security
      reason because most proxies did not do so and it was possible to poison
      proxy cache due to difference in query separator handling (see bugs.python.org/issue42967).
      
      To handle credentials we don't have any proxy here, and it is still
      perfectly valid to use ';' as credentials separator.
      
      -> Fix it with ';' -> '&' replace workaround, before feeding credentials
      string to parse_qsl.
      
      Amends: b9a42957.
      cf685fb5
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · b7a42db0
      Kirill Smelkov authored
      * master:
        go/internal/xtesting: Add missing X
        go/zodb/{fs1,zeo}: ~staticcheck
        go/zodb/btree: Fix missing return on data-consistency error
        go/zodb, go/zodb/btree: Fix go generate after rename on zodbtools side
      b7a42db0
    • Kirill Smelkov's avatar
      go/internal/xtesting: Add missing X · e7bea476
      Kirill Smelkov authored
      staticcheck says:
      
      	xtesting.go:386:2: this value of err is never used (SA4006)
      e7bea476
    • Kirill Smelkov's avatar
      go/zodb/{fs1,zeo}: ~staticcheck · 3ff4eb14
      Kirill Smelkov authored
      fs1/format.go:204:2: only the first constant in this group has an explicit type (SA9004)
      zeo/proto.go:56:2: only the first constant in this group has an explicit type (SA9004)
      3ff4eb14