Commits · 0ed7b1fcb9d165274d73e3a9a71a3911275fd994 · Stefane Fermigier / neo

21 Feb, 2018 5 commits

X noturbo deco timings after speeding up zlib decompression · 0ed7b1fc

Kirill Smelkov authored Feb 21, 2018

benchstat -alpha 0.07 20180220-deco-linux4.15.4-noturbo.txt 20180221-deco-noturbo.txt

name old pystone/s new pystone/s delta
deco/pystone 219k ± 2% 220k ± 1% ~ (p=0.310 n=5+5)

name old time/op new time/op delta
deco/sha1/py/1K 1.62µs ± 0% 1.83µs ± 1% +13.15% (p=0.008 n=5+5)
deco/sha1/go/1K 1.53µs ± 0% 1.53µs ± 0% ~ (p=0.635 n=5+4)
deco/sha1/py/4K 5.78µs ± 0% 6.68µs ± 1% +15.56% (p=0.008 n=5+5)
deco/sha1/go/4K 5.59µs ± 1% 5.59µs ± 0% ~ (p=1.000 n=5+5)
deco/sha1/py/2M 2.82ms ± 0% 3.19ms ± 0% +13.05% (p=0.008 n=5+5)
deco/sha1/go/2M 2.77ms ± 0% 2.78ms ± 0% +0.20% (p=0.008 n=5+5)
deco/disk/randread/direct/4K-min 104µs ± 2% 104µs ± 0% ~ (p=0.651 n=5+5)
deco/disk/randread/direct/4K-avg 143µs ± 0% 143µs ± 1% ~ (p=0.317 n=5+5)
deco/disk/randread/direct/2M-min 5.30ms ± 4% 5.38ms ± 0% ~ (p=0.175 n=5+4)
deco/disk/randread/direct/2M-avg 6.18ms ± 1% 6.19ms ± 2% ~ (p=0.651 n=5+5)
deco/disk/randread/pagecache/4K-min 584ns ± 0% 576ns ± 1% -1.37% (p=0.016 n=5+5)
deco/disk/randread/pagecache/4K-avg 981ns ± 0% 974ns ± 1% ~ (p=0.071 n=5+5)
deco/disk/randread/pagecache/2M-min 193µs ± 5% 196µs ± 4% ~ (p=0.690 n=5+5)
deco/disk/randread/pagecache/2M-avg 211µs ± 0% 212µs ± 0% ~ (p=0.119 n=5+5)

name old time/object new time/object delta
deco/fs1/zhash.py 17.9µs ±13% 17.9µs ±14% ~ (p=0.983 n=8+10)
deco/fs1/zhash.py-P16 132µs ±19% 139µs ±18% +5.39% (p=0.054 n=32+32)
deco/fs1/zhash.go 3.00µs ± 7% 2.95µs ± 8% ~ (p=0.307 n=10+10)
deco/fs1/zhash.go+prefetch128 3.54µs ±16% 3.92µs ±25% ~ (p=0.071 n=10+10)
deco/fs1/zhash.go-P16 12.7µs ±64% 13.9µs ±51% ~ (p=0.254 n=31+32)
deco/zeo/zhash.py 369µs ±11% 382µs ± 1% +3.53% (p=0.045 n=10+6)
deco/zeo/zhash.py-P16 3.01ms ±12% 3.10ms ± 9% +2.83% (p=0.024 n=31+32)
deco/neo/py/sqlite/zhash.py 378µs ±13% 368µs ± 9% ~ (p=0.436 n=10+10)
deco/neo/py/sqlite/zhash.py-P16 3.21ms ± 3% 3.20ms ± 3% -0.41% (p=0.032 n=32+32)
deco/neo/py/sqlite/zhash.go 235µs ±24% 177µs ± 7% -24.82% (p=0.063 n=10+10)
deco/neo/py/sqlite/zhash.go+prefetch128 139µs ± 8% 129µs ± 5% -7.03% (p=0.003 n=10+10)
deco/neo/py/sqlite/zhash.go-P16 2.37ms ± 7% 2.18ms ± 6% -8.16% (p=0.001 n=32+32)
deco/neo/py(!log)/sqlite/zhash.py 355µs ±14% 349µs ± 9% ~ (p=0.579 n=10+10)
deco/neo/py(!log)/sqlite/zhash.py-P16 3.01ms ± 4% 3.01ms ± 4% ~ (p=0.989 n=32+32)
deco/neo/py(!log)/sqlite/zhash.go 213µs ±23% 161µs ± 7% -24.49% (p=0.060 n=10+10)
deco/neo/py(!log)/sqlite/zhash.go+prefetch128 123µs ± 9% 113µs ± 1% -8.23% (p=0.000 n=10+10)
deco/neo/py(!log)/sqlite/zhash.go-P16 2.12ms ± 8% 1.96ms ± 3% -7.32% (p=0.001 n=32+32)
deco/neo/py/sql/zhash.py 429µs ± 8% 417µs ±11% ~ (p=0.442 n=8+8)
deco/neo/py/sql/zhash.py-P16 4.57ms ± 3% 4.55ms ± 4% ~ (p=0.257 n=32+32)
deco/neo/py/sql/zhash.go 281µs ±19% 227µs ± 9% -18.94% (p=0.063 n=10+10)
deco/neo/py/sql/zhash.go+prefetch128 201µs ± 7% 184µs ± 5% -8.40% (p=0.002 n=10+10)
deco/neo/py/sql/zhash.go-P16 3.34ms ± 5% 3.10ms ± 2% -7.36% (p=0.000 n=32+32)
deco/neo/py(!log)/sql/zhash.py 404µs ±14% 404µs ±10% ~ (p=0.959 n=8+8)
deco/neo/py(!log)/sql/zhash.py-P16 4.34ms ± 2% 4.35ms ± 3% ~ (p=0.763 n=32+32)
deco/neo/py(!log)/sql/zhash.go 261µs ±19% 209µs ±10% -20.01% (p=0.061 n=10+10)
deco/neo/py(!log)/sql/zhash.go+prefetch128 180µs ± 7% 161µs ± 5% -10.49% (p=0.000 n=10+10)
deco/neo/py(!log)/sql/zhash.go-P16 3.00ms ± 6% 2.76ms ± 3% -7.93% (p=0.000 n=32+32)
deco/neo/go/fs1/zhash.py 220µs ± 2% 222µs ± 2% ~ (p=0.219 n=9+10)
deco/neo/go/fs1/zhash.py-P16 2.00ms ± 7% 1.98ms ± 8% ~ (p=0.112 n=32+32)
deco/neo/go/fs1/zhash.go 50.3µs ±13% 50.3µs ±13% ~ (p=0.747 n=10+10)
deco/neo/go/fs1/zhash.go+prefetch128 20.6µs ±23% 21.2µs ±24% ~ (p=0.423 n=10+10)
deco/neo/go/fs1/zhash.go-P16 325µs ±12% 334µs ±12% ~ (p=0.120 n=30+32)
deco/neo/go/fs1(!sha1)/zhash.go 39.2µs ± 3% 38.5µs ± 3% ~ (p=0.137 n=10+10)
deco/neo/go/fs1(!sha1)/zhash.go+prefetch128 15.3µs ±16% 15.7µs ±12% ~ (p=0.196 n=10+10)
deco/neo/go/fs1(!sha1)/zhash.go-P16 270µs ±31% 262µs ±22% ~ (p=0.245 n=32+30)
deco/neo/go/sqlite/zhash.py 292µs ±11% 282µs ±10% ~ (p=0.165 n=10+10)
deco/neo/go/sqlite/zhash.py-P16 2.46ms ± 7% 2.45ms ± 7% ~ (p=0.546 n=32+32)
deco/neo/go/sqlite/zhash.go 159µs ±28% 111µs ±15% -30.02% (p=0.060 n=10+10)
deco/neo/go/sqlite/zhash.go+prefetch128 80.4µs ±34% 44.6µs ±25% -44.51% (p=0.001 n=10+10)
deco/neo/go/sqlite/zhash.go-P16 1.13ms ±41% 0.79ms ± 9% -30.00% (p=0.000 n=32+32)
deco/neo/go/sqlite(!sha1)/zhash.go 155µs ±28% 109µs ±15% -29.98% (p=0.061 n=10+10)
deco/neo/go/sqlite(!sha1)/zhash.go+prefetch128 78.9µs ±36% 41.5µs ±21% -47.34% (p=0.000 n=10+10)
deco/neo/go/sqlite(!sha1)/zhash.go-P16 1.14ms ±25% 0.73ms ±11% -35.37% (p=0.000 n=32+32)

0ed7b1fc

X client: Cleanup a bit around decompressing · bf213313
Kirill Smelkov authored Feb 21, 2018

bf213313

X xzlib: Switch to github.com/DataDog/czlib to zlib Decompression · 57524491

Kirill Smelkov authored Feb 21, 2018

name old time/op new time/op delta
deco/unzlib/py/null-1K 2.12µs ± 1% 2.11µs ± 2% ~ (p=0.841 n=5+5)
deco/unzlib/go/null-1K 1.89µs ± 1% 2.27µs ± 1% +20.03% (p=0.008 n=5+5)
deco/unzlib/py/null-4K 13.5µs ± 4% 13.3µs ± 0% ~ (p=0.310 n=5+5)
deco/unzlib/go/null-4K 8.54µs ± 0% 8.91µs ± 0% +4.43% (p=0.008 n=5+5)
deco/unzlib/py/null-2M 5.20ms ±10% 5.31ms ± 1% ~ (p=0.548 n=5+5)
deco/unzlib/go/null-2M 2.58ms ± 1% 3.87ms ± 0% +50.13% (p=0.008 n=5+5)
deco/unzlib/py/wczdata-avg 24.1µs ± 1% 23.9µs ± 0% ~ (p=0.114 n=4+4)
deco/unzlib/go/wczdata-avg 68.0µs ± 1% 20.9µs ± 0% -69.29% (p=0.008 n=5+5)
deco/unzlib/py/wczdata-max 23.5µs ± 1% 23.5µs ± 0% ~ (p=0.556 n=4+5)
deco/unzlib/go/wczdata-max 67.8µs ± 0% 20.7µs ± 1% -69.45% (p=0.008 n=5+5)
deco/unzlib/py/prod1-avg 4.47µs ± 2% 4.44µs ± 1% ~ (p=0.341 n=5+5)
deco/unzlib/go/prod1-avg 11.0µs ± 0% 4.1µs ± 1% -62.39% (p=0.016 n=5+4)
deco/unzlib/py/prod1-max 326µs ± 0% 325µs ± 0% ~ (p=0.095 n=5+5)
deco/unzlib/go/prod1-max 542µs ± 0% 262µs ± 0% -51.71% (p=0.008 n=5+5)

There is also unsafe interface with czlib.UnsafeDecompress & friends which I
had not tried because even using safe interface brings ~ 3x speedup.

57524491

X xzlib: Reuse zlib decoders · 9acb87a8

Kirill Smelkov authored Feb 21, 2018

name old time/op new time/op delta
deco/unzlib/py/null-1K 2.11µs ± 1% 2.12µs ± 1% ~ (p=0.690 n=5+5)
deco/unzlib/go/null-1K 5.87µs ± 1% 1.89µs ± 1% -67.77% (p=0.008 n=5+5)
deco/unzlib/py/null-4K 13.3µs ± 1% 13.5µs ± 4% ~ (p=0.690 n=5+5)
deco/unzlib/go/null-4K 12.6µs ± 0% 8.5µs ± 0% -32.50% (p=0.008 n=5+5)
deco/unzlib/py/null-2M 5.17ms ±11% 5.20ms ±10% ~ (p=1.000 n=5+5)
deco/unzlib/go/null-2M 2.61ms ± 1% 2.58ms ± 1% -1.32% (p=0.016 n=5+5)
deco/unzlib/py/wczdata-avg 20.9µs ±14% 24.1µs ± 1% +15.05% (p=0.016 n=5+4)
deco/unzlib/go/wczdata-avg 73.0µs ± 1% 68.0µs ± 1% -6.94% (p=0.008 n=5+5)
deco/unzlib/py/wczdata-max 23.6µs ± 0% 23.5µs ± 1% ~ (p=0.905 n=5+4)
deco/unzlib/go/wczdata-max 72.7µs ± 0% 67.8µs ± 0% -6.79% (p=0.008 n=5+5)
deco/unzlib/py/prod1-avg 4.53µs ± 3% 4.47µs ± 2% ~ (p=0.310 n=5+5)
deco/unzlib/go/prod1-avg 15.5µs ± 0% 11.0µs ± 0% -29.11% (p=0.008 n=5+5)
deco/unzlib/py/prod1-max 327µs ± 1% 326µs ± 0% ~ (p=1.000 n=5+5)
deco/unzlib/go/prod1-max 545µs ± 0% 542µs ± 0% -0.68% (p=0.008 n=5+5)

still on wczdata and prod1 much slower compared to py/c zlib.

9acb87a8

X unzlib benchmarks · 924831e7
Kirill Smelkov authored Feb 21, 2018

924831e7

20 Feb, 2018 3 commits

. · 446029d4
Kirill Smelkov authored Feb 21, 2018

446029d4

X master, storage: Speak both NEO and HTTP protocol on the same port · c4776192

Kirill Smelkov authored Feb 20, 2018

We can detect which protocol a client speaks by analyzing the first few
bytes it sends. Then we can use HTTP for administrative & debugging tasks.

In particular HTTP right now allows to collect profiles and traces:

https://golang.org/pkg/net/http/pprof/

and requests tracing for places where it is used:

https://godoc.org/golang.org/x/net/trace

Even though it is a pity that such wrapping retains artifact that the
connection actually used is a wrapper connection over accepted one, the
performance impact is negligable:

name old time/object new time/object delta
deco/neo/go/fs1(!sha1)/zhash.go 31.6µs ± 2% 31.6µs ± 2% ~ (p=0.897 n=28+29)

c4776192

X noturbo deco timings for 4.12.13, 4.14.13, 4.15.4 and 4.15.4-nopti · f069dd10

Kirill Smelkov authored Feb 20, 2018

Generally pti adds quite overhead:

benchstat -split dataset 20180220-deco-linux4.15.4-noturbo.txt 20180220-deco-linux4.15.4-noturbo-nopti.txt

name old pystone/s new pystone/s delta
deco/pystone 219k ± 2% 222k ± 1% ~ (p=0.095 n=5+5)

name old time/op new time/op delta
deco/sha1/py/1K 1.62µs ± 0% 1.62µs ± 0% ~ (p=0.246 n=5+5)
deco/sha1/go/1K 1.53µs ± 0% 1.53µs ± 0% ~ (p=0.294 n=5+5)
deco/sha1/py/4K 5.78µs ± 0% 5.78µs ± 0% ~ (p=0.937 n=5+5)
deco/sha1/go/4K 5.59µs ± 1% 5.58µs ± 0% ~ (p=0.119 n=5+5)
deco/sha1/py/2M 2.82ms ± 0% 2.82ms ± 0% ~ (p=0.056 n=5+5)
deco/sha1/go/2M 2.77ms ± 0% 2.77ms ± 0% ~ (p=0.151 n=5+5)
deco/disk/randread/direct/4K-min 104µs ± 2% 103µs ± 2% ~ (p=0.302 n=5+5)
deco/disk/randread/direct/4K-avg 143µs ± 0% 142µs ± 1% ~ (p=0.056 n=5+5)
deco/disk/randread/direct/2M-min 5.30ms ± 4% 5.44ms ± 5% ~ (p=0.238 n=5+5)
deco/disk/randread/direct/2M-avg 6.18ms ± 1% 6.11ms ± 1% -1.20% (p=0.048 n=5+5)
deco/disk/randread/pagecache/4K-min 584ns ± 0% 359ns ± 1% -38.64% (p=0.008 n=5+5)
deco/disk/randread/pagecache/4K-avg 981ns ± 0% 745ns ± 0% -24.05% (p=0.008 n=5+5)
deco/disk/randread/pagecache/2M-min 193µs ± 5% 195µs ± 4% ~ (p=0.690 n=5+5)
deco/disk/randread/pagecache/2M-avg 211µs ± 0% 211µs ± 0% ~ (p=0.389 n=5+5)

name old time/object new time/object delta
dataset:wczblk1-8
deco/fs1/zhash.py 20.0µs ± 2% 19.4µs ± 2% -3.00% (p=0.029 n=4+4)
deco/fs1/zhash.py-P16 136µs ±15% 130µs ±22% ~ (p=0.346 n=16+16)
deco/fs1/zhash.go 3.20µs ± 0% 2.70µs ± 0% -15.62% (p=0.000 n=5+4)
deco/fs1/zhash.go+prefetch128 3.82µs ± 7% 3.90µs ± 8% ~ (p=0.397 n=5+5)
deco/fs1/zhash.go-P16 13.9µs ±79% 10.7µs ±82% ~ (p=0.055 n=16+16)
deco/zeo/zhash.py 378µs ± 8% 360µs ± 8% ~ (p=0.222 n=5+5)
deco/zeo/zhash.py-P16 3.16ms ± 4% 3.06ms ± 5% -3.12% (p=0.007 n=14+16)
deco/neo/py/sqlite/zhash.py 399µs ± 7% 380µs ± 3% -4.75% (p=0.032 n=5+5)
deco/neo/py/sqlite/zhash.py-P16 3.30ms ± 1% 3.18ms ± 2% -3.67% (p=0.000 n=16+16)
deco/neo/py/sqlite/zhash.go 288µs ± 1% 265µs ± 1% -8.11% (p=0.008 n=5+5)
deco/neo/py/sqlite/zhash.go+prefetch128 147µs ± 2% 139µs ± 2% -5.61% (p=0.008 n=5+5)
deco/neo/py/sqlite/zhash.go-P16 2.52ms ± 0% 2.32ms ± 1% -7.87% (p=0.000 n=14+13)
deco/neo/py(!log)/sqlite/zhash.py 380µs ± 7% 357µs ± 2% -6.01% (p=0.032 n=5+5)
deco/neo/py(!log)/sqlite/zhash.py-P16 3.10ms ± 2% 3.05ms ± 2% -1.75% (p=0.000 n=15+16)
deco/neo/py(!log)/sqlite/zhash.go 261µs ± 0% 244µs ± 1% -6.51% (p=0.008 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go+prefetch128 131µs ± 2% 126µs ± 1% -4.17% (p=0.008 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go-P16 2.28ms ± 0% 2.17ms ± 0% -4.82% (p=0.000 n=14+16)
deco/neo/py/sql/zhash.py 446µs ± 3% 424µs ± 6% ~ (p=0.200 n=4+4)
deco/neo/py/sql/zhash.py-P16 4.71ms ± 0% 4.50ms ± 1% -4.28% (p=0.000 n=15+13)
deco/neo/py/sql/zhash.go 330µs ± 1% 307µs ± 2% -6.97% (p=0.008 n=5+5)
deco/neo/py/sql/zhash.go+prefetch128 213µs ± 2% 198µs ± 1% -7.12% (p=0.008 n=5+5)
deco/neo/py/sql/zhash.go-P16 3.51ms ± 0% 3.22ms ± 0% -8.27% (p=0.000 n=15+13)
deco/neo/py(!log)/sql/zhash.py 471µs ±41% 452µs ±43% ~ (p=0.310 n=5+5)
deco/neo/py(!log)/sql/zhash.py-P16 4.41ms ± 0% 4.22ms ± 0% -4.21% (p=0.000 n=15+15)
deco/neo/py(!log)/sql/zhash.go 310µs ± 1% 288µs ± 2% -7.10% (p=0.008 n=5+5)
deco/neo/py(!log)/sql/zhash.go+prefetch128 189µs ± 4% 179µs ± 2% -4.96% (p=0.016 n=5+5)
deco/neo/py(!log)/sql/zhash.go-P16 3.16ms ± 0% 2.95ms ± 0% -6.70% (p=0.000 n=16+16)
deco/neo/go/fs1/zhash.py 224µs ± 3% 205µs ± 2% -8.54% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.py-P16 2.04ms ± 6% 1.98ms ± 8% -2.92% (p=0.012 n=15+16)
deco/neo/go/fs1/zhash.go 56.6µs ± 1% 48.8µs ± 1% -13.82% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.go+prefetch128 24.3µs ± 4% 22.9µs ± 2% -5.76% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.go-P16 337µs ±10% 307µs ± 7% -8.94% (p=0.000 n=15+14)
deco/neo/go/fs1(!sha1)/zhash.go 40.1µs ± 1% 33.5µs ± 1% -16.40% (p=0.008 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go+prefetch128 16.5µs ± 8% 16.0µs ± 9% ~ (p=0.516 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go-P16 250µs ±20% 226µs ±16% -9.35% (p=0.001 n=15+16)
deco/neo/go/sqlite/zhash.py 316µs ± 2% 283µs ± 4% -10.47% (p=0.008 n=5+5)
deco/neo/go/sqlite/zhash.py-P16 2.55ms ± 3% 2.51ms ± 2% -1.61% (p=0.011 n=16+15)
deco/neo/go/sqlite/zhash.go 203µs ± 0% 192µs ± 2% -5.26% (p=0.008 n=5+5)
deco/neo/go/sqlite/zhash.go+prefetch128 106µs ± 2% 105µs ± 7% ~ (p=0.310 n=5+5)
deco/neo/go/sqlite/zhash.go-P16 1.39ms ± 7% 1.36ms ± 6% -2.48% (p=0.012 n=16+16)
deco/neo/go/sqlite(!sha1)/zhash.go 196µs ± 0% 185µs ± 1% -5.58% (p=0.016 n=4+5)
deco/neo/go/sqlite(!sha1)/zhash.go+prefetch128 104µs ± 3% 102µs ± 2% ~ (p=0.206 n=5+5)
deco/neo/go/sqlite(!sha1)/zhash.go-P16 1.38ms ± 4% 1.31ms ±11% -5.48% (p=0.000 n=16+16)
dataset:prod1-1024
deco/fs1/zhash.py 15.9µs ± 2% 15.3µs ± 3% -3.92% (p=0.029 n=4+4)
deco/fs1/zhash.py-P16 128µs ±11% 129µs ±21% ~ (p=0.780 n=16+16)
deco/fs1/zhash.go 2.80µs ± 0% 2.30µs ± 0% -17.86% (p=0.008 n=5+5)
deco/fs1/zhash.go+prefetch128 3.26µs ± 7% 3.14µs ± 8% ~ (p=0.246 n=5+5)
deco/fs1/zhash.go-P16 12.2µs ±50% 10.9µs ±34% ~ (p=0.154 n=16+15)
deco/zeo/zhash.py 360µs ± 6% 346µs ± 2% ~ (p=0.095 n=5+5)
deco/zeo/zhash.py-P16 2.90ms ± 8% 2.89ms ± 6% ~ (p=0.631 n=15+15)
deco/neo/py/sqlite/zhash.py 357µs ± 5% 331µs ± 7% -7.39% (p=0.032 n=5+5)
deco/neo/py/sqlite/zhash.py-P16 3.12ms ± 0% 3.02ms ± 0% -3.08% (p=0.000 n=16+16)
deco/neo/py/sqlite/zhash.go 182µs ± 0% 168µs ± 1% -7.37% (p=0.008 n=5+5)
deco/neo/py/sqlite/zhash.go+prefetch128 131µs ± 2% 124µs ± 3% -5.32% (p=0.008 n=5+5)
deco/neo/py/sqlite/zhash.go-P16 2.23ms ± 0% 2.10ms ± 0% -6.02% (p=0.000 n=14+15)
deco/neo/py(!log)/sqlite/zhash.py 330µs ± 4% 316µs ± 4% ~ (p=0.095 n=5+5)
deco/neo/py(!log)/sqlite/zhash.py-P16 2.91ms ± 1% 2.82ms ± 1% -3.34% (p=0.000 n=16+16)
deco/neo/py(!log)/sqlite/zhash.go 165µs ± 0% 154µs ± 1% -6.86% (p=0.008 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go+prefetch128 114µs ± 1% 108µs ± 0% -5.75% (p=0.008 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go-P16 1.96ms ± 0% 1.85ms ± 0% -5.30% (p=0.000 n=15+15)
deco/neo/py/sql/zhash.py 411µs ± 4% 373µs ± 1% -9.25% (p=0.029 n=4+4)
deco/neo/py/sql/zhash.py-P16 4.45ms ± 0% 4.25ms ± 0% -4.47% (p=0.000 n=15+15)
deco/neo/py/sql/zhash.go 231µs ± 1% 217µs ± 1% -6.01% (p=0.008 n=5+5)
deco/neo/py/sql/zhash.go+prefetch128 189µs ± 1% 180µs ± 2% -4.71% (p=0.008 n=5+5)
deco/neo/py/sql/zhash.go-P16 3.18ms ± 0% 2.98ms ± 0% -6.51% (p=0.000 n=15+14)
deco/neo/py(!log)/sql/zhash.py 384µs ± 3% 359µs ± 3% -6.66% (p=0.029 n=4+4)
deco/neo/py(!log)/sql/zhash.py-P16 4.28ms ± 0% 4.04ms ± 0% -5.45% (p=0.000 n=16+16)
deco/neo/py(!log)/sql/zhash.go 213µs ± 1% 196µs ± 1% -7.85% (p=0.008 n=5+5)
deco/neo/py(!log)/sql/zhash.go+prefetch128 171µs ± 1% 159µs ± 1% -7.19% (p=0.008 n=5+5)
deco/neo/py(!log)/sql/zhash.go-P16 2.83ms ± 0% 2.59ms ± 3% -8.68% (p=0.000 n=16+16)
deco/neo/go/fs1/zhash.py 218µs ± 1% 201µs ± 0% -8.14% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.py-P16 1.97ms ± 2% 1.88ms ± 8% -4.25% (p=0.000 n=15+16)
deco/neo/go/fs1/zhash.go 44.1µs ± 1% 37.7µs ± 1% -14.60% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.go+prefetch128 16.9µs ± 3% 16.2µs ± 4% ~ (p=0.056 n=5+5)
deco/neo/go/fs1/zhash.go-P16 313µs ± 8% 297µs ± 9% -4.97% (p=0.000 n=15+16)
deco/neo/go/fs1(!sha1)/zhash.go 38.2µs ± 1% 32.8µs ± 2% -14.28% (p=0.008 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go+prefetch128 14.0µs ± 3% 13.1µs ± 3% -6.70% (p=0.008 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go-P16 297µs ±12% 262µs ±17% -11.89% (p=0.000 n=15+16)
deco/neo/go/sqlite/zhash.py 268µs ± 4% 244µs ± 5% -9.10% (p=0.008 n=5+5)
deco/neo/go/sqlite/zhash.py-P16 2.36ms ± 2% 2.30ms ± 3% -2.50% (p=0.000 n=15+16)
deco/neo/go/sqlite/zhash.go 114µs ± 0% 105µs ± 0% -8.44% (p=0.008 n=5+5)
deco/neo/go/sqlite/zhash.go+prefetch128 54.8µs ± 2% 52.5µs ± 1% -4.13% (p=0.008 n=5+5)
deco/neo/go/sqlite/zhash.go-P16 873µs ± 2% 812µs ± 5% -7.03% (p=0.000 n=15+14)
deco/neo/go/sqlite(!sha1)/zhash.go 114µs ± 2% 103µs ± 2% -9.58% (p=0.008 n=5+5)
deco/neo/go/sqlite(!sha1)/zhash.go+prefetch128 53.5µs ± 3% 52.0µs ± 4% ~ (p=0.111 n=5+5)
deco/neo/go/sqlite(!sha1)/zhash.go-P16 891µs ± 3% 856µs ± 2% -3.97% (p=0.000 n=16+15)

however compare to 4.12 some things got also a bit slower even without
pti (and some got faster):

benchstat -split dataset 20180220-deco-linux4.12.13-noturbo.txt 20180220-deco-linux4.15.4-noturbo-nopti.txt

name old pystone/s new pystone/s delta
deco/pystone 211k ± 1% 222k ± 1% +5.19% (p=0.008 n=5+5)

name old time/op new time/op delta
deco/sha1/py/1K 1.69µs ± 0% 1.62µs ± 0% -3.86% (p=0.008 n=5+5)
deco/sha1/go/1K 1.59µs ± 0% 1.53µs ± 0% -3.84% (p=0.008 n=5+5)
deco/sha1/py/4K 6.01µs ± 0% 5.78µs ± 0% -3.93% (p=0.008 n=5+5)
deco/sha1/go/4K 5.81µs ± 0% 5.58µs ± 0% -3.89% (p=0.008 n=5+5)
deco/sha1/py/2M 2.93ms ± 0% 2.82ms ± 0% -3.65% (p=0.008 n=5+5)
deco/sha1/go/2M 2.88ms ± 0% 2.77ms ± 0% -3.82% (p=0.008 n=5+5)
deco/disk/randread/direct/4K-min 102µs ± 0% 103µs ± 2% ~ (p=0.135 n=5+5)
deco/disk/randread/direct/4K-avg 140µs ± 0% 142µs ± 1% +1.56% (p=0.008 n=5+5)
deco/disk/randread/direct/2M-min 4.95ms ± 2% 5.44ms ± 5% +9.94% (p=0.008 n=5+5)
deco/disk/randread/direct/2M-avg 5.48ms ± 3% 6.11ms ± 1% +11.54% (p=0.008 n=5+5)
deco/disk/randread/pagecache/4K-min 313ns ± 1% 359ns ± 1% +14.64% (p=0.008 n=5+5)
deco/disk/randread/pagecache/4K-avg 706ns ± 0% 745ns ± 0% +5.50% (p=0.016 n=4+5)
deco/disk/randread/pagecache/2M-min 200µs ± 5% 195µs ± 4% ~ (p=0.310 n=5+5)
deco/disk/randread/pagecache/2M-avg 221µs ± 1% 211µs ± 0% -4.58% (p=0.008 n=5+5)

name old time/object new time/object delta
dataset:wczblk1-8
deco/fs1/zhash.py 19.8µs ± 2% 19.4µs ± 2% -2.25% (p=0.048 n=5+4)
deco/fs1/zhash.py-P16 125µs ±30% 130µs ±22% ~ (p=0.468 n=16+16)
deco/fs1/zhash.go 2.70µs ± 0% 2.70µs ± 0% ~ (all equal)
deco/fs1/zhash.go+prefetch128 3.67µs ± 2% 3.90µs ± 8% ~ (p=0.175 n=4+5)
deco/fs1/zhash.go-P16 10.6µs ±79% 10.7µs ±82% ~ (p=0.830 n=14+16)
deco/zeo/zhash.py 384µs ± 9% 360µs ± 8% ~ (p=0.151 n=5+5)
deco/zeo/zhash.py-P16 3.29ms ± 5% 3.06ms ± 5% -7.01% (p=0.000 n=15+16)
deco/neo/py/sqlite/zhash.py 388µs ± 4% 380µs ± 3% ~ (p=0.421 n=5+5)
deco/neo/py/sqlite/zhash.py-P16 3.45ms ± 1% 3.18ms ± 2% -7.82% (p=0.000 n=16+16)
deco/neo/py/sqlite/zhash.go 273µs ± 1% 265µs ± 1% -2.94% (p=0.008 n=5+5)
deco/neo/py/sqlite/zhash.go+prefetch128 141µs ± 3% 139µs ± 2% ~ (p=0.286 n=5+5)
deco/neo/py/sqlite/zhash.go-P16 2.24ms ± 1% 2.32ms ± 1% +3.74% (p=0.000 n=12+13)
deco/neo/py(!log)/sqlite/zhash.py 357µs ± 1% 357µs ± 2% ~ (p=0.690 n=5+5)
deco/neo/py(!log)/sqlite/zhash.py-P16 3.29ms ± 2% 3.05ms ± 2% -7.35% (p=0.000 n=16+16)
deco/neo/py(!log)/sqlite/zhash.go 249µs ± 0% 244µs ± 1% -1.93% (p=0.008 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go+prefetch128 128µs ± 2% 126µs ± 1% ~ (p=0.056 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go-P16 2.14ms ± 0% 2.17ms ± 0% +1.35% (p=0.000 n=16+16)
deco/neo/py/sql/zhash.py 420µs ± 1% 424µs ± 6% ~ (p=0.686 n=4+4)
deco/neo/py/sql/zhash.py-P16 4.59ms ± 0% 4.50ms ± 1% -1.86% (p=0.000 n=14+13)
deco/neo/py/sql/zhash.go 312µs ± 1% 307µs ± 2% ~ (p=0.095 n=5+5)
deco/neo/py/sql/zhash.go+prefetch128 200µs ± 2% 198µs ± 1% ~ (p=0.206 n=5+5)
deco/neo/py/sql/zhash.go-P16 3.27ms ± 0% 3.22ms ± 0% -1.57% (p=0.000 n=14+13)
deco/neo/py(!log)/sql/zhash.py 411µs ± 2% 452µs ±43% ~ (p=0.730 n=4+5)
deco/neo/py(!log)/sql/zhash.py-P16 4.20ms ± 0% 4.22ms ± 0% +0.62% (p=0.000 n=14+15)
deco/neo/py(!log)/sql/zhash.go 292µs ± 1% 288µs ± 2% ~ (p=0.095 n=5+5)
deco/neo/py(!log)/sql/zhash.go+prefetch128 180µs ± 1% 179µs ± 2% ~ (p=0.548 n=5+5)
deco/neo/py(!log)/sql/zhash.go-P16 2.88ms ± 0% 2.95ms ± 0% +2.67% (p=0.000 n=15+16)
deco/neo/go/fs1/zhash.py 209µs ± 1% 205µs ± 2% -2.06% (p=0.016 n=5+5)
deco/neo/go/fs1/zhash.py-P16 1.89ms ± 8% 1.98ms ± 8% +4.75% (p=0.002 n=16+16)
deco/neo/go/fs1/zhash.go 45.2µs ± 1% 48.8µs ± 1% +7.78% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.go+prefetch128 23.3µs ± 5% 22.9µs ± 2% ~ (p=0.643 n=5+5)
deco/neo/go/fs1/zhash.go-P16 290µs ±10% 307µs ± 7% +6.06% (p=0.002 n=16+14)
deco/neo/go/fs1(!sha1)/zhash.go 30.2µs ± 2% 33.5µs ± 1% +10.91% (p=0.008 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go+prefetch128 14.6µs ±10% 16.0µs ± 9% +9.60% (p=0.024 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go-P16 183µs ±15% 226µs ±16% +23.93% (p=0.000 n=15+16)
deco/neo/go/sqlite/zhash.py 285µs ± 4% 283µs ± 4% ~ (p=0.548 n=5+5)
deco/neo/go/sqlite/zhash.py-P16 2.44ms ± 4% 2.51ms ± 2% +3.12% (p=0.000 n=15+15)
deco/neo/go/sqlite/zhash.go 191µs ± 1% 192µs ± 2% ~ (p=0.627 n=5+5)
deco/neo/go/sqlite/zhash.go+prefetch128 106µs ± 5% 105µs ± 7% ~ (p=0.421 n=5+5)
deco/neo/go/sqlite/zhash.go-P16 1.43ms ± 2% 1.36ms ± 6% -4.96% (p=0.000 n=16+16)
deco/neo/go/sqlite(!sha1)/zhash.go 186µs ± 1% 185µs ± 1% ~ (p=0.421 n=5+5)
deco/neo/go/sqlite(!sha1)/zhash.go+prefetch128 102µs ± 3% 102µs ± 2% ~ (p=0.841 n=5+5)
deco/neo/go/sqlite(!sha1)/zhash.go-P16 1.38ms ± 8% 1.31ms ±11% -5.27% (p=0.000 n=16+16)
dataset:prod1-1024
deco/fs1/zhash.py 15.8µs ± 1% 15.3µs ± 3% -3.01% (p=0.029 n=4+4)
deco/fs1/zhash.py-P16 127µs ±11% 129µs ±21% ~ (p=0.696 n=16+16)
deco/fs1/zhash.go 2.20µs ± 0% 2.30µs ± 0% +4.55% (p=0.008 n=5+5)
deco/fs1/zhash.go+prefetch128 3.08µs ± 4% 3.14µs ± 8% ~ (p=0.794 n=5+5)
deco/fs1/zhash.go-P16 8.99µs ±66% 10.93µs ±34% ~ (p=0.127 n=16+15)
deco/zeo/zhash.py 349µs ± 5% 346µs ± 2% ~ (p=0.548 n=5+5)
deco/zeo/zhash.py-P16 3.17ms ± 2% 2.89ms ± 6% -8.82% (p=0.000 n=12+15)
deco/neo/py/sqlite/zhash.py 350µs ± 5% 331µs ± 7% ~ (p=0.056 n=5+5)
deco/neo/py/sqlite/zhash.py-P16 3.31ms ± 1% 3.02ms ± 0% -8.82% (p=0.000 n=16+16)
deco/neo/py/sqlite/zhash.go 170µs ± 0% 168µs ± 1% -0.87% (p=0.008 n=5+5)
deco/neo/py/sqlite/zhash.go+prefetch128 125µs ± 1% 124µs ± 3% ~ (p=0.421 n=5+5)
deco/neo/py/sqlite/zhash.go-P16 2.11ms ± 0% 2.10ms ± 0% -0.61% (p=0.000 n=13+15)
deco/neo/py(!log)/sqlite/zhash.py 317µs ± 1% 316µs ± 4% ~ (p=1.000 n=5+5)
deco/neo/py(!log)/sqlite/zhash.py-P16 3.01ms ± 1% 2.82ms ± 1% -6.28% (p=0.000 n=16+16)
deco/neo/py(!log)/sqlite/zhash.go 153µs ± 0% 154µs ± 1% ~ (p=0.095 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go+prefetch128 107µs ± 1% 108µs ± 0% ~ (p=0.548 n=5+5)
deco/neo/py(!log)/sqlite/zhash.go-P16 1.83ms ± 0% 1.85ms ± 0% +1.06% (p=0.000 n=16+15)
deco/neo/py/sql/zhash.py 382µs ± 2% 373µs ± 1% ~ (p=0.114 n=4+4)
deco/neo/py/sql/zhash.py-P16 4.33ms ± 0% 4.25ms ± 0% -1.99% (p=0.000 n=15+15)
deco/neo/py/sql/zhash.go 217µs ± 1% 217µs ± 1% ~ (p=1.000 n=5+5)
deco/neo/py/sql/zhash.go+prefetch128 182µs ± 1% 180µs ± 2% ~ (p=0.135 n=5+5)
deco/neo/py/sql/zhash.go-P16 2.95ms ± 4% 2.98ms ± 0% +1.10% (p=0.019 n=16+14)
deco/neo/py(!log)/sql/zhash.py 372µs ± 4% 359µs ± 3% ~ (p=0.200 n=4+4)
deco/neo/py(!log)/sql/zhash.py-P16 4.15ms ± 0% 4.04ms ± 0% -2.43% (p=0.000 n=14+16)
deco/neo/py(!log)/sql/zhash.go 194µs ± 0% 196µs ± 1% +0.87% (p=0.008 n=5+5)
deco/neo/py(!log)/sql/zhash.go+prefetch128 158µs ± 1% 159µs ± 1% ~ (p=0.246 n=5+5)
deco/neo/py(!log)/sql/zhash.go-P16 2.62ms ± 0% 2.59ms ± 3% -1.32% (p=0.017 n=15+16)
deco/neo/go/fs1/zhash.py 201µs ± 2% 201µs ± 0% ~ (p=0.690 n=5+5)
deco/neo/go/fs1/zhash.py-P16 1.77ms ± 5% 1.88ms ± 8% +6.31% (p=0.000 n=16+16)
deco/neo/go/fs1/zhash.go 34.3µs ± 1% 37.7µs ± 1% +9.73% (p=0.008 n=5+5)
deco/neo/go/fs1/zhash.go+prefetch128 15.4µs ± 3% 16.2µs ± 4% +5.05% (p=0.032 n=5+5)
deco/neo/go/fs1/zhash.go-P16 304µs ± 8% 297µs ± 9% ~ (p=0.128 n=16+16)
deco/neo/go/fs1(!sha1)/zhash.go 28.9µs ± 2% 32.8µs ± 2% +13.58% (p=0.008 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go+prefetch128 12.7µs ± 5% 13.1µs ± 3% ~ (p=0.214 n=5+5)
deco/neo/go/fs1(!sha1)/zhash.go-P16 298µs ± 2% 262µs ±17% -12.17% (p=0.000 n=13+16)
deco/neo/go/sqlite/zhash.py 241µs ± 3% 244µs ± 5% ~ (p=0.548 n=5+5)
deco/neo/go/sqlite/zhash.py-P16 2.21ms ± 3% 2.30ms ± 3% +4.41% (p=0.000 n=16+16)
deco/neo/go/sqlite/zhash.go 101µs ± 0% 105µs ± 0% +3.43% (p=0.008 n=5+5)
deco/neo/go/sqlite/zhash.go+prefetch128 53.3µs ± 3% 52.5µs ± 1% ~ (p=0.452 n=5+5)
deco/neo/go/sqlite/zhash.go-P16 839µs ± 2% 812µs ± 5% -3.28% (p=0.000 n=14+14)
deco/neo/go/sqlite(!sha1)/zhash.go 100µs ± 0% 103µs ± 2% +2.67% (p=0.016 n=4+5)
deco/neo/go/sqlite(!sha1)/zhash.go+prefetch128 52.9µs ± 4% 52.0µs ± 4% ~ (p=0.310 n=5+5)
deco/neo/go/sqlite(!sha1)/zhash.go-P16 853µs ± 3% 856µs ± 2% ~ (p=0.953 n=16+15)

f069dd10

19 Feb, 2018 1 commit
- X sqlite: Switch to using github.com/gwenn/gosqlite directly · da01e01e
  Kirill Smelkov authored Feb 19, 2018
  
  da01e01e
15 Feb, 2018 10 commits
- . · c0502d41
  Kirill Smelkov authored Feb 16, 2018
  
  c0502d41
- . · 9206bfb6
  Kirill Smelkov authored Feb 15, 2018
  
  9206bfb6
- . · 686b8976
  Kirill Smelkov authored Feb 15, 2018
  
  686b8976
- . · bbc76236
  Kirill Smelkov authored Feb 15, 2018
  
  bbc76236
- . · c70467a6
  Kirill Smelkov authored Feb 15, 2018
  
  c70467a6
- . · aa17a79e
  Kirill Smelkov authored Feb 15, 2018
  
  aa17a79e
- . · 7a98ef72
  Kirill Smelkov authored Feb 15, 2018
  
  7a98ef72
- . · 9a4004fe
  Kirill Smelkov authored Feb 15, 2018
  
  9a4004fe
- . · 860c4019
  Kirill Smelkov authored Feb 15, 2018
  
  860c4019
- . · c9a3e4ad
  Kirill Smelkov authored Feb 15, 2018
  
  c9a3e4ad
14 Feb, 2018 3 commits
- . · d5a0a0ea
  Kirill Smelkov authored Feb 14, 2018
  
  d5a0a0ea
- X StorageBackend -> storage.Backend + registry system a-la zodb · d84bb116
  Kirill Smelkov authored Feb 14, 2018
  
  d84bb116
- X split storage backend from neo.Storage · 6991ec36
  Kirill Smelkov authored Feb 14, 2018
  
  6991ec36
13 Feb, 2018 11 commits
- . · fc2f4a1a
  Kirill Smelkov authored Feb 13, 2018
  
  fc2f4a1a
- . · c5f6dc77
  Kirill Smelkov authored Feb 13, 2018
  
  c5f6dc77
- . · 933a62ce
  Kirill Smelkov authored Feb 13, 2018
  
  933a62ce
- . · 6199789e
  Kirill Smelkov authored Feb 13, 2018
  
  6199789e
- . · 98197abb
  Kirill Smelkov authored Feb 13, 2018
  
  98197abb
- . · bce3fadc
  Kirill Smelkov authored Feb 13, 2018
  
  bce3fadc
- . · c9d2d4e8
  Kirill Smelkov authored Feb 13, 2018
  
  c9d2d4e8
- . · 8700e53e
  Kirill Smelkov authored Feb 13, 2018
  
  8700e53e
- . · b55e94bf
  Kirill Smelkov authored Feb 13, 2018
  
  b55e94bf
- X neonet: polish a bit · 8052ef1f
  Kirill Smelkov authored Feb 13, 2018
  
  8052ef1f
- X Goodbye neotools · a440d090
  Kirill Smelkov authored Feb 13, 2018
```
zodbtools provide additional service over zodb (various dumping
routines), but neotools just invokes neo.NewMaster and neo.NewStorage.

For a possible future `neo monitor` maybe bringing neotools back would
be needed, but for now there is simply no reason to keep them around.
```
  a440d090
12 Feb, 2018 5 commits
- X merge neo/server & neo/client -> neo · ccd74eb4
  Kirill Smelkov authored Feb 12, 2018
  
  ccd74eb4
- . · 98ac2433
  Kirill Smelkov authored Feb 12, 2018
  
  98ac2433
- X neonet: unexport PktBuf · 8dfe0341
  Kirill Smelkov authored Feb 12, 2018
```
not used outside neonet.
```
  8dfe0341
- X move network bits into neo/neonet/ · 889b8d42
  Kirill Smelkov authored Feb 12, 2018
  
  889b8d42
- X move protocol bits into neo/proto/ package · 47f069a5
  Kirill Smelkov authored Feb 12, 2018
  
  47f069a5
09 Feb, 2018 2 commits
- . · 7b4ae7bd
  Kirill Smelkov authored Feb 09, 2018
  
  7b4ae7bd
- . · a2ce1f4e
  Kirill Smelkov authored Feb 09, 2018
  
  a2ce1f4e