• Kirill Smelkov's avatar
    neotest: Draft software-release to run NEO/go & friends tests/benchmarks under webrunner · fd43ba62
    Kirill Smelkov authored
    We provide neotest SR which builds current NEO/go together with the
    tools which are needed to run the benchmarks.
    
    The SR does not yet provide automated service to run the tests
    automatically periodically and to upload the results to ERP5 (all marked
    as TODO).
    
    However with present state it provides all the infrastructure for people
    to try to run NEO tests in their webrunner: a `neotest` program is
    installed in buildout top-level bin/ . One should use it to run the
    tests.
    
    An example run output could be seen here:
    
    https://lab.nexedi.com/snippets/258
    
    With benchstat (https://godoc.org/golang.org/x/perf/cmd/benchstat) it can be summarized or compared to another run. Here is e.g. summarization:
    
    ```
    name                                                                 pystone/s
    vifibcloud-onlinenet-hosting-004/pystone                              156k ± 6%
    
    name                                                                 µs/op
    vifibcloud-onlinenet-hosting-004/sha1/py/1024B                        1.71 ± 6%
    vifibcloud-onlinenet-hosting-004/sha1/go/1024B                        2.26 ±73%
    vifibcloud-onlinenet-hosting-004/sha1/py/4096B                        6.39 ± 4%
    vifibcloud-onlinenet-hosting-004/sha1/go/4096B                        5.66 ± 2%
    
    name                                                                 us/op
    vifibcloud-onlinenet-hosting-004/disk/randread/direct/4K-min          75.7 ± 4%
    vifibcloud-onlinenet-hosting-004/disk/randread/direct/4K-avg          95.2 ± 1%
    vifibcloud-onlinenet-hosting-004/disk/randread/pagecache/4K-avg       1.84 ± 0%
    
    name                                                                 time/op
    vifibcloud-onlinenet-hosting-004/disk/randread/pagecache/4K-min      288ns ± 5%
    vifibcloud-onlinenet-hosting-004/disk/randread/pagecache/4K-avg      876ns ± 9%
    
    name                                                                 µs/object
    dataset:wczblk1-8
    vifibcloud-onlinenet-hosting-004/fs1/zhash.py                         22.2 ±11%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.py-P16                     40.6 ±19%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.go                         4.00 ±40%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.go+prefetch128             5.58 ±24%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.go-P16                     3.68 ±88%
    vifibcloud-onlinenet-hosting-004/zeo/zhash.py                          626 ±40%
    vifibcloud-onlinenet-hosting-004/zeo/zhash.py-P16                    2.17k ± 3%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.py                823 ±17%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.py-P16          2.04k ± 3%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.go                666 ± 9%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.go+prefetch128    138 ±10%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.go-P16          1.96k ± 0%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.py                 1.02k ±71%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.py-P16             3.09k ± 1%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.go                   602 ±29%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.go+prefetch128       213 ±14%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.go-P16             4.66k ± 1%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.py                       326 ±32%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.py-P16                   554 ± 7%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.go                      39.6 ± 7%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.go+prefetch128          24.3 ±18%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.go-P16                   213 ± 6%
    vifibcloud-onlinenet-hosting-004/neo/go(!sha1)/zhash.go               42.8 ±13%
    vifibcloud-onlinenet-hosting-004/neo/go(!sha1)/zhash.go+prefetch128   24.5 ±19%
    vifibcloud-onlinenet-hosting-004/neo/go(!sha1)/zhash.go-P16           70.2 ±62%
    dataset:prod1-1024
    vifibcloud-onlinenet-hosting-004/fs1/zhash.py                         18.8 ±22%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.py-P16                     33.6 ±21%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.go                         3.46 ±25%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.go+prefetch128             5.36 ±25%
    vifibcloud-onlinenet-hosting-004/fs1/zhash.go-P16                     2.56 ±45%
    vifibcloud-onlinenet-hosting-004/zeo/zhash.py                          617 ±23%
    vifibcloud-onlinenet-hosting-004/zeo/zhash.py-P16                    2.19k ± 1%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.py                717 ±46%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.py-P16          2.02k ± 1%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.go                297 ±14%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.go+prefetch128    130 ±10%
    vifibcloud-onlinenet-hosting-004/neo/py/sqlite/zhash.go-P16          1.86k ± 7%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.py                   680 ±61%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.py-P16             4.76k ± 1%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.go                   269 ±13%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.go+prefetch128       178 ± 8%
    vifibcloud-onlinenet-hosting-004/neo/py/sql/zhash.go-P16             3.03k ± 1%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.py                       227 ±10%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.py-P16                   654 ± 5%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.go                      32.4 ± 8%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.go+prefetch128          15.5 ±12%
    vifibcloud-onlinenet-hosting-004/neo/go/zhash.go-P16                   261 ±12%
    vifibcloud-onlinenet-hosting-004/neo/go(!sha1)/zhash.go               24.2 ± 1%
    vifibcloud-onlinenet-hosting-004/neo/go(!sha1)/zhash.go+prefetch128   13.9 ±25%
    vifibcloud-onlinenet-hosting-004/neo/go(!sha1)/zhash.go-P16            270 ±10%
    ```
    
    The infrastructure to build Go projects under SlapOS is also introduced along the way
    (was applied separetely as 1b540151).
    
    Please see details in the individual commit messages. 
    
    /cc @nexedi 
    
    P.S. the results are noisy becuase under regular webrunner I do not have root and cannot run e.g. `cpupower frequency-set -g performance`.
    
    /reviewed-on !242
    fd43ba62
Name
Last commit
Last update
component Loading commit data...
slapos Loading commit data...
software Loading commit data...
stack Loading commit data...
.gitignore Loading commit data...
CHANGES.rst Loading commit data...
MANIFEST.in Loading commit data...
README.rst Loading commit data...
README.software.rst Loading commit data...
schema.json Loading commit data...
setup.py Loading commit data...
sla.schema.json Loading commit data...
update-hash Loading commit data...
validate-schema Loading commit data...