Commit af0da20a authored by Kirill Smelkov's avatar Kirill Smelkov

py.bench: Automatically discover benchmarks in test files

Since the beginning (9bf03d9c "py.bench: New command to benchmark python
code similarly to `go test -bench`") py.bench was automatically
discovering benchmarks in bench_*.py files only. This was inherited from
wendelin.core which keeps its benchmarks in those files.

However in pygolang, following Go convention(*), we already have several
benchmarks that reside together with tests in same *_test.py files. And
currently just running py.bench does not discover them.

-> Let's fix this and teach py.bench to automatically discover
benchmarks in the test files by default as well.

Pytest's default is to look for tests in test_*.py and *_test.py (+).
Add those patterns and also keep bench_*.py for backward compatibility.

Before this patch running py.bench inside pygolang repository does not
run any benchmark at all. After the patch py.bench runs all the
benchmarks by default:

    (z-dev) kirr@deca:~/src/tools/go/pygolang$ py.bench
    ========================= test session starts ==========================
    platform linux2 -- Python 2.7.18, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
    rootdir: /home/kirr/src/tools/go/pygolang
    plugins: timeout-1.4.2, profiling-1.7.0, mock-2.0.0
    collected 18 items

    pymod: golang/golang_str_test.py
    Benchmarkstddecode              2000000 0.756 µs/op
    Benchmarkudecode                20000   74.359 µs/op
    Benchmarkstdencode              3000000 0.327 µs/op
    Benchmarkbencode                40000   32.613 µs/op

    pymod: golang/golang_test.py
    Benchmarkpyx_select_nogil       500000  2.051 µs/op
    Benchmarkpyx_go_nogil           90000   12.177 µs/op
    Benchmarkpyx_chan_nogil         600000  1.826 µs/op
    Benchmarkgo                     80000   13.267 µs/op
    Benchmarkchan                   500000  2.076 µs/op
    Benchmarkselect                 300000  3.835 µs/op
    Benchmarkdef                    30000000        0.035 µs/op
    Benchmarkfunc_def               40000   29.387 µs/op
    Benchmarkcall                   30000000        0.043 µs/op
    Benchmarkfunc_call              2000000 0.819 µs/op
    Benchmarktry_finally            20000000        0.096 µs/op
    Benchmarkdefer                  600000  1.755 µs/op

    pymod: golang/sync_test.py
    Benchmarkworkgroup_empty        40000   25.807 µs/op
    Benchmarkworkgroup_raise        40000   31.637 µs/op                     [100%]

    =========================== warnings summary ===========================

(*) see https://pkg.go.dev/cmd/go#hdr-Test_packages
(+) see https://docs.pytest.org/en/7.1.x/reference/reference.html#confval-python_files

/cc @jerome
parent 598eb479
#!/usr/bin/env python #!/usr/bin/env python
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright (C) 2014-2019 Nexedi SA and Contributors. # Copyright (C) 2014-2022 Nexedi SA and Contributors.
# Kirill Smelkov <kirr@nexedi.com> # Kirill Smelkov <kirr@nexedi.com>
# #
# This program is free software: you can Use, Study, Modify and Redistribute # This program is free software: you can Use, Study, Modify and Redistribute
...@@ -83,11 +83,11 @@ class XForkedFunc(ForkedFunc): ...@@ -83,11 +83,11 @@ class XForkedFunc(ForkedFunc):
# BenchPlugin is py.test plugin to collect & run benchmarks. # BenchPlugin is py.test plugin to collect & run benchmarks.
class BenchPlugin: class BenchPlugin:
# redirect python collector to bench_*.py and bench_*() # redirect python collector to bench_*() functions in test/bench files.
def pytest_configure(self, config): def pytest_configure(self, config):
# XXX a bit hacky # XXX a bit hacky
ini = config.inicfg ini = config.inicfg
ini['python_files'] = 'bench_*.py' ini['python_files'] = 'test_*.py *_test.py bench_*.py'
ini['python_classes'] = 'Bench' ini['python_classes'] = 'Bench'
ini['python_functions'] = 'bench_' ini['python_functions'] = 'bench_'
config._inicache.clear() config._inicache.clear()
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment