Commit c5c3071b authored by Kirill Smelkov's avatar Kirill Smelkov

pygolang v0.0.5

This release is driven by wendelin.core v2 needs with one of the changes
being that now most of the library was moved into nogil code and can be
used fully from inside nogil world(*). Python modules are now just wrappers
of their nogil counterparts. The way for Python and nogil worlds to
communicate is also provided.

The move to nogil required many other enhancements along the way. Please
see CHANGELOG for overview.

The move to nogil brought some speedup automatically.
Below are benchmark results of this release compared to pygolang v0.0.4
(1573d101) for python-level benchmarks (we have only those at present):

	(on i7@2.6GHz)

thread runtime:

    name             old time/op  new time/op  delta
    go               18.3µs ± 0%  18.3µs ± 1%     ~     (p=1.000 n=10+10)
    chan             2.91µs ± 3%  2.99µs ± 5%   +2.73%  (p=0.022 n=10+10)
    select           3.57µs ± 3%  3.57µs ± 4%     ~     (p=0.720 n=9+10)
    def              55.0ns ± 0%  54.0ns ± 0%   -1.82%  (p=0.002 n=8+10)
    func_def         43.8µs ± 2%  44.1µs ± 1%   +0.64%  (p=0.035 n=10+9)
    call             64.0ns ± 0%  66.3ns ± 1%   +3.59%  (p=0.000 n=10+10)
    func_call        1.05µs ± 1%  1.24µs ± 0%  +17.80%  (p=0.000 n=10+7)
    try_finally       138ns ± 0%   137ns ± 1%   -0.51%  (p=0.003 n=10+10)
    defer            2.32µs ± 1%  2.63µs ± 1%  +13.52%  (p=0.000 n=10+10)
    workgroup_empty  38.0µs ± 1%  24.1µs ± 1%  -36.43%  (p=0.000 n=10+10)
    workgroup_raise  47.7µs ± 1%  28.2µs ± 0%  -40.76%  (p=0.000 n=10+10)

gevent runtime:

    name             old time/op  new time/op  delta
    go               16.9µs ± 1%  17.2µs ± 2%   +1.94%  (p=0.000 n=10+10)
    chan             7.43µs ± 0%  7.82µs ± 0%   +5.34%  (p=0.000 n=10+7)
    select           10.5µs ± 0%  11.2µs ± 0%   +6.74%  (p=0.000 n=10+10)
    def              63.0ns ± 0%  57.6ns ± 1%   -8.57%  (p=0.000 n=9+10)
    func_def         44.0µs ± 1%  44.2µs ± 1%     ~     (p=0.063 n=10+10)
    call             67.0ns ± 0%  64.0ns ± 0%   -4.48%  (p=0.002 n=8+10)
    func_call        1.06µs ± 1%  1.23µs ± 1%  +16.50%  (p=0.000 n=10+10)
    try_finally       144ns ± 0%   136ns ± 0%   -5.90%  (p=0.000 n=10+10)
    defer            2.37µs ± 1%  2.61µs ± 1%  +10.07%  (p=0.000 n=10+10)
    workgroup_empty  57.0µs ± 0%  55.0µs ± 2%   -3.53%  (p=0.000 n=10+9)
    workgroup_raise  72.4µs ± 0%  69.6µs ± 6%   -3.95%  (p=0.035 n=9+10)

workgroup_* changes for thread runtime is the speedup I am talking about.
defer/func_call slowdown is due to added exception chaining. We did not
optimize Python-level defer yet, and if/when that would be needed, it should
be possible to optimize by moving pydefer implementation into Cython.

(*) go and channels were moved into nogil world in Pygolang v0.0.3 +
v0.0.4 . Now it is the rest of the library that was moved with packages
like context, time, sync etc.

wendelin.core v2 needs nogil to run pinner thread on client side to
support isolation property in cooperation with wcfs: since there is a
`client -> wcfs -> pinner` loop:

      - - - - - -
     |           |
        pinner <------.
     |           |   wcfs
        client -------^
     |           |
      - - - - - -
     client process

the pinner thread would deadlock if it tries to take the GIL because
client thread can be holding GIL already while accessing wcfs-mmaped
memory (think doing e.g. `x = A[i]` in Python).
parent 37ae291f
Pygolang change history
-----------------------
0.0.5 (2019-11-27)
~~~~~~~~~~~~~~~~~~
- Add support for typed Python channels. For
example `chan(dtype='C.int')` creates channel whose elements type is C `int`
instead of Python object. Besides providing runtime type-safety, this allows
to build interaction in between Python and nogil worlds (`commit 1`__, 2__,
3__, 4__, 5__, 6__, 7__, 8__, 9__, 10__, 11__).
__ https://lab.nexedi.com/kirr/pygolang/commit/f2847307
__ https://lab.nexedi.com/kirr/pygolang/commit/d6c8862d
__ https://lab.nexedi.com/kirr/pygolang/commit/2590e9a7
__ https://lab.nexedi.com/kirr/pygolang/commit/47111d3e
__ https://lab.nexedi.com/kirr/pygolang/commit/30561db4
__ https://lab.nexedi.com/kirr/pygolang/commit/f6fab7b5
__ https://lab.nexedi.com/kirr/pygolang/commit/2c8063f4
__ https://lab.nexedi.com/kirr/pygolang/commit/3121b290
__ https://lab.nexedi.com/kirr/pygolang/commit/77719d8a
__ https://lab.nexedi.com/kirr/pygolang/commit/69b80926
__ https://lab.nexedi.com/kirr/pygolang/commit/07f9430d
- Provide automatic memory management for C++/Cython/nogil classes.
Used approach complements `"Automatic multithreaded-safe memory managed
classes in Cython"` (Gwenaël Samain et al. 2019, `blog post`__) (`commit 1`__,
2__, 3__, 4__, 5__, 6__, 7__).
__ https://www.nexedi.com/blog/NXD-Document.Blog.Cypclass
__ https://lab.nexedi.com/kirr/pygolang/commit/e82b4fab
__ https://lab.nexedi.com/kirr/pygolang/commit/e614d641
__ https://lab.nexedi.com/kirr/pygolang/commit/af4a8d80
__ https://lab.nexedi.com/kirr/pygolang/commit/b2253abf
__ https://lab.nexedi.com/kirr/pygolang/commit/274afa3f
__ https://lab.nexedi.com/kirr/pygolang/commit/fd2a6fab
__ https://lab.nexedi.com/kirr/pygolang/commit/7f0672aa
- Provide minimal support for interfaces with empty and `error` interfaces
provided by base library (`commit 1`__, 2__).
__ https://lab.nexedi.com/kirr/pygolang/commit/5a99b769
__ https://lab.nexedi.com/kirr/pygolang/commit/45c8cddd
- Provide `sync.Mutex` and `sync.Sema` as part of both Python and Cython/nogil
API (`commit 1`__, 2__, 3__, 4__, 5__, 6__).
__ https://lab.nexedi.com/kirr/pygolang/commit/d99bb6b7
__ https://lab.nexedi.com/kirr/pygolang/commit/9c795ca7
__ https://lab.nexedi.com/kirr/pygolang/commit/34b7a1f4
__ https://lab.nexedi.com/kirr/pygolang/commit/2c1be15e
__ https://lab.nexedi.com/kirr/pygolang/commit/e6788170
__ https://lab.nexedi.com/kirr/pygolang/commit/548f2df1
- Provide C++/Cython/nogil API for `time` package. Python-level `time` becomes a
small wrapper around Cython/nogil one (`commit 1`__, 2__, 3__, 4__, 5__, 6__,
7__, 8__, 9__, 10__, 11__, 12__, 13__, 14__).
__ https://lab.nexedi.com/kirr/pygolang/commit/32f34607
__ https://lab.nexedi.com/kirr/pygolang/commit/0e838833
__ https://lab.nexedi.com/kirr/pygolang/commit/106c1b95
__ https://lab.nexedi.com/kirr/pygolang/commit/4f6a9e09
__ https://lab.nexedi.com/kirr/pygolang/commit/7c929b25
__ https://lab.nexedi.com/kirr/pygolang/commit/8c2ac5e9
__ https://lab.nexedi.com/kirr/pygolang/commit/a0ba1226
__ https://lab.nexedi.com/kirr/pygolang/commit/873cf8aa
__ https://lab.nexedi.com/kirr/pygolang/commit/8399ff2d
__ https://lab.nexedi.com/kirr/pygolang/commit/419c8950
__ https://lab.nexedi.com/kirr/pygolang/commit/1a9dae3b
__ https://lab.nexedi.com/kirr/pygolang/commit/b073f6df
__ https://lab.nexedi.com/kirr/pygolang/commit/0e6088ec
__ https://lab.nexedi.com/kirr/pygolang/commit/73182038
- Provide C++/Cython/nogil API for `context` package. Python-level `context`
becomes a small wrapper around Cython/nogil one (`commit 1`__, 2__, 3__, 4__,
5__, 6__, 7__, 8__, 9__, 10__, 11__, 12__, 13__, 14__, 15__).
__ https://lab.nexedi.com/kirr/pygolang/commit/149ae661
__ https://lab.nexedi.com/kirr/pygolang/commit/cc7069e0
__ https://lab.nexedi.com/kirr/pygolang/commit/223d7950
__ https://lab.nexedi.com/kirr/pygolang/commit/89381488
__ https://lab.nexedi.com/kirr/pygolang/commit/9662785b
__ https://lab.nexedi.com/kirr/pygolang/commit/34e3c404
__ https://lab.nexedi.com/kirr/pygolang/commit/ba2ab242
__ https://lab.nexedi.com/kirr/pygolang/commit/9869dc45
__ https://lab.nexedi.com/kirr/pygolang/commit/20761c55
__ https://lab.nexedi.com/kirr/pygolang/commit/f76c11f3
__ https://lab.nexedi.com/kirr/pygolang/commit/281defb2
__ https://lab.nexedi.com/kirr/pygolang/commit/66e1e756
__ https://lab.nexedi.com/kirr/pygolang/commit/9216e2db
__ https://lab.nexedi.com/kirr/pygolang/commit/2a359791
__ https://lab.nexedi.com/kirr/pygolang/commit/a6c1c984
- Provide C++/Cython/nogil API for `sync` package. Python-level `sync` becomes a
small wrapper around Cython/nogil one (`commit 1`__, 2__, 3__, 4__, 5__, 6__, 7__, 8__, 9__).
__ https://lab.nexedi.com/kirr/pygolang/commit/0fb53e33
__ https://lab.nexedi.com/kirr/pygolang/commit/b316e504
__ https://lab.nexedi.com/kirr/pygolang/commit/c5c576d2
__ https://lab.nexedi.com/kirr/pygolang/commit/5146a416
__ https://lab.nexedi.com/kirr/pygolang/commit/4fc6e49c
__ https://lab.nexedi.com/kirr/pygolang/commit/a36efe6d
__ https://lab.nexedi.com/kirr/pygolang/commit/4fb9b51c
__ https://lab.nexedi.com/kirr/pygolang/commit/33cf3113
__ https://lab.nexedi.com/kirr/pygolang/commit/6d94fccf
- Add `errors` package with `errors.New` to create new error with provided text (commit__).
__ https://lab.nexedi.com/kirr/pygolang/commit/a245ab56
- Add `fmt` package with `fmt.sprintf` and `fmt.errorf` to format text into
strings and errors (commit__).
__ https://lab.nexedi.com/kirr/pygolang/commit/309963f8
- Add `strings` package with utilities like `strings.has_prefix`,
`strings.split` and similar (commit__).
__ https://lab.nexedi.com/kirr/pygolang/commit/0efd4a9a
- Add `cxx` package with `cxx.dict` and `cxx.set` providing ergonomic interface
over STL hash map and set (commit__).
__ https://lab.nexedi.com/kirr/pygolang/commit/9785f2d3
- Teach `defer` to chain exceptions (PEP 3134) and adjust traceback dumps to
include exception cause/context even on Python2 (`commit 1`__, 2__, 3__, 4__, 5__).
__ https://lab.nexedi.com/kirr/pygolang/commit/6729fe92
__ https://lab.nexedi.com/kirr/pygolang/commit/bb9a94c3
__ https://lab.nexedi.com/kirr/pygolang/commit/7faaecbc
__ https://lab.nexedi.com/kirr/pygolang/commit/06cac90b
__ https://lab.nexedi.com/kirr/pygolang/commit/1477dd02
- Provide `defer` as part of C++ API too (`commit 1`__, 2__, 3__).
__ https://lab.nexedi.com/kirr/pygolang/commit/1d153a45
__ https://lab.nexedi.com/kirr/pygolang/commit/14a249cb
__ https://lab.nexedi.com/kirr/pygolang/commit/39f40159
- Provide `build_ext` as part of `pyx.build` package API. This allows projects
to customize the way their Pygolang-based extensions are built (`commit 1`__, 2__).
__ https://lab.nexedi.com/kirr/pygolang/commit/8f9e5619
__ https://lab.nexedi.com/kirr/pygolang/commit/b4feee6f
- Fix `recover` to clean current exception (`commit 1`__, 2__).
__ https://lab.nexedi.com/kirr/pygolang/commit/9e6ff8bd
__ https://lab.nexedi.com/kirr/pygolang/commit/5f76f363
- Fix `select` to not leak object reference on error path (commit__).
__ https://lab.nexedi.com/kirr/pygolang/commit/e9180de1
- Fix gevent runtime to preserve Python exception state during runtime calls
(`commit 1`__, 2__).
__ https://lab.nexedi.com/kirr/pygolang/commit/689dc862
__ https://lab.nexedi.com/kirr/pygolang/commit/47fac0a9
| |_| |_| |_| |_| |_| |_| |_| |_| *This release is driven by* |wendelin.core|_ *v2 needs*.
| |_| |_| |_| |_| |_| |_| |_| |_| *This release is dedicated to the memory of* |Бася|_.
.. |wendelin.core| replace:: *wendelin.core*
.. _wendelin.core: https://pypi.org/project/wendelin.core
.. |Бася| replace:: *Бася*
.. _Бася: https://navytux.spb.ru/memory/%D0%91%D0%B0%D1%81%D1%8F/
0.0.4 (2019-09-17)
~~~~~~~~~~~~~~~~~~
......@@ -120,9 +288,6 @@ Pygolang change history
__ https://lab.nexedi.com/kirr/pygolang/commit/3c55ca59
.. readme_renderer/pypi don't support `.. class:: align-center`
.. |_| unicode:: 0xA0 .. nbsp
|_| |_| |_| |_| |_| |_| |_| |_| *This release is dedicated to the memory of* |Вера Павловна Супрун|_.
.. |Вера Павловна Супрун| replace:: *Вера Павловна Супрун*
......@@ -276,3 +441,7 @@ Pygolang change history
- Initial release; `gimport` functionality only (commit__).
__ https://lab.nexedi.com/kirr/pygolang/commit/9c61f254
.. readme_renderer/pypi don't support `.. class:: align-center`
.. |_| unicode:: 0xA0 .. nbsp
......@@ -31,7 +31,7 @@ See also package golang.pyx which provides similar functionality for Cython nogi
from __future__ import print_function, absolute_import
__version__ = "0.0.4"
__version__ = "0.0.5"
__all__ = ['go', 'chan', 'select', 'default', 'nilchan', 'defer', 'panic', 'recover', 'func', 'gimport']
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment