- 18 Nov, 2019 2 commits
-
-
Kirill Smelkov authored
Gevent runtime uses python-level calls internally which might interfere with current python state. For example if current python exception is set, and e.g. go or makesema runtime call is made, the following happens: golang/golang_test.py::test_pyx_runtime_vs_pyexc RuntimeError: abc The above exception was the direct cause of the following exception: SystemError: <class 'gevent.__semaphore.Semaphore'> returned a result with an error set Exception ignored in: 'golang.runtime._runtime_gevent._sema_alloc' SystemError: <class 'gevent.__semaphore.Semaphore'> returned a result with an error set terminate called after throwing an instance of 'golang::PanicError' what(): makesema: alloc failed Fatal Python error: Aborted -> Fix all functions in the runtimes that work at Python level to save/restore Python-level exception on entry/exit. This is mostly gevent runtime, but also a couple of non-posix fallbacks in thread runtime. The bug was there from day 1 of runtimes - from ce8152a2 (pyx api: Provide sleep), f971a2a8 (pyx api: Provide go) and 69db91bf (libgolang: Add internal semaphores).
-
Kirill Smelkov authored
In the next patch we will fix runtime functions that work at Python level to preserve user Python exception state. Before that, as a preparatory step, restructure runtimes code, so that all such Python-accessing functions are grouped together.
-
- 15 Nov, 2019 1 commit
-
-
Kirill Smelkov authored
setuptools_dso hardcodes to use its own build_ext in its setup. We amend that build_ext and inject our own version carefully while setup() is run - see 7ae8c4f3 (pyx.build: Allow to combine C and C++ sources in one extension). However, currently, if user uses cmdclass = {'build_ext': ...} in his own setup, this will be ignored and overwritten by golang.pyx.build.build_ext . One example where this breaks things is wendelin.core which hooks into build_ext to first configure/build CCAN before any extension: https://lab.nexedi.com/nexedi/wendelin.core/blob/b26ba558/setup.py#L147-153 -> Fix it by taking into account what user could put into cmdclass.
-
- 14 Nov, 2019 17 commits
-
-
Kirill Smelkov authored
If defer was used multiple times in one function it was failing to compile. For example added test is failing like this: In file included from golang/runtime/libgolang_test.cpp:22: golang/runtime/libgolang_test.cpp: In function ‘void __test_defer(int*)’: ./golang/libgolang.h:457:59: error: redeclaration of ‘golang::_deferred _defer___COUNTER__’ #define defer(f) golang::_deferred _defer_ ## __COUNTER__ (f) ^ golang/runtime/libgolang_test.cpp:473:5: note: in expansion of macro ‘defer’ defer([&]() { ^~~~~ ./golang/libgolang.h:457:36: note: ‘golang::_deferred _defer___COUNTER__’ previously declared here #define defer(f) golang::_deferred _defer_ ## __COUNTER__ (f) ^~~~~~~ golang/runtime/libgolang_test.cpp:470:5: note: in expansion of macro ‘defer’ defer([&]() { ^~~~~ It turns out the __COUNTER__ was not expanded and they say you have to use 3 level of macro calls. Oh well... https://stackoverflow.com/a/27611869/9456786
-
Kirill Smelkov authored
std::function is frequently too long to type while func is more native to Go.
-
Kirill Smelkov authored
For example wendelin.core wants to accept Python-level context, transform it into C-level context and pass to pyx/nogil code.
-
Kirill Smelkov authored
Provide context-related functionality that can be used directly from C++ and Pyx/nogil codes. Python-level classes and functions become small wrappers around pyx/nogil ones. Like with timers (b073f6df "time: Move/Port timers to C++/Pyx nogil") and interfaces (5a99b769 "libgolang: Start providing interfaces") memory for objects dynamically allocated on heap is managed automatically.
-
Kirill Smelkov authored
Start new package that provides C++ amendments to be used by libgolang and its users. Current functionality: dict and set that wrap std::unordered_map and std::unordered_set into ergonomic interface. The code originates from wendelin.core: https://lab.nexedi.com/kirr/wendelin.core/blob/5a045ed1/wcfs/internal/wcfs_misc.h#L214-257 Pyx/nogil only.
-
Kirill Smelkov authored
With just one function - errors.New() - to create new error with given text. This will be soon used in Pyx/nogil context package to create context.canceled and context.deadlineExceeded errors. Pyx/nogil only for now.
-
Kirill Smelkov authored
- interface is analog of interface{} in Go. - error is analog of error in Go. For now the interfaces are implemented via classic C++ scheme via inheritance and additional vtable field in pointed object. In the future we might want to consider reworking that to Go-style interfaces without predeclaring which interfaces a class implements. Top-level documentation is TODO. Interfaces will be soon needed to describe Context interface in C context package.
-
Kirill Smelkov authored
- refptr==refptr (e.g. to compare whether returned error object is something particular) - adoptref() and newref() - object (exposed as gobject not to be confused with builtin pyx "object" referring to python object) All this will be soon used in C version of context package.
-
Kirill Smelkov authored
Briefly describe provided functionality instead of only referring to https://golang.org/pkg/context and https://blog.golang.org/context.
-
Kirill Smelkov authored
Thinking a bit more after e82b4fab (libgolang: Objects refcounting (initial draft)) and working on interfaces as part of context pyx -> C move, I've came to conclusion that the better name for on-heap objects managed by libgolang is just "object" without "ref" prefix. -> Rename refobj -> object and amend its documentation correspondingly.
-
Kirill Smelkov authored
So that refptr<T> could be used as keys in e.g. dict (unordered_map) or set (unordered_set). set<refptr> will soon be used by C version of context package.
-
Kirill Smelkov authored
We already provide select for array and std::initializer_list. However when cases are constructed at runtime dynamically both array and initializer_list are not convenient to use. -> Provide select() overload that accepts vector<_selcase>. This functionality will soon be used in C version of context package.
-
Kirill Smelkov authored
- indicated that WorkGroup is not provided in Go version (amends 40f3b90c "sync: Provide package documentation"). - fix typo in sync.pxd subject (amends 34b7a1f4 "golang: Expose Sema and Mutex as public Python and Cython/nogil API")
-
Kirill Smelkov authored
Every time tests are run under Python3 the following warnings are printed: golang/golang_test.py::test_func .../golang/golang_test.py:990: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() assert inspect.formatargspec(*inspect.getargspec(MyClass.zzz)) == '(self, v, x=2, **kkkkwww)' golang/golang_test.py::test_func .../golang/golang_test.py:990: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly assert inspect.formatargspec(*inspect.getargspec(MyClass.zzz)) == '(self, v, x=2, **kkkkwww)' However since we are not going to drop Python2 support soon, and there is no reffered "use-instead" functionality on Python2, let's simply silence the warning for now.
-
Kirill Smelkov authored
It will be some extra work to define general equality when moving context code into C. However that is unnecessary as usually a key is specific to one particular package, and value associated with that key is struct, or some other aggregate, that is holding all package-specific context values. Switching to compare keys by identity preserves that use-case while making transition to C easier.
-
Kirill Smelkov authored
We always store just one key in there. This will simplify transition to C.
-
Kirill Smelkov authored
This continues 2c8063f4 (*: Channels must be compared by ==, not by "is" even for nilchan) and changes comparision of .done() for different contexts from "is" to ==. The reason is that soon we are going to move contexts into C, and this way in context tree there potentially can be PyCtx -> CCtx -> PyCtx chains, and while all those contexts could have the same C-level channel, it would be difficult to propagate the knowledge that Python-level pychan wrapper should be also exactly the same. We already treat channel objects as pointers and require to compare them by ==, so there should be no loss here. However to keep performance of repeated select( ctx.done(), ... ) we continue to require that .done() for a particular python context object returns exactly the same channel object every time.
-
- 13 Nov, 2019 1 commit
-
-
Kirill Smelkov authored
Rearrange time C++ API so that e.g. time.Ticker is already refptr<time._Ticker>. This way time users don't need to write refptr all the time.
-
- 11 Nov, 2019 2 commits
-
-
Kirill Smelkov authored
https://bitbucket.org/pypy/pypy/commits/251b47698e8e https://bitbucket.org/pypy/pypy/issues/3096#comment-54707695 Which in turn fixes recover to clear current exception (see 9e6ff8bd "golang: Fix recover to clear current exception") for context.
-
Kirill Smelkov authored
e.g. Context -> PyContext, with_cancel -> pywith_cancel, etc. This will make the followup transition of context to pyx/nogil more clear.
-
- 07 Nov, 2019 10 commits
-
-
Kirill Smelkov authored
As of 20191107 imp is present in Python 3.8 source tree and is _used_ there - see e.g. https://github.com/python/cpython/blob/v3.8.0-66-g7c20888e713/Lib/pkgutil.py#L188-L192 And they say imp will be there until at least Python 4 - see e.g. https://github.com/python/cpython/commit/e4f41deccf94 This fixes test_defer_excchain_dump failure under python3-dbg, where E Failed: not equal: E Differences (unified diff with -expected +actual): E @@ -1,4 +1,6 @@ E +PYGOLANG/golang/_gopath.py:37: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses E + import imp was added into traceback printed on stderr. Fixes: bb9a94c3 (golang: Teach defer to chain exceptions (PEP 3134) even on Python2)
-
Kirill Smelkov authored
Provide time.Ticker, time.Timer and friends that can be used directly from C++ and Pyx/nogil codes. Python-level classes become small wrapper around pyx/nogil ones. This is the first patch that moves to Pyx/nogil classes that are dynamically allocated on heap. refptr<T> is used to automatically manage lifetime of such objects. At Pyx level exposed API is very similar to Python-one, while internally it uses refptr<T> and friends.
-
Kirill Smelkov authored
Refobj was just added in previous patch as the class that provides base functionality for reference-counted objects.
-
Kirill Smelkov authored
Since we are going to move more pygolang functionality into C++ (timers, context, ...), and since C++ does not have garbage collector, we will need to find a way to automatically manage memory in leak/error free way. Let's do this via refptr<T> smart pointer (inspired by WebKit's RefPtr<T>), which, similarly to chan<T> automatically manages pointed object's lifetime through reference counting. refptr<T> will be used in follow-up patches. Top-level documentation is TODO.
-
Kirill Smelkov authored
This will be needed in Python-wrappers for C-level code which exposes channels. For example in time package whose implementation will soon be moved to nogil world fully.
-
Kirill Smelkov authored
3121b290 (golang: Teach pychan to work with channels of C types, not only PyObjects) added ability to convert pychan to chan[T], but described that ability only briefly in README. Provide more details in that description before we add description and implementation for reflexive pychan <- chan[T] wrapping.
-
Kirill Smelkov authored
We already have some sync functionality implemented in C++ (e.g. sync.Once, sync.WaitGroup) and we are going to add more and also move timers implementation to C++. It is getting crowded for that functionality to still live in libgolang.{h,cpp} -> Split sync & time functionality into their own C++ packages (still built-into libgolang.so)
-
Kirill Smelkov authored
Briefly describe provided functionality instead of only referring to https://golang.org/pkg/time.
-
Kirill Smelkov authored
Briefly describe provided functionality instead of only referring to https://golang.org/pkg/sync.
-
Kirill Smelkov authored
Make built extensions depend on golang/{sync,time}.pxd and rebuild if those files change. A bit overkill to depend globally, but more correct, since an extension that is using pygolang can be using those packages. Before the patch - without those depends - since distutils does not implement proper dependency tracking, extensions - even those that actually depend on sync/time pxd, were not rebuilt. sync.pxd was missed to be added in 34b7a1f4 (golang: Expose Sema and Mutex as public Python and Cython/nogil API). time.pxd was missed to be added in ce8152a2 (pyx api: Provide sleep).
-
- 06 Nov, 2019 3 commits
-
-
Kirill Smelkov authored
Just cosmetics - easier to oversee the code.
-
Kirill Smelkov authored
Ticker: self -> pytx Timer: self -> pyt
-
Kirill Smelkov authored
e.g. tick -> pytick, Timer -> PyTimer. etc. This will make the followup transition of time to pyx/nogil more clear.
-
- 04 Nov, 2019 4 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
e.g. go -> pygo, select -> pyselect, etc. This will make the followup transition to non-py mode more clear.
-