- 16 May, 2019 2 commits
-
-
Kirill Smelkov authored
Add time.Timer, time.Ticker and convenience functions time.tick, time.after and time.after_func. These will be used in context to support deadlines and timeouts. While at time topic, also provide sleep and now from golang.time, so that there is no need to import both golang.time and stdlib's time in a file. Provide time constants in the module as they are useful to have and mirror constants provided by Go's time. Note: timers implementation is very suboptimal for now.
-
Kirill Smelkov authored
Python adds dirname of run program to sys.path . This way when golang_test_goleaked.py runs it can import modules located under golang/ just by their name. Until now this was not noticed, but in the next patch we are going to add golang.time module and if test program is run with golang/ in sys.path just plain `import time` won't import time from stdlib and instead import time from golang/ . Such behaviour can be mitigated by doing `from __future__ import absolute_import` and we do that, including in golang_test_goleaked.py (see 81dfefa0 "*: __future__ += absolute_imports; Use unified __future__ everywhere"). However that does not prevent modules - even modules from stdlib - who are doing `import time` and not doing future absolute_import to import golang's time instead of stdlib. For example on PyPy2 threading imports time and then the test breaks: Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/golang_test_goleaked.py", line 24, in <module> from golang import go, chan File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/__init__.py", line 38, in <module> import inspect, threading, collections, random, sys File "/usr/lib/pypy/lib-python/2.7/threading.py", line 15, in <module> from time import time as _time, sleep as _sleep File "/home/kirr/src/tools/go/pygolang/.tox/pypy-thread/site-packages/golang/time.py", line 30, in <module> from golang import go, chan, select, default, nilchan, panic ImportError: cannot import name 'go' -> Move the test program into a directory different from golang/ to avoid this trap.
-
- 14 May, 2019 1 commit
-
-
Kirill Smelkov authored
- we are going to introduce golang.time, and from inside there without `from __future__ import absolute_imports` it won't be possible to import needed stdlib's time. - we were already doing `from __future__ import print_function`, but only in some files. -> It makes sense to apply updated __future__ usage uniformly.
-
- 10 May, 2019 1 commit
-
-
Kirill Smelkov authored
- correct CHANGELOG title levels: since readme uses "----" for the first level and "~~~~" for the second level, "====" was interpreted as the third title level and "Pygolang change history" became sub-sub-section of "Additional packages and utilities" with section for each version conversely becoming first level. It was not very noticeable until 0c5f9d06 (readme: Push "Additional packages and utilities" into its own section) started to use "~~~~". -> Rework CHANGELOG titling to align with the one in README. - fix minor markup bits in README.
-
- 09 May, 2019 1 commit
-
-
Kirill Smelkov authored
-
- 03 May, 2019 3 commits
-
-
Kirill Smelkov authored
WorkGroup provides way to spawn goroutines that work on a common task and wait for their completion. It is modelled after https://godoc.org/golang.org/x/sync/errgroup but is not equal to it. See WorkGroup docstring for details.
-
Kirill Smelkov authored
Add sync.Once and sync.WaitGroup.
-
Kirill Smelkov authored
Add .Context, .background .with_cancel and .with_value. There is no support for deadline/timeout yet, since Go time package is not yet mirrored. Contrary to Go stdlib version, we also support context.merge that takes https://godoc.org/lab.nexedi.com/kirr/go123/xcontext#hdr-Merging_contexts for its inspiration. See https://blog.golang.org/context for overview of contexts.
-
- 02 May, 2019 2 commits
-
-
Kirill Smelkov authored
We are going to add more packages to mirror Go stdlib (e.g. next is context). Organize a dedicated section for what is additionally provided besides mirroring Go language and refer to that place only once from the top-level abstract.
-
Kirill Smelkov authored
Send/recv on the nil channel block forever; close panics. If a nil channel is used in select - corresponding case is never selected. Setting channel to nil is a usual idiom in Go to disable processing some cases in select. Nil channel is also used as "done" for e.g. context.Background() - for contexts that can be never canceled.
-
- 24 Mar, 2019 3 commits
-
-
Kirill Smelkov authored
It was deprecated and scheduled to be removed in 942ee900 (golang: Deprecate @method(cls) in favour of @func(cls)).
-
Kirill Smelkov authored
-
Kirill Smelkov authored
pypy-gevent runtests: commands[0] | gpython -m pytest gpython/ golang/ Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/bin/gpython", line 10, in <module> sys.exit(main()) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gpython/__init__.py", line 153, in main _ = monkey.patch_all() # XXX sys=True ? File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 976, in patch_all patch_thread(Event=Event, _warnings=_warnings) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 178, in ignores return func(*args, **kwargs) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 588, in patch_thread _patch_existing_locks(threading_mod) File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 492, in _patch_existing_locks if o._RLock__owner is not None: AttributeError: 'thread.RLock' object has no attribute '_RLock__owner' It will be fixed on next Gevent release, however until then the crash is there and we have to workaround: we can skip patching existing locks as it will be the same behaviour of next Gevent release, since its just not possible to patch locks created via instantiated C-level classes: https://github.com/gevent/gevent/commit/d0e04658 We are doing monkey-patching very early, so it should be safe.
-
- 20 Mar, 2019 1 commit
-
-
Kirill Smelkov authored
Channel tests are passing on an idle machine. However if there is another load - e.g. in the presence of simultaneous Firefox start, channel tests break, for example: def test_select(): ... # non-blocking try recv: ok ch = chan() done = chan() def _(): for i in range(N): ch.send(i) done.close() go(_) for i in range(N): tdelay() if i % 2: _, _rx = select( ch.recv, default, ) assert (_, _rx) == (0, i) else: _, _rx = select( ch.recv_, default, ) > assert (_, _rx) == (0, (i, True)) E assert (1, None) == (0, (320, True)) E At index 0 diff: 1 != 0 E Use -v to get the full diff golang_test.py:209: AssertionError The failure here is that it was default case selected, not ch.recv_. The default was selected because the sending goroutine was not fast to enqueue next send before we tried to receive. We were trying to make sure that the sender will be enqueued first via adding 1ms time delay before trying to receive, but in the presence of concurrent load spikes that turns out to be not enough. We could try to fix the test by increasing the time to sleep in tdelay, make the tests more slow and still not 100% reliable. However we can change the tests to instead actually wait for the condition that is semantically required: a sender enqueued on the channel. Do that everywhere where tdelay was used. Now tests are faster (it was ~3s total, now it is ~ 0.5s total) and pass irregardless of whether the machine is idle or otherwise loaded.
-
- 19 Mar, 2019 3 commits
-
-
Kirill Smelkov authored
Debian dropped Python3.6 from python3-distutils. Without it I'm no longer being able to run tests on that python version: (neo) (z-dev) (g.env) kirr@deco:~/src/tools/go/pygolang$ tox -e py36-thread execution failed: -- Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'distutils.sysconfig' GLOB sdist-make: /home/kirr/src/tools/go/pygolang/setup.py py36-thread create: /home/kirr/src/tools/go/pygolang/.tox/py36-thread ERROR: invocation failed (exit code 1), logfile: /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log ERROR: actionid: py36-thread msg: getenv cmdargs: '/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/virtualenv.py", line 25, in <module> import distutils.sysconfig ModuleNotFoundError: No module named 'distutils.sysconfig' Running virtualenv with interpreter /usr/bin/python3.6 ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread (see /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log)', 1) https://bugs.debian.org/918881#19
-
Kirill Smelkov authored
PyPy7 always pre-imports the time module. Without the change tests were failing: Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/bin/gpython", line 10, in <module> sys.exit(main()) File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/site-packages/gpython/__init__.py", line 145, in main '\n\n\t%s\n\nsys.modules:\n\n\t%s' % (bad, sysmodv)) RuntimeError: gpython: internal error: the following modules are pre-imported, but must be not: ['time'] sys.modules: ['__future__', '__main__', '__pypy__', '__pypy__._pypydatetime', '__pypy__.builders', '__pypy__.intop', '__pypy__.os', '__pypy__.thread', '__pypy__.time', '_ast', '_bootlocale', '_codecs', '_collections', '_collections_abc', '_continuation', '_csv', '_frozen_importlib', '_frozen_importlib_external', '_imp', '_io', '_locale', '_multibytecodec', '_operator', '_rawffi', '_rawffi.alt', '_signal', '_sre', '_structseq', '_thread', '_warnings', '_weakref', '_weakrefset', 'abc', 'array', 'builtins', 'codecs', 'copyreg', 'encodings', 'encodings.aliases', 'encodings.ascii', 'encodings.latin_1', 'encodings.utf_8', 'errno', 'gc', 'genericpath', 'gpython', 'marshal', 'os', 'os.path', 'posix', 'posixpath', 'pwd', 're', 'site', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'sys', 'time', 'unicodedata']
-
Kirill Smelkov authored
Debian started to ship PyPy3: https://packages.debian.org/search?keywords=pypy3
-
- 15 Mar, 2019 1 commit
-
-
Kirill Smelkov authored
Since we already have func (see 5146eb0b "Add support for defer & recover") we can use @func for both plain functions and for methods. For example instead of @func def my_function(...): ... @method(MyClass) <-- NOTE def my_method(self, ...): ... have it as @func def my_function(...): ... @func(MyClass) <-- NOTE def my_method(self, ...): ... which looks more similar to Go and exposes less golang keywords to a user.
-
- 13 Mar, 2019 1 commit
-
-
Kirill Smelkov authored
Pymain was not adding e.g. directory of executed file to sys.path, and as the result if there were e.g. 2 files dir/hello.py # imports world dir/world.py running `gpython dir/hello.py` would fail to import world. The case for interactive console was also failing to setup sys.argv as empty, so it was containing ['/path/to/gpython']. It was also, contrary to standard python, unconditionally showing '>>>' prompt even when stdin was not a tty. Fix all that and add a test to cover pymain functionality.
-
- 16 Jan, 2019 6 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Provide gpython interpreter, that sets UTF-8 as default encoding, integrates gevent and puts go, chan, select etc into builtin namespace. Please see details in the documentation added by this patch. pymain was taken from go123@7a476082 (go123: tracing/pyruntraced: New tool to run Python code with tracepoints activated (draft))
-
Kirill Smelkov authored
We describe golang/testdata/ in MANIFEST.in, and that makes testdata to be included into sdist. However testdata does not get installed. Since, we want to be able to run pygolang tests, even on installed package, we need the testdata files to be present there as well. Fix it. See https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files for setuptools-related details.
-
Kirill Smelkov authored
Move version definition from setup.py to golang module so that it is available at runtime with plain golang.__version__ .
-
Kirill Smelkov authored
A comma was missing - this way last element was 'funcgimport', not 'func' and 'gimport', and so `from golang import *` was failing.
-
Kirill Smelkov authored
.../pygolang/golang/strconv_test.py:75: PytestDeprecationWarning: raises(..., 'code(as_a_string)') is deprecated, use the context manager form or use `exec()` directly ... See https://docs.pytest.org/en/latest/deprecations.html#raises-warns-with-a-string-as-the-second-argument
-
- 30 Dec, 2018 1 commit
-
-
Kirill Smelkov authored
Until now it was not included by `python setup.py sdist`. On this topic, also include README.rst explicitly, since even if it is currently implicitly included, that might change some day and break us.
-
- 13 Dec, 2018 4 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
This are functions to decode quotation that was produced by strconv.quote().
-
Kirill Smelkov authored
Move quote implementation from gcompat to strconv. Make quote work on both unicode|bytes string input and produce the same output type. Preserve qq (still remaining in gcompat) to always produce str, since that is used in prints.
-
Kirill Smelkov authored
Stop hiding qq and benchmarking bits in changelog only - add their description into additional sections in readme. Since py.bench description is now in readme, there is no need to keep py.bench example in the changelog.
-
- 31 Oct, 2018 1 commit
-
-
Kirill Smelkov authored
All tests pass.
-
- 30 Oct, 2018 6 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
`defer` allows to schedule a cleanup to be executed when current function returns. It is similar to `try`/`finally` but does not force the cleanup part to be far away in the end. For example:: wc = wcfs.join(zurl) │ wc = wcfs.join(zurl) defer(wc.close) │ try: │ ... ... │ ... ... │ ... ... │ finally: │ wc.close() For completeness there is `recover` and `panic` that allow to program with Go-style error handling, for example:: def _(): r = recover() if r is not None: print("recovered. error was: %s" % (r,)) defer(_) ... panic("aaa") But `recover` and `panic` are probably of less utility since they can be practically natively modelled with `try`/`except`. If `defer` is used, the function that uses it must be wrapped with `@func` or `@method` decorators. The implementation is partly inspired by work of Denis Kolodin: - https://habr.com/post/191786 - https://stackoverflow.com/a/43028386/9456786
-
Kirill Smelkov authored
While the second phase of select is running we queue send/recv cases to corresponding channels. At some point - when some of the cases are already queued - a peer goroutine might try to send/recv on that channel. And it will succeed because a waiter was queued to the channel. At the same time select is continuing its enqueue loop and before enqueuing to a channel it tries to send/recv there. If that channel became just ready (i.e. just after select poll phase) the try to send/recv will succeed. This means that actually 2 select cases could be executed at the same time. Fix it by carefully checking whether some case already won before trying to send/recv on a channel. This fixes the test failures that were demonstrated by previous 2 patches.
-
Kirill Smelkov authored
Previous select-select test uses 2 channels and recreats them every iteration. However the bug we just saw in b51b8d5d (select: Run tests more thoroughly) is of parasitic nature - where misbehaviour depends on what state has been left there from previous select and whether the race to get to that state fast enough succeeded. So add a more secialized test which tries to trigger the parasitic effects that depend on previous state left by select on the channels. The test, similarly to b51b8d5d, currently fails.
-
Kirill Smelkov authored
With more iterations it triggers a bug: kirr@deco:~/src/tools/go/pygolang$ py.test -vsx ============================================== test session starts =============================================== platform linux2 -- Python 2.7.15+, pytest-3.6.4, py-1.6.0, pluggy-0.6.0 -- /usr/bin/python2 cachedir: .pytest_cache rootdir: /home/kirr/src/tools/go/pygolang, inifile: collected 7 items golang/_gopath_test.py::test_import_module PASSED golang/_gopath_test.py::test_import_package PASSED golang/gcompat_test.py::test_qq PASSED golang/golang_test.py::test_go PASSED golang/golang_test.py::test_chan PASSED golang/golang_test.py::test_select Exception in thread Thread-117: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/home/kirr/src/tools/go/pygolang/golang/golang_test.py", line 320, in _ assert (_, _rx) == (1, 'b') AssertionError: assert (1, 'xxx1') == (1, 'b') At index 1 diff: 'xxx1' != 'b' Full diff: - (1, 'xxx1') + (1, 'b')
-
Kirill Smelkov authored
This test was failing when pygolang was not dev-mode installed (`pip install -e .`): kirr@deco:~/src/tools/go/pygolang$ py.test =========================================== test session starts ============================================ platform linux2 -- Python 2.7.15+, pytest-3.6.4, py-1.6.0, pluggy-0.6.0 rootdir: /home/kirr/src/tools/go/pygolang, inifile: collected 8 items golang/_gopath_test.py .. [ 25%] golang/gcompat_test.py . [ 37%] golang/golang_test.py F.... [100%] ================================================= FAILURES ================================================= _________________________________________________ test_go __________________________________________________ def test_go(): # leaked goroutine behaviour check: done in separate process because we need # to test process termination exit there. subprocess.check_call([sys.executable, > dirname(__file__) + "/golang_test_goleaked.py"]) golang/golang_test.py:38: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ popenargs = (['/usr/bin/python2', '/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py'],) kwargs = {}, retcode = 1 cmd = ['/usr/bin/python2', '/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py'] def check_call(*popenargs, **kwargs): """Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute. The arguments are the same as for the Popen constructor. Example: check_call(["ls", "-l"]) """ retcode = call(*popenargs, **kwargs) if retcode: cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] > raise CalledProcessError(retcode, cmd) E CalledProcessError: Command '['/usr/bin/python2', '/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py']' returned non-zero exit status 1 /usr/lib/python2.7/subprocess.py:190: CalledProcessError ------------------------------------------- Captured stderr call ------------------------------------------- Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py", line 23, in <module> from golang import go, chan ImportError: No module named golang ==================================== 1 failed, 7 passed in 0.10 seconds ==================================== Fix it by injecting top-level pygolang/ into $PYTHONPATH when testing via external processes. Fixes 69cef96e (go: Don't allow leaked goroutines to prevent program to exit)
-
- 25 Oct, 2018 1 commit
-
-
Kirill Smelkov authored
There was no test for @method so far and that's why it went unnoticed. But on Python3 it breaks on f.func_name: In [3]: def f(): pass In [4]: f.func_name --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-4-662dcbac1531> in <module>() ----> 1 f.func_name AttributeError: 'function' object has no attribute 'func_name' Fix it by using f.__name__ which works on both py2 and py3. Add test for @method to make sure it doesn't break unnoticed.
-
- 24 Oct, 2018 1 commit
-
-
Kirill Smelkov authored
This is the Go behaviour, as demonstratd by the following program: ---- 8< ---- package main import ( "fmt" "time" ) func work(w int) { for i := 0; ; i++ { fmt.Printf("w%d: %d\n", w, i) time.Sleep(1*time.Second) } } func main() { for i := 0; i < 100; i++ { go work(i) } time.Sleep(3*time.Second) println("main: exit") } ---- 8< ----
-
- 04 Jul, 2018 1 commit
-
-
Kirill Smelkov authored
-