pygolang:6d9c27b1897d4a28a28e227c8db04cbc3ae13e75 commitshttps://lab.nexedi.com/nexedi/pygolang/-/commits/6d9c27b1897d4a28a28e227c8db04cbc3ae13e752019-05-09T23:07:26+03:00https://lab.nexedi.com/nexedi/pygolang/-/commit/6d9c27b1897d4a28a28e227c8db04cbc3ae13e75pygolang v0.0.12019-05-09T23:07:26+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/9ee7ba91771c8a84be6444f50937a1185c84e12esync += WorkGroup2019-05-03T16:55:18+03:00Kirill Smelkovkirr@nexedi.com
WorkGroup provides way to spawn goroutines that work on a common task
and wait for their completion. It is modelled after
<a href="https://godoc.org/golang.org/x/sync/errgroup" rel="nofollow noreferrer noopener" target="_blank">https://godoc.org/golang.org/x/sync/errgroup</a> but is not equal to it.
See WorkGroup docstring for details.https://lab.nexedi.com/nexedi/pygolang/-/commit/e6bea2cfdae4be96c92a383eee7ba956c498eb30sync: New package that mirrors Go's sync2019-05-03T16:51:19+03:00Kirill Smelkovkirr@nexedi.com
Add sync.Once and sync.WaitGroup.https://lab.nexedi.com/nexedi/pygolang/-/commit/e9567c7bd274f40c6eae9f82c44d34f032c3997dcontext: New package that mirrors Go's context2019-05-03T16:41:21+03:00Kirill Smelkovkirr@nexedi.com
Add .Context, .background .with_cancel and .with_value. There is no
support for deadline/timeout yet, since Go time package is not yet
mirrored.
Contrary to Go stdlib version, we also support context.merge that takes
<a href="https://godoc.org/lab.nexedi.com/kirr/go123/xcontext#hdr-Merging_contexts" rel="nofollow noreferrer noopener" target="_blank">https://godoc.org/lab.nexedi.com/kirr/go123/xcontext#hdr-Merging_contexts</a>
for its inspiration.
See <a href="https://blog.golang.org/context" rel="nofollow noreferrer noopener" target="_blank">https://blog.golang.org/context</a> for overview of contexts.https://lab.nexedi.com/nexedi/pygolang/-/commit/0c5f9d06da60d6a019d9b66a2d4563fa930d8102readme: Push "Additional packages and utilities" into its own section2019-05-02T12:50:42+03:00Kirill Smelkovkirr@nexedi.com
We are going to add more packages to mirror Go stdlib (e.g. next is
context). Organize a dedicated section for what is additionally provided
besides mirroring Go language and refer to that place only once from the
top-level abstract.https://lab.nexedi.com/nexedi/pygolang/-/commit/2aad64bb954d28a28171b92deba8eb3f7c72a22agolang: Add support for nil channel2019-05-02T12:50:20+03:00Kirill Smelkovkirr@nexedi.com
Send/recv on the nil channel block forever; close panics.
If a nil channel is used in select - corresponding case is never selected.
Setting channel to nil is a usual idiom in Go to disable processing some
cases in select. Nil channel is also used as "done" for e.g.
context.Background() - for contexts that can be never canceled.https://lab.nexedi.com/nexedi/pygolang/-/commit/262f89861b826bb26ece1627299c39f8cb6dadb4golang: Kill @method2019-03-24T13:50:20+03:00Kirill Smelkovkirr@nexedi.com
It was deprecated and scheduled to be removed in <a href="/kirr/pygolang/-/commit/942ee9009d2e0a5d5cc174e72243591eee86ba55" data-original="942ee900" data-link="false" data-link-reference="false" data-project="1158" data-commit="942ee9009d2e0a5d5cc174e72243591eee86ba55" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="golang: Deprecate @method(cls) in favour of @func(cls)" class="gfm gfm-commit has-tooltip">942ee900</a> (golang:
Deprecate @method(cls) in favour of @func(cls)).https://lab.nexedi.com/nexedi/pygolang/-/commit/0f60aa7736900e493881b49a3437535ef65b0d53pygolang v0.0.0.dev82019-03-24T13:23:00+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/704d99f0b49e34ac1573fce32b669bf59c0540c0gpython: Workaround PyPy2 vs Gevent patch_thread crash2019-03-24T12:00:04+03:00Kirill Smelkovkirr@nexedi.com
pypy-gevent runtests: commands[0] | gpython -m pytest gpython/ golang/
Traceback (most recent call last):
File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/bin/gpython", line 10, in <module>
sys.exit(main())
File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gpython/__init__.py", line 153, in main
_ = monkey.patch_all() # XXX sys=True ?
File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 976, in patch_all
patch_thread(Event=Event, _warnings=_warnings)
File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 178, in ignores
return func(*args, **kwargs)
File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 588, in patch_thread
_patch_existing_locks(threading_mod)
File "/home/kirr/src/tools/go/pygolang/.tox/pypy-gevent/site-packages/gevent/monkey.py", line 492, in _patch_existing_locks
if o._RLock__owner is not None:
AttributeError: 'thread.RLock' object has no attribute '_RLock__owner'
It will be fixed on next Gevent release, however until then the crash is there
and we have to workaround: we can skip patching existing locks as it will be
the same behaviour of next Gevent release, since its just not possible to patch
locks created via instantiated C-level classes:
<a href="https://github.com/gevent/gevent/commit/d0e04658" rel="nofollow noreferrer noopener" target="_blank">https://github.com/gevent/gevent/commit/d0e04658</a>
We are doing monkey-patching very early, so it should be safe.https://lab.nexedi.com/nexedi/pygolang/-/commit/731f39e38a31305e2a5dc9a65f9a9585bc27ee92golang: Deflake channel tests2019-03-20T07:57:31+03:00Kirill Smelkovkirr@nexedi.com
Channel tests are passing on an idle machine. However if there is
another load - e.g. in the presence of simultaneous Firefox start,
channel tests break, for example:
def test_select():
...
# non-blocking try recv: ok
ch = chan()
done = chan()
def _():
for i in range(N):
ch.send(i)
done.close()
go(_)
for i in range(N):
tdelay()
if i % 2:
_, _rx = select(
ch.recv,
default,
)
assert (_, _rx) == (0, i)
else:
_, _rx = select(
ch.recv_,
default,
)
> assert (_, _rx) == (0, (i, True))
E assert (1, None) == (0, (320, True))
E At index 0 diff: 1 != 0
E Use -v to get the full diff
golang_test.py:209: AssertionError
The failure here is that it was default case selected, not ch.recv_. The
default was selected because the sending goroutine was not fast to
enqueue next send before we tried to receive. We were trying to make
sure that the sender will be enqueued first via adding 1ms time delay
before trying to receive, but in the presence of concurrent load spikes
that turns out to be not enough.
We could try to fix the test by increasing the time to sleep in tdelay,
make the tests more slow and still not 100% reliable. However we can
change the tests to instead actually wait for the condition that is
semantically required: a sender enqueued on the channel.
Do that everywhere where tdelay was used.
Now tests are faster (it was ~3s total, now it is ~ 0.5s total) and pass
irregardless of whether the machine is idle or otherwise loaded.https://lab.nexedi.com/nexedi/pygolang/-/commit/469f21a9527becce8dc12134ad08d16cde7f28cdtox -= Python3.62019-03-19T22:59:06+03:00Kirill Smelkovkirr@nexedi.com
Debian dropped Python3.6 from python3-distutils. Without it I'm no
longer being able to run tests on that python version:
(neo) (z-dev) (g.env) kirr@deco:~/src/tools/go/pygolang$ tox -e py36-thread
execution failed: -- Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'distutils.sysconfig'
GLOB sdist-make: /home/kirr/src/tools/go/pygolang/setup.py
py36-thread create: /home/kirr/src/tools/go/pygolang/.tox/py36-thread
ERROR: invocation failed (exit code 1), logfile: /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log
ERROR: actionid: py36-thread
msg: getenv
cmdargs: '/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread'
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/virtualenv.py", line 25, in <module>
import distutils.sysconfig
ModuleNotFoundError: No module named 'distutils.sysconfig'
Running virtualenv with interpreter /usr/bin/python3.6
ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread (see /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log)', 1)
<a href="https://bugs.debian.org/918881#19" rel="nofollow noreferrer noopener" target="_blank">https://bugs.debian.org/918881#19</a>https://lab.nexedi.com/nexedi/pygolang/-/commit/e847c550b299c23419974f5bb269e282dbd57cb9gpython: Don't check for time not being pre-imported on PyPy2019-03-19T21:15:33+03:00Kirill Smelkovkirr@nexedi.com
PyPy7 always pre-imports the time module. Without the change tests were
failing:
Traceback (most recent call last):
File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/bin/gpython", line 10, in <module>
sys.exit(main())
File "/home/kirr/src/tools/go/pygolang/.tox/pypy3-gevent/site-packages/gpython/__init__.py", line 145, in main
'\n\n\t%s\n\nsys.modules:\n\n\t%s' % (bad, sysmodv))
RuntimeError: gpython: internal error: the following modules are pre-imported, but must be not:
['time']
sys.modules:
['__future__', '__main__', '__pypy__', '__pypy__._pypydatetime', '__pypy__.builders', '__pypy__.intop', '__pypy__.os', '__pypy__.thread', '__pypy__.time', '_ast', '_bootlocale', '_codecs', '_collections', '_collections_abc', '_continuation', '_csv', '_frozen_importlib', '_frozen_importlib_external', '_imp', '_io', '_locale', '_multibytecodec', '_operator', '_rawffi', '_rawffi.alt', '_signal', '_sre', '_structseq', '_thread', '_warnings', '_weakref', '_weakrefset', 'abc', 'array', 'builtins', 'codecs', 'copyreg', 'encodings', 'encodings.aliases', 'encodings.ascii', 'encodings.latin_1', 'encodings.utf_8', 'errno', 'gc', 'genericpath', 'gpython', 'marshal', 'os', 'os.path', 'posix', 'posixpath', 'pwd', 're', 'site', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'sys', 'time', 'unicodedata']https://lab.nexedi.com/nexedi/pygolang/-/commit/da68a8ae2d70d163e28a57ace140daba948f6356tox += PyPy32019-03-19T21:14:23+03:00Kirill Smelkovkirr@nexedi.com
Debian started to ship PyPy3:
<a href="https://packages.debian.org/search?keywords=pypy3" rel="nofollow noreferrer noopener" target="_blank">https://packages.debian.org/search?keywords=pypy3</a>https://lab.nexedi.com/nexedi/pygolang/-/commit/942ee9009d2e0a5d5cc174e72243591eee86ba55golang: Deprecate @method(cls) in favour of @func(cls)2019-03-15T16:20:27+03:00Kirill Smelkovkirr@nexedi.com
Since we already have func (see <a href="/nexedi/pygolang/-/commit/5146eb0bd4945465435e6343bf81685133287ad3" data-original="5146eb0b" data-link="false" data-link-reference="false" data-project="1156" data-commit="5146eb0bd4945465435e6343bf81685133287ad3" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="Add support for defer & recover" class="gfm gfm-commit has-tooltip">5146eb0b</a> "Add support for defer &
recover") we can use @func for both plain functions and for methods.
For example instead of
@func
def my_function(...):
...
@method(MyClass) <-- NOTE
def my_method(self, ...):
...
have it as
@func
def my_function(...):
...
@func(MyClass) <-- NOTE
def my_method(self, ...):
...
which looks more similar to Go and exposes less golang keywords to a user.https://lab.nexedi.com/nexedi/pygolang/-/commit/6b4990f68920df24422b98a02efd877b19c29dc2gpython: Fix pymain to properly setup sys.path[0]2019-03-13T19:40:17+03:00Kirill Smelkovkirr@nexedi.com
Pymain was not adding e.g. directory of executed file to sys.path, and
as the result if there were e.g. 2 files
dir/hello.py # imports world
dir/world.py
running `gpython dir/hello.py` would fail to import world.
The case for interactive console was also failing to setup sys.argv as
empty, so it was containing ['/path/to/gpython']. It was also, contrary
to standard python, unconditionally showing '>>>' prompt even when stdin
was not a tty.
Fix all that and add a test to cover pymain functionality.https://lab.nexedi.com/nexedi/pygolang/-/commit/411bdd7d999f038e5107c0ac6305bb8e0bc98fbapygolang v0.0.0.dev72019-01-16T16:36:00+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/32a21d5b36579ac2488edfc675567eeb364a3e13gpython: Python interpreter with support for lightweight threads2019-01-16T16:27:21+03:00Kirill Smelkovkirr@nexedi.com
Provide gpython interpreter, that sets UTF-8 as default encoding,
integrates gevent and puts go, chan, select etc into builtin namespace.
Please see details in the documentation added by this patch.
pymain was taken from <a href="https://lab.nexedi.com/kirr/go123/commit/7a476082" data-original="https://lab.nexedi.com/kirr/go123/commit/7a476082" data-link="false" data-link-reference="true" data-project="480" data-commit="7a47608267cbb79125e93ee23b7a9601cf297c6a" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="tracing/pyruntraced: New tool to run Python code with tracepoints activated (draft)" class="gfm gfm-commit has-tooltip">go123@7a476082</a>
(go123: tracing/pyruntraced: New tool to run Python code with tracepoints activated (draft))https://lab.nexedi.com/nexedi/pygolang/-/commit/fda78bceacc3b9605665a53e73e82ae0eb464a53setup: Install golang/testdata/... as well2019-01-16T16:22:39+03:00Kirill Smelkovkirr@nexedi.com
We describe golang/testdata/ in MANIFEST.in, and that makes testdata to
be included into sdist. However testdata does not get installed.
Since, we want to be able to run pygolang tests, even on installed
package, we need the testdata files to be present there as well.
Fix it.
See <a href="https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files" rel="nofollow noreferrer noopener" target="_blank">https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files</a>
for setuptools-related details.https://lab.nexedi.com/nexedi/pygolang/-/commit/1685fede532d9a834a5567cb77eeb74e97705178golang: Provide __version__2019-01-16T16:22:39+03:00Kirill Smelkovkirr@nexedi.com
Move version definition from setup.py to golang module so that it is
available at runtime with plain golang.__version__ .https://lab.nexedi.com/nexedi/pygolang/-/commit/2506fa5f072e76e8badec4c779988d461e44c126golang: Fix typo in __all__2019-01-16T16:22:39+03:00Kirill Smelkovkirr@nexedi.com
A comma was missing - this way last element was 'funcgimport', not
'func' and 'gimport', and so `from golang import *` was failing.https://lab.nexedi.com/nexedi/pygolang/-/commit/f94a8f868710971b639dd5411d486895c334d931*_test: pytest.raises(string) is deprecated2019-01-16T16:22:34+03:00Kirill Smelkovkirr@nexedi.com
.../pygolang/golang/strconv_test.py:75: PytestDeprecationWarning: raises(..., 'code(as_a_string)') is deprecated, use the context manager form or use `exec()` directly
...
See <a href="https://docs.pytest.org/en/latest/deprecations.html#raises-warns-with-a-string-as-the-second-argument" rel="nofollow noreferrer noopener" target="_blank">https://docs.pytest.org/en/latest/deprecations.html#raises-warns-with-a-string-as-the-second-argument</a>https://lab.nexedi.com/nexedi/pygolang/-/commit/50213c5432c6b6dbb6ac2c3a4cdcfc78bff44ba6dist: Don't forget to include COPYING into release tarball2018-12-30T13:33:41+03:00Kirill Smelkovkirr@nexedi.com
Until now it was not included by `python setup.py sdist`.
On this topic, also include README.rst explicitly, since even if it is
currently implicitly included, that might change some day and break us.https://lab.nexedi.com/nexedi/pygolang/-/commit/1ebbbfeece98cfad0bbc138dc42b15d2ba5c96e5pygolang v0.0.0.dev62018-12-13T15:42:15+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/ed6b789587973b45813b48889291e96d0bb2a421strconv += unquote(), unquote_next()2018-12-13T15:36:32+03:00Kirill Smelkovkirr@nexedi.com
This are functions to decode quotation that was produced by
strconv.quote().https://lab.nexedi.com/nexedi/pygolang/-/commit/f09701b0d01209d46b08f979f945a9051e7c9e80strconv: New package + expose .quote() there2018-12-13T15:36:32+03:00Kirill Smelkovkirr@nexedi.com
Move quote implementation from gcompat to strconv. Make quote work on both
unicode|bytes string input and produce the same output type. Preserve qq
(still remaining in gcompat) to always produce str, since that is used
in prints.https://lab.nexedi.com/nexedi/pygolang/-/commit/94bae8e87bddfb9681e845b4a0ffa5dde3a6a817readme: Overhaul2018-12-13T15:36:24+03:00Kirill Smelkovkirr@nexedi.com
Stop hiding qq and benchmarking bits in changelog only - add their
description into additional sections in readme.
Since py.bench description is now in readme, there is no need to keep
py.bench example in the changelog.https://lab.nexedi.com/nexedi/pygolang/-/commit/c859940b54a7086c2b83b47e1c6aa094f95b462atox: Test under PyPy too2018-10-31T21:29:33+03:00Kirill Smelkovkirr@nexedi.com
All tests pass.https://lab.nexedi.com/nexedi/pygolang/-/commit/8424abe01275a92740d927b15685312b0e612b89pygolang v0.0.0.dev52018-10-30T19:05:59+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/5146eb0bd4945465435e6343bf81685133287ad3Add support for defer & recover2018-10-30T18:57:49+03:00Kirill Smelkovkirr@nexedi.com
`defer` allows to schedule a cleanup to be executed when current function
returns. It is similar to `try`/`finally` but does not force the cleanup part
to be far away in the end. For example::
wc = wcfs.join(zurl) │ wc = wcfs.join(zurl)
defer(wc.close) │ try:
│ ...
... │ ...
... │ ...
... │ finally:
│ wc.close()
For completeness there is `recover` and `panic` that allow to program with
Go-style error handling, for example::
def _():
r = recover()
if r is not None:
print("recovered. error was: %s" % (r,))
defer(_)
...
panic("aaa")
But `recover` and `panic` are probably of less utility since they can be
practically natively modelled with `try`/`except`.
If `defer` is used, the function that uses it must be wrapped with `@func` or
`@method` decorators.
The implementation is partly inspired by work of Denis Kolodin:
- <a href="https://habr.com/post/191786" rel="nofollow noreferrer noopener" target="_blank">https://habr.com/post/191786</a>
- <a href="https://stackoverflow.com/a/43028386/9456786" rel="nofollow noreferrer noopener" target="_blank">https://stackoverflow.com/a/43028386/9456786</a>https://lab.nexedi.com/nexedi/pygolang/-/commit/f0b592b460ff3e2a1e03776de142713181d8b9a7select: Don't let both a queued and a tried cases win at the same time2018-10-30T17:59:37+03:00Kirill Smelkovkirr@nexedi.com
While the second phase of select is running we queue send/recv cases to
corresponding channels. At some point - when some of the cases are
already queued - a peer goroutine might try to send/recv on that
channel. And it will succeed because a waiter was queued to the channel.
At the same time select is continuing its enqueue loop and before enqueuing
to a channel it tries to send/recv there. If that channel became just ready
(i.e. just after select poll phase) the try to send/recv will succeed. This
means that actually 2 select cases could be executed at the same time.
Fix it by carefully checking whether some case already won before trying
to send/recv on a channel.
This fixes the test failures that were demonstrated by previous 2 patches.https://lab.nexedi.com/nexedi/pygolang/-/commit/2fc6797c619ac888cb0fcf286fe541a7a091c5bcselect: Another select-select test which reuses channels during iterations2018-10-30T17:50:16+03:00Kirill Smelkovkirr@nexedi.com
Previous select-select test uses 2 channels and recreats them every
iteration. However the bug we just saw in <a href="/nexedi/pygolang/-/commit/b51b8d5d64f020f2b0892eb0f9d2c3a4968c39e3" data-original="b51b8d5d" data-link="false" data-link-reference="false" data-project="1156" data-commit="b51b8d5d64f020f2b0892eb0f9d2c3a4968c39e3" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="select: Run tests more thoroughly" class="gfm gfm-commit has-tooltip">b51b8d5d</a> (select: Run tests
more thoroughly) is of parasitic nature - where misbehaviour depends on
what state has been left there from previous select and whether the race
to get to that state fast enough succeeded.
So add a more secialized test which tries to trigger the parasitic
effects that depend on previous state left by select on the channels.
The test, similarly to <a href="/nexedi/pygolang/-/commit/b51b8d5d64f020f2b0892eb0f9d2c3a4968c39e3" data-original="b51b8d5d" data-link="false" data-link-reference="false" data-project="1156" data-commit="b51b8d5d64f020f2b0892eb0f9d2c3a4968c39e3" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="select: Run tests more thoroughly" class="gfm gfm-commit has-tooltip">b51b8d5d</a>, currently fails.https://lab.nexedi.com/nexedi/pygolang/-/commit/b51b8d5d64f020f2b0892eb0f9d2c3a4968c39e3select: Run tests more thoroughly2018-10-30T17:43:08+03:00Kirill Smelkovkirr@nexedi.com
With more iterations it triggers a bug:
kirr@deco:~/src/tools/go/pygolang$ py.test -vsx
============================================== test session starts ===============================================
platform linux2 -- Python 2.7.15+, pytest-3.6.4, py-1.6.0, pluggy-0.6.0 -- /usr/bin/python2
cachedir: .pytest_cache
rootdir: /home/kirr/src/tools/go/pygolang, inifile:
collected 7 items
golang/_gopath_test.py::test_import_module PASSED
golang/_gopath_test.py::test_import_package PASSED
golang/gcompat_test.py::test_qq PASSED
golang/golang_test.py::test_go PASSED
golang/golang_test.py::test_chan PASSED
golang/golang_test.py::test_select Exception in thread Thread-117:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/kirr/src/tools/go/pygolang/golang/golang_test.py", line 320, in _
assert (_, _rx) == (1, 'b')
AssertionError: assert (1, 'xxx1') == (1, 'b')
At index 1 diff: 'xxx1' != 'b'
Full diff:
- (1, 'xxx1')
+ (1, 'b')https://lab.nexedi.com/nexedi/pygolang/-/commit/ec929991e525546b718216ec51fcfb0f1f043a42go: Fix test_go when running in-tree without pygolang being installed2018-10-30T13:59:05+03:00Kirill Smelkovkirr@nexedi.com
This test was failing when pygolang was not dev-mode installed (`pip install -e .`):
kirr@deco:~/src/tools/go/pygolang$ py.test
=========================================== test session starts ============================================
platform linux2 -- Python 2.7.15+, pytest-3.6.4, py-1.6.0, pluggy-0.6.0
rootdir: /home/kirr/src/tools/go/pygolang, inifile:
collected 8 items
golang/_gopath_test.py .. [ 25%]
golang/gcompat_test.py . [ 37%]
golang/golang_test.py F.... [100%]
================================================= FAILURES =================================================
_________________________________________________ test_go __________________________________________________
def test_go():
# leaked goroutine behaviour check: done in separate process because we need
# to test process termination exit there.
subprocess.check_call([sys.executable,
> dirname(__file__) + "/golang_test_goleaked.py"])
golang/golang_test.py:38:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python2', '/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py'],)
kwargs = {}, retcode = 1
cmd = ['/usr/bin/python2', '/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py']
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the Popen constructor. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E CalledProcessError: Command '['/usr/bin/python2', '/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py']' returned non-zero exit status 1
/usr/lib/python2.7/subprocess.py:190: CalledProcessError
------------------------------------------- Captured stderr call -------------------------------------------
Traceback (most recent call last):
File "/home/kirr/src/tools/go/pygolang/golang/golang_test_goleaked.py", line 23, in <module>
from golang import go, chan
ImportError: No module named golang
==================================== 1 failed, 7 passed in 0.10 seconds ====================================
Fix it by injecting top-level pygolang/ into $PYTHONPATH when testing
via external processes.
Fixes <a href="/kirr/pygolang/-/commit/69cef96e15124663025546b2938c6555a229d92d" data-original="69cef96e" data-link="false" data-link-reference="false" data-project="1158" data-commit="69cef96e15124663025546b2938c6555a229d92d" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="go: Don't allow leaked goroutines to prevent program to exit" class="gfm gfm-commit has-tooltip">69cef96e</a> (go: Don't allow leaked goroutines to prevent program to exit)https://lab.nexedi.com/nexedi/pygolang/-/commit/ab69e0fa1d2b266738f7227dafd757787aca9870@method: Fix it for Python32018-10-25T15:01:44+03:00Kirill Smelkovkirr@nexedi.com
There was no test for @method so far and that's why it went unnoticed.
But on Python3 it breaks on f.func_name:
In [3]: def f(): pass
In [4]: f.func_name
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-4-662dcbac1531> in <module>()
----> 1 f.func_name
AttributeError: 'function' object has no attribute 'func_name'
Fix it by using f.__name__ which works on both py2 and py3.
Add test for @method to make sure it doesn't break unnoticed.https://lab.nexedi.com/nexedi/pygolang/-/commit/69cef96e15124663025546b2938c6555a229d92dgo: Don't allow leaked goroutines to prevent program to exit2018-10-24T13:46:59+03:00Kirill Smelkovkirr@nexedi.com
This is the Go behaviour, as demonstratd by the following program:
---- 8< ----
package main
import (
"fmt"
"time"
)
func work(w int) {
for i := 0; ; i++ {
fmt.Printf("w%d: %d\n", w, i)
time.Sleep(1*time.Second)
}
}
func main() {
for i := 0; i < 100; i++ {
go work(i)
}
time.Sleep(3*time.Second)
println("main: exit")
}
---- 8< ----https://lab.nexedi.com/nexedi/pygolang/-/commit/f2905909f67e47d6079665fb9cba7d3bf80c49d1pygolang v0.0.0.dev42018-07-04T16:19:54+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/9bf03d9c99312c94672b22f13bb3813f7902cbb3py.bench: New command to benchmark python code similarly to `go test -bench`2018-07-04T16:07:00+03:00Kirill Smelkovkirr@nexedi.com
+ corresponding bits in golang.testing package.
py.bench description follows:
"""
py.bench, similarly to py.test, discovers bench_* functions and Bench* classes
and then runs each discovered benchmark via golang.testing.benchmark. Similarly
to `go test -bench`, benchmarking results are printed in Go benchmark format.
For example, running py.bench on the following code::
def bench_add(b):
x, y = 1, 2
for i in xrange(b.N):
x + y
gives something like::
$ py.bench --count=3 x.py
...
pymod: bench_add.py
Benchmarkadd 50000000 0.020 µs/op
Benchmarkadd 50000000 0.020 µs/op
Benchmarkadd 50000000 0.020 µs/op
"""
The implementation is based on t/py.bench from Wendelin.core - see
following commits for its history:
lab.nexedi.com/nexedi/wendelin.core/commit/51f252d4
lab.nexedi.com/nexedi/wendelin.core/commit/074ce24d
lab.nexedi.com/nexedi/wendelin.core/commit/ed13c3f9
lab.nexedi.com/nexedi/wendelin.core/commit/fc08766d
lab.nexedi.com/nexedi/wendelin.core/commit/5a1ed45a
lab.nexedi.com/nexedi/wendelin.core/commit/bcab1246https://lab.nexedi.com/nexedi/pygolang/-/commit/75443f647aabe465dae9941078f9381c9d408a96pygolang v0.0.0.dev32018-07-02T19:47:29+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/622ccd829dcc7d9469a69d3f2b455ef14bb98522Test via tox under both Python2 and Python32018-07-02T19:44:41+03:00Kirill Smelkovkirr@nexedi.comhttps://lab.nexedi.com/nexedi/pygolang/-/commit/e01e5c2f085d7e232a8b4f8d92e78c64173b89e3channels: Make them work with Python32018-07-02T18:57:03+03:00Kirill Smelkovkirr@nexedi.com
Python3 complains that there is no .im_func & friends of a method.
Workaround it all with compat code.