- 17 Sep, 2019 1 commit
-
-
Kirill Smelkov authored
- ThreadSanitizer helps to detect races and some memory errors, - AddressSanitizer helps to detect memory errors, - Python debug builds help to detect e.g reference counting errors. Adding all those tools to testing coverage discovers e.g. the following bugs (not a full list): ---- 8< ---- py27-thread-tsan: WARNING: ThreadSanitizer: data race (pid=7143) Write of size 8 at 0x7b1400000650 by main thread: #0 free ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:649 (libtsan.so.0+0x2b46a) #1 free ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:643 (libtsan.so.0+0x2b46a) #2 golang::_chan::decref() golang/runtime/libgolang.cpp:470 (liblibgolang.so.0.1+0x47f2) #3 _chanxdecref golang/runtime/libgolang.cpp:452 (liblibgolang.so.0.1+0x484a) #4 _test_go_c golang/runtime/libgolang_test_c.c:86 (_golang_test.so+0x13a2e) #5 __pyx_pf_6golang_12_golang_test_12test_go_c golang/_golang_test.cpp:3340 (_golang_test.so+0xcbaa) #6 __pyx_pw_6golang_12_golang_test_13test_go_c golang/_golang_test.cpp:3305 (_golang_test.so+0xcbaa) #7 PyEval_EvalFrameEx <null> (python2.7+0xf68b4) Previous read of size 8 at 0x7b1400000650 by thread T8: #0 golang::Sema::acquire() golang/runtime/libgolang.cpp:164 (liblibgolang.so.0.1+0x410a) #1 golang::Mutex::lock() golang/runtime/libgolang.cpp:175 (liblibgolang.so.0.1+0x4c82) #2 golang::_chan::close() golang/runtime/libgolang.cpp:754 (liblibgolang.so.0.1+0x4c82) #3 _chanclose golang/runtime/libgolang.cpp:732 (liblibgolang.so.0.1+0x4d1a) #4 _work golang/runtime/libgolang_test_c.c:92 (_golang_test.so+0x136cc) #5 <null> <null> (python2.7+0x1929e3) Thread T8 (tid=7311, finished) created by main thread at: #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:915 (libtsan.so.0+0x2be1b) #1 PyThread_start_new_thread <null> (python2.7+0x19299f) #2 _taskgo golang/runtime/libgolang.cpp:119 (liblibgolang.so.0.1+0x3f68) #3 _test_go_c golang/runtime/libgolang_test_c.c:84 (_golang_test.so+0x13a1c) #4 __pyx_pf_6golang_12_golang_test_12test_go_c golang/_golang_test.cpp:3340 (_golang_test.so+0xcbaa) #5 __pyx_pw_6golang_12_golang_test_13test_go_c golang/_golang_test.cpp:3305 (_golang_test.so+0xcbaa) #6 PyEval_EvalFrameEx <null> (python2.7+0xf68b4) py37-thread-asan: ==22205==ERROR: AddressSanitizer: heap-use-after-free on address 0x607000002cd0 at pc 0x7fd3732a7679 bp 0x7fd3723c8c50 sp 0x7fd3723c8c48 READ of size 8 at 0x607000002cd0 thread T7 #0 0x7fd3732a7678 in golang::Sema::acquire() golang/runtime/libgolang.cpp:164 #1 0x7fd3732a8644 in golang::Mutex::lock() golang/runtime/libgolang.cpp:175 #2 0x7fd3732a8644 in golang::_chan::close() golang/runtime/libgolang.cpp:754 #3 0x7fd3724004b2 in golang::chan<golang::structZ>::close() const golang/libgolang.h:323 #4 0x7fd3724004b2 in operator() golang/runtime/libgolang_test.cpp:262 #5 0x7fd3724004b2 in __invoke_impl<void, _test_chan_vs_stackdeadwhileparked()::<lambda()>&> /usr/include/c++/8/bits/invoke.h:60 #6 0x7fd3724004b2 in __invoke<_test_chan_vs_stackdeadwhileparked()::<lambda()>&> /usr/include/c++/8/bits/invoke.h:95 #7 0x7fd3724004b2 in __call<void> /usr/include/c++/8/functional:400 #8 0x7fd3724004b2 in operator()<> /usr/include/c++/8/functional:484 #9 0x7fd3724004b2 in _M_invoke /usr/include/c++/8/bits/std_function.h:297 #10 0x7fd3723fdc6e in std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687 #11 0x7fd3723fdc6e in operator() golang/libgolang.h:273 #12 0x7fd3723fdc6e in _FUN golang/libgolang.h:271 #13 0x62ddf3 (/home/kirr/src/tools/go/pygolang-master/.tox/py37-thread-asan/bin/python3+0x62ddf3) #14 0x7fd377393fa2 in start_thread /build/glibc-vjB4T1/glibc-2.28/nptl/pthread_create.c:486 #15 0x7fd376eda4ce in clone (/lib/x86_64-linux-gnu/libc.so.6+0xf94ce) 0x607000002cd0 is located 16 bytes inside of 72-byte region [0x607000002cc0,0x607000002d08) freed by thread T0 here: #0 0x7fd377519fb0 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0) #1 0x7fd372401335 in golang::chan<golang::structZ>::~chan() golang/libgolang.h:292 #2 0x7fd372401335 in _test_chan_vs_stackdeadwhileparked() golang/runtime/libgolang_test.cpp:222 previously allocated by thread T0 here: #0 0x7fd37751a518 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9518) #1 0x7fd3732a7d0b in zalloc golang/runtime/libgolang.cpp:1185 #2 0x7fd3732a7d0b in _makechan golang/runtime/libgolang.cpp:413 Thread T7 created by T0 here: #0 0x7fd377481db0 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x50db0) #1 0x62df39 in PyThread_start_new_thread (/home/kirr/src/tools/go/pygolang-master/.tox/py37-thread-asan/bin/python3+0x62df39) SUMMARY: AddressSanitizer: heap-use-after-free golang/runtime/libgolang.cpp:164 in golang::Sema::acquire() Shadow bytes around the buggy address: 0x0c0e7fff8540: fa fa fa fa fd fd fd fd fd fd fd fd fd fa fa fa 0x0c0e7fff8550: fa fa fd fd fd fd fd fd fd fd fd fa fa fa fa fa 0x0c0e7fff8560: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd 0x0c0e7fff8570: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd 0x0c0e7fff8580: fd fd fd fd fd fa fa fa fa fa fd fd fd fd fd fd =>0x0c0e7fff8590: fd fd fd fa fa fa fa fa fd fd[fd]fd fd fd fd fd 0x0c0e7fff85a0: fd fa fa fa fa fa 00 00 00 00 00 00 00 00 00 fa 0x0c0e7fff85b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c0e7fff85c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c0e7fff85d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c0e7fff85e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ---- 8< ---- The bugs will be addressed in the followup patches.
-
- 26 Aug, 2019 1 commit
-
-
Kirill Smelkov authored
... and everything that whole pygolang depends on in pygolang[all]. The reason we do this: currently golang.perf.benchlib needs NumPy, but it would be not ok to depend whole pygolang on numpy. Similarly pybench needs pytest, but it would be not ok to require pytest unconditionally. Thus allow users to specify what they need from pygolang via extras, like pygolang[x.perf.benchlib] or pygolang[cmd.pybench], with pygolang[all] serving as merge point for all subpackages dependencies. TODO in the future we'll, hopefully, also provide accumulation points, like pygolang[x.perf] which contains everything pygolang[x.perf.*] depends on. Note: core functionality and gpython are provided always unconditionally.
-
- 08 Jul, 2019 1 commit
-
-
Kirill Smelkov authored
-> Use .[test] to refer to them. https://stackoverflow.com/a/41398850/9456786
-
- 24 May, 2019 1 commit
-
-
Kirill Smelkov authored
This reverts commit 469f21a9. Even though Debian stopped shipping Python3.6 I'm now building it myself to have test coverage for two latest Python releases.
-
- 19 Mar, 2019 2 commits
-
-
Kirill Smelkov authored
Debian dropped Python3.6 from python3-distutils. Without it I'm no longer being able to run tests on that python version: (neo) (z-dev) (g.env) kirr@deco:~/src/tools/go/pygolang$ tox -e py36-thread execution failed: -- Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'distutils.sysconfig' GLOB sdist-make: /home/kirr/src/tools/go/pygolang/setup.py py36-thread create: /home/kirr/src/tools/go/pygolang/.tox/py36-thread ERROR: invocation failed (exit code 1), logfile: /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log ERROR: actionid: py36-thread msg: getenv cmdargs: '/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/virtualenv.py", line 25, in <module> import distutils.sysconfig ModuleNotFoundError: No module named 'distutils.sysconfig' Running virtualenv with interpreter /usr/bin/python3.6 ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('/usr/bin/python3 -m virtualenv --python /usr/bin/python3.6 py36-thread (see /home/kirr/src/tools/go/pygolang/.tox/py36-thread/log/py36-thread-0.log)', 1) https://bugs.debian.org/918881#19
-
Kirill Smelkov authored
Debian started to ship PyPy3: https://packages.debian.org/search?keywords=pypy3
-
- 16 Jan, 2019 1 commit
-
-
Kirill Smelkov authored
Provide gpython interpreter, that sets UTF-8 as default encoding, integrates gevent and puts go, chan, select etc into builtin namespace. Please see details in the documentation added by this patch. pymain was taken from kirr/go123@7a476082 (go123: tracing/pyruntraced: New tool to run Python code with tracepoints activated (draft))
-
- 31 Oct, 2018 1 commit
-
-
Kirill Smelkov authored
All tests pass.
-
- 02 Jul, 2018 1 commit
-
-
Kirill Smelkov authored
-