Start using Cython and providing Cython/nogil API

Start transforming Pygolang from being pure Python project into project
that uses both Python and Cython and provides both Python and Cython API
to its users.

For Cython API there is a goal for it to be usable independently of
Python GIL - i.e. independently of Python runtime and limitations
imposed by Python's global interpreter lock. The motivation for this
work is wendelin.core v2, which, due to its design, would deadlock if it
tries to take the GIL in its pinner thread.

This commit brings bare minimum establishing libraries and build system.
It:

- Introduces C/C++ library libgolang. Libgolang will be serving most of
  nogil functionality and Pyx API will be a small wrapper over it. At
  this step Libgolang comes only with C-level panic;

- Introduces Pyx package golang that is aliased to Pyx package
  golang._golang . Users can do `from golang cimport ...` similarly to how
  they do py-level `from golang import ...`. At this step golang.pyx
  only wraps panic and provides way to transform C-level panic into Python
  exception;

- Introduces Py package golang.pyx.build . This package, similarly to
  golang.org/pkg/go/build, serves as the system to build Pyx packages -
  both external packages that use Pygolang in pyx mode, and to build
  Pygolang itself. The build, internally, is served by setuptools_dso
  and cython. An example how to use golang.pyx.build from external
  project comes in golang/pyx/testprog/golang_pyx_user/ and is used by
  tests to verify golang.pyx.build functionality;

- Introduces Pygolang build dependency to cython and setuptools_dso.

.gitignore

@@ -6,6 +6,10 @@ build/
 
 *.o
 *.so
+*.so.*
+*.dylib
+*.dll
+*.pyd
 
 /dist/
 *.egg-info

MANIFEST.in

-include COPYING README.rst CHANGELOG.rst tox.ini
-recursive-include golang    *.py
+include COPYING README.rst CHANGELOG.rst tox.ini pyproject.toml
+include golang/libgolang.h
+include golang/runtime/libgolang.cpp
+recursive-include golang    *.py *.pxd *.pyx
 recursive-include gpython   *.py

README.rst

-========================================
- Pygolang - Go-like features for Python
-========================================
+===================================================
+ Pygolang - Go-like features for Python and Cython
+===================================================
 
 Package `golang` provides Go-like features for Python:
 
@@ -11,8 +11,12 @@ Package `golang` provides Go-like features for Python:
 - `defer` allows to schedule a cleanup from the main control flow.
 - `gimport` allows to import python modules by full path in a Go workspace.
 
+Package `golang.pyx` provides__ similar features for Cython/nogil.
+
+__ `Cython/nogil API`_
+
 Additional packages and utilities are also provided__ to close other gaps
-between Python and Go environments.
+between Python/Cython and Go environments.
 
 __ `Additional packages and utilities`_
 
@@ -162,13 +166,54 @@ will import either
 
 located in `src/` under `$GOPATH`.
 
+
+Cython/nogil API
+----------------
+
+Cython package `golang` provides *nogil* API with
+features that mirror corresponding Python package. Cython API is not only
+faster compared to Python version, but also, due to *nogil* property, allows to
+build concurrent systems without limitations imposed by Python's GIL. All that
+while still programming in Python-like language. Brief description of
+Cython/nogil API follows:
+
+`panic` stops normal execution of current goroutine by throwing a C-level
+exception. On Python/C boundaries C-level exceptions have to be converted to
+Python-level exceptions with `topyexc`. For example::
+
+   cdef void _do_something() nogil:
+      ...
+      panic("bug")   # hit a bug
+
+   # do_something is called by Python code - it is thus on Python/C boundary
+   cdef void do_something() nogil except +topyexc:
+      _do_something()
+
+   def pydo_something():
+      with nogil:
+         do_something()
+
+
+See |libgolang.h|_ and |golang.pxd|_ for details of the API.
+See also |testprog/golang_pyx_user/|_ for demo project that uses Pygolang in
+Cython/nogil mode.
+
+.. |libgolang.h| replace:: `libgolang.h`
+.. _libgolang.h: https://lab.nexedi.com/kirr/pygolang/tree/master/golang/libgolang.h
+
+.. |golang.pxd| replace:: `golang.pxd`
+.. _golang.pxd: https://lab.nexedi.com/kirr/pygolang/tree/master/golang/_golang.pxd
+
+.. |testprog/golang_pyx_user/| replace:: `testprog/golang_pyx_user/`
+.. _testprog/golang_pyx_user/: https://lab.nexedi.com/kirr/pygolang/tree/master/golang/pyx/testprog/golang_pyx_user
+
 --------
 
 Additional packages and utilities
 ---------------------------------
 
 The following additional packages and utilities are also provided to close gaps
-between Python and Go environments:
+between Python/Cython and Go environments:
 
 .. contents::
    :local:

golang/.gitignore

+/_golang.cpp

golang/__init__.pxd

+# cython: language_level=2
+# Copyright (C) 2019  Nexedi SA and Contributors.
+#                     Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+"""Package golang.pyx provides Go-like features for Cython/nogil and runtime for golang.py.
+
+See _golang.pxd for package documentation.
+"""
+
+# redirect cimport: golang -> golang._golang
+#
+# we do this because we cannot put pyx code into __init__.pyx - else Python and
+# other tools (e.g. setuptools) fail to recognize golang/ as Python package.
+from golang._golang cimport *

golang/__init__.py

@@ -26,6 +26,7 @@
 - `gimport` allows to import python modules by full path in a Go workspace.
 
 See README for thorough overview.
+See also package golang.pyx which provides similar functionality for Cython nogil.
 """
 
 from __future__ import print_function, absolute_import
@@ -42,12 +43,9 @@ import six
 from golang._pycompat import im_class
 
 
-# panic stops normal execution of current goroutine.
-def panic(arg):
-    raise _PanicError(arg)
-
-class _PanicError(Exception):
-    pass
+from ._golang import    \
+    _PanicError,            \
+    pypanic     as panic
 
 
 # @func is a necessary decorator for functions for selected golang features to work.

golang/_golang.pxd

+# cython: language_level=2
+# Copyright (C) 2019  Nexedi SA and Contributors.
+#                     Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+"""Package golang.pyx provides Go-like features for Cython/nogil and runtime for golang.py.
+
+Cython/nogil API
+----------------
+
+- `panic` stops normal execution of current goroutine by throwing a C-level exception.
+
+Everything in Cython/nogil API do not depend on Python runtime and in
+particular can be used in nogil code.
+
+See README for thorough overview.
+See libgolang.h for API details.
+See also package golang.py which provides similar functionality for Python.
+
+
+Golang.py runtime
+-----------------
+
+In addition to Cython/nogil API, golang.pyx provides runtime for golang.py:
+
+- Python-level panic is represented by pypanic.
+"""
+
+
+# nogil pyx-level golang API.
+#
+# NOTE even though many functions may panic (= throw C++ exception) nothing is
+# annotated with `except +`. Reason: `f() except +` tells Cython to wrap every
+# call to f with try/catch and convert C++ exception into Python one. And once
+# you have a Python-level exception you are in Python world. However we want
+# nogil golang.pyx API to be usable without Python at all.
+#
+# -> golang.pyx users need to add `except +topyexc` to their functions that are
+# on the edge of Python/nogil world.
+cdef extern from "golang/libgolang.h" namespace "golang" nogil:
+    void panic(const char *)
+    const char *recover()
+
+
+# ---- python bits ----
+
+cdef void topyexc() except *
+cpdef pypanic(arg)

golang/_golang.pyx

+# -*- coding: utf-8 -*-
+# cython: language_level=2
+# distutils: language = c++
+# distutils: depends = libgolang.h
+#
+# Copyright (C) 2018-2019  Nexedi SA and Contributors.
+#                          Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+"""_golang.pyx provides Python interface to libgolang.{h,cpp}.
+
+See _golang.pxd for package overview.
+"""
+
+from __future__ import print_function, absolute_import
+
+from cpython cimport PY_MAJOR_VERSION
+from cython cimport final
+
+# ---- panic ----
+
+@final
+cdef class _PanicError(Exception):
+    pass
+
+# panic stops normal execution of current goroutine.
+cpdef pypanic(arg):
+    raise _PanicError(arg)
+
+# topyexc converts C-level panic/exc to python panic/exc.
+# (see usage in e.g. *_pyexc functions in "misc")
+cdef void topyexc() except *:
+    # TODO use libunwind/libbacktrace/libstacktrace/... to append C-level traceback
+    #      from where it panicked till topyexc user.
+    # TODO install C-level traceback dump as std::terminate handler.
+    #
+    # recover_ is declared `except +` - if it was another - not panic -
+    # exception, it will be converted to py exc by cython automatically.
+    arg = recover_()
+    if arg != NULL:
+        pyarg = <bytes>arg
+        if PY_MAJOR_VERSION >= 3:
+            pyarg = pyarg.decode("utf-8")
+        pypanic(pyarg)
+
+cdef extern from "golang/libgolang.h" nogil:
+    const char *recover_ "golang::recover" () except +

golang/libgolang.h

+#ifndef _NXD_LIBGOLANG_H
+#define _NXD_LIBGOLANG_H
+
+// Copyright (C) 2018-2019  Nexedi SA and Contributors.
+//                          Kirill Smelkov <kirr@nexedi.com>
+//
+// This program is free software: you can Use, Study, Modify and Redistribute
+// it under the terms of the GNU General Public License version 3, or (at your
+// option) any later version, as published by the Free Software Foundation.
+//
+// You can also Link and Combine this program with other software covered by
+// the terms of any of the Free Software licenses or any of the Open Source
+// Initiative approved licenses and Convey the resulting work. Corresponding
+// source of such a combination shall include the source code for all other
+// software used.
+//
+// This program is distributed WITHOUT ANY WARRANTY; without even the implied
+// warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+//
+// See COPYING file for full licensing terms.
+// See https://www.nexedi.com/licensing for rationale and options.
+
+// Library Libgolang provides Go-like features for C and C++.
+//
+// Library Libgolang provides Go-like
+// features. The library consists of high-level type-safe C++ API,
+// and low-level unsafe C API.
+//
+// The primary motivation for Libgolang is to serve as runtime for golang.pyx -
+// - Cython part of Pygolang project. However Libgolang is independent of
+// Python and should be possible to use in standalone C/C++ projects.
+//
+// Brief description of Libgolang API follows:
+//
+// C++-level API
+//
+//  - `panic` throws exception that represent C-level panic.
+//
+// For example:
+//
+//      if (<bug condition>)
+//          panic("bug");
+//
+//
+// C-level API
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+
+// DSO symbols visibility (based on https://gcc.gnu.org/wiki/Visibility)
+#if defined _WIN32 || defined __CYGWIN__
+  #ifdef BUILDING_LIBGOLANG
+    #define LIBGOLANG_API __declspec(dllexport)
+  #else
+    #define LIBGOLANG_API __declspec(dllimport)
+  #endif
+#elif __GNUC__ >= 4
+    #define LIBGOLANG_API __attribute__ ((visibility ("default")))
+#else
+    #define LIBGOLANG_API
+#endif
+
+
+// ---- C-level API that is always available ----
+// (most of the functions are documented in libgolang.cpp)
+
+#ifdef  __cplusplus
+namespace golang {
+extern "C" {
+#endif
+
+#ifdef __cplusplus
+    [[noreturn]]
+#else
+    _Noreturn
+#endif
+LIBGOLANG_API void panic(const char *arg);
+LIBGOLANG_API const char *recover(void);
+
+#ifdef __cplusplus
+}}
+#endif
+
+#endif  // _NXD_LIBGOLANG_H

golang/pyx/build.py

+# Copyright (C) 2019  Nexedi SA and Contributors.
+#                     Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+"""Package build provides infrastructure to build Cython Pygolang-based packages.
+
+Use `setup` and `Extension` to build packages. For example::
+
+    from golang.pyx.build import setup, Extension
+
+    setup(
+        name        = 'mypkg',
+        description = 'my Pygolang/Cython-based package',
+        ...
+        ext_modules = [Extension('mypkg.mymod', ['mypkg/mymod.pyx'])],
+    )
+"""
+
+from __future__ import print_function, absolute_import
+
+# pygolang uses setuptools_dso.DSO to build libgolang; all extensions link to it.
+import setuptools_dso
+
+import sys, pkgutil, platform, sysconfig
+from os.path import dirname, join, exists
+from distutils.errors import DistutilsError
+
+# Error represents a build error.
+class Error(DistutilsError):
+    pass
+
+# _PyPkg provides information about 1 py package.
+class _PyPkg:
+    # .name - full package name, e.g. "golang.time"
+    # .path - filesystem path of the package
+    #         (file for module, directory for pkg/__init__.py)
+    pass
+
+# _findpkg finds specified python package and returns information about it.
+#
+# e.g. _findpkg("golang") -> _PyPkg("/path/to/pygolang/golang")
+def _findpkg(pkgname):  # -> _PyPkg
+    pkg = pkgutil.get_loader(pkgname)
+    if pkg is None: # package not found
+        raise Error("package %r not found" % (pkgname,))
+    path = pkg.get_filename()
+    if path.endswith("__init__.py"):
+        path = dirname(path) # .../pygolang/golang/__init__.py -> .../pygolang/golang
+    pypkg = _PyPkg()
+    pypkg.name = pkgname
+    pypkg.path = path
+    return pypkg
+
+# setup should be used instead of setuptools.setup
+def setup(**kw):
+    return setuptools_dso.setup(**kw)
+
+# Extension should be used to build extensions that use pygolang.
+#
+# For example:
+#
+#   setup(
+#       ...
+#       ext_modules = [Extension('mypkg.mymod', ['mypkg/mymod.pyx'])],
+#   )
+def Extension(name, sources, **kw):
+    # find pygolang root
+    gopkg = _findpkg("golang")
+    pygo  = dirname(gopkg.path) # .../pygolang/golang -> .../pygolang
+    if pygo == '':
+        pygo = '.'
+
+    kw = kw.copy()
+
+    # prepend -I<pygolang> so that e.g. golang/libgolang.h is found
+    incv = kw.get('include_dirs', [])[:]
+    incv.insert(0, pygo)
+    kw['include_dirs'] = incv
+
+    # link with libgolang.so
+    dsov = kw.get('dsos', [])[:]
+    dsov.insert(0, 'golang.runtime.libgolang')
+    kw['dsos'] = dsov
+
+    # default language to C++ (chan[T] & co are accessible only via C++)
+    lang = kw.setdefault('language', 'c++')
+
+    # default to C++11 (chan[T] & co require C++11 features)
+    if lang == 'c++':
+        ccextra = kw.get('extra_compile_args', [])[:]
+        ccextra.insert(0, '-std=c++11') # if another -std=... was already there - it will override us
+        kw['extra_compile_args'] = ccextra
+
+    # some depends to workaround a bit lack of proper dependency tracking in
+    # setuptools/distutils.
+    dependv = kw.get('depends', [])[:]
+    dependv.append('%s/golang/libgolang.h'  % pygo)
+    dependv.append('%s/golang/_golang.pxd'  % pygo)
+    dependv.append('%s/golang/__init__.pxd' % pygo)
+    kw['depends'] = dependv
+
+    # workaround pip bug that for virtualenv case headers are installed into
+    # not-searched include path. https://github.com/pypa/pip/issues/4610
+    # (without this e.g. "greenlet/greenlet.h" is not found)
+    venv_inc = join(sys.prefix, 'include', 'site', 'python' + sysconfig.get_python_version())
+    if exists(venv_inc):
+        kw['include_dirs'].append(venv_inc)
+
+    ext = setuptools_dso.Extension(name, sources, **kw)
+    return ext

golang/pyx/build_test.py

+# -*- coding: utf-8 -*-
+# Copyright (C) 2019  Nexedi SA and Contributors.
+#                     Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+
+from __future__ import print_function, absolute_import
+
+from golang.golang_test import pyrun, pyout
+from os.path import dirname
+
+# verify that we can build/run external package that uses pygolang in pyx mode.
+def test_pyx_build():
+    pyxuser = dirname(__file__) + "/testprog/golang_pyx_user"
+    pyrun(["setup.py", "build_ext", "-i"], cwd=pyxuser)
+
+    # run built test.
+    _ = pyout(["-c",
+        # XXX `import golang` is a hack: it dynamically loads _golang.so -> libgolang.so,
+        # and this way dynamic linker already has libgolang.so DSO found and in
+        # memory for all further imports. If we don't, current state of setuptools_dso
+        # is that `import pyxuser.test` will fail finding libgolang.so.
+        "import golang;" +
+        "from pyxuser import test; test.main()"], cwd=pyxuser)
+    assert _ == b"test.pyx: OK\n"

golang/pyx/testprog/golang_pyx_user/pyproject.toml

+[build-system]
+requires = ["pygolang[pyx.build]"]

golang/pyx/testprog/golang_pyx_user/pyxuser/.gitignore

+/test.cpp

golang/pyx/testprog/golang_pyx_user/pyxuser/test.pyx

+# cython: language_level=2
+# distutils: language=c++
+#
+# Copyright (C) 2019  Nexedi SA and Contributors.
+#                     Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+
+# Small program that uses a bit of golang.pyx nogil features, mainly to verify
+# that external project can build against golang in pyx mode.
+
+from golang cimport topyexc
+from libc.stdio cimport printf
+
+cdef nogil:
+
+    void _main() except +topyexc:
+        printf("test.pyx: OK\n")
+
+def main():
+    _main()

golang/pyx/testprog/golang_pyx_user/setup.py

+# Copyright (C) 2019  Nexedi SA and Contributors.
+#                     Kirill Smelkov <kirr@nexedi.com>
+#
+# This program is free software: you can Use, Study, Modify and Redistribute
+# it under the terms of the GNU General Public License version 3, or (at your
+# option) any later version, as published by the Free Software Foundation.
+#
+# You can also Link and Combine this program with other software covered by
+# the terms of any of the Free Software licenses or any of the Open Source
+# Initiative approved licenses and Convey the resulting work. Corresponding
+# source of such a combination shall include the source code for all other
+# software used.
+#
+# This program is distributed WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See COPYING file for full licensing terms.
+# See https://www.nexedi.com/licensing for rationale and options.
+"""Demo package that links to and uses golang in pyx mode."""
+
+from golang.pyx.build import setup, Extension
+
+setup(
+    name        = 'golang_pyx_user',
+    description = 'test project that uses pygolang in pyx mode',
+
+    ext_modules = [Extension('pyxuser.test', ['pyxuser/test.pyx'])],
+)

golang/runtime/libgolang.cpp

+// Copyright (C) 2018-2019  Nexedi SA and Contributors.
+//                          Kirill Smelkov <kirr@nexedi.com>
+//
+// This program is free software: you can Use, Study, Modify and Redistribute
+// it under the terms of the GNU General Public License version 3, or (at your
+// option) any later version, as published by the Free Software Foundation.
+//
+// You can also Link and Combine this program with other software covered by
+// the terms of any of the Free Software licenses or any of the Open Source
+// Initiative approved licenses and Convey the resulting work. Corresponding
+// source of such a combination shall include the source code for all other
+// software used.
+//
+// This program is distributed WITHOUT ANY WARRANTY; without even the implied
+// warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+//
+// See COPYING file for full licensing terms.
+// See https://www.nexedi.com/licensing for rationale and options.
+
+// Library Libgolang provides Go-like features for C and C++.
+// See libgolang.h for library overview.
+
+// Pygolang C part: provides runtime implementation of panic, etc.
+//
+// C++ (not C) is used:
+// - to implement C-level panic (via C++ exceptions).
+
+#include "golang/libgolang.h"
+
+#include <exception>
+#include <string>
+
+using std::exception;
+using std::string;
+
+namespace golang {
+
+// ---- panic ----
+
+struct PanicError : exception {
+    const char *arg;
+};
+
+// panic throws exception that represents C-level panic.
+// the exception can be caught at C++ level via try/catch and recovered via recover.
+[[noreturn]] void panic(const char *arg) {
+    PanicError _; _.arg = arg;
+    throw _;
+}
+
+// recover recovers from exception thrown by panic.
+// it returns: !NULL - there was panic with that argument. NULL - there was no panic.
+// if another exception was thrown - recover rethrows it.
+const char *recover() {
+    // if PanicError was thrown - recover from it
+    try {
+        throw;
+    } catch (PanicError &exc) {
+        return exc.arg;
+    }
+
+    return NULL;
+}
+
+
+// bug indicates internal bug in golang implementation.
+struct Bug : exception {
+    const string msg;
+
+    virtual const char *what() const throw() {
+        return msg.c_str();
+    }
+
+    Bug(const string &msg) : msg("BUG: " + msg) {}
+};
+
+[[noreturn]] void bug(const char *msg) {
+    throw Bug(msg);
+}
+
+}   // golang::

pyproject.toml

+[build-system]
+requires = ["setuptools", "wheel", "setuptools_dso >= 1.2", "cython"]

setup.py

@@ -17,12 +17,30 @@
 #
 # See COPYING file for full licensing terms.
 # See https://www.nexedi.com/licensing for rationale and options.
-from setuptools import setup, find_packages
+
+from setuptools import find_packages
+# setuptools has Library but this days it is not well supported and test for it
+# has been killed https://github.com/pypa/setuptools/commit/654c26f78a30
+# -> use setuptools_dso instead.
+from setuptools_dso import DSO
 from setuptools.command.install_scripts import install_scripts as _install_scripts
 from setuptools.command.develop import develop as _develop
 from os.path import dirname, join
 import sys, re
 
+# reuse golang.pyx.build to build pygolang extensions.
+# we have to be careful and inject synthetic golang package in order to be
+# able to import golang.pyx.build without built/working golang.
+import imp, pkgutil
+golang = imp.new_module('golang')
+golang.__package__ = 'golang'
+golang.__path__    = ['golang']
+golang.__file__    = 'golang/__init__.py'
+golang.__loader__  = pkgutil.ImpLoader('golang', None, 'golang/__init__.py',
+                                       [None, None, imp.PY_SOURCE])
+sys.modules['golang'] = golang
+from golang.pyx.build import setup, Extension as Ext
+
 # read file content
 def readfile(path):
     with open(path, 'r') as f:
@@ -141,6 +159,7 @@ class develop(XInstallGPython, _develop):
 # requirements of packages under "golang." namespace
 R = {
     'cmd.pybench':      {'pytest'},
+    'pyx.build':        {'setuptools', 'wheel', 'cython', 'setuptools_dso >= 1.2'},
     'x.perf.benchlib':  {'numpy'},
 }
 # TODO generate `a.b -> a`, e.g. x.perf = join(x.perf.*); x = join(x.*)
@@ -158,7 +177,7 @@ for k in sorted(R.keys()):
 setup(
     name        = 'pygolang',
     version     = version,
-    description = 'Go-like features for Python',
+    description = 'Go-like features for Python and Cython',
     long_description = '%s\n----\n\n%s' % (
                             readfile('README.rst'), readfile('CHANGELOG.rst')),
     long_description_content_type  = 'text/x-rst',
@@ -167,9 +186,21 @@ setup(
     author      = 'Kirill Smelkov',
     author_email= 'kirr@nexedi.com',
 
-    keywords    = 'golang go channel goroutine concurrency GOPATH python import gpython gevent',
+    keywords    = 'golang go channel goroutine concurrency GOPATH python import gpython gevent cython nogil GIL',
 
     packages    = find_packages(),
+
+    x_dsos      = [DSO('golang.runtime.libgolang', ['golang/runtime/libgolang.cpp'],
+                        depends         = ['golang/libgolang.h'],
+                        include_dirs    = ['.'],
+                        define_macros   = [('BUILDING_LIBGOLANG', None)],
+                        extra_compile_args = ['-std=c++11'],
+                        soversion       = '0.1')],
+    ext_modules = [
+                    Ext('golang._golang',
+                        ['golang/_golang.pyx']),
+                  ],
+
     include_package_data = True,
 
     install_requires = ['gevent', 'six', 'decorator'],