• Kirill Smelkov's avatar
    libgolang: Add internal semaphores · 69db91bf
    Kirill Smelkov authored
    - Add semaphore alloc/free/acquire/release functionality to libgolang runtime;
    - Implement semaphores for thread and gevent runtimes.
    
      * Thread runtime uses PyThread_acquire_lock/PyThread_release_lock +
        PyThread_acquire_lock/PyThread_release_lock, which, if used
        carefully, do not depend on GIL and on e.g. POSIX are tiny wrappers
        around sem_init(process-private) + sem_post/sem_wait(*).
    
      * Gevent runtime uses geven't Semaphore in Pyx mode.
    
    - Add Sema and Mutex classes that use semaphores provided by a runtime
      in a RAII style.
    - Add with_lock(mu) that mimics `with mu` in Python.
    
    Sema and Mutex will be used in channels implementation in the followup
    patch.
    
    (*) during late testing a bug was found in CPython2 and PyPy semaphore
    implementations on Darwin (technically speaking on POSIX with
    _POSIX_SEMAPHORES undefined). Quoting the patch:
    
        FIXME On Darwin, even though this is considered as POSIX, Python uses
        mutex+condition variable to implement its lock, and, as of 20190828, Py2.7
        implementation, even though similar issue was fixed for Py3 in 2012, contains
        synchronization bug: the condition is signalled after mutex unlock while the
        correct protocol is to signal condition from under mutex:
    
          https://github.com/python/cpython/blob/v2.7.16-127-g0229b56d8c0/Python/thread_pthread.h#L486-L506
          https://github.com/python/cpython/commit/187aa545165d (py3 fix)
    
        PyPy has the same bug for both pypy2 and pypy3:
    
          https://bitbucket.org/pypy/pypy/src/578667b3fef9/rpython/translator/c/src/thread_pthread.c#lines-443:465
          https://bitbucket.org/pypy/pypy/src/5b42890d48c3/rpython/translator/c/src/thread_pthread.c#lines-443:465
    
        This way when Pygolang is used with buggy Python/darwin, the bug leads to
        frequently appearing deadlocks, while e.g. CPython3/darwin works ok.
    
        -> TODO maintain our own semaphore code.
    
    So eventually we'll have push down and maintain our own semaphores,
    at least for platforms we care, not to be beaten by CPython runtime bugs.
    69db91bf
_runtime_thread.pyx 7.33 KB