Commit ff97a1c2 authored by Michael Davidsaver's avatar Michael Davidsaver

doc

parent 54e2ce01
# setuptools extension for building non-Python Dynamic Shared Objects # setuptools extension for building non-Python Dynamic Shared Objects
Building non-python shared libraries (eg. libY.so or Y.dll) for inclusion in a Python Wheel. Building non-python shared libraries (eg. `libY.so`, `libY.dylib`, or `Y.dll`) for inclusion in a Python Wheel.
This extension provides at alternative to bundling externally built This extension provides at alternative to bundling externally built
libraries in Python Wheel packages. This replaces an external libraries in Python Wheel packages. This replaces an external
build system (eg. Makefile). build system (eg. Makefile), allowing non-python libraries to be
built from source within the python ecosystem.
See [example/setup.py](example/setup.py) for usage. See [example/setup.py](example/setup.py) for a working example.
![setuptools-dso](https://github.com/mdavidsaver/setuptools_dso/workflows/setuptools-dso/badge.svg) ![setuptools-dso](https://github.com/mdavidsaver/setuptools_dso/workflows/setuptools-dso/badge.svg)
## Use case and mechanics ## Usage
The [example/](example/setup.py) demonstrates building a non-python library,
and linking it with a python extension module.
### pyproject.toml
To properly support `pip install ...`, it is recommended to include a
[`pyproject.toml`](https://www.python.org/dev/peps/pep-0518/)
file containing at least:
```
[build-system]
requires = ["setuptools", "wheel", "setuptools_dso"]
```
This ensures that `setuptools_dso` is available to be imported by `setup.py`.
### Building a DSO
The [source](example/src/) files while make up the non-python `demo` library are: `mylib.h`, `foo.c`, `bar.cpp`.
This library will be expressed as a `setuptools_dso.DSO` object.
The first argument is a directory path and library base name encoded like a python module.
eg. the result of `dsodemo.lib.demo` will be eg. `dsodemo/lib/libdemo.so` or `dsodemo\lib\demo.dll`
installed in the python module tree along-side any other python code or C extensions.
Note that there need not be a `dsodemo/lib/__init__.py` as `dsodemo.lib` need not be a python package.
```py
from setuptools_dso import DSO, Extension, setup
dso = DSO('dsodemo.lib.demo', ['src/foo.c', 'src/bar.cpp'], ...)
setup(
...
x_dsos = [dso],
zip_safe = False, # setuptools_dso is not compatible with eggs!
)
```
The `DSO` constructor understands all of the same keyword arguments as `setuptools.Extension`
and [`distutils.core.Extension`](https://docs.python.org/3/distutils/apiref.html#distutils.core.Extension),
with the addition of `dsos=[...]`, `soversion='...'`, and `lang_compile_args={'...':'...'}`.
The `dsos=` argument is a list of other `DSO` names (eg. `'dsodemo.lib.demo'`) to allow
one `DSO` to be linked against others.
eg. `dsos=['some.lib.foo']` will result in something like `gcc ... -L.../some/lib -lfoo`.
### Building an Extension
`setuptools_dso.Extension` is a wrapper around `setuptools.Extension` which adds the `dsos=[...]` keyword argument.
This allows a C extension module to be linked against a `DSO` by name.
The named `DSO` may be built by the same `setup.py`, or may already be present in `$PYTHONPATH`.
```py
from setuptools_dso import DSO, Extension, setup
ext = Extension('dsodemo.ext.dtest', ['src/extension.cpp'],
dsos=['dsodemo.lib.demo'],
)
setup(
...
ext_modules = [ext],
zip_safe = False, # setuptools_dso is not compatible with eggs!
)
```
### Runtime
Some additional runtime preparation is need to in order to find the `'dsodemo.lib.demo'` library
when the `dsodemo.ext.dtest` Extension is imported on Windows.
The example places this in [`example/src/dsodemo/__init__.py`](example/src/dsodemo/__init__.py)
to ensure it always runs before the extension library is loaded.
```
import sys, os
def fixpath():
path = os.environ.get('PATH', '').split(os.pathsep)
libdir = os.path.join(os.path.dirname(__file__), 'lib')
path.append(libdir)
os.environ['PATH'] = os.pathsep.join(path)
if hasattr(os, 'add_dll_directory'):
os.add_dll_directory(libdir)
if sys.platform == "win32":
fixpath()
```
## Mechanics
Libraries build with setuptools-dso will typically be linked, directly or indirectly, to python extension modules. Libraries build with setuptools-dso will typically be linked, directly or indirectly, to python extension modules.
Such libraries should loaded implicitly when the extension module is imported by the python runtime. Such libraries should loaded implicitly when the extension module is imported by the python runtime.
...@@ -26,30 +119,32 @@ capabilities of the executable format used. ...@@ -26,30 +119,32 @@ capabilities of the executable format used.
For ELF (Executable and Linking Format) targets the RPATH mechanism is used with the magic $ORIGIN/ prefix. For ELF (Executable and Linking Format) targets the RPATH mechanism is used with the magic $ORIGIN/ prefix.
Note that on Linux runtime linking is really a function of the libc (aka glibc). Take as an example the following installed module tree:
Imagine a directory tree:
```
* modulea * modulea
* __init__.py * __init__.py
* _ext.so * _ext.so
* lib * lib
* libsup.so * libsup.so
* libsup.so.0 * libsup.so.0
```
The supporting library is then linked with a library name of "libsup.so.0" The supporting library is then linked with a library name of `libsup.so.0`
(eg. "gcc -o libsup.so.0 -shared -Wl,-soname,libsup.so.0 ...") (eg. `gcc -o libsup.so.0 -shared -Wl,-soname,libsup.so.0 ...`)
When linking a dependent library or extension module "-Wl,-rpath,'$ORIGIN/lib'" When linking a dependent library or extension module `-Wl,-rpath,'$ORIGIN/lib'`
is given since the relative path from _ext.so to libsup.so.0 is './lib'. is given since the relative path from _ext.so to libsup.so.0 is './lib'.
Note that on Linux runtime linking is really a function of the libc (eg. glibc).
### Mach-O (OSX) ### Mach-O (OSX)
Historically Mach-O did not support anything like RPATH. Historically Mach-O did not support anything like RPATH.
However, it now does (cf. 'man dyld') However, it now does (cf. 'man dyld')
The equivalence with GCC+ELF is for "-Wl,-soname,libsup.so.0" to become "-install\_name @rpath/libsup.0.dylib". The equivalence with GCC+ELF is for `-Wl,-soname,libsup.so.0` to become `-install\_name @rpath/libsup.0.dylib`.
Then "-Wl,-rpath,'$ORIGIN/lib'" becomes "-Wl,-rpath,@loader_path/lib". Then `-Wl,-rpath,'$ORIGIN/lib'` becomes `-Wl,-rpath,@loader_path/lib`.
### PE (Windows) ### PE (Windows)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment