Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
S
setuptools_dso
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Kirill Smelkov
setuptools_dso
Commits
ff97a1c2
Commit
ff97a1c2
authored
Jan 22, 2021
by
Michael Davidsaver
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
doc
parent
54e2ce01
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
107 additions
and
12 deletions
+107
-12
README.md
README.md
+107
-12
No files found.
README.md
View file @
ff97a1c2
# setuptools extension for building non-Python Dynamic Shared Objects
# setuptools extension for building non-Python Dynamic Shared Objects
Building non-python shared libraries (eg.
libY.so or Y.dll
) for inclusion in a Python Wheel.
Building non-python shared libraries (eg.
`libY.so`
,
`libY.dylib`
, or
`Y.dll`
) for inclusion in a Python Wheel.
This extension provides at alternative to bundling externally built
This extension provides at alternative to bundling externally built
libraries in Python Wheel packages. This replaces an external
libraries in Python Wheel packages. This replaces an external
build system (eg. Makefile).
build system (eg. Makefile), allowing non-python libraries to be
built from source within the python ecosystem.
See
[
example/setup.py
](
example/setup.py
)
for
usag
e.
See
[
example/setup.py
](
example/setup.py
)
for
a working exampl
e.


## Use case and mechanics
## Usage
The
[
example/
](
example/setup.py
)
demonstrates building a non-python library,
and linking it with a python extension module.
### pyproject.toml
To properly support
`pip install ...`
, it is recommended to include a
[
`pyproject.toml`
](
https://www.python.org/dev/peps/pep-0518/
)
file containing at least:
```
[build-system]
requires = ["setuptools", "wheel", "setuptools_dso"]
```
This ensures that
`setuptools_dso`
is available to be imported by
`setup.py`
.
### Building a DSO
The
[
source
](
example/src/
)
files while make up the non-python
`demo`
library are:
`mylib.h`
,
`foo.c`
,
`bar.cpp`
.
This library will be expressed as a
`setuptools_dso.DSO`
object.
The first argument is a directory path and library base name encoded like a python module.
eg. the result of
`dsodemo.lib.demo`
will be eg.
`dsodemo/lib/libdemo.so`
or
`dsodemo\lib\demo.dll`
installed in the python module tree along-side any other python code or C extensions.
Note that there need not be a
`dsodemo/lib/__init__.py`
as
`dsodemo.lib`
need not be a python package.
```
py
from
setuptools_dso
import
DSO
,
Extension
,
setup
dso
=
DSO
(
'dsodemo.lib.demo'
,
[
'src/foo.c'
,
'src/bar.cpp'
],
...)
setup
(
...
x_dsos
=
[
dso
],
zip_safe
=
False
,
# setuptools_dso is not compatible with eggs!
)
```
The
`DSO`
constructor understands all of the same keyword arguments as
`setuptools.Extension`
and
[
`distutils.core.Extension`
](
https://docs.python.org/3/distutils/apiref.html#distutils.core.Extension
)
,
with the addition of
`dsos=[...]`
,
`soversion='...'`
, and
`lang_compile_args={'...':'...'}`
.
The
`dsos=`
argument is a list of other
`DSO`
names (eg.
`'dsodemo.lib.demo'`
) to allow
one
`DSO`
to be linked against others.
eg.
`dsos=['some.lib.foo']`
will result in something like
`gcc ... -L.../some/lib -lfoo`
.
### Building an Extension
`setuptools_dso.Extension`
is a wrapper around
`setuptools.Extension`
which adds the
`dsos=[...]`
keyword argument.
This allows a C extension module to be linked against a
`DSO`
by name.
The named
`DSO`
may be built by the same
`setup.py`
, or may already be present in
`$PYTHONPATH`
.
```
py
from
setuptools_dso
import
DSO
,
Extension
,
setup
ext
=
Extension
(
'dsodemo.ext.dtest'
,
[
'src/extension.cpp'
],
dsos
=
[
'dsodemo.lib.demo'
],
)
setup
(
...
ext_modules
=
[
ext
],
zip_safe
=
False
,
# setuptools_dso is not compatible with eggs!
)
```
### Runtime
Some additional runtime preparation is need to in order to find the
`'dsodemo.lib.demo'`
library
when the
`dsodemo.ext.dtest`
Extension is imported on Windows.
The example places this in
[
`example/src/dsodemo/__init__.py`
](
example/src/dsodemo/__init__.py
)
to ensure it always runs before the extension library is loaded.
```
import sys, os
def fixpath():
path = os.environ.get('PATH', '').split(os.pathsep)
libdir = os.path.join(os.path.dirname(__file__), 'lib')
path.append(libdir)
os.environ['PATH'] = os.pathsep.join(path)
if hasattr(os, 'add_dll_directory'):
os.add_dll_directory(libdir)
if sys.platform == "win32":
fixpath()
```
## Mechanics
Libraries build with setuptools-dso will typically be linked, directly or indirectly, to python extension modules.
Libraries build with setuptools-dso will typically be linked, directly or indirectly, to python extension modules.
Such libraries should loaded implicitly when the extension module is imported by the python runtime.
Such libraries should loaded implicitly when the extension module is imported by the python runtime.
...
@@ -26,30 +119,32 @@ capabilities of the executable format used.
...
@@ -26,30 +119,32 @@ capabilities of the executable format used.
For ELF (Executable and Linking Format) targets the RPATH mechanism is used with the magic $ORIGIN/ prefix.
For ELF (Executable and Linking Format) targets the RPATH mechanism is used with the magic $ORIGIN/ prefix.
Note that on Linux runtime linking is really a function of the libc (aka glibc).
Take as an example the following installed module tree:
Imagine a directory tree:
```
* modulea
* modulea
* __init__.py
* __init__.py
* _ext.so
* _ext.so
* lib
* lib
* libsup.so
* libsup.so
* libsup.so.0
* libsup.so.0
```
The supporting library is then linked with a library name of
"libsup.so.0"
The supporting library is then linked with a library name of
`libsup.so.0`
(eg.
"gcc -o libsup.so.0 -shared -Wl,-soname,libsup.so.0 ..."
)
(eg.
`gcc -o libsup.so.0 -shared -Wl,-soname,libsup.so.0 ...`
)
When linking a dependent library or extension module
"-Wl,-rpath,'$ORIGIN/lib'"
When linking a dependent library or extension module
`-Wl,-rpath,'$ORIGIN/lib'`
is given since the relative path from _ext.so to libsup.so.0 is './lib'.
is given since the relative path from _ext.so to libsup.so.0 is './lib'.
Note that on Linux runtime linking is really a function of the libc (eg. glibc).
### Mach-O (OSX)
### Mach-O (OSX)
Historically Mach-O did not support anything like RPATH.
Historically Mach-O did not support anything like RPATH.
However, it now does (cf. 'man dyld')
However, it now does (cf. 'man dyld')
The equivalence with GCC+ELF is for
"-Wl,-soname,libsup.so.0" to become "-install
\_
name @rpath/libsup.0.dylib"
.
The equivalence with GCC+ELF is for
`-Wl,-soname,libsup.so.0`
to become
`-install\_name @rpath/libsup.0.dylib`
.
Then
"-Wl,-rpath,'$ORIGIN/lib'" becomes "-Wl,-rpath,@loader_path/lib"
.
Then
`-Wl,-rpath,'$ORIGIN/lib'`
becomes
`-Wl,-rpath,@loader_path/lib`
.
### PE (Windows)
### PE (Windows)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment