1. 02 Oct, 2020 1 commit
  2. 01 Oct, 2020 1 commit
  3. 20 May, 2020 2 commits
  4. 15 May, 2020 3 commits
    • Kirill Smelkov's avatar
      build_dso: Fix support for develop install (pip install -e) · 84e06348
      Kirill Smelkov authored
      When a package is installed in development mode by `pip install -e`, it
      calls `python setup.py develop` under the covers
      
          https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode
      
      However it currently skips build_dso in that mode which leads to build
      failure if anything - e.g. extension - is depending on a dso:
      
          + pip install -v -e .
          Created temporary directory: /tmp/pip-ephem-wheel-cache-OJgoBO
          Created temporary directory: /tmp/pip-req-tracker-CrRj5z
          Created requirements tracker '/tmp/pip-req-tracker-CrRj5z'
          Created temporary directory: /tmp/pip-install-hiu8rq
          Obtaining file:///home/kirr/src/tools/py/pypa/setuptools_dso/example
            Added file:///home/kirr/src/tools/py/pypa/setuptools_dso/example to build tracker '/tmp/pip-req-tracker-CrRj5z'
            Running setup.py (path:/home/kirr/src/tools/py/pypa/setuptools_dso/example/setup.py) egg_info for package from file:///home/kirr/src/tools/py/pypa/setuptools_dso/example
              Running command python setup.py egg_info
              running egg_info
              creating src/dsodemo.egg-info
              writing src/dsodemo.egg-info/PKG-INFO
              writing top-level names to src/dsodemo.egg-info/top_level.txt
              writing dependency_links to src/dsodemo.egg-info/dependency_links.txt
              writing manifest file 'src/dsodemo.egg-info/SOURCES.txt'
              reading manifest file 'src/dsodemo.egg-info/SOURCES.txt'
              writing manifest file 'src/dsodemo.egg-info/SOURCES.txt'
            Source in /home/kirr/src/tools/py/pypa/setuptools_dso/example has version 0.1, which satisfies requirement dsodemo==0.1 from file:///home/kirr/src/tools/py/pypa/setuptools_dso/example
            Removed dsodemo==0.1 from file:///home/kirr/src/tools/py/pypa/setuptools_dso/example from build tracker '/tmp/pip-req-tracker-CrRj5z'
          Installing collected packages: dsodemo
            Running setup.py develop for dsodemo
              Running command /home/kirr/src/tools/py/pypa/setuptools_dso/env/bin/python -c "import setuptools, tokenize;__file__='/home/kirr/src/tools/py/pypa/setuptools_dso/example/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps
              running develop
              Checking .pth file support in /home/kirr/src/tools/py/pypa/setuptools_dso/env/lib/python2.7/site-packages/
              /home/kirr/src/tools/py/pypa/setuptools_dso/env/bin/python -E -c pass
              TEST PASSED: /home/kirr/src/tools/py/pypa/setuptools_dso/env/lib/python2.7/site-packages/ appears to support .pth files
              running egg_info
              writing src/dsodemo.egg-info/PKG-INFO
              writing top-level names to src/dsodemo.egg-info/top_level.txt
              writing dependency_links to src/dsodemo.egg-info/dependency_links.txt
              reading manifest file 'src/dsodemo.egg-info/SOURCES.txt'
              writing manifest file 'src/dsodemo.egg-info/SOURCES.txt'
              running build_ext
              Traceback (most recent call last):
                File "<string>", line 1, in <module>
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/example/setup.py", line 26, in <module>
                  x_dsos = [dso],
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools_dso/__init__.py", line 37, in setup
                  _setup(**kws)
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in setup
                  return distutils.core.setup(**attrs)
                File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
                  dist.run_commands()
                File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
                  self.run_command(cmd)
                File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
                  cmd_obj.run()
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools/command/develop.py", line 38, in run
                  self.install_for_development()
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools/command/develop.py", line 140, in install_for_development
                  self.run_command('build_ext')
                File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
                  self.distribution.run_command(command)
                File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
                  cmd_obj.run()
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools_dso/dsocmd.py", line 395, in run
                  _build_ext.run(self)
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools/command/build_ext.py", line 78, in run
                  _build_ext.run(self)
                File "/usr/lib/python2.7/distutils/command/build_ext.py", line 340, in run
                  self.build_extensions()
                File "/usr/lib/python2.7/distutils/command/build_ext.py", line 449, in build_extensions
                  self.build_extension(ext)
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools_dso/dsocmd.py", line 406, in build_extension
                  self.dso2lib_pre(ext)
                File "/home/kirr/src/tools/py/pypa/setuptools_dso/env/local/lib/python2.7/site-packages/setuptools_dso/dsocmd.py", line 146, in dso2lib_pre
                  raise RuntimeError("Unable to find DSO %s needed by extension %s"%(dso, ext.name))
              RuntimeError: Unable to find DSO dsodemo.lib.demo needed by extension dsodemo.ext.dtest
      
      -> Fix it by making build_ext to depend on build_dso.
      84e06348
    • Kirill Smelkov's avatar
      build_dso: Factor-out .sub_commands dependency injection into separate function · d7cb4fe0
      Kirill Smelkov authored
      In the next patch we will need to inject build_ext -> build_dso
      dependency and will use this infrastructure.
      d7cb4fe0
    • Kirill Smelkov's avatar
      example: Don't import build_dso and build_ext · 326ffa47
      Kirill Smelkov authored
      - they are not used in the example
      - they are injected as command classes by setuptools_dso.setup itself
      326ffa47
  5. 02 May, 2020 1 commit
  6. 01 May, 2020 3 commits
  7. 30 Apr, 2020 1 commit
    • Mehul Tikekar's avatar
      Use @rpath as install_name prefix on MacOS · dd2cf303
      Mehul Tikekar authored
      @loader_path prefix requires dependent libraries to be in the same
      directory or to use install_name_tool to change the name after linking.
      With this fix, linker's -rpath can be used instead.
      dd2cf303
  8. 25 Jan, 2020 1 commit
  9. 24 Jan, 2020 5 commits
  10. 06 Jan, 2020 2 commits
  11. 04 Jan, 2020 1 commit
  12. 06 Dec, 2019 1 commit
  13. 04 Dec, 2019 1 commit
    • Kirill Smelkov's avatar
      build_dso: Take package_dir into account · 40b492ab
      Kirill Smelkov authored
      package_dir needs to be taken into account for inplace builds, because
      then inplace directory of a package != directory of the package when it
      is installed. For example setuptools_dso itself uses
      `package_dir={'':'src'}` in its setup.py and this way for inplace build the
      directory of setuptools_dso package is not setuptools_dso/ but
      src/setuptools_dso/ instead.
      
      However `build_dso --inplace` was installing built libraries without
      taking package_dir into account. For example with amended example/ test,
      libdemo.so was installed into dsodemo/lib/libdemo.so instead of
      src/dsodemo/lib/libdemo.so, while dtest.so was correctly installed into
      src/dsodemo/ext/dtest.so . As the result it was not working:
      
          + python -m dsodemo.cli
          Traceback (most recent call last):
            File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
              "__main__", fname, loader, pkg_name)
            File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
              exec code in run_globals
            File "/home/kirr/src/tools/py/pypa/setuptools_dso/example/src/dsodemo/cli.py", line 15, in <module>
              from .ext import dtest
          ImportError: libdemo.so.1.0: cannot open shared object file: No such file or directory
          .../setuptools_dso$ ll example/
          итого 16
          drwxr-xr-x 4 kirr kirr 4096 дек  4 18:12 build
          drwxr-xr-x 3 kirr kirr 4096 дек  4 18:12 dsodemo
          -rwxr-xr-x 1 kirr kirr  639 дек  4 18:12 setup.py
          drwxr-xr-x 3 kirr kirr 4096 дек  4 18:06 src
          .../setuptools_dso$ ll example/dsodemo/lib/
          итого 136
          -rwxr-xr-x 1 kirr kirr 68888 дек  4 18:12 libdemo.so
          -rwxr-xr-x 1 kirr kirr 68888 дек  4 18:12 libdemo.so.1.0
          .../setuptools_dso$ ll example/src/dsodemo/ext/
          итого 124
          -rwxr-xr-x 1 kirr kirr 119000 дек  4 18:12 dtest.so
          -rw-r--r-- 1 kirr kirr      0 июл 30 21:37 __init__.py
          -rw-r--r-- 1 kirr kirr    110 дек  4 18:12 __init__.pyc
          .../setuptools_dso$ ldd example/src/dsodemo/ext/dtest.so
                  linux-vdso.so.1 (0x00007fffae3fd000)
                  libdemo.so.1.0 => not found
                  libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1c3bf20000)
                  libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1c3bd9d000)
                  libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1c3bd83000)
                  libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1c3bd62000)
                  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1c3bba1000)
                  /lib64/ld-linux-x86-64.so.2 (0x00007f1c3c149000)
          .../setuptools_dso$ objdump -x example/src/dsodemo/ext/dtest.so |grep PATH
            RUNPATH              \$ORIGIN/../lib:$ORIGIN/../lib
      
      -> Fix it by taking package_dir option into account in --inplace dsocmd handler.
      
      Note1: The code to do so mimics build_ext from setuptools:
      
          https://github.com/pypa/setuptools/blob/v42.0.2-0-g1d03fdc9/setuptools/command/build_ext.py#L89-L107
      
      Note2: Sources of Extension and DSO are specified with full path from project
      top-level, because it already works this way for unmodified Extension even in
      presence of package_dir.
      40b492ab
  14. 18 Nov, 2019 1 commit
  15. 17 Nov, 2019 2 commits
  16. 15 Nov, 2019 2 commits
  17. 26 Sep, 2019 1 commit
  18. 21 Aug, 2019 2 commits
  19. 15 Aug, 2019 5 commits
    • Michael Davidsaver's avatar
      1.1 · 99ab78cd
      Michael Davidsaver authored
      99ab78cd
    • Kirill Smelkov's avatar
      build_dso: Fix incremental rebuild · 024cc109
      Kirill Smelkov authored
      The code was trying to create solibbase symlink pointing to -> outbaselib, e.g.
      
          libdemo.so.1.0 -> build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0
      
      However if the source file in worktree already exists, symlinking was
      failing with an error. Fix it by making sure that source file does not
      exist before calling os.symlink.
      
      Also there was related thinko in log.ingf describing who points to
      who/where, which is corrected along the way.
      
      Without the fix newly added test fails as
      
          ...
          running build_dso
          Building DSOs
          building 'dsodemo.lib.demo' DSO as build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0
          creating build
          creating build/temp.linux-x86_64-2.7
          x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBUILD_FOO -c foo.c -o build/temp.linux-x86_64-2.7/foo.o -DALL -DISC
          x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBUILD_FOO -c bar.cpp -o build/temp.linux-x86_64-2.7/bar.o -DALL -DISCXX
          cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
          creating build/lib.linux-x86_64-2.7
          creating build/lib.linux-x86_64-2.7/dsodemo
          creating build/lib.linux-x86_64-2.7/dsodemo/lib
          x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/foo.o build/temp.linux-x86_64-2.7/bar.o -o build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0 -Wl,-h,libdemo.so.1.0
          symlink libdemo.so.1.0 -> build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0
          creating dsodemo/lib
          copying build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0 -> dsodemo/lib
          copying build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so -> dsodemo/lib
          running build_dso
          Building DSOs
          building 'dsodemo.lib.demo' DSO as build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0
          x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBUILD_FOO -c foo.c -o build/temp.linux-x86_64-2.7/foo.o -DALL -DISC
          x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DBUILD_FOO -c bar.cpp -o build/temp.linux-x86_64-2.7/bar.o -DALL -DISCXX
          cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
          x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/foo.o build/temp.linux-x86_64-2.7/bar.o -o build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0 -Wl,-h,libdemo.so.1.0
          symlink libdemo.so.1.0 -> build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0
          error: [Errno 17] File exists
      024cc109
    • Kirill Smelkov's avatar
      build_dso: Factor out basename(solib) computation · a3c37836
      Kirill Smelkov authored
      It is used in several places and will be used more in the next patch.
      a3c37836
    • Kirill Smelkov's avatar
      Fix inplace build · e249f9de
      Kirill Smelkov authored
      build_dso for inplace mode copies DSO built in build/ into worktree.
      However if the destination directory does not exist yet, copy_file
      fails. Fix it by making sure before copying that destination directory
      exits.
      
      Without the fix added test fails as follows:
      
          ...
          creating build/lib.linux-x86_64-2.7
          creating build/lib.linux-x86_64-2.7/dsodemo
          creating build/lib.linux-x86_64-2.7/dsodemo/lib
          x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-IbFBHb/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/foo.o build/temp.linux-x86_64-2.7/bar.o -o build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0 -Wl,-h,libdemo.so.1.0
          symlink libdemo.so.1.0 -> build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0
          copying build/lib.linux-x86_64-2.7/dsodemo/lib/libdemo.so.1.0 -> dsodemo/lib
          error: [Errno 2] No such file or directory: 'dsodemo/lib/libdemo.so.1.0'
      e249f9de
    • Kirill Smelkov's avatar
      .gitignore += *.so.* · d9a47177
      Kirill Smelkov authored
      This covers .so files created with soversion, e.g. there can be
      libdemo.so and libdemo.so.1.0 . The latter is covered by added patern.
      
      No need to change dylib and dll patterns - according to current
      setuptools_dso code, the soversion is either used before '.dylib'
      suffix, or, for '.dll' case is not present in file name at all.
      d9a47177
  20. 13 Oct, 2018 1 commit
  21. 18 Jul, 2018 1 commit
  22. 17 Jul, 2018 1 commit
  23. 15 Jul, 2018 1 commit