1. 31 Jul, 2024 34 commits
    • Xavier Thompson's avatar
      [fix] zc.recipe.egg: Reinstall missing develop egg metadata · 544f8244
      Xavier Thompson authored
      Let zc.recipe.egg:develop reinstall if .egg-info or .dist-info is
      missing from the folder referenced by .egg-link.
      544f8244
    • Xavier Thompson's avatar
      [feat] Let recipe.update access installed paths · ef8c6d07
      Xavier Thompson authored
      Store a part's installed paths in `self[part].installed_files` before
      calling `recipe.update()`, to allow the part's recipe to read them if
      needed by looking up `options.installed_files`.
      
      Delete this attribute just after the call to ensure this data remains
      private to the part.
      
      This will allow zc.recipe.egg:develop to remember in `update()` which
      .egg-link it installed in `install()`; inferring this would otherwise
      require essentially reinstalling the egg from the setup path, as that
      is all the recipe gets as input.
      
      Storing this under `__buildout_installed__` in the part's options was
      considered, but some recipes take all the recorded key/value pairs as
      input, so it would be a breaking change.
      ef8c6d07
    • Xavier Thompson's avatar
      [feat] Enable build of pyproject.toml projects · 77716e17
      Xavier Thompson authored
      Enable zc.buildout.easy_install.build, which builds a project manually
      from an unpacked sdist archive and is used by zc.recipe.egg:custom, to
      build a project which only has a pyproject.toml but no setup.py.
      77716e17
    • Xavier Thompson's avatar
      [fix] Sign develop eggs with directory hash · 28a2f87c
      Xavier Thompson authored
      Mark .dist-info eggs installed by buildout as develop to distinguish
      them from .dist-info eggs from the system and decide how to sign the
      egg accordingly (with directory hash or with version).
      28a2f87c
    • Xavier Thompson's avatar
      [test] Fix tests after PEP 625 · ad48dd46
      Xavier Thompson authored
      Adapt tests to ignore warning generated by setuptools not finding
      zc.recipe.egg on first index lookup, because it's indexed as
      zc-recipe-egg, because since PEP 625 the sdist filename is
      zc_recipe_egg-xyz.tar.gz.
      ad48dd46
    • Xavier Thompson's avatar
      [wkrd] Workaround package index after PEP 625 · cf9c2983
      Xavier Thompson authored
      Since PEP 625, sdist filenames replace . and - in the project name by
      _, such as e.g. zc.buildout==3.0.1 yielding zc_buildout-3.0.1.tar.gz.
      
      However, when looking up available dists in a package index, when an
      sdist with such a normalized filename is found, setuptools interprets
      the filename by replacing _ with - and uses that as the dist's name;
      e.g. zc_buildout-3.0.1.tar.gz yields zc-buildout. This causes lookups
      for the name with . (e.g.: zc.buildout) to fail.
      
      Workaround by also looking up the name with . replaced by _, and then
      fixing the names of found dists back to expected name.
      cf9c2983
    • Xavier Thompson's avatar
      69ea2779
    • Xavier Thompson's avatar
      b51efb6f
    • Xavier Thompson's avatar
      [wkrd] Use pip install --editable --user · a4303921
      Xavier Thompson authored
      Prior to pip 21.1, pip install --editable --target fails because it
      results in wrong parameters being passed to setup.py develop by pip.
      
      Prior to setuptools 45.2.0, both pip install --editable --target and
      pip install --editable --prefix fail because the temporary install
      directory used internally by pip is not added to PYTHONPATH prior
      to pip calling setup.py develop. In later version setuptools emits a
      warning instead of an error.
      
      Temporarily override PYTHONUSERBASE to point to the target directory,
      so as to emulate --prefix=<dir> with PYTHONUSERBASE=<dir> and --user.
      
      This is needed for Python2 because pip 21.1 and setuptools 45.2.0 are
      both Python3 only.
      a4303921
    • Xavier Thompson's avatar
      [tool] Gitignore *.dist-info · c1b47cc5
      Xavier Thompson authored
      c1b47cc5
    • Xavier Thompson's avatar
      [fixup] Fix .egg-link resolution for PEP 660 case · 4725b1cb
      Xavier Thompson authored
      Fixup "Use pip install --editable in easy_install.develop"
      
      Support namespace packages where `spec.submodule_earch_locations` is
      a `_NamespacePath` object instead of a simple `list` and add support
      for cases where the layout of the source project does not follow the
      same structure as the package tree - meaning some custom magic might
      be involved in making editable imports work as intended.
      4725b1cb
    • Xavier Thompson's avatar
      [feat] Use pip install --editable in easy_install.develop · a5f13df6
      Xavier Thompson authored
      Instead of running python setup.py develop directly. This will allow
      using zc.buildout.easy_install.develop on recent projects that have
      only a pyproject.toml. It also fixes develop leaving build artifacts
      in the source directory that caused later runs to do nothing, e.g.
      preventing develop-eggs to be rebuilt when a build dependency passed
      in setup-eggs option of zc.recipe.egg:develop changed.
      
      A verbosity parameter to tune verbosity of pip is added, with adjusted
      values for the case of buildout:develop and of zc.recipe.egg:develop,
      so as to remain close to the previous behavior with regards to logs.
      a5f13df6
    • Xavier Thompson's avatar
      [feat] zc.recipe.egg: Reinstall when setup-eggs versions change · c22e43c8
      Xavier Thompson authored
      Trigger uninstall + install of eggs installed with zc.recipe.egg:custom
      or :develop when pinned versions of setup-eggs have changed. To achieve
      this the versions of setup-eggs are included in the section: this makes
      them part of its signature so that when they change, buildout will call
      `uninstall` and `install` for this section instead of just `update`.
      
      Unlike other zc.recipe.egg entry points, :custom stores the path of the
      installed egg; thus `uninstall` will remove it fully, leaving `install`
      to reinstall it cleanly from scrach.
      
      In the case of :develop, `uninstall` matters little as only the path of
      the installed `.egg-link` is stored. Instead `install` must be fixed to
      actually rebuild the egg in-place in the source directory and `develop`
      should do nothing.
      
      The main issue lies in `zc.buildout.easy_install.develop`: depending on
      the build process, it may leave build artifacts in the source directory
      that cause future runs to do nothing.
      c22e43c8
    • Xavier Thompson's avatar
      3f9210b6
    • Xavier Thompson's avatar
      [fix] Fix invalid specifier in test · 37693a69
      Xavier Thompson authored
      Pinning zc.buildout = >.1 is now invalid, so use >0.1 instead.
      37693a69
    • Xavier Thompson's avatar
      [fix] Fix pip Py2 deprecation filter in tests · 0e2d9a8e
      Xavier Thompson authored
      Adapt filter in tests for pip emitting Python2.7 deprecation warnings
      to pip >= 20.2.2 and < 21.0 - these versions seem to accidentally add
      a redundant bit of message to the warning.
      0e2d9a8e
    • Xavier Thompson's avatar
      [fix] Ignore setuptools deprecation warnings · d08e3d74
      Xavier Thompson authored
      The testing framework, some individual tests and 'buildout:develop'
      call python setup.py <command> manually, which is deprecated by
      setuptools. Depending on the setuptools version, suppress the
      warning for now to avoid polluting the logs.
      d08e3d74
    • Xavier Thompson's avatar
      [fix] Fix logging filters for Python2 · 3a0add1d
      Xavier Thompson authored
      3a0add1d
    • Xavier Thompson's avatar
      340b9e1d
    • Xavier Thompson's avatar
      [fix] Adapt to setuptools>=65.6.0 logging on root · 6122a6ea
      Xavier Thompson authored
      INFO logs from setuptools which were previously not emitted because
      setuptool's ad-hoc legacy logger defaulted to WARNING and above may
      now be emitted because setuptools now logs to the root logger, thus
      the global root logger's level configuration applied.
      
      This caused undesired 'root: <some setuptools info>' messages to be
      emitted and caused many tests to fail due to unexpected outputs.
      6122a6ea
    • Xavier Thompson's avatar
      6ff3ab07
    • Julien Muchembled's avatar
    • Julien Muchembled's avatar
      [feat] download: add support for slapos.libnetworkcache · 09230efc
      Julien Muchembled authored
      When specifying an alternate URL as fallback, the main URL is always
      used for both downloading & uploading from/to networkcache.
      09230efc
    • Xavier Thompson's avatar
      [feat] Propagate libnetworkcache installation · b78c33af
      Xavier Thompson authored
      If slapos.libnetworkcache is importable, install it in bootstrap and
      in buildout upgrade - the places where bin/buildout is (re)generated
      - as though it were a dependency of zc.buildout.
      
      This is a hack to propagate libnetworkcache as a soft dependency.
      b78c33af
    • Julien Muchembled's avatar
      [feat] download: add netrc file support · 2e3258e9
      Julien Muchembled authored
      Like for URL that contain credentials, we still skip auth challenge
      because it's faster and:
      - we only support one auth scheme (basic)
      - netrc provides no way to specify realms, which seem anyway to be
        less and less used (https://stackoverflow.com/q/69303610 reports
        that recent browsers don't display them anymore)
      
      See merge request !25
      2e3258e9
    • Julien Muchembled's avatar
      [feat] Extend Download API to use an alternate URL as fallback · 00c26491
      Julien Muchembled authored
      This retries with the alternate URL in case of HTTPError with the main
      one.
      
      Used by slapos.recipe.build:download* and slapos.recipe.cmmi recipes.
      00c26491
    • Julien Muchembled's avatar
      [fix] Rewrite 'urlretrieve' helper to fix various download-related issues · 648fcdd6
      Julien Muchembled authored
      - Py3: stop using legacy API of urllib.request and
             fix download of http(s) URLs containing user:passwd@
      - Py2: avoid OOM when downloading huge files
      
      This is implemented as a method in case we want to make it configurable
      via [buildout].
      648fcdd6
    • Julien Muchembled's avatar
      [fix/opti] download: clean-up, fix, optimization · dd7262ae
      Julien Muchembled authored
      An optimization is to avoid temporary file when possible: a rename
      (or hard link) is not always possible (different mount points).
      
      Another one is to not check md5sum twice when using cache file.
      
      Fall-back mode is ignored if an MD5 checksum is given.
      
      In case of checksum mismatch for a cached path, remove it and
      download again, mainly to cover the following cases:
      - the url content changes and the user updates the checksum
      - buildout killed while downloading directly to cache
        (see above optimization)
      - shutil.copyfile is interrupted
      dd7262ae
    • Jérome Perrin's avatar
      [fixup] Ignore _profile_base_location_ when computing signatures · c9edcc49
      Jérome Perrin authored
      We want two identical sections at different URL to be able to produce
      same signature. This feature is useful for slapos.recipe.cmmi's
      shared parts.
      
      This commit may fixup "Support ${:_profile_base_location_}." but is
      purposefuly kept separate because it concerns parts signatures which
      are an orthogonal feature.
      c9edcc49
    • Kazuhiko Shiozaki's avatar
      1bd4c5c3
    • Kazuhiko Shiozaki's avatar
      e32f7f36
    • Xavier Thompson's avatar
      e26ed197
    • Xavier Thompson's avatar
      17beb7b3
    • Xavier Thompson's avatar
      [fix] Support local version label for zc.buildout · c78f0ef9
      Xavier Thompson authored
      When there is no pinned version for zc.buildout itself, buildout
      adds a ">=<current-version>" requirement to prevent accidental
      downgrading. If the current version has a local version label,
      this produced an invalid version specifier. To fix this, only
      the public part of the current version is used.
      c78f0ef9
  2. 03 Jun, 2024 2 commits
  3. 24 May, 2024 4 commits