Commit 419087c4 authored by Jason R. Coombs's avatar Jason R. Coombs

Merge branch 'master' into debt/drop-py35

parents 38de858e 898ae046
[bumpversion]
current_version = 47.1.1
current_version = 50.3.2
commit = True
tag = True
......
[run]
source=
pkg_resources
setuptools
omit=
*/_vendor/*
[report]
---
name: Setuptools warns about Python 2 incompatibility
about: Report the issue where setuptools 45 or later stops working on Python 2
title: Incompatible install in (summarize your environment)
labels: Python 2
assignees: ''
---
<!--
Please DO NOT SUBMIT this template without first investigating the issue and answering the questions below. This template is intended mainly for developers of systems and not for end users. If you are an end user experiencing the warning, please work with your system maintainers (starting with the project you're trying to use) to report the issue.
If you did not intend to use this template, but only meant to file a blank issue, just hit the back button and click "Open a blank issue".
Setuptools 45 dropped support for Python 2 with a strenuous warning and Setuptools 47 fails to run on Python 2.
In most cases, using pip 9 or later to install Setuptools from PyPI or any index supporting the Requires-Python metadata will do the right thing and install Setuptools 44.x on Python 2.
If you've come to file an issue, it's probably because some process managed to bypass these protections.
Your first course of action should be to reason about how you managed to get an unsupported version of Setuptools on Python 2. Please complete the sections below and provide any other detail about your environment that will help us help you.
-->
## Prerequisites
<!-- These are the recommended workarounds for the issue. Please
try them first. -->
- [ ] Read [Python 2 Sunset docs](https://setuptools.readthedocs.io/en/latest/python%202%20sunset.html).
- [ ] Python 2 is required for this application.
- [ ] I maintain the software that installs Setuptools (if not, please contact that project).
- [ ] Setuptools installed with pip 9 or later.
- [ ] Pinning Setuptools to `setuptools<45` in the environment was unsuccessful.
## Environment Details
- Operating System and version:
- Python version:
- Python installed how:
- Virtualenv version (if using virtualenv): n/a
Command(s) that triggered the warning/error (and output):
```
```
Command(s) used to install setuptools (and output):
```
```
Output of `pip --version` when installing setuptools:
```
```
## Other notes
......@@ -19,28 +19,54 @@ jobs:
# max-parallel: 5
matrix:
python-version:
- 3.9
- 3.8
- pypy3
- 3.7
- 3.6
os:
- ubuntu-latest
- ubuntu-18.04
- ubuntu-16.04
- macOS-latest
# - windows-2019
# - windows-2016
include:
# Dev versions (deadsnakes)
- os: ubuntu-20.04
python-version: 3.9-dev
- os: ubuntu-20.04
python-version: 3.8-dev
env:
NETWORK_REQUIRED: 1
PYTHON_VERSION: ${{ matrix.python-version }}
TOX_PARALLEL_NO_SPINNER: 1
TOXENV: python
USE_DEADSNAKES: false
steps:
- uses: actions/checkout@master
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1.1.1
- name: Set flag to use deadsnakes
if: >-
endsWith(env.PYTHON_VERSION, '-beta') ||
endsWith(env.PYTHON_VERSION, '-dev')
run: |
from __future__ import print_function
python_version = '${{ env.PYTHON_VERSION }}'.replace('-beta', '')
print('::set-env name=PYTHON_VERSION::{ver}'.format(ver=python_version))
print('::set-env name=USE_DEADSNAKES::true')
shell: python
- name: Set up Python ${{ env.PYTHON_VERSION }} (deadsnakes)
uses: deadsnakes/action@v1.0.0
if: fromJSON(env.USE_DEADSNAKES) && true || false
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v2.1.1
if: >-
!fromJSON(env.USE_DEADSNAKES) && true || false
with:
python-version: ${{ matrix.python-version }}
python-version: ${{ env.PYTHON_VERSION }}
- name: Log Python version
run: >-
python --version
......@@ -72,9 +98,9 @@ jobs:
run: >-
python -m pip freeze --all
- name: Adjust TOXENV for PyPy
if: startsWith(matrix.python-version, 'pypy')
if: startsWith(env.PYTHON_VERSION, 'pypy')
run: >-
echo "::set-env name=TOXENV::${{ matrix.python-version }}"
echo "::set-env name=TOXENV::${{ env.PYTHON_VERSION }}"
- name: Log env vars
run: >-
env
......@@ -90,6 +116,7 @@ jobs:
python -m
tox
--parallel auto
--parallel-live
--notest
--skip-missing-interpreters false
- name: Test with tox
......@@ -97,5 +124,6 @@ jobs:
python -m
tox
--parallel auto
--parallel-live
--
--cov
-vvvvv
......@@ -5,7 +5,6 @@ jobs:
fast_finish: true
include:
- python: pypy3
env: DISABLE_COVERAGE=1 # Don't run coverage on pypy (too slow).
- python: 3.6
- python: 3.7
- &latest_py3
......@@ -13,16 +12,32 @@ jobs:
- <<: *latest_py3
env: LANG=C
- python: 3.8-dev
- python: 3.9-dev
- <<: *latest_py3
env: TOXENV=docs DISABLE_COVERAGE=1
env: TOXENV=docs
- arch: ppc64le
python: pypy3
- arch: ppc64le
python: 3.5
- &latest_py3_ppc
arch: ppc64le
python: 3.8
- <<: *latest_py3_ppc
env: LANG=C
- arch: ppc64le
python: 3.9-dev
allow_failures:
# suppress failures due to pypa/setuptools#2000
- python: pypy3
- <<: *latest_py3
env: TOXENV=docs DISABLE_COVERAGE=1
env: TOXENV=docs
cache: pip
before_install:
- python tools/ppc64le-patch.py
install:
# ensure we have recent pip/setuptools/wheel
......@@ -38,22 +53,8 @@ install:
script:
- export NETWORK_REQUIRED=1
- |
( # Run testsuite.
if [ -z "$DISABLE_COVERAGE" ]
then
tox -- --cov
else
tox
fi
)
- tox
after_success:
- |
( # Upload coverage data.
if [ -z "$DISABLE_COVERAGE" ]
then
export TRAVIS_JOB_NAME="${TRAVIS_PYTHON_VERSION} (LANG=$LANG)" CODECOV_ENV=TRAVIS_JOB_NAME
tox -e coverage,codecov
fi
)
- export TRAVIS_JOB_NAME="${TRAVIS_PYTHON_VERSION} (LANG=$LANG)" CODECOV_ENV=TRAVIS_JOB_NAME
- tox -e coverage,codecov
This diff is collapsed.
......@@ -54,4 +54,4 @@ Code of Conduct
Everyone interacting in the setuptools project's codebases, issue trackers,
chat rooms, and mailing lists is expected to follow the
`PyPA Code of Conduct <https://www.pypa.io/en/latest/code-of-conduct/>`_.
`PSF Code of Conduct <https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md>`_.
import sys
import os
import re
import importlib
import warnings
is_pypy = '__pypy__' in sys.builtin_module_names
def warn_distutils_present():
if 'distutils' not in sys.modules:
return
if is_pypy and sys.version_info < (3, 7):
# PyPy for 3.6 unconditionally imports distutils, so bypass the warning
# https://foss.heptapod.net/pypy/pypy/-/blob/be829135bc0d758997b3566062999ee8b23872b4/lib-python/3/site.py#L250
return
warnings.warn(
"Distutils was imported before Setuptools, but importing Setuptools "
"also replaces the `distutils` module in `sys.modules`. This may lead "
"to undesirable behaviors or errors. To avoid these issues, avoid "
"using distutils directly, ensure that setuptools is installed in the "
"traditional way (e.g. not an editable install), and/or make sure "
"that setuptools is always imported before distutils.")
def clear_distutils():
if 'distutils' not in sys.modules:
return
warnings.warn("Setuptools is replacing distutils.")
mods = [name for name in sys.modules if re.match(r'distutils\b', name)]
for name in mods:
del sys.modules[name]
def enabled():
"""
Allow selection of distutils by environment variable.
"""
which = os.environ.get('SETUPTOOLS_USE_DISTUTILS', 'stdlib')
return which == 'local'
def ensure_local_distutils():
clear_distutils()
distutils = importlib.import_module('setuptools._distutils')
distutils.__name__ = 'distutils'
sys.modules['distutils'] = distutils
# sanity check that submodules load as expected
core = importlib.import_module('distutils.core')
assert '_distutils' in core.__file__, core.__file__
def do_override():
"""
Ensure that the local copy of distutils is preferred over stdlib.
See https://github.com/pypa/setuptools/issues/417#issuecomment-392298401
for more motivation.
"""
if enabled():
warn_distutils_present()
ensure_local_distutils()
class DistutilsMetaFinder:
def find_spec(self, fullname, path, target=None):
if path is not None:
return
method_name = 'spec_for_{fullname}'.format(**locals())
method = getattr(self, method_name, lambda: None)
return method()
def spec_for_distutils(self):
import importlib.abc
import importlib.util
class DistutilsLoader(importlib.abc.Loader):
def create_module(self, spec):
return importlib.import_module('setuptools._distutils')
def exec_module(self, module):
pass
return importlib.util.spec_from_loader('distutils', DistutilsLoader())
def spec_for_pip(self):
"""
Ensure stdlib distutils when running under pip.
See pypa/pip#8761 for rationale.
"""
if self.pip_imported_during_build():
return
clear_distutils()
self.spec_for_distutils = lambda: None
@staticmethod
def pip_imported_during_build():
"""
Detect if pip is being imported in a build script. Ref #2355.
"""
import traceback
return any(
frame.f_globals['__file__'].endswith('setup.py')
for frame, line in traceback.walk_stack(None)
)
DISTUTILS_FINDER = DistutilsMetaFinder()
def add_shim():
sys.meta_path.insert(0, DISTUTILS_FINDER)
def remove_shim():
try:
sys.meta_path.remove(DISTUTILS_FINDER)
except ValueError:
pass
__import__('_distutils_hack').do_override()
......@@ -8,16 +8,22 @@ environment:
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
APPVEYOR_JOB_NAME: "python36-x64-vs2015"
PYTHON: "C:\\Python36-x64"
APPVEYOR_JOB_NAME: "Python38-x64-vs2015"
PYTHON: "C:\\Python38-x64"
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
APPVEYOR_JOB_NAME: "python36-x64-vs2017"
PYTHON: "C:\\Python36-x64"
APPVEYOR_JOB_NAME: "Python38-x64-vs2017"
PYTHON: "C:\\Python38-x64"
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
APPVEYOR_JOB_NAME: "python36-x64-vs2019"
PYTHON: "C:\\Python36-x64"
APPVEYOR_JOB_NAME: "Python38-x64-vs2019"
PYTHON: "C:\\Python38-x64"
- APPVEYOR_JOB_NAME: "python37-x64"
PYTHON: "C:\\Python37-x64"
- APPVEYOR_JOB_NAME: "python36-x64"
PYTHON: "C:\\Python36-x64"
- APPVEYOR_JOB_NAME: "python35-x64"
PYTHON: "C:\\Python35-x64"
PYTEST_ADDOPTS: "--cov"
TOX_TESTENV_PASSENV: "PYTEST_ADDOPTS"
install:
# symlink python from a directory with a space
......@@ -35,9 +41,14 @@ test_script:
- python -m pip install --disable-pip-version-check --upgrade pip setuptools wheel
- pip install --upgrade tox tox-venv virtualenv
- pip freeze --all
- tox -- --cov
- tox -- --junit-xml=test-results.xml
after_test:
- tox -e coverage,codecov
on_finish:
- ps: |
$wc = New-Object 'System.Net.WebClient'
$wc.UploadFile("https://ci.appveyor.com/api/testresults/junit/$($env:APPVEYOR_JOB_ID)", (Resolve-Path .\test-results.xml))
version: '{build}'
......@@ -76,6 +76,7 @@ stages:
env:
TWINE_PASSWORD: $(PyPI-token)
TIDELIFT_TOKEN: $(Tidelift-token)
GITHUB_TOKEN: $(Github-token)
displayName: 'publish to PyPI'
condition: contains(variables['Build.SourceBranch'], 'tags')
......@@ -5,8 +5,6 @@ environment by creating a minimal egg-info directory and then invoking the
egg-info command to flesh out the egg-info directory.
"""
from __future__ import unicode_literals
import os
import sys
import textwrap
......
Fixed a typo in Sphinx docs that made docs dev section disappear
as a result of PR #2426 -- by :user:`webknjaz`
Fixed inconsistent RST title nesting levels caused by #2399
-- by :user:`webknjaz`
.. _Adding change notes with your PRs:
Adding change notes with your PRs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It is very important to maintain a log for news of how
updating to the new version of the software will affect
end-users. This is why we enforce collection of the change
fragment files in pull requests as per `Towncrier philosophy`_.
The idea is that when somebody makes a change, they must record
the bits that would affect end-users only including information
that would be useful to them. Then, when the maintainers publish
a new release, they'll automatically use these records to compose
a change log for the respective version. It is important to
understand that including unnecessary low-level implementation
related details generates noise that is not particularly useful
to the end-users most of the time. And so such details should be
recorded in the Git history rather than a changelog.
Alright! So how to add a news fragment?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
``setuptools`` uses `towncrier <https://pypi.org/project/towncrier/>`_
for changelog management.
To submit a change note about your PR, add a text file into the
``changelog.d/`` folder. It should contain an
explanation of what applying this PR will change in the way
end-users interact with the project. One sentence is usually
enough but feel free to add as many details as you feel necessary
for the users to understand what it means.
**Use the past tense** for the text in your fragment because,
combined with others, it will be a part of the "news digest"
telling the readers **what changed** in a specific version of
the library *since the previous version*. You should also use
reStructuredText syntax for highlighting code (inline or block),
linking parts of the docs or external sites.
If you wish to sign your change, feel free to add ``-- by
:user:`github-username``` at the end (replace ``github-username``
with your own!).
Finally, name your file following the convention that Towncrier
understands: it should start with the number of an issue or a
PR followed by a dot, then add a patch type, like ``change``,
``doc``, ``misc`` etc., and add ``.rst`` as a suffix. If you
need to add more than one fragment, you may add an optional
sequence number (delimited with another period) between the type
and the suffix.
In general the name will follow ``<pr_number>.<category>.rst`` pattern,
where the categories are:
- ``change``: Any backwards compatible code change
- ``breaking``: Any backwards-compatibility breaking change
- ``doc``: A change to the documentation
- ``misc``: Changes internal to the repo like CI, test and build changes
- ``deprecation``: For deprecations of an existing feature or behavior
A pull request may have more than one of these components, for example
a code change may introduce a new feature that deprecates an old
feature, in which case two fragments should be added. It is not
necessary to make a separate documentation fragment for documentation
changes accompanying the relevant code changes.
Examples for adding changelog entries to your Pull Requests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File :file:`changelog.d/2395.doc.1.rst`:
.. code-block:: rst
Added a ``:user:`` role to Sphinx config -- by :user:`webknjaz`
File :file:`changelog.d/1354.misc.rst`:
.. code-block:: rst
Added ``towncrier`` for changelog managment -- by :user:`pganssle`
File :file:`changelog.d/2355.change.rst`:
.. code-block:: rst
When pip is imported as part of a build, leave :py:mod:`distutils`
patched -- by :user:`jaraco`
.. tip::
See :file:`pyproject.toml` for all available categories
(``tool.towncrier.type``).
.. _Towncrier philosophy:
https://towncrier.readthedocs.io/en/actual-freaking-docs/#philosophy
......@@ -14,13 +14,16 @@ def pytest_addoption(parser):
collect_ignore = [
'tests/manual_test.py',
'setuptools/tests/mod_with_constant.py',
'setuptools/_distutils',
'_distutils_hack',
'setuptools/extern',
'pkg_resources/extern',
'pkg_resources/tests/data',
'setuptools/_vendor',
'pkg_resources/_vendor',
]
if sys.version_info < (3,):
collect_ignore.append('setuptools/lib2to3_ex.py')
collect_ignore.append('setuptools/_imp.py')
if sys.version_info < (3, 6):
collect_ignore.append('docs/conf.py') # uses f-strings
collect_ignore.append('pavement.py')
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html web pickle htmlhelp latex changes linkcheck
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " changes to make an overview over all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"
clean:
-rm -rf build/*
html:
mkdir -p build/html build/doctrees
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) build/html
@echo
@echo "Build finished. The HTML pages are in build/html."
pickle:
mkdir -p build/pickle build/doctrees
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) build/pickle
@echo
@echo "Build finished; now you can process the pickle files."
web: pickle
json:
mkdir -p build/json build/doctrees
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) build/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
mkdir -p build/htmlhelp build/doctrees
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) build/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in build/htmlhelp."
latex:
mkdir -p build/latex build/doctrees
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) build/latex
@echo
@echo "Build finished; the LaTeX files are in build/latex."
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
"run these through (pdf)latex."
changes:
mkdir -p build/changes build/doctrees
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) build/changes
@echo
@echo "The overview file is in build/changes."
linkcheck:
mkdir -p build/linkcheck build/doctrees
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) build/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in build/linkcheck/output.txt."
......@@ -56,7 +56,8 @@ setuptools, the content would be::
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
Use ``setuptools``' `declarative config`_ to specify the package information::
Use ``setuptools``' :ref:`declarative config <declarative config>` to
specify the package information::
[metadata]
name = meowpkg
......
......@@ -10,23 +10,25 @@ import os
cwd=os.path.join(os.path.dirname(__file__), os.path.pardir),
)
# -- Project information -----------------------------------------------------
github_url = 'https://github.com'
github_sponsors_url = f'{github_url}/sponsors'
# -- General configuration --
extensions = ['jaraco.packaging.sphinx', 'rst.linker']
extensions = [
'sphinx.ext.extlinks', # allows to create custom roles easily
'jaraco.packaging.sphinx',
'rst.linker',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.txt'
# The master toctree document.
master_doc = 'index'
# A list of glob-style patterns that should be excluded
# when looking for source files.
exclude_patterns = ['requirements.txt']
# List of directories, relative to source directory, that shouldn't be searched
# for source files.
exclude_trees = []
......@@ -34,6 +36,11 @@ exclude_trees = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# -- Options for extlinks extension ---------------------------------------
extlinks = {
'user': (f'{github_sponsors_url}/%s', '@'), # noqa: WPS323
}
# -- Options for HTML output --
# The theme to use for HTML and HTML Help pages. Major themes that come with
......@@ -101,7 +108,7 @@ link_files = {
url='http://bugs.jython.org/issue{jython}',
),
dict(
pattern=r'Python #(?P<python>\d+)',
pattern=r'(Python #|bpo-)(?P<python>\d+)',
url='http://bugs.python.org/issue{python}',
),
dict(
......@@ -128,6 +135,10 @@ link_files = {
pattern=r'setuptools_svn #(?P<setuptools_svn>\d+)',
url='{GH}/jaraco/setuptools_svn/issues/{setuptools_svn}',
),
dict(
pattern=r'pypa/distutils#(?P<distutils>\d+)',
url='{GH}/pypa/distutils/issues/{distutils}',
),
dict(
pattern=r'^(?m)((?P<scm_version>v?\d+(\.\d+){1,2}))\n[-=]+\n',
with_scm='{text}\n{rev[timestamp]:%d %b %Y}\n',
......@@ -135,3 +146,12 @@ link_files = {
],
),
}
# Be strict about any broken references:
nitpicky = True
# Ref: https://github.com/python-attrs/attrs/pull/571/files\
# #diff-85987f48f1258d9ee486e3191495582dR82
default_role = 'any'
Porting from Distutils
======================
Setuptools and the PyPA have a `stated goal <https://github.com/pypa/packaging-problems/issues/127>`_ to make Setuptools the reference API for distutils.
Since the 49.1.2 release, Setuptools includes a local, vendored copy of distutils (from late copies of CPython) that is disabled by default. To enable the use of this copy of distutils when invoking setuptools, set the enviroment variable:
SETUPTOOLS_USE_DISTUTILS=local
This behavior is planned to become the default.
Prefer Setuptools
-----------------
As Distutils is deprecated, any usage of functions or objects from distutils is similarly discouraged, and Setuptools aims to replace or deprecate all such uses. This section describes the recommended replacements.
``distutils.core.setup`` → ``setuptools.setup``
``distutils.cmd.Command`` → ``setuptools.Command``
``distutils.log`` → (no replacement yet)
``distutils.version.*`` → ``packaging.version.*``
If a project relies on uses of ``distutils`` that do not have a suitable replacement above, please search the `Setuptools issue tracker <https://github.com/pypa/setuptools/issues/>`_ and file a request, describing the use-case so that Setuptools' maintainers can investigate. Please provide enough detail to help the maintainers understand how distutils is used, what value it provides, and why that behavior should be supported.
"Eggsecutable" Scripts
----------------------
.. deprecated:: 45.3.0
Occasionally, there are situations where it's desirable to make an ``.egg``
file directly executable. You can do this by including an entry point such
as the following::
setup(
# other arguments here...
entry_points={
"setuptools.installation": [
"eggsecutable = my_package.some_module:main_func",
]
}
)
Any eggs built from the above setup script will include a short executable
prelude that imports and calls ``main_func()`` from ``my_package.some_module``.
The prelude can be run on Unix-like platforms (including Mac and Linux) by
invoking the egg with ``/bin/sh``, or by enabling execute permissions on the
``.egg`` file. For the executable prelude to run, the appropriate version of
Python must be available via the ``PATH`` environment variable, under its
"long" name. That is, if the egg is built for Python 2.3, there must be a
``python2.3`` executable present in a directory on ``PATH``.
IMPORTANT NOTE: Eggs with an "eggsecutable" header cannot be renamed, or
invoked via symlinks. They *must* be invoked using their original filename, in
order to ensure that, once running, ``pkg_resources`` will know what project
and version is in use. The header script will check this and exit with an
error if the ``.egg`` file has been renamed or is invoked via a symlink that
changes its base name.
\ No newline at end of file
======================================================
Guides on backward compatibility & deprecated practice
======================================================
``Setuptools`` has undergone tremendous changes since its first debut. As its
development continues to roll forward, many of the practice and mechanisms it
had established are now considered deprecated. But they still remain relevant
as a plethora of libraries continue to depend on them. Many people also find
it necessary to equip themselves with the knowledge to better support backward
compatibility. This guide aims to provide the essential information for such
objectives.
.. toctree::
:maxdepth: 1
python3
python_eggs
easy_install
distutils-legacy
functionalities
......@@ -61,33 +61,14 @@ jump to the in-depth discussion about any subject referenced.
Making a pull request
---------------------
When making a pull request, please include a short summary of the changes
and a reference to any issue tickets that the PR is intended to solve.
All PRs with code changes should include tests. All changes should include a
changelog entry.
``setuptools`` uses `towncrier <https://pypi.org/project/towncrier/>`_
for changelog management, so when making a PR, please add a news fragment in the
``changelog.d/`` folder. Changelog files are written in reStructuredText and
should be a 1 or 2 sentence description of the substantive changes in the PR.
They should be named ``<pr_number>.<category>.rst``, where the categories are:
- ``change``: Any backwards compatible code change
- ``breaking``: Any backwards-compatibility breaking change
- ``doc``: A change to the documentation
- ``misc``: Changes internal to the repo like CI, test and build changes
- ``deprecation``: For deprecations of an existing feature or behavior
A pull request may have more than one of these components, for example a code
change may introduce a new feature that deprecates an old feature, in which
case two fragments should be added. It is not necessary to make a separate
documentation fragment for documentation changes accompanying the relevant
code changes. See the following for an example news fragment:
.. code-block:: bash
$ cat changelog.d/1288.change.rst
Add support for maintainer in PKG-INFO
When making a pull request, please
:ref:`include a short summary of the changes <Adding change notes
with your PRs>` and a reference to any issue tickets that the PR is
intended to solve.
All PRs with code changes should include tests. All changes should
include a changelog entry.
.. include:: ../../changelog.d/README.rst
-------------------
Auto-Merge Requests
......@@ -145,5 +126,5 @@ setuptools from source. Eventually, this limitation may be lifted as
PEP 517/518 reach ubiquitous adoption, but for now, Setuptools
cannot declare dependencies other than through
``setuptools/_vendor/vendored.txt`` and
``pkg_reosurces/_vendor/vendored.txt`` and refreshed by way of
``pkg_resources/_vendor/vendored.txt`` and refreshed by way of
``paver update_vendored`` (pavement.py).
......@@ -31,5 +31,4 @@ setuptools changes. You have been warned.
:maxdepth: 1
developer-guide
formats
releases
Documentation
=============
Setuptools is a fully-featured, actively-maintained, and stable library
designed to facilitate packaging Python projects.
Documentation content:
.. toctree::
:maxdepth: 1
User guide <userguide/index>
build_meta
pkg_resources
references/keywords
roadmap
setuptools
Development guide <development/index>
Backward compatibility & deprecated practice <deprecated/index>
Changelog <history>
Welcome to Setuptools' documentation!
=====================================
Setuptools is a fully-featured, actively-maintained, and stable library
designed to facilitate packaging Python projects, where packaging includes:
- Python package and module definitions
- Distribution package metadata
- Test hooks
- Project installation
- Platform-specific details
- Python 3 support
Documentation content:
.. toctree::
:maxdepth: 2
setuptools
pkg_resources
python3
development
roadmap
Deprecated: Easy Install <easy_install>
history
========
Keywords
========
``name``
A string specifying the name of the package.
......@@ -189,7 +193,7 @@
discovery of services or plugins provided by a project. See :ref:`Dynamic
Discovery of Services and Plugins` for details and examples of the format
of this argument. In addition, this keyword is used to support
:ref:`Automatic Script Creation`.
:ref:`Automatic Script Creation <entry_points>`.
``extras_require``
A dictionary mapping names of "extras" (optional features of your project)
......@@ -282,7 +286,7 @@
this argument. The named class must be instantiable with no arguments, and
its instances must support the ``loadTestsFromNames()`` method as defined
in the Python ``unittest`` module's ``TestLoader`` class. Setuptools will
pass only one test "name" in the `names` argument: the value supplied for
pass only one test "name" in the ``names`` argument: the value supplied for
the ``test_suite`` argument. The loader you specify may interpret this
string in any way it likes, as there are no restrictions on what may be
contained in a ``test_suite`` string.
......@@ -317,15 +321,15 @@
``use_2to3``
Convert the source code from Python 2 to Python 3 with 2to3 during the
build process. See :doc:`python3` for more details.
build process. See :doc:`../deprecated/python3` for more details.
``convert_2to3_doctests``
List of doctest source files that need to be converted with 2to3.
See :doc:`python3` for more details.
See :doc:`../deprecated/python3` for more details.
``use_2to3_fixers``
A list of modules to search for additional fixers to be used during
the 2to3 conversion. See :doc:`python3` for more details.
the 2to3 conversion. See :doc:`../deprecated/python3` for more details.
``use_2to3_exclude_fixers``
List of fixer names to be skipped.
......
==================================================
Building and Distributing Packages with Setuptools
==================================================
``Setuptools`` is a collection of enhancements to the Python ``distutils``
that allow developers to more easily build and
distribute Python packages, especially ones that have dependencies on other
packages.
Packages built and distributed using ``setuptools`` look to the user like
ordinary Python packages based on the ``distutils``.
Feature Highlights:
* Create `Python Eggs <http://peak.telecommunity.com/DevCenter/PythonEggs>`_ -
a single-file importable distribution format
* Enhanced support for accessing data files hosted in zipped packages.
* Automatically include all packages in your source tree, without listing them
individually in setup.py
* Automatically include all relevant files in your source distributions,
without needing to create a ``MANIFEST.in`` file, and without having to force
regeneration of the ``MANIFEST`` file when your source tree changes.
* Automatically generate wrapper scripts or Windows (console and GUI) .exe
files for any number of "main" functions in your project. (Note: this is not
a py2exe replacement; the .exe files rely on the local Python installation.)
* Transparent Cython support, so that your setup.py can list ``.pyx`` files and
still work even when the end-user doesn't have Cython installed (as long as
you include the Cython-generated C in your source distribution)
* Command aliases - create project-specific, per-user, or site-wide shortcut
names for commonly used commands and options
* Deploy your project in "development mode", such that it's available on
``sys.path``, yet can still be edited directly from its source checkout.
* Easily extend the distutils with new commands or ``setup()`` arguments, and
distribute/reuse your extensions for multiple projects, without copying code.
* Create extensible applications and frameworks that automatically discover
extensions, using simple "entry points" declared in a project's setup script.
* Full support for PEP 420 via ``find_namespace_packages()``, which is also backwards
compatible to the existing ``find_packages()`` for Python >= 3.3.
.. contents:: **Table of Contents**
-----------------
Developer's Guide
-----------------
TRANSITIONAL NOTE
~~~~~~~~~~~~~~~~~
Setuptools automatically calls ``declare_namespace()`` for you at runtime,
but future versions may *not*. This is because the automatic declaration
feature has some negative side effects, such as needing to import all namespace
packages during the initialization of the ``pkg_resources`` runtime, and also
the need for ``pkg_resources`` to be explicitly imported before any namespace
packages work at all. In some future releases, you'll be responsible
for including your own declaration lines, and the automatic declaration feature
will be dropped to get rid of the negative side effects.
During the remainder of the current development cycle, therefore, setuptools
will warn you about missing ``declare_namespace()`` calls in your
``__init__.py`` files, and you should correct these as soon as possible
before the compatibility support is removed.
Namespace packages without declaration lines will not work
correctly once a user has upgraded to a later version, so it's important that
you make this change now in order to avoid having your code break in the field.
Our apologies for the inconvenience, and thank you for your patience.
setup.cfg-only projects
=======================
.. versionadded:: 40.9.0
If ``setup.py`` is missing from the project directory when a :pep:`517`
build is invoked, ``setuptools`` emulates a dummy ``setup.py`` file containing
only a ``setuptools.setup()`` call.
.. note::
:pep:`517` doesn't support editable installs so this is currently
incompatible with ``pip install -e .``, as :pep:`517` does not support editable installs.
This means that you can have a Python project with all build configuration
specified in ``setup.cfg``, without a ``setup.py`` file, if you **can rely
on** your project always being built by a :pep:`517`/:pep:`518` compatible
frontend.
To use this feature:
* Specify build requirements and :pep:`517` build backend in
``pyproject.toml``.
For example:
.. code-block:: toml
[build-system]
requires = [
"setuptools >= 40.9.0",
"wheel",
]
build-backend = "setuptools.build_meta"
* Use a :pep:`517` compatible build frontend, such as ``pip >= 19`` or ``pep517``.
.. warning::
As :pep:`517` is new, support is not universal, and frontends that
do support it may still have bugs. For compatibility, you may want to
put a ``setup.py`` file containing only a ``setuptools.setup()``
invocation.
Configuration API
=================
Some automation tools may wish to access data from a configuration file.
``Setuptools`` exposes a ``read_configuration()`` function for
parsing ``metadata`` and ``options`` sections into a dictionary.
.. code-block:: python
from setuptools.config import read_configuration
conf_dict = read_configuration("/home/user/dev/package/setup.cfg")
By default, ``read_configuration()`` will read only the file provided
in the first argument. To include values from other configuration files
which could be in various places, set the ``find_others`` keyword argument
to ``True``.
If you have only a configuration file but not the whole package, you can still
try to get data out of it with the help of the ``ignore_option_errors`` keyword
argument. When it is set to ``True``, all options with errors possibly produced
by directives, such as ``attr:`` and others, will be silently ignored.
As a consequence, the resulting dictionary will include no such options.
Mailing List and Bug Tracker
============================
Please use the `distutils-sig mailing list`_ for questions and discussion about
setuptools, and the `setuptools bug tracker`_ ONLY for issues you have
confirmed via the list are actual bugs, and which you have reduced to a minimal
set of steps to reproduce.
.. _distutils-sig mailing list: http://mail.python.org/pipermail/distutils-sig/
.. _setuptools bug tracker: https://github.com/pypa/setuptools/
This source diff could not be displayed because it is too large. You can view the blob instead.
This diff is collapsed.
====================
Data Files Support
====================
The distutils have traditionally allowed installation of "data files", which
are placed in a platform-specific location. However, the most common use case
for data files distributed with a package is for use *by* the package, usually
by including the data files in the package directory.
Setuptools offers three ways to specify data files to be included in your
packages. First, you can simply use the ``include_package_data`` keyword,
e.g.::
from setuptools import setup, find_packages
setup(
...
include_package_data=True
)
This tells setuptools to install any data files it finds in your packages.
The data files must be specified via the distutils' ``MANIFEST.in`` file.
(They can also be tracked by a revision control system, using an appropriate
plugin. See the section below on :ref:`Adding Support for Revision
Control Systems` for information on how to write such plugins.)
If you want finer-grained control over what files are included (for example,
if you have documentation files in your package directories and want to exclude
them from installation), then you can also use the ``package_data`` keyword,
e.g.::
from setuptools import setup, find_packages
setup(
...
package_data={
# If any package contains *.txt or *.rst files, include them:
"": ["*.txt", "*.rst"],
# And include any *.msg files found in the "hello" package, too:
"hello": ["*.msg"],
}
)
The ``package_data`` argument is a dictionary that maps from package names to
lists of glob patterns. The globs may include subdirectory names, if the data
files are contained in a subdirectory of the package. For example, if the
package tree looks like this::
setup.py
src/
mypkg/
__init__.py
mypkg.txt
data/
somefile.dat
otherdata.dat
The setuptools setup file might look like this::
from setuptools import setup, find_packages
setup(
...
packages=find_packages("src"), # include all packages under src
package_dir={"": "src"}, # tell distutils packages are under src
package_data={
# If any package contains *.txt files, include them:
"": ["*.txt"],
# And include any *.dat files found in the "data" subdirectory
# of the "mypkg" package, also:
"mypkg": ["data/*.dat"],
}
)
Notice that if you list patterns in ``package_data`` under the empty string,
these patterns are used to find files in every package, even ones that also
have their own patterns listed. Thus, in the above example, the ``mypkg.txt``
file gets included even though it's not listed in the patterns for ``mypkg``.
Also notice that if you use paths, you *must* use a forward slash (``/``) as
the path separator, even if you are on Windows. Setuptools automatically
converts slashes to appropriate platform-specific separators at build time.
If datafiles are contained in a subdirectory of a package that isn't a package
itself (no ``__init__.py``), then the subdirectory names (or ``*``) are required
in the ``package_data`` argument (as shown above with ``"data/*.dat"``).
When building an ``sdist``, the datafiles are also drawn from the
``package_name.egg-info/SOURCES.txt`` file, so make sure that this is removed if
the ``setup.py`` ``package_data`` list is updated before calling ``setup.py``.
(Note: although the ``package_data`` argument was previously only available in
``setuptools``, it was also added to the Python ``distutils`` package as of
Python 2.4; there is `some documentation for the feature`__ available on the
python.org website. If using the setuptools-specific ``include_package_data``
argument, files specified by ``package_data`` will *not* be automatically
added to the manifest unless they are listed in the MANIFEST.in file.)
__ https://docs.python.org/3/distutils/setupscript.html#installing-package-data
Sometimes, the ``include_package_data`` or ``package_data`` options alone
aren't sufficient to precisely define what files you want included. For
example, you may want to include package README files in your revision control
system and source distributions, but exclude them from being installed. So,
setuptools offers an ``exclude_package_data`` option as well, that allows you
to do things like this::
from setuptools import setup, find_packages
setup(
...
packages=find_packages("src"), # include all packages under src
package_dir={"": "src"}, # tell distutils packages are under src
include_package_data=True, # include everything in source control
# ...but exclude README.txt from all packages
exclude_package_data={"": ["README.txt"]},
)
The ``exclude_package_data`` option is a dictionary mapping package names to
lists of wildcard patterns, just like the ``package_data`` option. And, just
as with that option, a key of ``""`` will apply the given pattern(s) to all
packages. However, any files that match these patterns will be *excluded*
from installation, even if they were listed in ``package_data`` or were
included as a result of using ``include_package_data``.
In summary, the three options allow you to:
``include_package_data``
Accept all data files and directories matched by ``MANIFEST.in``.
``package_data``
Specify additional patterns to match files that may or may
not be matched by ``MANIFEST.in`` or found in source control.
``exclude_package_data``
Specify patterns for data files and directories that should *not* be
included when a package is installed, even if they would otherwise have
been included due to the use of the preceding options.
NOTE: Due to the way the distutils build process works, a data file that you
include in your project and then stop including may be "orphaned" in your
project's build directories, requiring you to run ``setup.py clean --all`` to
fully remove them. This may also be important for your users and contributors
if they track intermediate revisions of your project using Subversion; be sure
to let them know when you make changes that remove files from inclusion so they
can run ``setup.py clean --all``.
.. _Accessing Data Files at Runtime:
Accessing Data Files at Runtime
-------------------------------
Typically, existing programs manipulate a package's ``__file__`` attribute in
order to find the location of data files. However, this manipulation isn't
compatible with PEP 302-based import hooks, including importing from zip files
and Python Eggs. It is strongly recommended that, if you are using data files,
you should use the :ref:`ResourceManager API` of ``pkg_resources`` to access
them. The ``pkg_resources`` module is distributed as part of setuptools, so if
you're using setuptools to distribute your package, there is no reason not to
use its resource management API. See also `Importlib Resources`_ for
a quick example of converting code that uses ``__file__`` to use
``pkg_resources`` instead.
.. _Importlib Resources: https://docs.python.org/3/library/importlib.html#module-importlib.resources
Non-Package Data Files
----------------------
Historically, ``setuptools`` by way of ``easy_install`` would encapsulate data
files from the distribution into the egg (see `the old docs
<https://github.com/pypa/setuptools/blob/52aacd5b276fedd6849c3a648a0014f5da563e93/docs/setuptools.txt#L970-L1001>`_). As eggs are deprecated and pip-based installs
fall back to the platform-specific location for installing data files, there is
no supported facility to reliably retrieve these resources.
Instead, the PyPA recommends that any data files you wish to be accessible at
run time be included in the package.
.. _declarative config:
-----------------------------------------
Configuring setup() using setup.cfg files
-----------------------------------------
.. note:: New in 30.3.0 (8 Dec 2016).
.. important::
If compatibility with legacy builds (i.e. those not using the :pep:`517`
build API) is desired, a ``setup.py`` file containing a ``setup()`` function
call is still required even if your configuration resides in ``setup.cfg``.
``Setuptools`` allows using configuration files (usually :file:`setup.cfg`)
to define a package’s metadata and other options that are normally supplied
to the ``setup()`` function (declarative config).
This approach not only allows automation scenarios but also reduces
boilerplate code in some cases.
.. note::
This implementation has limited compatibility with the distutils2-like
``setup.cfg`` sections used by the ``pbr`` and ``d2to1`` packages.
Namely: only metadata-related keys from ``metadata`` section are supported
(except for ``description-file``); keys from ``files``, ``entry_points``
and ``backwards_compat`` are not supported.
.. code-block:: ini
[metadata]
name = my_package
version = attr: src.VERSION
description = My package description
long_description = file: README.rst, CHANGELOG.rst, LICENSE.rst
keywords = one, two
license = BSD 3-Clause License
classifiers =
Framework :: Django
License :: OSI Approved :: BSD License
Programming Language :: Python :: 3
Programming Language :: Python :: 3.5
[options]
zip_safe = False
include_package_data = True
packages = find:
scripts =
bin/first.py
bin/second.py
install_requires =
requests
importlib; python_version == "2.6"
[options.package_data]
* = *.txt, *.rst
hello = *.msg
[options.extras_require]
pdf = ReportLab>=1.2; RXP
rest = docutils>=0.3; pack ==1.1, ==1.3
[options.packages.find]
exclude =
src.subpackage1
src.subpackage2
[options.data_files]
/etc/my_package =
site.d/00_default.conf
host.d/00_default.conf
data = data/img/logo.png, data/svg/icon.svg
Metadata and options are set in the config sections of the same name.
* Keys are the same as the keyword arguments one provides to the ``setup()``
function.
* Complex values can be written comma-separated or placed one per line
in *dangling* config values. The following are equivalent:
.. code-block:: ini
[metadata]
keywords = one, two
[metadata]
keywords =
one
two
* In some cases, complex values can be provided in dedicated subsections for
clarity.
* Some keys allow ``file:``, ``attr:``, ``find:``, and ``find_namespace:`` directives in
order to cover common usecases.
* Unknown keys are ignored.
Using a ``src/`` layout
=======================
One commonly used package configuration has all the module source code in a
subdirectory (often called the ``src/`` layout), like this::
├── src
│   └── mypackage
│   ├── __init__.py
│   └── mod1.py
├── setup.py
└── setup.cfg
You can set up your ``setup.cfg`` to automatically find all your packages in
the subdirectory like this:
.. code-block:: ini
# This example contains just the necessary options for a src-layout, set up
# the rest of the file as described above.
[options]
package_dir=
=src
packages=find:
[options.packages.find]
where=src
Specifying values
=================
Some values are treated as simple strings, some allow more logic.
Type names used below:
* ``str`` - simple string
* ``list-comma`` - dangling list or string of comma-separated values
* ``list-semi`` - dangling list or string of semicolon-separated values
* ``bool`` - ``True`` is 1, yes, true
* ``dict`` - list-comma where keys are separated from values by ``=``
* ``section`` - values are read from a dedicated (sub)section
Special directives:
* ``attr:`` - Value is read from a module attribute. ``attr:`` supports
callables and iterables; unsupported types are cast using ``str()``.
In order to support the common case of a literal value assigned to a variable
in a module containing (directly or indirectly) third-party imports,
``attr:`` first tries to read the value from the module by examining the
module's AST. If that fails, ``attr:`` falls back to importing the module.
* ``file:`` - Value is read from a list of files and then concatenated
.. note::
The ``file:`` directive is sandboxed and won't reach anything outside
the directory containing ``setup.py``.
Metadata
--------
.. note::
The aliases given below are supported for compatibility reasons,
but their use is not advised.
============================== ================= ================= =============== =====
Key Aliases Type Minimum Version Notes
============================== ================= ================= =============== =====
name str
version attr:, file:, str 39.2.0 (1)
url home-page str
download_url download-url str
project_urls dict 38.3.0
author str
author_email author-email str
maintainer str
maintainer_email maintainer-email str
classifiers classifier file:, list-comma
license str
license_file str
license_files list-comma
description summary file:, str
long_description long-description file:, str
long_description_content_type str 38.6.0
keywords list-comma
platforms platform list-comma
provides list-comma
requires list-comma
obsoletes list-comma
============================== ================= ================= =============== =====
.. note::
A version loaded using the ``file:`` directive must comply with PEP 440.
It is easy to accidentally put something other than a valid version
string in such a file, so validation is stricter in this case.
Notes:
1. The ``version`` file attribute has only been supported since 39.2.0.
Options
-------
======================= =================================== =============== =====
Key Type Minimum Version Notes
======================= =================================== =============== =====
zip_safe bool
setup_requires list-semi
install_requires list-semi
extras_require section
python_requires str
entry_points file:, section
use_2to3 bool
use_2to3_fixers list-comma
use_2to3_exclude_fixers list-comma
convert_2to3_doctests list-comma
scripts list-comma
eager_resources list-comma
dependency_links list-comma
tests_require list-semi
include_package_data bool
packages find:, find_namespace:, list-comma
package_dir dict
package_data section (1)
exclude_package_data section
namespace_packages list-comma
py_modules list-comma
data_files dict 40.6.0
======================= =================================== =============== =====
.. note::
**packages** - The ``find:`` and ``find_namespace:`` directive can be further configured
in a dedicated subsection ``options.packages.find``. This subsection
accepts the same keys as the ``setuptools.find_packages`` and the
``setuptools.find_namespace_packages`` function:
``where``, ``include``, and ``exclude``.
**find_namespace directive** - The ``find_namespace:`` directive is supported since Python >=3.3.
Notes:
1. In the ``package_data`` section, a key named with a single asterisk (``*``)
refers to all packages, in lieu of the empty string used in ``setup.py``.
=====================================
Dependencies Management in Setuptools
=====================================
There are three types of dependency styles offered by setuptools:
1) build system requirement, required dependency and 3) optional
dependency.
.. Note::
Packages that are added to dependency can be optionally specified with the
version by following `PEP 440 <https://www.python.org/dev/peps/pep-0440/>`_
.. contents::
Build system requirement
========================
Package requirement
-------------------
After organizing all the scripts and files and getting ready for packaging,
there needs to be a way to tell Python what programs it need to actually
do the packgaging (in our case, ``setuptools`` of course). Usually,
you also need the ``wheel`` package as well since it is recommended that you
upload a ``.whl`` file to PyPI alongside your ``.tar.gz`` file. Unlike the
other two types of dependency keyword, this one is specified in your
``pyproject.toml`` file (if you have forgot what this is, go to
:doc:`quickstart` or (WIP)):
.. code-block:: ini
[build-system]
requires = ["setuptools", "wheel"]
#...
.. note::
This used to be accomplished with the ``setup_requires`` keyword but is
now considered deprecated in favor of the PEP 517 style described above.
To peek into how this legacy keyword is used, consult our :doc:`guide on
deprecated practice (WIP) <../deprecated/index>`
.. _Declaring Dependencies:
Declaring required dependency
=============================
This is where a package declares its core dependencies, without which it won't
be able to run. ``setuptools`` support automatically download and install
these dependencies when the package is installed. Although there is more
finess to it, let's start with a simple example.
.. code-block:: ini
[options]
#...
install_requires =
docutils
BazSpam ==1.1
.. code-block:: python
setup(
#...,
install_requires = [
'docutils',
'BazSpam ==1.1'
]
)
When your project is installed (e.g. using pip), all of the dependencies not
already installed will be located (via PyPI), downloaded, built (if necessary),
and installed and 2) Any scripts in your project will be installed with wrappers
that verify the availability of the specified dependencies at runtime.
Platform specific dependencies
------------------------------
Setuptools offer the capability to evaluate certain conditions before blindly
installing everything listed in ``install_requires``. This is great for platform
specific dependencies. For example, the ``enum`` package was added in Python
3.4, therefore, package that depends on it can elect to install it only when
the Python version is older than 3.4. To accomplish this
.. code-block:: ini
[options]
#...
install_requires =
enum34;python_version<'3.4'
.. code-block:: python
setup(
#...
install_requires=[
"enum34;python_version<'3.4'",]
)
Similarly, if you also wish to declare ``pywin32`` with a minimal version of 1.0
and only install it if the user is using a Windows operating system:
.. code-block:: ini
[options]
#...
install_requires =
enum34;python_version<'3.4'
pywin32 >= 1.0;platform_system=='Windows'
.. code-block:: python
setup(
#...
install_requires=[
"enum34;python_version<'3.4'",
"pywin32 >= 1.0;platform_system=='Windows'"
]
)
The environmental markers that may be used for testing platform types are
detailed in `PEP 508 <https://www.python.org/dev/peps/pep-0508/>`_.
Dependencies that aren't in PyPI
--------------------------------
.. warning::
Dependency links support has been dropped by pip starting with version
19.0 (released 2019-01-22).
If your project depends on packages that don't exist on PyPI, you may still be
able to depend on them, as long as they are available for download as:
- an egg, in the standard distutils ``sdist`` format,
- a single ``.py`` file, or
- a VCS repository (Subversion, Mercurial, or Git).
You just need to add some URLs to the ``dependency_links`` argument to
``setup()``.
The URLs must be either:
1. direct download URLs,
2. the URLs of web pages that contain direct download links, or
3. the repository's URL
In general, it's better to link to web pages, because it is usually less
complex to update a web page than to release a new version of your project.
You can also use a SourceForge ``showfiles.php`` link in the case where a
package you depend on is distributed via SourceForge.
If you depend on a package that's distributed as a single ``.py`` file, you
must include an ``"#egg=project-version"`` suffix to the URL, to give a project
name and version number. (Be sure to escape any dashes in the name or version
by replacing them with underscores.) EasyInstall will recognize this suffix
and automatically create a trivial ``setup.py`` to wrap the single ``.py`` file
as an egg.
In the case of a VCS checkout, you should also append ``#egg=project-version``
in order to identify for what package that checkout should be used. You can
append ``@REV`` to the URL's path (before the fragment) to specify a revision.
Additionally, you can also force the VCS being used by prepending the URL with
a certain prefix. Currently available are:
- ``svn+URL`` for Subversion,
- ``git+URL`` for Git, and
- ``hg+URL`` for Mercurial
A more complete example would be:
``vcs+proto://host/path@revision#egg=project-version``
Be careful with the version. It should match the one inside the project files.
If you want to disregard the version, you have to omit it both in the
``requires`` and in the URL's fragment.
This will do a checkout (or a clone, in Git and Mercurial parlance) to a
temporary folder and run ``setup.py bdist_egg``.
The ``dependency_links`` option takes the form of a list of URL strings. For
example, this will cause a search of the specified page for eggs or source
distributions, if the package's dependencies aren't already installed:
.. code-block:: ini
[options]
#...
dependency_links = http://peak.telecommunity.com/snapshots/
.. code-block:: python
setup(
#...
dependency_links=[
"http://peak.telecommunity.com/snapshots/"
],
)
Optional dependencies
=====================
Setuptools allows you to declare dependencies that only get installed under
specific circumstances. These dependencies are specified with ``extras_require``
keyword and are only installed if another package depends on it (either
directly or indirectly) This makes it convenient to declare dependencies for
ancillary functions such as "tests" and "docs".
.. note::
``tests_require`` is now deprecated
For example, Package-A offers optional PDF support and requires two other
dependencies for it to work:
.. code-block:: ini
[metadata]
name = Package-A
[options.extras_require]
PDF = ReportLab>=1.2; RXP
.. code-block:: python
setup(
name="Project-A",
#...
extras_require={
"PDF": ["ReportLab>=1.2", "RXP"],
}
)
The name ``PDF`` is an arbitary identifier of such a list of dependencies, to
which other components can refer and have them installed. There are two common
use cases.
First is the console_scripts entry point:
.. code-block:: ini
[metadata]
name = Project A
#...
[options]
#...
entry_points=
[console_scripts]
rst2pdf = project_a.tools.pdfgen [PDF]
rst2html = project_a.tools.htmlgen
.. code-block:: python
setup(
name = "Project-A"
#...,
entry_points={
"console_scripts": [
"rst2pdf = project_a.tools.pdfgen [PDF]",
"rst2html = project_a.tools.htmlgen",
],
}
)
When the script ``rst2pdf`` is run, it will trigger the installation of
the two dependencies ``PDF`` maps to.
The second use case is that other package can use this "extra" for their
own dependencies. For example, if "Project-B" needs "project A" with PDF support
installed, it might declare the dependency like this:
.. code-block:: ini
[metadata]
name = Project-B
#...
[options]
#...
install_requires =
Project-A[PDF]
.. code-block:: python
setup(
name="Project-B",
install_requires=["Project-A[PDF]"],
...
)
This will cause ReportLab to be installed along with project A, if project B is
installed -- even if project A was already installed. In this way, a project
can encapsulate groups of optional "downstream dependencies" under a feature
name, so that packages that depend on it don't have to know what the downstream
dependencies are. If a later version of Project A builds in PDF support and
no longer needs ReportLab, or if it ends up needing other dependencies besides
ReportLab in order to provide PDF support, Project B's setup information does
not need to change, but the right packages will still be installed if needed.
.. note::
Best practice: if a project ends up not needing any other packages to
support a feature, it should keep an empty requirements list for that feature
in its ``extras_require`` argument, so that packages depending on that feature
don't break (due to an invalid feature name).
Python requirement
==================
In some cases, you might need to specify the minimum required python version.
This is handled with the ``python_requires`` keyword supplied to ``setup.cfg``
or ``setup.py``.
Example WIP
"Development Mode"
==================
Under normal circumstances, the ``distutils`` assume that you are going to
build a distribution of your project, not use it in its "raw" or "unbuilt"
form. If you were to use the ``distutils`` that way, you would have to rebuild
and reinstall your project every time you made a change to it during
development.
Another problem that sometimes comes up with the ``distutils`` is that you may
need to do development on two related projects at the same time. You may need
to put both projects' packages in the same directory to run them, but need to
keep them separate for revision control purposes. How can you do this?
Setuptools allows you to deploy your projects for use in a common directory or
staging area, but without copying any files. Thus, you can edit each project's
code in its checkout directory, and only need to run build commands when you
change a project's C extensions or similarly compiled files. You can even
deploy a project into another project's checkout directory, if that's your
preferred way of working (as opposed to using a common independent staging area
or the site-packages directory).
To do this, use the ``setup.py develop`` command. It works very similarly to
``setup.py install``, except that it doesn't actually install anything.
Instead, it creates a special ``.egg-link`` file in the deployment directory,
that links to your project's source code. And, if your deployment directory is
Python's ``site-packages`` directory, it will also update the
``easy-install.pth`` file to include your project's source code, thereby making
it available on ``sys.path`` for all programs using that Python installation.
If you have enabled the ``use_2to3`` flag, then of course the ``.egg-link``
will not link directly to your source code when run under Python 3, since
that source code would be made for Python 2 and not work under Python 3.
Instead the ``setup.py develop`` will build Python 3 code under the ``build``
directory, and link there. This means that after doing code changes you will
have to run ``setup.py build`` before these changes are picked up by your
Python 3 installation.
In addition, the ``develop`` command creates wrapper scripts in the target
script directory that will run your in-development scripts after ensuring that
all your ``install_requires`` packages are available on ``sys.path``.
You can deploy the same project to multiple staging areas, e.g. if you have
multiple projects on the same machine that are sharing the same project you're
doing development work.
When you're done with a given development task, you can remove the project
source from a staging area using ``setup.py develop --uninstall``, specifying
the desired staging area if it's not the default.
There are several options to control the precise behavior of the ``develop``
command; see the section on the :ref:`develop <develop>` command below for more details.
Note that you can also apply setuptools commands to non-setuptools projects,
using commands like this::
python -c "import setuptools; with open('setup.py') as f: exec(compile(f.read(), 'setup.py', 'exec'))" develop
That is, you can simply list the normal setup commands and options following
the quoted part.
This diff is collapsed.
.. _`entry_points`:
============
Entry Points
============
Packages may provide commands to be run at the console (console scripts),
such as the ``pip`` command. These commands are defined for a package
as a specific kind of entry point in the ``setup.cfg`` or
``setup.py``.
Console Scripts
===============
First consider an example without entry points. Imagine a package
defined thus:
.. code-block:: bash
timmins/
timmins/__init__.py
timmins/__main__.py
setup.cfg # or setup.py
#other necessary files
with ``__init__.py`` as:
.. code-block:: python
def helloworld():
print("Hello world")
and ``__main__.py`` providing a hook:
.. code-block:: python
from . import hello_world
if __name__ == '__main__':
hello_world()
After installing the package, the function may be invoked through the
`runpy <https://docs.python.org/3/library/runpy.html>`_ module:
.. code-block:: bash
python -m timmins
Adding a console script entry point allows the package to define a
user-friendly name for installers of the package to execute. Installers
like pip will create wrapper scripts to execute a function. In the
above example, to create a command ``hello-world`` that invokes
``timmins.hello_world``, add a console script entry point to
``setup.cfg``:
.. code-block:: ini
[options.entry_points]
console_scripts =
hello-world = timmins:hello_world
After installing the package, a user may invoke that function by simply calling
``hello-world`` on the command line.
The syntax for entry points is specified as follows:
.. code-block::
<name> = [<package>.[<subpackage>.]]<module>[:<object>.<object>]
where ``name`` is the name for the script you want to create, the left hand
side of ``:`` is the module that contains your function and the right hand
side is the object you want to invoke (e.g. a function).
In addition to ``console_scripts``, Setuptools supports ``gui_scripts``, which
will launch a GUI application without running in a terminal window.
.. _dynamic discovery of services and plugins:
Advertising Behavior
====================
Console scripts are one use of the more general concept of entry points. Entry
points more generally allow a packager to advertise behavior for discovery by
other libraries and applications. This feature enables "plug-in"-like
functionality, where one library solicits entry points and any number of other
libraries provide those entry points.
A good example of this plug-in behavior can be seen in
`pytest plugins <https://docs.pytest.org/en/latest/writing_plugins.html>`_,
where pytest is a test framework that allows other libraries to extend
or modify its functionality through the ``pytest11`` entry point.
The console scripts work similarly, where libraries advertise their commands
and tools like ``pip`` create wrapper scripts that invoke those commands.
For a project wishing to solicit entry points, Setuptools recommends the
`importlib.metadata <https://docs.python.org/3/library/importlib.metadata.html>`_
module (part of stdlib since Python 3.8) or its backport,
`importlib_metadata <https://pypi.org/project/importlib_metadata>`_.
For example, to find the console script entry points from the example above:
.. code-block:: python
>>> from importlib import metadata
>>> eps = metadata.entry_points()['console_scripts']
``eps`` is now a list of ``EntryPoint`` objects, one of which corresponds
to the ``hello-world = timmins:hello_world`` defined above. Each ``EntryPoint``
contains the ``name``, ``group``, and ``value``. It also supplies a ``.load()``
method to import and load that entry point (module or object).
.. code-block:: ini
[options.entry_points]
my.plugins =
hello-world = timmins:hello_world
Then, a different project wishing to load 'my.plugins' plugins could run
the following routine to load (and invoke) such plugins:
.. code-block:: python
>>> from importlib import metadata
>>> eps = metadata.entry_points()['my.plugins']
>>> for ep in eps:
... plugin = ep.load()
... plugin()
The project soliciting the entry points needs not to have any dependency
or prior knowledge about the libraries implementing the entry points, and
downstream users are able to compose functionality by pulling together
libraries implementing the entry points.
Dependency Management
=====================
Some entry points may require additional dependencies to properly function.
For such an entry point, declare in square brakets any number of dependency
``extras`` following the entry point definition. Such entry points will only
be viable if their extras were declared and installed. See the
:doc:`guide on dependencies management <dependency_management>` for
more information on defining extra requirements. Consider from the
above example:
.. code-block:: ini
[options.entry_points]
console_scripts =
hello-world = timmins:hello_world [pretty-printer]
In this case, the ``hello-world`` script is only viable if the ``pretty-printer``
extra is indicated, and so a plugin host might exclude that entry point
(i.e. not install a console script) if the relevant extra dependencies are not
installed.
This diff is collapsed.
========================================================
Using setuptools to package and distribute your project
========================================================
``setuptools`` offers a variety of functionalities that make it easy to
build and distribute your python package. Here we provide an overview on
the commonly used ones.
==================================================
Building and Distributing Packages with Setuptools
==================================================
``Setuptools`` is a collection of enhancements to the Python ``distutils``
that allow developers to more easily build and
distribute Python packages, especially ones that have dependencies on other
packages.
Packages built and distributed using ``setuptools`` look to the user like
ordinary Python packages based on the ``distutils``.
Transition to PEP517
====================
Since setuptools no longer serves as the default build tool, one must explicitly
opt in (by providing a :file:`pyproject.toml` file) to use this library. The user
facing part is provided by tools such as pip and
backend interface is described :doc:`in this document <../build_meta>`. The
quickstart provides an overview of the new workflow.
.. toctree::
:maxdepth: 1
quickstart
package_discovery
entry_point
dependency_management
datafiles
development_mode
distribution
extension
declarative_config
keywords
commands
functionalities_rewrite
miscellaneous
New and Changed ``setup()`` Keywords
====================================
The following keyword arguments to ``setup()`` are added or changed by
``setuptools``. All of them are optional; you do not have to supply them
unless you need the associated ``setuptools`` feature.
``include_package_data``
If set to ``True``, this tells ``setuptools`` to automatically include any
data files it finds inside your package directories that are specified by
your ``MANIFEST.in`` file. For more information, see the section on
:ref:`Including Data Files`.
``exclude_package_data``
A dictionary mapping package names to lists of glob patterns that should
be *excluded* from your package directories. You can use this to trim back
any excess files included by ``include_package_data``. For a complete
description and examples, see the section on :ref:`Including Data Files`.
``package_data``
A dictionary mapping package names to lists of glob patterns. For a
complete description and examples, see the section on :ref:`Including
Data Files`. You do not need to use this option if you are using
``include_package_data``, unless you need to add e.g. files that are
generated by your setup script and build process. (And are therefore not
in source control or are files that you don't want to include in your
source distribution.)
``zip_safe``
A boolean (True or False) flag specifying whether the project can be
safely installed and run from a zip file. If this argument is not
supplied, the ``bdist_egg`` command will have to analyze all of your
project's contents for possible problems each time it builds an egg.
``install_requires``
A string or list of strings specifying what other distributions need to
be installed when this one is. See the section on :ref:`Declaring
Dependencies` for details and examples of the format of this argument.
``entry_points``
A dictionary mapping entry point group names to strings or lists of strings
defining the entry points. Entry points are used to support dynamic
discovery of services or plugins provided by a project. See :ref:`Dynamic
Discovery of Services and Plugins` for details and examples of the format
of this argument. In addition, this keyword is used to support
:ref:`Automatic Script Creation <entry_points>`.
``extras_require``
A dictionary mapping names of "extras" (optional features of your project)
to strings or lists of strings specifying what other distributions must be
installed to support those features. See the section on :ref:`Declaring
Dependencies` for details and examples of the format of this argument.
``python_requires``
A string corresponding to a version specifier (as defined in PEP 440) for
the Python version, used to specify the Requires-Python defined in PEP 345.
``setup_requires``
A string or list of strings specifying what other distributions need to
be present in order for the *setup script* to run. ``setuptools`` will
attempt to obtain these (using pip if available) before processing the
rest of the setup script or commands. This argument is needed if you
are using distutils extensions as part of your build process; for
example, extensions that process setup() arguments and turn them into
EGG-INFO metadata files.
(Note: projects listed in ``setup_requires`` will NOT be automatically
installed on the system where the setup script is being run. They are
simply downloaded to the ./.eggs directory if they're not locally available
already. If you want them to be installed, as well as being available
when the setup script is run, you should add them to ``install_requires``
**and** ``setup_requires``.)
``dependency_links``
A list of strings naming URLs to be searched when satisfying dependencies.
These links will be used if needed to install packages specified by
``setup_requires`` or ``tests_require``. They will also be written into
the egg's metadata for use during install by tools that support them.
``namespace_packages``
A list of strings naming the project's "namespace packages". A namespace
package is a package that may be split across multiple project
distributions. For example, Zope 3's ``zope`` package is a namespace
package, because subpackages like ``zope.interface`` and ``zope.publisher``
may be distributed separately. The egg runtime system can automatically
merge such subpackages into a single parent package at runtime, as long
as you declare them in each project that contains any subpackages of the
namespace package, and as long as the namespace package's ``__init__.py``
does not contain any code other than a namespace declaration. See the
section below on :ref:`Namespace Packages` for more information.
``test_suite``
A string naming a ``unittest.TestCase`` subclass (or a package or module
containing one or more of them, or a method of such a subclass), or naming
a function that can be called with no arguments and returns a
``unittest.TestSuite``. If the named suite is a module, and the module
has an ``additional_tests()`` function, it is called and the results are
added to the tests to be run. If the named suite is a package, any
submodules and subpackages are recursively added to the overall test suite.
Specifying this argument enables use of the :ref:`test <test>` command to run the
specified test suite, e.g. via ``setup.py test``. See the section on the
:ref:`test <test>` command below for more details.
New in 41.5.0: Deprecated the test command.
``tests_require``
If your project's tests need one or more additional packages besides those
needed to install it, you can use this option to specify them. It should
be a string or list of strings specifying what other distributions need to
be present for the package's tests to run. When you run the ``test``
command, ``setuptools`` will attempt to obtain these (using pip if
available). Note that these required projects will *not* be installed on
the system where the tests are run, but only downloaded to the project's setup
directory if they're not already installed locally.
New in 41.5.0: Deprecated the test command.
.. _test_loader:
``test_loader``
If you would like to use a different way of finding tests to run than what
setuptools normally uses, you can specify a module name and class name in
this argument. The named class must be instantiable with no arguments, and
its instances must support the ``loadTestsFromNames()`` method as defined
in the Python ``unittest`` module's ``TestLoader`` class. Setuptools will
pass only one test "name" in the ``names`` argument: the value supplied for
the ``test_suite`` argument. The loader you specify may interpret this
string in any way it likes, as there are no restrictions on what may be
contained in a ``test_suite`` string.
The module name and class name must be separated by a ``:``. The default
value of this argument is ``"setuptools.command.test:ScanningLoader"``. If
you want to use the default ``unittest`` behavior, you can specify
``"unittest:TestLoader"`` as your ``test_loader`` argument instead. This
will prevent automatic scanning of submodules and subpackages.
The module and class you specify here may be contained in another package,
as long as you use the ``tests_require`` option to ensure that the package
containing the loader class is available when the ``test`` command is run.
New in 41.5.0: Deprecated the test command.
``eager_resources``
A list of strings naming resources that should be extracted together, if
any of them is needed, or if any C extensions included in the project are
imported. This argument is only useful if the project will be installed as
a zipfile, and there is a need to have all of the listed resources be
extracted to the filesystem *as a unit*. Resources listed here
should be "/"-separated paths, relative to the source root, so to list a
resource ``foo.png`` in package ``bar.baz``, you would include the string
``bar/baz/foo.png`` in this argument.
If you only need to obtain resources one at a time, or you don't have any C
extensions that access other files in the project (such as data files or
shared libraries), you probably do NOT need this argument and shouldn't
mess with it. For more details on how this argument works, see the section
below on :ref:`Automatic Resource Extraction`.
``use_2to3``
Convert the source code from Python 2 to Python 3 with 2to3 during the
build process. See :doc:`../deprecated/python3` for more details.
``convert_2to3_doctests``
List of doctest source files that need to be converted with 2to3.
See :doc:`../deprecated/python3` for more details.
``use_2to3_fixers``
A list of modules to search for additional fixers to be used during
the 2to3 conversion. See :doc:`../deprecated/python3` for more details.
``project_urls``
An arbitrary map of URL names to hyperlinks, allowing more extensible
documentation of where various resources can be found than the simple
``url`` and ``download_url`` options provide.
.. _Automatic Resource Extraction:
Automatic Resource Extraction
-----------------------------
If you are using tools that expect your resources to be "real" files, or your
project includes non-extension native libraries or other files that your C
extensions expect to be able to access, you may need to list those files in
the ``eager_resources`` argument to ``setup()``, so that the files will be
extracted together, whenever a C extension in the project is imported.
This is especially important if your project includes shared libraries *other*
than distutils-built C extensions, and those shared libraries use file
extensions other than ``.dll``, ``.so``, or ``.dylib``, which are the
extensions that setuptools 0.6a8 and higher automatically detects as shared
libraries and adds to the ``native_libs.txt`` file for you. Any shared
libraries whose names do not end with one of those extensions should be listed
as ``eager_resources``, because they need to be present in the filesystem when
he C extensions that link to them are used.
The ``pkg_resources`` runtime for compressed packages will automatically
extract *all* C extensions and ``eager_resources`` at the same time, whenever
*any* C extension or eager resource is requested via the ``resource_filename()``
API. (C extensions are imported using ``resource_filename()`` internally.)
This ensures that C extensions will see all of the "real" files that they
expect to see.
Note also that you can list directory resource names in ``eager_resources`` as
well, in which case the directory's contents (including subdirectories) will be
extracted whenever any C extension or eager resource is requested.
Please note that if you're not sure whether you need to use this argument, you
don't! It's really intended to support projects with lots of non-Python
dependencies and as a last resort for crufty projects that can't otherwise
handle being compressed. If your package is pure Python, Python plus data
files, or Python plus C, you really don't need this. You've got to be using
either C or an external program that needs "real" files in your project before
there's any possibility of ``eager_resources`` being relevant to your project.
Defining Additional Metadata
----------------------------
Some extensible applications and frameworks may need to define their own kinds
of metadata to include in eggs, which they can then access using the
``pkg_resources`` metadata APIs. Ordinarily, this is done by having plugin
developers include additional files in their ``ProjectName.egg-info``
directory. However, since it can be tedious to create such files by hand, you
may want to create a distutils extension that will create the necessary files
from arguments to ``setup()``, in much the same way that ``setuptools`` does
for many of the ``setup()`` arguments it adds. See the section below on
:ref:`Creating ``distutils\`\` Extensions` for more details, especially the
subsection on :ref:`Adding new EGG-INFO Files`.
Setting the ``zip_safe`` flag
-----------------------------
For some use cases (such as bundling as part of a larger application), Python
packages may be run directly from a zip file.
Not all packages, however, are capable of running in compressed form, because
they may expect to be able to access either source code or data files as
normal operating system files. So, ``setuptools`` can install your project
as a zipfile or a directory, and its default choice is determined by the
project's ``zip_safe`` flag.
You can pass a True or False value for the ``zip_safe`` argument to the
``setup()`` function, or you can omit it. If you omit it, the ``bdist_egg``
command will analyze your project's contents to see if it can detect any
conditions that would prevent it from working in a zipfile. It will output
notices to the console about any such conditions that it finds.
Currently, this analysis is extremely conservative: it will consider the
project unsafe if it contains any C extensions or datafiles whatsoever. This
does *not* mean that the project can't or won't work as a zipfile! It just
means that the ``bdist_egg`` authors aren't yet comfortable asserting that
the project *will* work. If the project contains no C or data files, and does
no ``__file__`` or ``__path__`` introspection or source code manipulation, then
there is an extremely solid chance the project will work when installed as a
zipfile. (And if the project uses ``pkg_resources`` for all its data file
access, then C extensions and other data files shouldn't be a problem at all.
See the :ref:`Accessing Data Files at Runtime` section above for more information.)
However, if ``bdist_egg`` can't be *sure* that your package will work, but
you've checked over all the warnings it issued, and you are either satisfied it
*will* work (or if you want to try it for yourself), then you should set
``zip_safe`` to ``True`` in your ``setup()`` call. If it turns out that it
doesn't work, you can always change it to ``False``, which will force
``setuptools`` to install your project as a directory rather than as a zipfile.
In the future, as we gain more experience with different packages and become
more satisfied with the robustness of the ``pkg_resources`` runtime, the
"zip safety" analysis may become less conservative. However, we strongly
recommend that you determine for yourself whether your project functions
correctly when installed as a zipfile, correct any problems if you can, and
then make an explicit declaration of ``True`` or ``False`` for the ``zip_safe``
flag, so that it will not be necessary for ``bdist_egg`` to try to guess
whether your project can work as a zipfile.
.. _`package_discovery`:
========================================
Package Discovery and Namespace Package
========================================
.. note::
a full specification for the keyword supplied to ``setup.cfg`` or
``setup.py`` can be found at :doc:`keywords reference <keywords>`
.. note::
the examples provided here are only to demonstrate the functionality
introduced. More metadata and options arguments need to be supplied
if you want to replicate them on your system. If you are completely
new to setuptools, the :doc:`quickstart section <quickstart>` is a good
place to start.
``Setuptools`` provide powerful tools to handle package discovery, including
support for namespace package. Normally, you would specify the package to be
included manually in the following manner:
.. code-block:: ini
[options]
#...
packages =
mypkg1
mypkg2
.. code-block:: python
setup(
#...
packages = ['mypkg1', 'mypkg2']
)
This can get tiresome reallly quickly. To speed things up, we introduce two
functions provided by setuptools:
.. code-block:: ini
[options]
packages = find:
#or
packages = find_namespace:
.. code-block:: python
from setuptools import find_packages
#or
from setuptools import find_namespace_packages
Using ``find:`` or ``find_packages``
====================================
Let's start with the first tool. ``find:`` (``find_packages``) takes a source
directory and two lists of package name patterns to exclude and include, and
then return a list of ``str`` representing the packages it could find. To use
it, consider the following directory
.. code-block:: bash
mypkg/
src/
pkg1/__init__.py
pkg2/__init__.py
additional/__init__.py
setup.cfg #or setup.py
To have your setup.cfg or setup.py to automatically include packages found
in ``src`` that starts with the name ``pkg`` and not ``additional``:
.. code-block:: ini
[options]
packages = find:
package_dir =
=src
[options.packages.find]
where = src
include = pkg*
exclude = additional
.. code-block:: python
setup(
#...
packages = find_packages(
where = 'src',
include = ['pkg*',],
exclude = ['additional',]
),
package_dir = {"":"src"}
#...
)
.. _Namespace Packages:
Using ``find_namespace:`` or ``find_namespace_packages``
========================================================
``setuptools`` provides the ``find_namespace:`` (``find_namespace_packages``)
which behaves similarly to ``find:`` but works with namespace package. Before
diving in, it is important to have a good understanding of what namespace
packages are. Here is a quick recap:
Suppose you have two packages named as follows:
.. code-block:: bash
/Users/Desktop/timmins/foo/__init__.py
/Library/timmins/bar/__init__.py
If both ``Desktop`` and ``Library`` are on your ``PYTHONPATH``, then a
namespace package called ``timmins`` will be created automatically for you when
you invoke the import mechanism, allowing you to accomplish the following
.. code-block:: python
>>> import timmins.foo
>>> import timmins.bar
as if there is only one ``timmins`` on your system. The two packages can then
be distributed separately and installed individually without affecting the
other one. Suppose you are packaging the ``foo`` part:
.. code-block:: bash
foo/
src/
timmins/foo/__init__.py
setup.cfg # or setup.py
and you want the ``foo`` to be automatically included, ``find:`` won't work
because timmins doesn't contain ``__init__.py`` directly, instead, you have
to use ``find_namespace:``:
.. code-block:: ini
[options]
package_dir =
=src
packages = find_namespace:
[options.packages.find_namespace]
where = src
When you install the zipped distribution, ``timmins.foo`` would become
available to your interpreter.
You can think of ``find_namespace:`` as identical to ``find:`` except it
would count a directory as a package even if it doesn't contain ``__init__.py``
file directly. As a result, this creates an interesting side effect. If you
organize your package like this:
.. code-block:: bash
foo/
timmins/
foo/__init__.py
setup.cfg # or setup.py
tests/
test_foo/__init__.py
a naive ``find_namespace:`` would include tests as part of your package to
be installed. A simple way to fix it is to adopt the aforementioned
``src`` layout.
Legacy Namespace Packages
=========================
The fact you can create namespace package so effortlessly above is credited
to `PEP 420 <https://www.python.org/dev/peps/pep-0420/>`_. It use to be more
cumbersome to accomplish the same result. Historically, there were two methods
to create namespace packages. One is the ``pkg_resources`` style supported by
``setuptools`` and the other one being ``pkgutils`` style offered by
``pkgutils`` module in Python. Both are now considered deprecated despite the
fact they still linger in many existing packages. These two differ in many
subtle yet significant aspects and you can find out more on `Python packaging
user guide <https://packaging.python.org/guides/packaging-namespace-packages/>`_
``pkg_resource`` style namespace package
----------------------------------------
This is the method ``setuptools`` directly supports. Starting with the same
layout, there are two pieces you need to add to it. First, an ``__init__.py``
file directly under your namespace package directory that contains the
following:
.. code-block:: python
__import__("pkg_resources").declare_namespace(__name__)
And the ``namespace_packages`` keyword in your ``setup.cfg`` or ``setup.py``:
.. code-block:: ini
[options]
namespace_packages = timmins
.. code-block:: python
setup(
# ...
namespace_packages = ['timmins']
)
And your directory should look like this
.. code-block:: bash
/foo/
src/
timmins/
__init__.py
foo/__init__.py
setup.cfg #or setup.py
Repeat the same for other packages and you can achieve the same result as
the previous section.
``pkgutil`` style namespace package
-----------------------------------
This method is almost identical to the ``pkg_resource`` except that the
``namespace_packages`` declaration is omitted and the ``__init__.py``
file contains the following:
.. code-block:: python
__path__ = __import__('pkgutil').extend_path(__path__, __name__)
The project layout remains the same and ``setup.cfg`` remains the same.
This diff is collapsed.
......@@ -22,7 +22,11 @@ def rewrite_packaging(pkg_files, new_root):
"""
for file in pkg_files.glob('*.py'):
text = file.text()
text = re.sub(r' (pyparsing|six)', rf' {new_root}.\1', text)
text = re.sub(r' (pyparsing)', rf' {new_root}.\1', text)
text = text.replace(
'from six.moves.urllib import parse',
'from urllib import parse',
)
file.write_text(text)
......@@ -50,6 +54,7 @@ def install(vendor):
subprocess.check_call(install_args)
remove_all(vendor.glob('*.dist-info'))
remove_all(vendor.glob('*.egg-info'))
remove_all(vendor.glob('six.py'))
(vendor / '__init__.py').write_text('')
......
This diff is collapsed.
......@@ -4,18 +4,24 @@
from __future__ import absolute_import, division, print_function
__all__ = [
"__title__", "__summary__", "__uri__", "__version__", "__author__",
"__email__", "__license__", "__copyright__",
"__title__",
"__summary__",
"__uri__",
"__version__",
"__author__",
"__email__",
"__license__",
"__copyright__",
]
__title__ = "packaging"
__summary__ = "Core utilities for Python packages"
__uri__ = "https://github.com/pypa/packaging"
__version__ = "16.8"
__version__ = "20.4"
__author__ = "Donald Stufft and individual contributors"
__email__ = "donald@stufft.io"
__license__ = "BSD or Apache License, Version 2.0"
__copyright__ = "Copyright 2014-2016 %s" % __author__
__license__ = "BSD-2-Clause or Apache-2.0"
__copyright__ = "Copyright 2014-2019 %s" % __author__
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
packaging==16.8
packaging==20.4
pyparsing==2.2.1
six==1.10.0
appdirs==1.4.3
......@@ -62,5 +62,5 @@ class VendorImporter:
sys.meta_path.append(self)
names = 'packaging', 'pyparsing', 'six', 'appdirs'
names = 'packaging', 'pyparsing', 'appdirs'
VendorImporter(__name__, names).install()
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment