...
 
Commits (1259)
.installed.cfg
bin/
build/
develop-eggs/
eggs/
parts/
python*/
*.pyc
*.egg
*.egg-info
dist
.tox
language: python
env:
- PYTHON_VER=2.6
- PYTHON_VER=2.7
- PYTHON_VER=3.3
- PYTHON_VER=3.4
- PYTHON_VER=3.5
sudo: false
cache:
directories:
- pythons
# Note on this cache directories: "make build" downloads pythons elsewhere,
# but installs the compiled bin/lib/share in pythons/pythonx.y/ .
notifications:
email:
- buildout-development@googlegroups.com
install:
- ls -al pythons
- deactivate
- make build
script: make test
[buildout]
extends = buildout.cfg
[versions]
zope.interface = 3.8.0
zope.exceptions = 3.7.1
2.5.2+slapos005
---------------
Fix a regression introduced in 2.5.2+slapos004 with the review of:
- Add referred parts' hash strings in __buildout_signature__, that invokes
rebuild of a part when one of its (recursive) dependencies are modified.
2.5.2+slapos004
---------------
Rebase to refresh some patches and fix commits that were wrongly cherry-picked
from Nexedi 1.x branch. Changed:
- Give the same permission but write as owner to group and other.
- Make buildout.rmtree working with symlink as a path argument.
- Add referred parts' hash strings in __buildout_signature__, that invokes
rebuild of a part when one of its (recursive) dependencies are modified.
- Write .installed.cfg only once, in safe way and only if there's any change.
- Escape $$ character to $.
2.5.2+slapos003
---------------
- Expand sys.path when PYTHONEXTRAPATH is present for develop.
2.5.2+slapos002
---------------
- Fix a bug where slapos.libnetworkcache is lost when buildout script
is upgraded.
2.5.2+slapos001
---------------
- Rebase with 2.5.2
- In verbose mode, save .installed.cfg each time a part is installed.
2.5.1+slapos001
---------------
- Rebase with 2.5.1
2.5.0+slapos001
---------------
- Rebase with 2.5.0
- Fix a bug in $$ escape handling.
2.4.7+slapos001
---------------
- Rebase with 2.4.7.
- Support Python 3.
2.4.5+slapos001
---------------
- Support ${:_profile_base_location_}.
- Support on the fly patches in easy_install.
- Merge PYTHONEXTRAPATH value into PYTHONPATH in easy_install so that we can
use some eggs that are required to build an egg.
- Support network cache in download.py and easy_install.py
- Cache downloaded data in zc/buildout/buildout.py:_open() in memory
to accelerate remote extends.
- Give the same permission but write as owner to group and other.
- Make buildout.rmtree working with symlink as a path argument.
https://bugs.launchpad.net/zc.buildout/+bug/144228
- Respect specified versions of initial eggs in bootstrap.
- Keep develop-eggs directory in bootstrap.
- Add referred parts' hash strings in __buildout_signature__, that invokes
rebuild of a part when one of its (recursive) dependencies are modified.
- Allow assigning non-string values to section keys. Restricted to selected
python base types.
- Write .installed.cfg only once, in safe way and only if there's any change.
- Put only one [buildout] section in .installed.cfg.
- Escape $$ character to $.
- Compare Options with sorted result, if possible.
- Add '--dry-run' option.
- Add '--skip-signature-check' option.
not applied (still required?)
-----------------------------
- 83fc3a2 Always use build() in easy_install.py to install eggs.
(it's not required if the commit below is not applied, I guess)
- b7cd5ca Include '.postN' in generated egg's version so that version pinning with 'N.N.N.postN' works.
- 1761c65 Workaround M2Crypto bug of https redirection.
This diff is collapsed.
Zope Foundation and Contributors
\ No newline at end of file
Developing buildout itself
**************************
When you're developing buildout itself, you need to know two things:
- Use a clean python *without* setuptools installed. Otherwise many tests
will find your already-installed setuptools, leading to test differences
when setuptools' presence is explicitly tested.
- Also the presence of ``~/.buildout/default.cfg`` may interfere with the
tests so you may want to temporarily rename it so that it does not get in
the way.
- Don't bootstrap with ``python bootstrap/bootstrap.py`` but with ``python
dev.py``.
- Run buildout with -U, to ignore user (default) settings which can interfere
with using the development version
For your convenience we provide a Makefile to build various Python versions
in subdirectories of the buildout checkout. To use these and run the tests
with them do::
make PYTHON_VER=2.6 build
make PYTHON_VER=2.6 test
make PYTHON_VER=2.7 build
make PYTHON_VER=2.7 test
make PYTHON_VER=3.2 build
make PYTHON_VER=3.2 test
The actual Python compilation is only done once and then re-used. So on
subsequent builds, only the development buildout itself needs to be redone.
Releases: zc.buildout, zc.recipe.egg and bootstrap.py
-----------------------------------------------------
Buildout consists of two python packages that are released separately:
zc.buildout and zc.recipe.egg. zc.recipe.egg is changed much less often than
zc.buildout.
zc.buildout's setup.py and changelog is in the same directory as this
``DEVELOPERS.txt`` and the code is in ``src/zc/buildout``.
zc.recipe.egg, including setup.py and a separate changelog, is in the
``zc.recipe.egg_`` subdirectory.
A third item is the bootstrap.py file in ``bootstrap/bootstrap.py``. The
canonical location is at https://bootstrap.pypa.io/bootstrap-buildout.py,
(though it is unfortunate that it isn't named just ``bootstrap.py``). This
file is pulled automatically every 15 minutes from the bootstrap-release
branch. When a new buildout release is made, **the releaser** should update
the bootstrap-release branch, too.
The http://downloads.buildout.org/2/bootstrap.py location doesn't need to be
updated manually: it is a redirect now, to
https://bootstrap.pypa.io/bootstrap-buildout.py .
If there are changes to bootstrap.py, be sure to update the date in the
``__version__`` attribute and to record the bootstrap change (including the
new date/version) in zc.buildout's changelog.
When releasing, make sure you also build a (universal) wheel in addition to
the regular .tar.gz::
$ python setup.py sdist bdist_wheel upload
You can also use zest.releaser to release it. If you've installed it as
``zest.releaser[recommended]`` it builds the wheel for you and uploads it via
https (via twine).
Zope Public License (ZPL) Version 2.1
A copyright notice accompanies this license document that identifies the
copyright holders.
This license has been certified as open source. It has also been designated as
GPL compatible by the Free Software Foundation (FSF).
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions in source code must retain the accompanying copyright
notice, this list of conditions, and the following disclaimer.
2. Redistributions in binary form must reproduce the accompanying copyright
notice, this list of conditions, and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Names of the copyright holders must not be used to endorse or promote
products derived from this software without prior written permission from the
copyright holders.
4. The right to distribute this software or to use it for any purpose does not
give you the right to use Servicemarks (sm) or Trademarks (tm) of the
copyright
holders. Use of them is covered by separate agreement with the copyright
holders.
5. If any files are modified, you must cause the modified files to carry
prominent notices stating that you changed the files and the date of any
change.
Disclaimer
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY EXPRESSED
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
include *.py
include *.rst
include *.txt
recursive-include bootstrap *.py
recursive-include doc *.txt
recursive-include specifications *.txt
recursive-include src/zc *.test
recursive-include src/zc *.txt
HERE = $(shell pwd)
PYTHON_VER ?= 2.7
PYTHON_PATH = $(HERE)/pythons/$(PYTHON_VER)
PYTHON_BUILD_DIR = $(HERE)/python_builds
ifeq ($(PYTHON_VER),2.6)
PYTHON_MINOR ?= 2.6.8
endif
ifeq ($(PYTHON_VER),2.7)
PYTHON_MINOR ?= 2.7.3
endif
ifeq ($(PYTHON_VER),3.2)
PYTHON_MINOR ?= 3.2.3
endif
ifeq ($(PYTHON_VER),3.3)
PYTHON_MINOR ?= 3.3.0
endif
ifeq ($(PYTHON_VER),3.4)
PYTHON_MINOR ?= 3.4.2
PYTHON_CONFIGURE_ARGS ?= --without-ensurepip
endif
ifeq ($(PYTHON_VER),3.5)
PYTHON_MINOR ?= 3.5.0
PYTHON_CONFIGURE_ARGS ?= --without-ensurepip
endif
ifndef PYTHON_MINOR
$(error Please specify desired PYTHON_MINOR for Python $(PYTHON_VER))
endif
PYTHON_ARCHIVE ?= Python-$(PYTHON_MINOR)
PYTHON_DOWNLOAD = https://www.python.org/ftp/python/$(PYTHON_MINOR)/$(PYTHON_ARCHIVE).tgz
PYTHON_EXE = python$(PYTHON_VER)
.PHONY: all build test
BUILD_DIRS = $(PYTHON_PATH) bin build develop-eggs eggs parts
all: build
$(PYTHON_PATH)/bin/$(PYTHON_EXE):
@echo "Installing Python"
mkdir -p $(PYTHON_PATH)
mkdir -p $(PYTHON_BUILD_DIR)
cd $(PYTHON_BUILD_DIR) && \
curl --progress-bar --location $(PYTHON_DOWNLOAD) | tar -zx
ifeq ($(PYTHON_VER),2.6)
cd $(PYTHON_BUILD_DIR) && \
curl --progress-bar -L https://raw.github.com/collective/buildout.python/ad45adb78bfa37542d62a394392d5146fce5af34/src/issue12012-sslv2-py26.patch > ssl.patch
cd $(PYTHON_BUILD_DIR)/$(PYTHON_ARCHIVE) && \
patch -p0 < ../ssl.patch
endif
cd $(PYTHON_BUILD_DIR)/$(PYTHON_ARCHIVE) && \
./configure --prefix $(PYTHON_PATH) $(PYTHON_CONFIGURE_ARGS) >/dev/null 2>&1 && \
make >/dev/null 2>&1 && \
make install >/dev/null 2>&1
@echo "Finished installing Python"
build: $(PYTHON_PATH)/bin/$(PYTHON_EXE)
$(PYTHON_PATH)/bin/$(PYTHON_EXE) dev.py
clean:
rm -rf $(BUILD_DIRS) $(PYTHON_BUILD_DIR)
test:
$(HERE)/bin/test -1 -v
********
Buildout
********
.. image:: https://secure.travis-ci.org/buildout/buildout.png?branch=master
:width: 82px
:height: 13px
:alt: Travis CI build report
:target: https://travis-ci.org/buildout/buildout
.. image:: http://winbot.zope.org/buildstatusimage?builder=zc_buildout_dev%20py_270_win32%20master&number=-1
:width: 82px
:height: 13px
:alt: winbot build report
:target: http://winbot.zope.org/builders/zc_buildout_dev%20py_270_win32%20master/builds/-1
Buildout is a project designed to solve 2 problems:
1. Application-centric assembly and deployment
*Assembly* runs the gamut from stitching together libraries to
create a running program, to production deployment configuration of
applications, and associated systems and tools (e.g. run-control
scripts, cron jobs, logs, service registration, etc.).
Buildout might be confused with build tools like make or ant, but
it is a little higher level and might invoke systems like make or
ant to get its work done.
Buildout might be confused with systems like puppet or chef, but it
is more application focused. Systems like puppet or chef might
use buildout to get their work done.
Buildout is also somewhat Python-centric, even though it can be
used to assemble and deploy non-python applications. It has some
special features for assembling Python programs. It's scripted with
Python, unlike, say puppet or chef, which are scripted with Ruby.
2. Repeatable assembly of programs from Python software distributions
Buildout puts great effort toward making program assembly a highly
repeatable process, whether in a very open-ended development mode,
where dependency versions aren't locked down, or in a deployment
environment where dependency versions are fully specified. You
should be able to check buildout into a VCS and later check it out.
Two checkouts built at the same time in the same environment should
always give the same result, regardless of their history. Among
other things, after a buildout, all dependencies should be at the
most recent version consistent with any version specifications
expressed in the buildout.
Buildout supports applications consisting of multiple programs,
with different programs in an application free to use different
versions of Python distributions. This is in contrast with a
Python installation (real or virtual), where, for any given
distribution, there can only be one installed.
To learn more about buildout, including how to use it, see
http://buildout.org/.
Installation
************
There are a number of ways to install buildout. You can install it as
you would any other package, using pip or easy_install. In this case,
you'll get a buildout command that you can use to build projects. To
build a project, just use::
buildout
from a project directory.
Buildout's (stubborn) philosophy, however, is that projects should be
self-contained, and not require changes to a shared Python
installation. To avoid changing a shared Python installation you can
download a bootstrap script that, when run, will install buildout
locally in your project.
The bootstrap script for buildout version 2 is at:
https://bootstrap.pypa.io/bootstrap-buildout.py
So, for example, to install buildout 2 in a project, you might::
wget https://bootstrap.pypa.io/bootstrap-buildout.py
python bootstrap-buildout.py
Then to build your project, you can just run::
bin/buildout
from the project directory.
buildout 2 is somewhat backward-incompatible with version 1. Most
projects will probably work fine with either. If you need to keep
using version 1, however, specify a version requirement when you use
pip or easy_install, or use the version 1 bootstrap script at:
http://downloads.buildout.org/1/bootstrap.py
##############################################################################
#
# Copyright (c) 2006 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Bootstrap a buildout-based project
Simply run this script in a directory containing a buildout.cfg.
The script accepts buildout command-line options, so you can
use the -c option to specify an alternate configuration file.
"""
import os
import shutil
import sys
import tempfile
from optparse import OptionParser
__version__ = '2015-07-01'
# See zc.buildout's changelog if this version is up to date.
tmpeggs = tempfile.mkdtemp(prefix='bootstrap-')
usage = '''\
[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
Bootstraps a buildout-based project.
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
Note that by using --find-links to point to local resources, you can keep
this script from going over the network.
'''
parser = OptionParser(usage=usage)
parser.add_option("--version",
action="store_true", default=False,
help=("Return bootstrap.py version."))
parser.add_option("-t", "--accept-buildout-test-releases",
dest='accept_buildout_test_releases',
action="store_true", default=False,
help=("Normally, if you do not specify a --buildout-version, "
"the bootstrap script and buildout gets the newest "
"*final* versions of zc.buildout and its recipes and "
"extensions for you. If you use this flag, "
"bootstrap and buildout will get the newest releases "
"even if they are alphas or betas."))
parser.add_option("-c", "--config-file",
help=("Specify the path to the buildout configuration "
"file to be used."))
parser.add_option("-f", "--find-links",
help=("Specify a URL to search for buildout releases"))
parser.add_option("--allow-site-packages",
action="store_true", default=False,
help=("Let bootstrap.py use existing site packages"))
parser.add_option("--buildout-version",
help="Use a specific zc.buildout version")
parser.add_option("--setuptools-version",
help="Use a specific setuptools version")
parser.add_option("--setuptools-to-dir",
help=("Allow for re-use of existing directory of "
"setuptools versions"))
options, args = parser.parse_args()
if options.version:
print("bootstrap.py version %s" % __version__)
sys.exit(0)
######################################################################
# load/install setuptools
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
ez = {}
if os.path.exists('ez_setup.py'):
exec(open('ez_setup.py').read(), ez)
else:
exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
if not options.allow_site_packages:
# ez_setup imports site, which adds site packages
# this will remove them from the path to ensure that incompatible versions
# of setuptools are not in the path
import site
# inside a virtualenv, there is no 'getsitepackages'.
# We can't remove these reliably
if hasattr(site, 'getsitepackages'):
for sitepackage_path in site.getsitepackages():
# Strip all site-packages directories from sys.path that
# are not sys.prefix; this is because on Windows
# sys.prefix is a site-package directory.
if sitepackage_path != sys.prefix:
sys.path[:] = [x for x in sys.path
if sitepackage_path not in x]
setup_args = dict(to_dir=tmpeggs, download_delay=0)
if options.setuptools_version is not None:
setup_args['version'] = options.setuptools_version
if options.setuptools_to_dir is not None:
setup_args['to_dir'] = options.setuptools_to_dir
ez['use_setuptools'](**setup_args)
import setuptools
import pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
if path not in pkg_resources.working_set.entries:
pkg_resources.working_set.add_entry(path)
######################################################################
# Install buildout
ws = pkg_resources.working_set
setuptools_path = ws.find(
pkg_resources.Requirement.parse('setuptools')).location
# Fix sys.path here as easy_install.pth added before PYTHONPATH
cmd = [sys.executable, '-c',
'import sys; sys.path[0:0] = [%r]; ' % setuptools_path +
'from setuptools.command.easy_install import main; main()',
'-mZqNxd', tmpeggs]
find_links = os.environ.get(
'bootstrap-testing-find-links',
options.find_links or
('http://downloads.buildout.org/'
if options.accept_buildout_test_releases else None)
)
if find_links:
cmd.extend(['-f', find_links])
requirement = 'zc.buildout'
version = options.buildout_version
if version is None and not options.accept_buildout_test_releases:
# Figure out the most recent final version of zc.buildout.
import setuptools.package_index
_final_parts = '*final-', '*final'
def _final_version(parsed_version):
try:
return not parsed_version.is_prerelease
except AttributeError:
# Older setuptools
for part in parsed_version:
if (part[:1] == '*') and (part not in _final_parts):
return False
return True
index = setuptools.package_index.PackageIndex(
search_path=[setuptools_path])
if find_links:
index.add_find_links((find_links,))
req = pkg_resources.Requirement.parse(requirement)
if index.obtain(req) is not None:
best = []
bestv = None
for dist in index[req.project_name]:
distv = dist.parsed_version
if _final_version(distv):
if bestv is None or distv > bestv:
best = [dist]
bestv = distv
elif distv == bestv:
best.append(dist)
if best:
best.sort()
version = best[-1].version
if version:
requirement = '=='.join((requirement, version))
cmd.append(requirement)
import subprocess
if subprocess.call(cmd) != 0:
raise Exception(
"Failed to execute command:\n%s" % repr(cmd)[1:-1])
######################################################################
# Import and run buildout
ws.add_entry(tmpeggs)
ws.require(requirement)
import zc.buildout.buildout
if not [a for a in args if '=' not in a]:
args.append('bootstrap')
# if -c was provided, we push it back into args for buildout' main function
if options.config_file is not None:
args[0:0] = ['-c', options.config_file]
zc.buildout.buildout.main(args)
shutil.rmtree(tmpeggs)
[buildout]
develop = zc.recipe.egg_ .
parts = test oltest py
versions = versions
[versions]
zope.interface = 4.1.3
zope.exceptions = 4.0.8
zc.recipe.testrunner = 2.0.0
zope.testing = 4.5.0
[py]
recipe = zc.recipe.egg
eggs = zc.buildout
zc.recipe.egg
zope.testing
interpreter = py
[test]
recipe = zc.recipe.testrunner
eggs =
zc.buildout[test]
zc.recipe.egg
# Tests that can be run wo a network
[oltest]
recipe = zc.recipe.testrunner
eggs =
zc.buildout[test]
zc.recipe.egg
defaults =
[
'-t',
'!(bootstrap|selectingpython|selecting-python)',
]
##############################################################################
#
# Copyright (c) 2005 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Bootstrap the buildout project itself.
This is different from a normal boostrapping process because the
buildout egg itself is installed as a develop egg.
"""
import os, shutil, sys, subprocess
for d in 'eggs', 'develop-eggs', 'bin', 'parts':
if not os.path.exists(d):
os.mkdir(d)
if os.path.isdir('build'):
shutil.rmtree('build')
######################################################################
# Make sure we have a relatively clean environment
try:
import pkg_resources, setuptools
except ImportError:
pass
else:
raise SystemError(
"Buildout development with a pre-installed setuptools or "
"distribute is not supported.")
######################################################################
# Install distribute
ez = {}
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
# XXX use a more permanent ez_setup.py URL when available.
exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
ez['use_setuptools'](to_dir='eggs', download_delay=0)
import pkg_resources, setuptools
setuptools_path = os.path.dirname(os.path.dirname(setuptools.__file__))
######################################################################
# Install buildout
if subprocess.call(
[sys.executable] +
['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
env=dict(os.environ, PYTHONPATH=setuptools_path)):
raise RuntimeError("buildout build failed.")
pkg_resources.working_set.add_entry('src')
import zc.buildout.easy_install
zc.buildout.easy_install.scripts(
['zc.buildout'], pkg_resources.working_set , sys.executable, 'bin')
bin_buildout = os.path.join('bin', 'buildout')
if sys.platform.startswith('java'):
# Jython needs the script to be called twice via sys.executable
assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0
if sys.version_info < (2, 6):
bin_buildout = [bin_buildout, '-c2.4.cfg']
sys.exit(subprocess.Popen(bin_buildout).wait())
This diff is collapsed.
This diff is collapsed.
[bdist_wheel]
universal = 1
[check-manifest]
ignore =
*.cfg
.travis.yml
Makefile
zc.recipe.egg_/*
zc.recipe.egg_
tox.ini
[zest.releaser]
create-wheel = yes
extra-message = [ci skip]
##############################################################################
#
# Copyright (c) 2006-2009 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
name = "zc.buildout"
version = '2.5.2+slapos005'
import os
from setuptools import setup
def read(*rnames):
return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
doc_intro = """
Below, you'll find doctest-based documentation. It was an experiment
in reusing tests as documentation. The experiment didn't go that
well, but there may be details below that aren't easy to find on
buildout.org yet.
.. contents ::
doctest-based Documentation
***************************
"""
long_description=(
read('README.rst')
+ doc_intro +
read('src', 'zc', 'buildout', 'buildout.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'repeatable.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'download.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'downloadcache.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'extends-cache.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'setup.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'update.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'debugging.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'meta-recipes.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'testing.txt')
+ '\n' +
read('src', 'zc', 'buildout', 'easy_install.txt')
+ '\n' +
read('CHANGES.rst')
# + '\n' +
# 'Download\n'
# '**********************\n'
)
entry_points = """
[console_scripts]
buildout = %(name)s.buildout:main
[zc.buildout]
debug = %(name)s.testrecipes:Debug
""" % dict(name=name)
setup(
name = name,
version = version,
author = "Jim Fulton",
author_email = "jim@zope.com",
description = "System for managing development buildouts",
long_description=long_description,
license = "ZPL 2.1",
keywords = "development build",
url='http://buildout.org',
packages = ['zc', 'zc.buildout'],
package_dir = {'': 'src'},
namespace_packages = ['zc'],
install_requires = [
'setuptools>=8.0',
],
include_package_data = True,
entry_points = entry_points,
extras_require = dict(test=['zope.testing', 'manuel']),
zip_safe=False,
classifiers = [
'Intended Audience :: Developers',
'License :: OSI Approved :: Zope Public License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Topic :: Software Development :: Build Tools',
'Topic :: Software Development :: Libraries :: Python Modules',
],
)
This directory is (an experimental) place to manage Launchpad
specification bodies. Files here are referenced (via svn.zope.org
URLs) from the buildout project specifications in Launchpad,
https://features.launchpad.net/products/zc.buildout.
Repeatable (taggable) buildouts
===============================
It's important to be able to tag a buildout in a software repository
in such a way that, months, or even years later, the buildout tag can
be checked out and used to construct the same collection of parts,
with the same versions. (Note that parts could still behave
differently due to changes in parts of the environment, such as system
libraries, not controlled by the buildout.)
A feature of the buildout is it's use of eggs and the automatic
resolution of dependencies. The latest versions of dependencies are
automatically downloaded and installed. This is great during
development or when using the buildout for casual software
development, but it doesn't work very well for reproducing an old
buildout.
What's needed is some way to, when needed, record information about
the versions of eggs (and any other bits) who's versions are
determined dynamically.
Proposal
--------
We'll add a buildout option, create-repeatable. The option will
specify a file into which option information should be saved to create
a repeatable buildout. The data will be saved in a form that can be
used by the buildout or recipes in a later run. To make a tagged
buildout, a user would run the buildout with the create-repeatable
option set to a file name and then modify the buildout to be
extended-by this file.
Consider the following example buildout.cfg::
[buildout]
parts = foo
[foo]
recipe = zc.recipe.eggs
eggs = foo
eek
Now assume that:
- The current version of foo is 1.1
- Foo depends on bar =, which depends on baz. The current versions of
bar and bas are 1.1 and 2.1.
- The current version of eek is 1.5
- eek depends on ook, which is as version 1.3.
- zc.recipe.egg is at version 1.0b5
If we run the buildout with the command-line option::
buildout:create-repeatable=reapeatable.cfg
we'll get a repeatable.cfg file that looks something like::
[foo]
recipe = zc.recipe.eggs ==1.0b5
static = true
eggs = foo ==1.1
bar ==1.1
baz ==2.1
eek ==1.5
ook ==1.3
The file contains options for the foo part. The buildout software
itself added an entry for the recipe that fixes the recipe version
at the version used by the buildout.
The zc.recipe.eggs recipe added the eggs option that lists the
specific releases that were assembled.
Finally the buildout.cfg file can be modified to use the
repeatable.cfg file::
[buildout]
parts = foo
extended-by: repeatable.cfg
[foo]
recipe = zc.recipe.eggs
eggs = foo
eek
When the buildout is run, the options in repeatable.cfg will override
the ones in buildout.cfg, providing a repeatable buildout
Python API
----------
The recipe API will grow a repeatable method that is called after the
install method and is passed a dictionary that a recipe can store
option data in. A recipe instance will only be able to provide repeatable data
for it's part.
try:
__import__('pkg_resources').declare_namespace(__name__)
except:
# bootstrapping
pass
##############################################################################
#
# Copyright (c) 2006 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Buildout package
"""
class UserError(Exception):
"""Errors made by a user
"""
def __str__(self):
return " ".join(map(str, self.args))
Allow hosts
-----------
On some environments the links visited by `zc.buildout` can be forbidden
by paranoiac firewalls. These URL might be on the chain of links
visited by `zc.buildout` whether they are defined in the `find-links` option
or by various eggs in their `url`, `download_url` and `dependency_links` metadata.
It is even harder to track that package_index works like a spider and
might visit links and go to other location.
The `allow-hosts` option provides a way to prevent this, and
works exactly like the one provided in `easy_install`
(see `easy_install allow-hosts option`_).
You can provide a list of allowed host, together with wildcards::
[buildout]
...
allow-hosts =
*.python.org
example.com
Let's create a develop egg in our buildout that specifies
`dependency_links` which points to a server in the outside world::
>>> mkdir(sample_buildout, 'allowdemo')
>>> write(sample_buildout, 'allowdemo', 'dependencydemo.py',
... 'import eggrecipekss.core')
>>> write(sample_buildout, 'allowdemo', 'setup.py',
... '''from setuptools import setup; setup(
... name='allowdemo', py_modules=['dependencydemo'],
... install_requires = 'kss.core',
... dependency_links = ['http://dist.plone.org'],
... zip_safe=True, version='1')
... ''')
Now let's configure the buildout to use the develop egg,
together with some rules that disallow any web site but PyPI and
local files::
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... develop = allowdemo
... parts = eggs
... allow-hosts =
... pypi.python.org
...
... [eggs]
... recipe = zc.recipe.egg:eggs
... eggs = allowdemo
... ''')
Now we can run the buildout and make sure all attempts to dist.plone.org fails::
>>> print_(system(buildout), end='') # doctest: +ELLIPSIS
Develop: '/sample-buildout/allowdemo'
Installing eggs...
...
While:
Installing eggs.
Getting distribution for 'kss.core'.
Error: Couldn't find a distribution for 'kss.core'.
That's what we wanted : this will prevent any attempt to access
unwanted domains. For instance, some packages are listing in their
links `svn://` links. These can lead to error in some cases, and
can therefore be protected like this::
XXX (showcase with a svn:// file)
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... develop = allowdemo
... parts = eggs
... allow-hosts =
... ^(!svn://).*
...
... [eggs]
... recipe = zc.recipe.egg:eggs
... eggs = allowdemo
... ''')
Now we can run the buildout and make sure all attempts to dist.plone.org fails::
>>> print_(system(buildout), end='') # doctest: +ELLIPSIS
Develop: '/sample-buildout/allowdemo'
Installing eggs...
...
While:
Installing eggs.
Getting distribution for 'kss.core'.
Error: Couldn't find a distribution for 'kss.core'.
Test for issues
---------------
Test for 1.0.5 breakage as in https://bugs.launchpad.net/zc.buildout/+bug/239212::
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... parts=python
... foo = ${python:interpreter}
...
... [python]
... recipe=zc.recipe.egg
... eggs=zc.buildout
... interpreter=python
... ''')
>>> print_('XX'); print_(system(buildout), end='') # doctest: +ELLIPSIS
X...
Unused options for buildout: 'foo'.
Installing python.
Generated script '/sample-buildout/bin/buildout'.
Generated interpreter '/sample-buildout/bin/python'.
The bug 239212 above would have got us an *AttributeError* on
*buildout._allow_hosts*. This was fixed in this changeset:
http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/buildout.py?rev=87309&r1=87277&r2=87309
Make sure the bootstrap script actually works::
>>> import os, sys
>>> from os.path import dirname, join
>>> import zc.buildout
>>> bootstrap_py = join(
... dirname(
... dirname(
... dirname(
... dirname(zc.buildout.__file__)
... )
... )
... ),
... 'bootstrap', 'bootstrap.py')
>>> sample_buildout = tmpdir('sample')
>>> os.chdir(sample_buildout)
>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''')
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py')); print_('X') # doctest: +ELLIPSIS
X...
Creating directory '/sample/eggs'.
Creating directory '/sample/bin'.
Creating directory '/sample/parts'.
Creating directory '/sample/develop-eggs'.
Generated script '/sample/bin/buildout'.
...
>>> ls(sample_buildout)
d bin
- bootstrap.py
- buildout.cfg
d develop-eggs
d eggs
d parts
>>> ls(sample_buildout, 'bin')
- buildout
>>> print_('X'); ls(sample_buildout, 'eggs') # doctest: +ELLIPSIS
X...
d zc.buildout-...egg
By default it gets the latest version:
>>> buildout_script = join(sample_buildout, 'bin', 'buildout')
>>> if sys.platform.startswith('win'):
... buildout_script += '-script.py'
>>> print_(open(buildout_script).read()) # doctest: +ELLIPSIS
#...
sys.path[0:0] = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-22.0.0...egg',
]...
Now trying the `--buildout-version` option, that let you define a version for
`zc.buildout`.
Let's try with an unknown version::
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --buildout-version UNKNOWN')); print_('X') # doctest: +ELLIPSIS
...
X
...
No local packages or download links found for zc.buildout==UNKNOWN
...
Now let's try with `2.0.0`, which happens to exist::
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --buildout-version 2.0.0')); print_('X')
... # doctest: +ELLIPSIS
X...Generated script '/sample/bin/buildout'...X
Let's make sure the generated `buildout` script uses it::
>>> print_(open(buildout_script).read()) # doctest: +ELLIPSIS
#...
sys.path[0:0] = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-2.0.0...egg',
]...
Now trying the `--setuptools-version` option, that let you define a version for
`setuptools`.
Now let's try with `8.0`, which happens to exist::
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setuptools-version 8.0')); print_('X')
... # doctest: +ELLIPSIS
X...Generated script '/sample/bin/buildout'...X
Let's make sure the generated `buildout` script uses it::
>>> print_(open(buildout_script).read()) # doctest: +ELLIPSIS
#...
sys.path[0:0] = [
'/sample/eggs/setuptools-8.0...egg',
'/sample/eggs/zc.buildout-...egg',
]...
Now let's try specifying both `zc.buildout` and `setuptools` to versions
which happens to exist::
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setuptools-version 8.0 --buildout-version 2.0.0')); print_('X')
... # doctest: +ELLIPSIS
X...Generated script '/sample/bin/buildout'...X
Let's make sure the generated `buildout` script uses it::
>>> print_(open(buildout_script).read()) # doctest: +ELLIPSIS
#...
sys.path[0:0] = [
'/sample/eggs/setuptools-8.0...egg',
'/sample/eggs/zc.buildout-2.0.0...egg',
]...
For a completely offline install we want to avoid downloading ez_setup.py,
specify the setuptools version, and to reuse the setuptools zipfile.
>>> try:
... from urllib.request import urlopen
... except ImportError:
... from urllib2 import urlopen
>>> ez_setup = urlopen('https://bootstrap.pypa.io/ez_setup.py').read()
>>> write('ez_setup.py',
... '''print("Using local ez_setup.py")
... ''' + ez_setup.decode('ascii'))
>>> os.path.exists('setuptools-14.3.zip')
False
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setuptools-version 14.3 --buildout-version 2.0.0 '+
... '--setuptools-to-dir .')); print_('X')
... # doctest: +ELLIPSIS
X...Using local ez_setup.py...Generated script '/sample/bin/buildout'...X
>>> os.path.exists('setuptools-14.3.zip')
True
>>> print_(open(buildout_script).read()) # doctest: +ELLIPSIS
#...
sys.path[0:0] = [
'/sample/eggs/setuptools-14.3...egg',
'/sample/eggs/zc.buildout-2.0.0...egg',
]...
You can ask bootstrap.py for its version. This is really the day the last
change was made. A date leads to less confusion than a version number separate
from buildout's own version number. Similarly, tracking buildout's version
number leads to bootstraps with a new version number but without changes or to
bootstraps with version number from an already-older buildout. So: a date is
best.
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --version')); print_('X')
... # doctest: +ELLIPSIS
X...2015...X
Some people pass buildout settings to bootstrap.
>>> import os, sys
>>> from os.path import dirname, join
>>> import zc.buildout
>>> bootstrap_py = join(
... dirname(
... dirname(
... dirname(
... dirname(zc.buildout.__file__)
... )
... )
... ),
... 'bootstrap', 'bootstrap.py')
>>> top = tmpdir('top')
>>> mkdir(top, 'buildout')
>>> os.chdir(top)
>>> write('buildout', 'buildout.cfg',
... '''
... [buildout]
... parts =
... ''')
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable) +
... ' bootstrap.py buildout:directory=' + top +
... ' -c'+join('buildout', 'buildout.cfg')
... )); print_('X') # doctest: +ELLIPSIS
X...
Creating directory '/top/eggs'.
Creating directory '/top/bin'.
Creating directory '/top/parts'.
Creating directory '/top/develop-eggs'.
Generated script '/top/bin/buildout'.
...
X
They might do it with init, but no worries:
>>> os.chdir('..')
>>> remove(top)
>>> top = tmpdir('top')
>>> os.chdir(top)
>>> mkdir(top, 'buildout')
>>> os.chdir(top)
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print_('X'); print_(system(
... zc.buildout.easy_install._safe_arg(sys.executable) +
... ' bootstrap.py buildout:directory=' + top +
... ' -c'+join('buildout', 'buildout.cfg') +
... ' init'
... )); print_('X') # doctest: +ELLIPSIS
X...
Creating '/top/buildout/buildout.cfg'.
Creating directory '/top/eggs'.
Creating directory '/top/bin'.
Creating directory '/top/parts'.
Creating directory '/top/develop-eggs'.
Generated script '/top/bin/buildout'.
...
X
This diff is collapsed.
This diff is collapsed.
##############################################################################
#
# Copyright Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
# The following copied from Python 2 config parser because:
# - The py3 configparser isn't backward compatible
# - Both strip option values in undesirable ways
# - dict of dicts is a much simpler api
import re
import textwrap
import logging
logger = logging.getLogger('zc.buildout')
class Error(Exception):
"""Base class for ConfigParser exceptions."""
def _get_message(self):
"""Getter for 'message'; needed only to override deprecation in
BaseException."""
return self.__message
def _set_message(self, value):
"""Setter for 'message'; needed only to override deprecation in
BaseException."""
self.__message = value
# BaseException.message has been deprecated since Python 2.6. To prevent
# DeprecationWarning from popping up over this pre-existing attribute, use
# a new property that takes lookup precedence.
message = property(_get_message, _set_message)
def __init__(self, msg=''):
self.message = msg
Exception.__init__(self, msg)
def __repr__(self):
return self.message
__str__ = __repr__
class ParsingError(Error):
"""Raised when a configuration file does not follow legal syntax."""
def __init__(self, filename):
Error.__init__(self, 'File contains parsing errors: %s' % filename)
self.filename = filename
self.errors = []
def append(self, lineno, line):
self.errors.append((lineno, line))
self.message += '\n\t[line %2d]: %s' % (lineno, line)
class MissingSectionHeaderError(ParsingError):
"""Raised when a key-value pair is found before any section header."""
def __init__(self, filename, lineno, line):
Error.__init__(
self,
'File contains no section headers.\nfile: %s, line: %d\n%r' %
(filename, lineno, line))
self.filename = filename
self.lineno = lineno
self.line = line
# This regex captures either sections headers with optional trailing comment
# separated by a semicolon or a hash. Section headers can have an optional
# expression. Expressions and comments can contain brackets but no verbatim '#'
# and ';' : these need to be escaped.
# A title line with an expression has the general form:
# [section_name: some Python expression] #; some comment
# This regex leverages the fact that the following is a valid Python expression:
# [some Python expression] # some comment
# and that section headers are also delimited by [brackets] that are also [list]
# delimiters.
# So instead of doing complex parsing to balance brackets in an expression, we
# capture just enough from a header line to collect then remove the section_name
# and colon expression separator keeping only a list-enclosed expression and
# optional comments. The parsing and validation of this Python expression can be
# entirely delegated to Python's eval. The result of the evaluated expression is
# the always returned wrapped in a list with a single item that contains the
# original expression
section_header = re.compile(
r'(?P<head>\[)'
r'\s*'
r'(?P<name>[^\s#[\]:;{}]+)'
r'\s*'
r'(:(?P<expression>[^#;]*))?'
r'\s*'
r'(?P<tail>]'
r'\s*'
r'([#;].*)?$)'
).match
option_start = re.compile(
r'(?P<name>[^\s{}[\]=:]+\s*[-+]?)'
r'='
r'(?P<value>.*)$').match
leading_blank_lines = re.compile(r"^(\s*\n)+")
def parse(fp, fpname, exp_globals=dict):
"""Parse a sectioned setup file.
The sections in setup files contain a title line at the top,
indicated by a name in square brackets (`[]'), plus key/value
options lines, indicated by `name: value' format lines.
Continuations are represented by an embedded newline then
leading whitespace. Blank lines, lines beginning with a '#',
and just about everything else are ignored.
The title line is in the form [name] followed by an optional trailing
comment separated by a semicolon `;' or a hash `#' character.
Optionally the title line can have the form `[name:expression]' where
expression is an arbitrary Python expression. Sections with an expression
that evaluates to False are ignored. Semicolon `;' an hash `#' characters
must be string-escaped in expression literals.
exp_globals is a callable returning a mapping of defaults used as globals
during the evaluation of a section conditional expression.
"""
sections = {}
# the current section condition, possibly updated from a section expression
section_condition = True
context = None
cursect = None # None, or a dictionary
blockmode = None
optname = None
lineno = 0
e = None # None, or an exception
while True:
line = fp.readline()
if not line:
break # EOF
lineno = lineno + 1
if line[0] in '#;':
continue # comment
if line[0].isspace() and cursect is not None and optname:
if not section_condition:
#skip section based on its expression condition
continue
# continuation line
if blockmode:
line = line.rstrip()
else:
line = line.strip()
if not line:
continue
cursect[optname] = "%s\n%s" % (cursect[optname], line)
else:
header = section_header(line)
if header:
# reset to True when starting a new section
section_condition = True
sectname = header.group('name')
head = header.group('head') # the starting [
expression = header.group('expression')