Commit f91c2a08 authored by Kazuhiko Shiozaki's avatar Kazuhiko Shiozaki

Merge tag '1.7.1' into rebase_1.7.1

Conflicts:
	.gitignore
	setup.py
	src/zc/buildout/buildout.py
	src/zc/buildout/download.py
	src/zc/buildout/easy_install.py
	src/zc/buildout/testing.py
	src/zc/buildout/tests.py
parents 6a520b75 beaa8cb1
*pyc
*pyo
.installed.cfg
bin/
build/
develop-eggs/
dist/
eggs/
parts/
src/zc.buildout.egg-info/
z3c.recipe.scripts_/src/z3c.recipe.scripts.egg-info/
zc.recipe.egg_/src/zc.recipe.egg.egg-info/
*.pyc
*.egg
*.egg-info
dist
language: python
env:
- PYTHON_VER=2.4
- PYTHON_VER=2.5
- PYTHON_VER=2.6
- PYTHON_VER=2.7
branches:
only:
- 1.6.x
install:
- deactivate
- make build
script: make test
Change History
**************
1.6.0 (unreleased)
1.7.1 (2013-02-21)
==================
Fixed: Constraints intended to prevent upgrading to
buildout-2-compatible recipes weren't expressed correctly,
leading to unintendional use of zc.recipe.egg-2.0.0a3.
1.7.0 (2013-01-11)
==================
- Unless version requirements are specified, buildout won't upgrade
itself past version 1.
- Versions in versions sections can now be simple constraints, like
<2.0dev in addition to being simple versions.
This is used to prevent upgrading zc.recipe.egg and
zc.recipe.testrunner past version 1.
- If buildout is bootstrapped with a non-final release, it
won't downgrade itself to a final release.
- Fix: distribute 0.6.33 broke Python 2.4 compatibility
- remove `data_files` from `setup.py`, which was installing README.txt
in current directory during installation
[Domen Kožar]
- Windows fix: use cli-64.exe/cli.exe depending on 64/32 bit
and try cli.exe if cli-64.exe is not found,
fixing 9c6be7ac6d218f09e33725e07dccc4af74d8cf97
- Windows fix: `buildout init` was broken, re.sub does not like a
single backslash
- fixed all builds on travis-ci
[Domen Kožar]
- use os._exit insted of sys.exit after ugrade forking
[Domen Kožar]
- Revert cfa0478937d16769c268bf51e60e69cd3ead50f3, it only broke a feature
[Domen Kožar]
1.6.3 (2012-08-22)
==================
- Fix Windows regression (see: https://github.com/buildout/buildout/commit/90bc44f9bffd0d9eb09aacf08c6a4c2fed797319
and https://github.com/buildout/buildout/commit/e65b7bfbd7c7ccd556a278016a16b63ae8ef782b)
[aclark4life]
1.6.2 (2012-08-21)
==================
- Fix Windows regression (see: https://github.com/buildout/buildout/commit/cfa0478937d16769c268bf51e60e69cd3ead50f3)
[aclark4life]
1.6.1 (2012-08-18)
==================
- `bootstrap.py -d init` would invoke buildout with arguments
`init bootstrap` leading into installation of bootstrap package.
now bootstrap.py first runs any commands passed, then tries to
bootstrap. (Domen Kožar)
- fix Python 2.4 support (Domen Kožar)
- added travis-ci testing (Domen Kožar)
1.6.0 (2012-08-15)
==================
- The buildout init command now accepts distribution requirements and
......@@ -25,6 +94,17 @@ Change History
- Made sure to download extended configuration files only once per buildout
run even if they are referenced multiple times (patch by Rafael Monnerat).
- Ported speedup optimization patch by Ross Patterson to 1.5.x series.
Improved patch to calculate required_by packages in linear time
in verbose mode (-v). Running relatively simple Buildout envornment
yielded in running time improvement from 30 seconds to 10 seconds.
(Domen Kožar, Ross Patterson)
- Removed unnecessary pyc recompilation with optimization flags. Running
Buildout with pre-downloaded ~300 packages that were installed in empty
eggs repository yielded in running time improvement from 1126 seconds
to 348 seconds. (Domen Kožar)
Bugs fixed:
- In the download module, fixed the handling of directories that are pointed
......@@ -36,6 +116,9 @@ Bugs fixed:
- https://bugs.launchpad.net/bugs/697913 : Buildout doesn't honor exit code
from scripts. Fixed.
- Handle both addition and subtraction of elements (+= and -=) on the same key
in the same section.
1.5.2 (2010-10-11)
==================
......
include *.txt
recursive-include src *.txt
exclude MANIFEST.in buildout.cfg .bzrignore
exclude MANIFEST.in buildout.cfg .bzrignore .gitignore
HERE = $(shell pwd)
PYTHON_VER ?= 2.7
PYTHON_MINOR ?= 2.7.3
PYTHON_PATH = $(HERE)/python$(PYTHON_VER)
# see http://lipyrary.blogspot.com/2011/05/how-to-compile-python-on-ubuntu-1104.html
ARCH = $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
ifeq ($(PYTHON_VER),2.6)
PYTHON_MINOR = 2.6.8
endif
ifeq ($(PYTHON_VER),2.5)
PYTHON_MINOR = 2.5.6
endif
ifeq ($(PYTHON_VER),2.4)
PYTHON_MINOR = 2.4.6
endif
PYTHON_ARCHIVE ?= Python-$(PYTHON_MINOR)
PYTHON_DOWNLOAD = http://www.python.org/ftp/python/$(PYTHON_MINOR)/$(PYTHON_ARCHIVE).tgz
PYTHON_EXE = python$(PYTHON_VER)
.PHONY: all build test
BUILD_DIRS = $(PYTHON_PATH) bin build develop-eggs eggs parts
all: build
$(PYTHON_PATH):
@echo "Installing Python"
# see http://mail.python.org/pipermail/python-bugs-list/2007-April/038211.html
sudo mv /usr/include/sqlite3.h /tmp/
sudo apt-get install zlibc
mkdir -p $(PYTHON_PATH)
cd $(PYTHON_PATH) && \
curl --progress-bar $(PYTHON_DOWNLOAD) | tar -zx
ifeq ($(PYTHON_VER),2.6)
cd $(PYTHON_PATH) && \
curl --progress-bar https://raw.github.com/collective/buildout.python/master/src/issue12012-sslv2-py26.txt > ssl.txt
cd $(PYTHON_PATH)/$(PYTHON_ARCHIVE) && \
patch -p0 < ../ssl.txt
endif
cd $(PYTHON_PATH)/$(PYTHON_ARCHIVE) && \
./configure LDFLAGS="-L/usr/lib/$(ARCH) -L/lib/$(ARCH)" --prefix $(PYTHON_PATH) --with-zlib=$(PYTHON_PATH)/include
ifeq ($(PYTHON_VER),2.4)
sed -i "s/^#zlib/zlib/g" $(PYTHON_PATH)/$(PYTHON_ARCHIVE)/Modules/Setup
endif
cd $(PYTHON_PATH)/$(PYTHON_ARCHIVE) && \
make && \
make install >/dev/null 2>&1
@echo "Finished installing Python"
build: $(PYTHON_PATH)
$(PYTHON_PATH)/bin/$(PYTHON_EXE) dev.py
clean:
rm -rf $(BUILD_DIRS)
test:
$(HERE)/bin/test -v
......@@ -135,7 +135,7 @@ If I need to run a previous version of zc.buildout, I use the
$ python bootstrap.py --version 1.1.3
The `zc.buildout project <http://svn.zope.org/zc.buildout/trunk>`_
The `zc.buildout project <https://github.com/buildout/buildout>`_
is a slightly more complex example of this type of buildout.
Install egg-based scripts
......
......@@ -32,7 +32,7 @@ the biggest is that Buildout itself supports usage with a system Python.
This can work if you follow a couple of simple rules.
1. Use the new bootstrap.py (available from
svn://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py).
http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py).
2. Use buildout recipes that have been upgraded to work with zc.buildout 1.5
and higher. Specifically, they should use
......@@ -188,7 +188,7 @@ If you want to support the other arguments (``include_site_packages``,
``exec_sitecustomize``, ``script_initialization``, as well as the
``allowed-eggs-from-site-packages`` option), you might want to look at
some of the code in
svn://svn.zope.org/repos/main/zc.buildout/trunk/z3c.recipe.scripts\_/src/z3c/recipe/scripts/scripts.py .
https://github.com/buildout/buildout/blob/1.6.x/z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py .
You might even be able to adopt some of it by subclassing or delegating.
The Scripts class in that file is the closest to what you might be used
to from zc.recipe.egg.
......
......@@ -143,10 +143,6 @@ parser.add_option("-c", None, action="store", dest="config_file",
options, args = parser.parse_args()
# if -c was provided, we push it back into args for buildout's main function
if options.config_file is not None:
args += ['-c', options.config_file]
if options.eggs:
eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
else:
......@@ -159,8 +155,7 @@ if options.setup_source is None:
options.setup_source = setuptools_source
if options.accept_buildout_test_releases:
args.append('buildout:accept-buildout-test-releases=true')
args.append('bootstrap')
args.insert(0, 'buildout:accept-buildout-test-releases=true')
try:
import pkg_resources
......@@ -177,6 +172,8 @@ except ImportError:
setup_args['download_base'] = options.download_base
if options.use_distribute:
setup_args['no_fake'] = True
if sys.version_info[:2] == (2, 4):
setup_args['version'] = '0.6.32'
ez['use_setuptools'](**setup_args)
if 'pkg_resources' in sys.modules:
reload(sys.modules['pkg_resources'])
......@@ -199,6 +196,8 @@ if not has_broken_dash_S:
find_links = options.buildout_download_base or options.download_base
if not find_links:
find_links = os.environ.get('bootstrap-testing-find-links')
if not find_links and options.accept_buildout_test_releases:
find_links = 'http://downloads.buildout.org/'
if find_links:
cmd.extend(['-f', quote(find_links)])
......@@ -235,6 +234,8 @@ if version is None and not options.accept_buildout_test_releases:
bestv = None
for dist in index[req.project_name]:
distv = dist.parsed_version
if distv >= pkg_resources.parse_version('2dev'):
continue
if _final_version(distv):
if bestv is None or distv > bestv:
best = [dist]
......@@ -244,8 +245,12 @@ if version is None and not options.accept_buildout_test_releases:
if best:
best.sort()
version = best[-1].version
if version:
requirement = '=='.join((requirement, version))
requirement += '=='+version
else:
requirement += '<2dev'
cmd.append(requirement)
if is_jython:
......@@ -264,6 +269,16 @@ if exitcode != 0:
ws.add_entry(eggs_dir)
ws.require(requirement)
import zc.buildout.buildout
# If there isn't already a command in the args, add bootstrap
if not [a for a in args if '=' not in a]:
args.append('bootstrap')
# if -c was provided, we push it back into args for buildout's main function
if options.config_file is not None:
args[0:0] = ['-c', options.config_file]
zc.buildout.buildout.main(args)
if not options.eggs: # clean up temporary egg directory
shutil.rmtree(eggs_dir)
[buildout]
develop = zc.recipe.egg_ z3c.recipe.scripts_ .
develop =
zc.recipe.egg_
z3c.recipe.scripts_
.
parts = test oltest py
versions = versions
[py]
recipe = z3c.recipe.scripts
......@@ -28,3 +32,9 @@ defaults =
'!(bootstrap|selectingpython|selecting-python)',
]
# python2.4 compatibility
[versions]
zope.interface = 3.6.7
zope.exceptions = 3.7.1
zope.testrunner = 4.0.1
......@@ -12,7 +12,7 @@
#
##############################################################################
name = "zc.buildout"
version = "1.6.0-dev-SlapOS-014"
version = "1.7.1-dev-SlapOS-001"
import os
from setuptools import setup
......@@ -77,8 +77,6 @@ setup(
license = "ZPL 2.1",
keywords = "development build",
url='http://git.erp5.org/gitweb/slapos.buildout.git',
data_files = [('.', ['README.txt'])],
packages = ['zc', 'zc.buildout'],
package_dir = {'': 'src'},
namespace_packages = ['zc'],
......
......@@ -86,7 +86,7 @@ selected the (generated-locally) 99.99 egg rather than the also-available
"...
buildout_paths = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-99.99-pyN.N.egg'
'/sample/eggs/zc.buildout-1.99.99-pyN.N.egg'
]
...
......@@ -106,11 +106,11 @@ Here's an example.
"...
buildout_paths = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-100.0b1-pyN.N.egg'
'/sample/eggs/zc.buildout-1.100.0b1-pyN.N.egg'
]
...
Notice we are now using zc.buildout 100.0b1, a non-final release.
Notice we are now using zc.buildout 1.100.0b1, a non-final release.
The buildout script remembers the decision to accept early releases, and
alerts the user.
......@@ -141,7 +141,7 @@ As the note says, to undo, you just need to re-run bootstrap without
"...
buildout_paths = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-99.99-pyN.N.egg'
'/sample/eggs/zc.buildout-1.99.99-pyN.N.egg'
]
...
>>> ('buildout:accept-buildout-test-releases=true' in
......@@ -232,7 +232,7 @@ Let's make sure the generated ``site.py`` uses it::
"...
buildout_paths = [
'/sample/eggs/distribute-...egg',
'/sample/eggs/zc.buildout-99.99-pyN.N.egg'
'/sample/eggs/zc.buildout-1.99.99-pyN.N.egg'
]
...
......@@ -347,3 +347,15 @@ Here's the entire help text.
or betas.
-c CONFIG_FILE Specify the path to the buildout configuration file to
be used.
Rebootstrap and create config file.
>>> remove('buildout.cfg')
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable) +
... ' bootstrap.py init') # doctest: +NORMALIZE_WHITESPACE
Creating '/sample/buildout.cfg'.
>>> cat('buildout.cfg') # doctest: +NORMALIZE_WHITESPACE
[buildout]
parts =
Bootstrap.py won't bootstrap buildout 2 even if there's a 2 egg in the
user's egg cache:
>>> import os
>>> home = os.environ['HOME']
>>> mkdir(home)
>>> mkdir(home, '.buildout')
>>> write(home, '.buildout', 'default.cfg', """
... [buildout]
... eggs-directory = %s
... """ % sample_eggs)
>>> import os, sys
>>> from os.path import dirname, join
>>> import zc.buildout
>>> bootstrap_py = join(
... dirname(
... dirname(
... dirname(
... dirname(zc.buildout.__file__)
... )
... )
... ),
... 'bootstrap', 'bootstrap.py')
>>> sample_buildout = tmpdir('sample')
>>> os.chdir(sample_buildout)
>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''')
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py'); print 'X' # doctest: +ELLIPSIS
X...
Creating directory '/sample/bin'.
Creating directory '/sample/parts'.
Creating directory '/sample/develop-eggs'.
Generated script '/sample/bin/buildout'.
...
The bootstrap process prefers final versions of zc.buildout, so it has
selected the (generated-locally) 99.99 egg rather than the also-available
100.0b1 egg. We can see that in the buildout script's site.py.
>>> buildout_site_py = join(
... sample_buildout, 'parts', 'buildout', 'site.py')
>>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
"...
buildout_paths = [
'/sample_eggs/setuptools-...egg',
'/sample_eggs/zc.buildout-1.99.99-pyN.N.egg'
]
...
It hasn't chosen buildout2.
Some people pass buildout settings to bootstrap.
>>> import os, sys
>>> from os.path import dirname, join
>>> import zc.buildout
>>> bootstrap_py = join(
... dirname(
... dirname(
... dirname(
... dirname(zc.buildout.__file__)
... )
... )
... ),
... 'bootstrap', 'bootstrap.py')
>>> top = tmpdir('top')
>>> mkdir(top, 'buildout')
>>> os.chdir(top)
>>> write('buildout', 'buildout.cfg',
... '''
... [buildout]
... parts =
... ''')
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable) +
... ' bootstrap.py buildout:directory=' + top +
... ' -c'+join('buildout', 'buildout.cfg')
... ); print 'X' # doctest: +ELLIPSIS
X
Creating directory '/top/bin'.
Creating directory '/top/parts'.
Creating directory '/top/eggs'.
Creating directory '/top/develop-eggs'.
Generated script '/top/bin/buildout'.
...
X
They might do it with init, but no worries:
>>> remove(top)
>>> top = tmpdir('top')
>>> mkdir(top, 'buildout')
>>> os.chdir(top)
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable) +
... ' bootstrap.py buildout:directory=' + top +
... ' -c'+join('buildout', 'buildout.cfg') +
... ' init'
... ); print 'X' # doctest: +ELLIPSIS
X
Creating '/top/buildout/buildout.cfg'.
Creating directory '/top/bin'.
Creating directory '/top/parts'.
Creating directory '/top/eggs'.
Creating directory '/top/develop-eggs'.
Generated script '/top/bin/buildout'.
...
X
......@@ -355,10 +355,14 @@ class Buildout(UserDict.DictMixin):
self._setup_logging()
versions = options.get('versions')
if versions:
zc.buildout.easy_install.default_versions(dict(self[versions]))
self.versions = {
'zc.recipe.egg': '<2dev',
'zc.recipe.testrunner': '<2dev',
}
versions_option = options.get('versions')
if versions_option:
self.versions.update(self[versions_option])
zc.buildout.easy_install.default_versions(self.versions)
self.offline = options.get_bool('offline')
if self.offline:
......@@ -504,7 +508,7 @@ class Buildout(UserDict.DictMixin):
# Get a base working set for our distributions that corresponds to the
# stated desires in the configuration.
distributions = ['setuptools', 'zc.buildout']
distributions = ['setuptools', 'zc.buildout<2dev']
if LIBNETWORKCACHE_ENABLED:
distributions.append('slapos.libnetworkcache')
if options.get('offline') == 'true':
......@@ -599,11 +603,14 @@ class Buildout(UserDict.DictMixin):
def _init_config(self, config_file, args):
print 'Creating %r.' % config_file
f = open(config_file, 'w')
sep = re.compile(r'[\\/]')
if args:
sep = re.compile(r'[\\/]')
ossep = os.path.sep
if ossep == '\\':
ossep = '\\\\' # re.sub does not like a single backslash
eggs = '\n '.join(a for a in args if not sep.search(a))
paths = '\n '.join(
sep.sub(os.path.sep, a) for a in args if sep.search(a))
sep.sub(ossep, a) for a in args if sep.search(a))
f.write('[buildout]\n'
'parts = py\n'
'\n'
......@@ -1057,13 +1064,22 @@ class Buildout(UserDict.DictMixin):
if not self.newest:
return
options = self['buildout']
specs = ['zc.buildout']
if zc.buildout.easy_install.is_distribute:
specs.append('distribute')
else:
specs.append('setuptools')
# Prevent downgrading due to prefer-final:
options = self['buildout']
if not ('zc.buildout-version' in options
or
'zc.buildout' in self.versions):
v = pkg_resources.working_set.find(
pkg_resources.Requirement.parse('zc.buildout')
).version
options['zc.buildout-version'] = '>=' + v + ', <2dev'
ws = zc.buildout.easy_install.install(
[
(spec + ' ' + options.get(spec+'-version', '')).strip()
......@@ -1106,7 +1122,7 @@ class Buildout(UserDict.DictMixin):
if sys.platform == 'win32':
should_run += '-script.py'
if (realpath(os.path.abspath(sys.argv[0])) != should_run):
if realpath(os.path.abspath(sys.argv[0])) != should_run:
self._logger.debug("Running %r.", realpath(sys.argv[0]))
self._logger.debug("Local buildout is %r.", should_run)
self._logger.warn("Not upgrading because not running a local "
......@@ -1165,10 +1181,10 @@ class Buildout(UserDict.DictMixin):
)
# Restart
args = map(zc.buildout.easy_install._safe_arg, sys.argv)
args = list(sys.argv)
if not __debug__:
args.insert(0, '-O')
args.insert(0, zc.buildout.easy_install._safe_arg(sys.executable))
args.insert(0, sys.executable)
# We want to make sure that our new site.py is used for rerunning
# buildout, so we put the partsdir in PYTHONPATH for our restart.
# This overrides any set PYTHONPATH, but since we generally are
......@@ -1176,7 +1192,9 @@ class Buildout(UserDict.DictMixin):
# library) then that should be fine.
env = os.environ.copy()
env['PYTHONPATH'] = partsdir
sys.exit(subprocess.Popen(args, env=env).wait())
# windows: Popen will quote args itself if needed
# see subprocess.list2cmdline
os._exit(subprocess.Popen(args, env=env).wait())
def _load_extensions(self):
__doing__ = 'Loading extensions.'
......@@ -1806,19 +1824,26 @@ def _dists_sig(dists):
return result
def _update_section(s1, s2):
# Base section 2 on section 1; section 1 is copied, with key-value pairs
# in section 2 overriding those in section 1. If there are += or -=
# operators in section 2, process these to add or substract items (delimited
# by newlines) from the preexisting values.
s2 = s2.copy() # avoid mutating the second argument, which is unexpected
for k, v in s2.items():
# Sort on key, then on the addition or substraction operator (+ comes first)
for k, v in sorted(s2.items(), key=lambda x: (x[0].rstrip(' +'), x[0][-1])):
v2, note2 = v
if k.endswith('+'):
key = k.rstrip(' +')
v1, note1 = s1.get(key, ("", ""))
# Find v1 in s2 first; it may have been defined locally too.
v1, note1 = s2.get(key, s1.get(key, ("", "")))
newnote = ' [+] '.join((note1, note2)).strip()
s2[key] = "\n".join((v1).split('\n') +
v2.split('\n')), newnote
del s2[k]
elif k.endswith('-'):
key = k.rstrip(' -')
v1, note1 = s1.get(key, ("", ""))
# Find v1 in s2 first; it may have been set by a += operation first
v1, note1 = s2.get(key, s1.get(key, ("", "")))
newnote = ' [-] '.join((note1, note2)).strip()
s2[key] = ("\n".join(
[v for v in v1.split('\n')
......
......@@ -440,6 +440,10 @@ the mkdir recipe to support multiple paths:
... pass
... """)
..
>>> clean_up_pyc(sample_buildout, 'recipes', 'mkdir.py')
If there is an error creating a path, the install method will exit and
leave previously created paths in place:
......@@ -549,6 +553,10 @@ Let's fix the recipe:
... pass
... """)
..
>>> clean_up_pyc(sample_buildout, 'recipes', 'mkdir.py')
And put back the typo:
>>> write(sample_buildout, 'buildout.cfg',
......@@ -578,11 +586,6 @@ When we rerun the buildout:
...
OSError: [Errno 17] File exists: '/sample-buildout/bin'
.. Wait for the file to really disappear. My linux is weird.
>>> wait_until("foo goes away", lambda : not os.path.exists('foo'),
... timeout=200)
we get the same error, but we don't get the directory left behind:
>>> os.path.exists('foo')
......@@ -636,7 +639,7 @@ recipe:
..
>>> remove(sample_buildout, 'recipes', 'mkdir.pyc')
>>> clean_up_pyc(sample_buildout, 'recipes', 'mkdir.py')
We returned by calling created, taking advantage of the fact that it
returns the registered paths. We did this for illustrative purposes.
......@@ -1192,12 +1195,21 @@ This is illustrated below; first we define a base configuration.
...
... [part2]
... recipe =
... option = b1 b2 b3 b4
... option = b1
... b2
... b3
... b4
...
... [part3]
... recipe =
... option = c1 c2
...
... [part4]
... recipe =
... option = d2
... d3
... d5
...
... """)
Extending this configuration, we can "adjust" the values set in the
......@@ -1214,16 +1226,22 @@ base configuration file.
...
... # removing values
... [part2]
... option -= b1 b2
... option -= b1
... b2
...
... # alt. spelling
... [part3]
... option+=c3 c4 c5
...
... # normal assignment
... # combining both adding and removing
... [part4]
... option = h1 h2
... option += d1
... d4
... option -= d5
...
... # normal assignment
... [part5]
... option = h1 h2
... """)
An additional extension.
......@@ -1239,7 +1257,9 @@ An additional extension.
...
... # removing values
... [part2]
... option -= b1 b2 b3
... option -= b1
... b2
... b3
...
... """)
......@@ -1274,13 +1294,13 @@ Set up a buildout configuration for this extension.
... """)
>>> os.chdir(sample_buildout)
>>> print system(os.path.join(sample_buildout, 'bin', 'buildout')),
>>> print system(os.path.join(sample_buildout, 'bin', 'buildout')), # doctest: +ELLIPSIS
Develop: '/sample-buildout/demo'
Uninstalling myfiles.
Getting distribution for 'recipes'.
zip_safe flag not set; analyzing archive contents...
Got recipes 0.0.0.
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
warning: install_lib: 'build/lib...' does not exist -- no Python modules to install
Verify option values.
......@@ -1293,7 +1313,7 @@ Verify option values.
... """)
>>> print system(os.path.join('bin', 'buildout')),
['a1 a2/na3 a4/na5', 'b1 b2 b3 b4', 'c1 c2/nc3 c4 c5', 'h1 h2']
['a1 a2/na3 a4/na5', 'b4', 'c1 c2/nc3 c4 c5', 'd2/nd3/nd1/nd4', 'h1 h2']
Develop: '/sample-buildout/demo'
Annotated sections output shows which files are responsible for which
......@@ -1317,7 +1337,7 @@ operations.
/sample-buildout/base.cfg
<BLANKLINE>
[part2]
option= b1 b2 b3 b4
option= b4
/sample-buildout/base.cfg
-= /sample-buildout/extension1.cfg
-= /sample-buildout/extension2.cfg
......@@ -1333,6 +1353,17 @@ operations.
/sample-buildout/base.cfg
<BLANKLINE>
[part4]
option= d2
d3
d1
d4
/sample-buildout/base.cfg
+= /sample-buildout/extension1.cfg
-= /sample-buildout/extension1.cfg
recipe=
/sample-buildout/base.cfg
<BLANKLINE>
[part5]
option= h1 h2
/sample-buildout/extension1.cfg
......@@ -2335,7 +2366,7 @@ database is shown.
... """)
>>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE
Installing 'zc.buildout', 'setuptools'.
Installing 'zc.buildout >=1.9a1, <2dev', 'setuptools'.
We have a develop egg: zc.buildout X.X.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = V.V
......@@ -2369,6 +2400,7 @@ database is shown.
unzip = false
use-dependency-links = true
verbosity = 20
zc.buildout-version = >=1.9a1, <2dev
<BLANKLINE>
All of these options can be overridden by configuration files or by
......@@ -2659,7 +2691,7 @@ or paths to use:
>>> print system(buildout + ' -csetup.cfg init demo other ./src'),
Creating '/sample-bootstrapped/setup.cfg'.
Generated script '/sample-bootstrapped/bin/buildout'.
Getting distribution for 'zc.recipe.egg'.
Getting distribution for 'zc.recipe.egg<2dev'.
Got zc.recipe.egg 1.3.3dev.
Installing py.
Getting distribution for 'demo'.
......
......@@ -219,11 +219,15 @@ class Download(object):
self.shacache_key_file,
self.shadir_cert_file,
self.shadir_key_file)
finally:
os.close(handle)
except:
os.remove(tmp_path)
raise
except IOError, e:
os.remove(tmp_path)
raise zc.buildout.UserError("Error downloading extends for URL"
" %s: %r" % (url, e[1:3]))
except Exception, e:
os.remove(tmp_path)
raise
finally:
os.close(handle)
if path:
shutil.move(tmp_path, path)
......
......@@ -30,6 +30,7 @@ import setuptools.archive_util
import setuptools.command.setopt
import setuptools.package_index
import shutil
import struct
import subprocess
import sys
import tempfile
......@@ -59,6 +60,7 @@ logger = logging.getLogger('zc.buildout.easy_install')
url_match = re.compile('[a-z0-9+.-]+://').match
is_win32 = sys.platform == 'win32'
is_64 = struct.calcsize("P") == 8
is_jython = sys.platform.startswith('java')
is_distribute = (
pkg_resources.Requirement.parse('setuptools').key=='distribute')
......@@ -174,10 +176,6 @@ def _get_version_info(executable):
return eval(stdout.strip())
class IncompatibleVersionError(zc.buildout.UserError):
"""A specified version is incompatible with a given requirement.
"""
_versions = {sys.executable: '%d.%d' % sys.version_info[:2]}
def _get_version(executable):
try:
......@@ -1049,17 +1047,9 @@ class Installer:
def _constrain(self, requirement):
if is_distribute and requirement.key == 'setuptools':
requirement = pkg_resources.Requirement.parse('distribute')
version = self._versions.get(requirement.project_name)
if version:
if version not in requirement:
logger.error("The version, %s, is not consistent with the "
"requirement, %r.", version, str(requirement))
raise IncompatibleVersionError("Bad version", version)
requirement = pkg_resources.Requirement.parse(
"%s[%s] ==%s" % (requirement.project_name,
','.join(requirement.extras),
version))
constraint = self._versions.get(requirement.project_name)
if constraint:
requirement = _constrained_requirement(constraint, requirement)
return requirement
......@@ -1383,12 +1373,19 @@ def develop(setup, dest,
args[1] == '-v'
if log_level < logging.DEBUG:
logger.debug("in: %r\n%s", directory, ' '.join(args))
try:
subprocess.check_call([_safe_arg(executable)] + args)
except subprocess.CalledProcessError:
raise zc.buildout.UserError("Installing develop egg failed")
# XXX It looks like someone tried to get clever with "bad" develop eggs, but this
# currently fails on Windows
#p = subprocess.Popen(
# [_safe_arg(executable)] + args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
#if p.wait() > 0:
# raise zc.buildout.UserError("Installing develop egg failed: %s" % p.stderr.read())
if is_jython:
assert subprocess.Popen([_safe_arg(executable)] + args).wait() == 0
else:
assert os.spawnl(os.P_WAIT, executable, _safe_arg(executable),
*args) == 0
return _copyeggs(tmp3, dest, '.egg-link', undo)
finally:
......@@ -1655,7 +1652,15 @@ def _write_script(full_name, contents, logged_type):
script_name += '-script.py'
# Generate exe file and give the script a magic name.
exe = full_name + '.exe'
new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
if is_64:
resource = 'cli-64.exe'
else:
resource = 'cli.exe'
try:
new_data = pkg_resources.resource_string('setuptools', resource)
except IOError:
# setuptools has just cli.exe, no matter if 64/32 bit
new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
if not os.path.exists(exe) or (open(exe, 'rb').read() != new_data):
# Only write it if it's different.
open(exe, 'wb').write(new_data)
......@@ -2052,8 +2057,9 @@ class MissingDistribution(zc.buildout.UserError):
return "Couldn't find a distribution for %r." % str(req)
def _log_requirement(ws, req):
ws = list(ws)
ws.sort()
if not logger.isEnabledFor(logging.DEBUG):
return
for dist in ws:
if req in dist.requires():
logger.debug(" required by %s." % dist)
......@@ -2098,15 +2104,111 @@ def redo_pyc(egg):
py_compile.compile(filepath)
except py_compile.PyCompileError:
logger.warning("Couldn't compile %s", filepath)
else:
# Recompile under other optimization. :)
args = [_safe_arg(sys.executable)]
if __debug__:
args.append('-O')
args.extend(['-m', 'py_compile', _safe_arg(filepath)])
if is_jython:
subprocess.call([sys.executable, args])
else:
os.spawnv(os.P_WAIT, sys.executable, args)
def _constrained_requirement(constraint, requirement):
return pkg_resources.Requirement.parse(
"%s[%s]%s" % (
requirement.project_name,
','.join(requirement.extras),
_constrained_requirement_constraint(constraint, requirement)
)
)
class IncompatibleConstraintError(zc.buildout.UserError):
"""A specified version is incompatible with a given requirement.
"""
IncompatibleVersionError = IncompatibleConstraintError # Backward compatibility
def bad_constraint(constraint, requirement):
logger.error("The constraint, %s, is not consistent with the "
"requirement, %r.", constraint, str(requirement))
raise IncompatibleConstraintError("Bad constraint", constraint, requirement)
_parse_constraint = re.compile(r'([<>]=?)(\S+)').match
_comparef = {
'>' : lambda x, y: x > y,
'>=': lambda x, y: x >= y,
'<' : lambda x, y: x < y,
'<=': lambda x, y: x <= y,
}
_opop = {'<': '>', '>': '<'}
_opeqop = {'<': '>=', '>': '<='}
def _constrained_requirement_constraint(constraint, requirement):
# Simple cases:
# No specs tp merge with:
if not requirement.specs:
if not constraint[0] in '<=>':
constraint = '==' + constraint
return constraint
# Simple single-version constraint:
if constraint[0] not in '<>':
if constraint.startswith('='):
assert constraint.startswith('==')
constraint = constraint[2:]
if constraint in requirement:
return '=='+constraint
bad_constraint(constraint, requirement)
# OK, we have a complex constraint (<. <=, >=, or >) and specs.
# In many cases, the spec needs to filter constraints.
# In other cases, the constraints need to limit the constraint.
specs = requirement.specs
cop, cv = _parse_constraint(constraint).group(1, 2)
pcv = pkg_resources.parse_version(cv)
# Special case, all of the specs are == specs:
if not [op for (op, v) in specs if op != '==']:
# There aren't any non-== specs.
# See if any of the specs satisfy the constraint:
specs = [op+v for (op, v) in specs
if _comparef[cop](pkg_resources.parse_version(v), pcv)]
if specs:
return ','.join(specs)
bad_constraint(constraint, requirement)
cop0 = cop[0]
# Normalize specs by splitting >= and <= specs. We meed tp do this
# becaise these have really weird semantics. Also cache parsed
# versions, which we'll need for comparisons:
specs = []
for op, v in requirement.specs:
pv = pkg_resources.parse_version(v)
if op == _opeqop[cop0]:
specs.append((op[0], v, pv))
specs.append(('==', v, pv))
else:
specs.append((op, v, pv))
# Error if there are opposite specs that conflict with the constraint
# and there are no equal specs that satisfy the constraint:
if [v for (op, v, pv) in specs
if op == _opop[cop0] and _comparef[_opop[cop0]](pv, pcv)
]:
eqspecs = [op+v for (op, v, pv) in specs
if _comparef[cop](pv, pcv)]
if eqspecs:
# OK, we do, use these:
return ','.join(eqspecs)
bad_constraint(constraint, requirement)
# We have a combination of range constraints and eq specs that
# satisfy the requirement.
# Return the constraint + the filtered specs
return ','.join(
op+v
for (op, v) in (
[(cop, cv)] +
[(op, v) for (op, v, pv) in specs if _comparef[cop](pv, pcv)]
)
)
......@@ -319,13 +319,13 @@ we'll get an error:
... versions = dict(demo='0.2', demoneeded='1.0'))
Traceback (most recent call last):
...
IncompatibleVersionError: Bad version 0.2
IncompatibleConstraintError: Bad constraint 0.2 demo>0.2
>>> print handler
zc.buildout.easy_install DEBUG
Installing 'demo >0.2'.
zc.buildout.easy_install ERROR
The version, 0.2, is not consistent with the requirement, 'demo>0.2'.
The constraint, 0.2, is not consistent with the requirement, 'demo>0.2'.
>>> handler.clear()
......
......@@ -124,7 +124,7 @@ about versions used. If we run the buildout in verbose mode without
specifying a versions section:
>>> print system(buildout+' buildout:versions= -v'), # doctest: +ELLIPSIS
Installing 'zc.buildout', 'setuptools'.
Installing 'zc.buildout >=1.99, <2dev', 'setuptools'.
We have a develop egg: zc.buildout 1.0.0.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6
......@@ -146,7 +146,7 @@ that we can fix them in a versions section.
If we run the buildout with the versions section:
>>> print system(buildout+' -v'), # doctest: +ELLIPSIS
Installing 'zc.buildout', 'setuptools'.
Installing 'zc.buildout >=1.99, <2dev', 'setuptools'.
We have a develop egg: zc.buildout 1.0.0.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6
......
......@@ -103,6 +103,17 @@ def write(dir, *args):
fsync(f.fileno())
f.close()
def clean_up_pyc(*path):
base, filename = os.path.join(*path[:-1]), path[-1]
if filename.endswith('.py'):
filename += 'c' # .py -> .pyc
for path in (
os.path.join(base, filename),
os.path.join(base, '__pycache__', filename),
):
if os.path.exists(path):
remove(path)
## FIXME - check for other platforms
MUST_CLOSE_FDS = not sys.platform.startswith('win')
......@@ -144,8 +155,7 @@ def _runsetup(setup, executable, *args):
setup = os.path.join(setup, 'setup.py')
d = os.path.dirname(setup)
args = [zc.buildout.easy_install._safe_arg(arg)
for arg in args]
args = list(args)
args.insert(0, '-q')
env = dict(os.environ)
if executable == sys.executable:
......@@ -156,9 +166,9 @@ def _runsetup(setup, executable, *args):
try:
os.chdir(d)
p = subprocess.Popen(
[zc.buildout.easy_install._safe_arg(executable), setup] + args,
[executable, setup] + args,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
close_fds=True, env=env)
close_fds=MUST_CLOSE_FDS, env=env)
out = p.stdout.read()
if p.wait():
print out
......@@ -176,12 +186,13 @@ def bdist_egg(setup, executable, dest):
def sys_install(setup, dest):
_runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
'--install-scripts', dest,
'--record', os.path.join(dest, '__added_files__'),
'--single-version-externally-managed')
def find_python(version):
env_friendly_version = ''.join(version.split('.'))
e = os.environ.get('PYTHON%s' % env_friendly_version)
if e is not None:
return e
......@@ -471,6 +482,7 @@ def buildoutSetUp(test):
buildout = os.path.join(sample, 'bin', 'buildout'),
wait_until = wait_until,
make_py = make_py,
clean_up_pyc = clean_up_pyc,
get_filename_from_url = zc.buildout.networkcache.get_filename_from_url,
get_filename_from_upload_network_cached = get_filename_from_upload_network_cached,
))
......
This diff is collapsed.
......@@ -8,10 +8,11 @@ new_releases folder:
>>> ls(new_releases)
d setuptools
- setuptools-99.99-py2.4.egg
- setuptools-1.99.99-py2.4.egg
d zc.buildout
- zc.buildout-100.0b1-pyN.N.egg
- zc.buildout-99.99-py2.4.egg
- zc.buildout-1.100.0b1-pyN.N.egg
- zc.buildout-1.99.99-py2.4.egg
- zc.buildout-2.0.0-pyN.N.egg
Let's update the sample buildout.cfg to look in this area:
......@@ -65,27 +66,30 @@ Now if we run the buildout, the buildout will upgrade itself to the
new versions found in new releases:
>>> print system(buildout),
Getting distribution for 'zc.buildout'.
Got zc.buildout 99.99.
Getting distribution for 'zc.buildout>=1.99, <2dev'.
Got zc.buildout 1.99.99.
Getting distribution for 'setuptools'.
Got setuptools 99.99.
Got setuptools 1.99.99.
Upgraded:
zc.buildout version 99.99,
setuptools version 99.99;
zc.buildout version 1.99.99,
setuptools version 1.99.99;
restarting.
Generated script '/sample-buildout/bin/buildout'.
Develop: '/sample-buildout/showversions'
Installing show-versions.
zc.buildout 99.99
setuptools 99.99
zc.buildout 1.99.99
setuptools 1.99.99
Notice that, even though we have a newer beta version of zc.buildout
available, the final "99.99" was selected. If you want to get non-final
available, the final "1.99.99" was selected. If you want to get non-final
versions, specify a specific version in your buildout's versions
section, you typically want to use the --accept-buildout-test-releases
option to the bootstrap script, which internally uses the
``accept-buildout-test-releases = true`` discussed below.
Also, even thought there's a later final version, buildout won't
upgrade itself past version 1.
Our buildout script's site.py has been updated to use the new eggs:
>>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
......@@ -95,13 +99,13 @@ Our buildout script's site.py has been updated to use the new eggs:
"""Add site packages, as determined by zc.buildout.
<BLANKLINE>
See original_addsitepackages, below, for the original version."""
setuptools_path = '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
setuptools_path = '/sample-buildout/eggs/setuptools-1.99.99-pyN.N.egg'
sys.path.append(setuptools_path)
known_paths.add(os.path.normcase(setuptools_path))
import pkg_resources
buildout_paths = [
'/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
'/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
'/sample-buildout/eggs/zc.buildout-1.99.99-pyN.N.egg',
'/sample-buildout/eggs/setuptools-1.99.99-pyN.N.egg'
]
for path in buildout_paths:
sitedir, sitedircase = makepath(path)
......@@ -130,9 +134,9 @@ running the buildout will install earlier versions of these packages:
... index = %(new_releases)s
... parts = show-versions
... develop = showversions
... zc.buildout-version = < 99
... setuptools-version = < 99
... distribute-version = < 99
... zc.buildout-version = < 1.99
... setuptools-version = < 1.99
... distribute-version = < 1.99
...
... [show-versions]
... recipe = showversions
......@@ -200,10 +204,10 @@ directory:
Creating directory '/sample_buildout2/parts'.
Creating directory '/sample_buildout2/eggs'.
Creating directory '/sample_buildout2/develop-eggs'.
Getting distribution for 'zc.buildout'.
Got zc.buildout 99.99.
Getting distribution for 'zc.buildout>=1.99, <2dev'.
Got zc.buildout 1.99.99.
Getting distribution for 'setuptools'.
Got setuptools 99.99.
Got setuptools 1.99.99.
Not upgrading because not running a local buildout command.
>>> ls('bin')
......@@ -233,11 +237,11 @@ We'll mimic this by passing the argument actually in the command line.
>>> print system(buildout +
... ' buildout:accept-buildout-test-releases=true'),
... # doctest: +NORMALIZE_WHITESPACE
Getting distribution for 'zc.buildout'.
Got zc.buildout 100.0b1.
Getting distribution for 'zc.buildout>=1.99, <2dev'.
Got zc.buildout 1.100.0b1.
Upgraded:
zc.buildout version 100.0b1,
setuptools version 99.99;
zc.buildout version 1.100.0b1,
setuptools version 1.99.99;
restarting.
Generated script '/sample-buildout/bin/buildout'.
NOTE: Accepting early releases of build system packages. Rerun bootstrap
......@@ -245,8 +249,8 @@ We'll mimic this by passing the argument actually in the command line.
behavior.
Develop: '/sample-buildout/showversions'
Updating show-versions.
zc.buildout 100.0b1
setuptools 99.99
zc.buildout 1.100.0b1
setuptools 1.99.99
The buildout script shows the change.
......@@ -267,10 +271,10 @@ caught (displaying a warning) and the rest of the buildout update process
should continue.
>>> version = sys.version_info[0:2]
>>> egg = new_releases + '/zc.buildout-99.99-py%s.%s.egg ' % version
>>> copy_egg = new_releases + '/zc.buildout-1000-py%s.%s.egg ' % version
>>> system('cp ' + egg + copy_egg)
''
>>> egg = new_releases + '/zc.buildout-1.99.99-py%s.%s.egg' % version
>>> copy_egg = new_releases + '/zc.buildout-1.1000-py%s.%s.egg' % version
>>> import shutil
>>> shutil.copy(egg, copy_egg)
Create a broken egg
......@@ -290,7 +294,5 @@ Create a broken egg
... eggs = broken
... """ % dict(new_releases=new_releases))
>>> import subprocess
>>> subprocess.check_call([buildout])
Traceback (most recent call last):
...
CalledProcessError: Command '['/sample-buildout/bin/buildout']' returned non-zero exit status 1
>>> subprocess.call([buildout])
1
......@@ -179,7 +179,7 @@ take a look at bin/buildout:
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
sys.exit(zc.buildout.buildout.main())
That's the old-style buildout script: no changes for users with this issue.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment