Skip to content

S
slapos.buildout
Source
Select Git revision
...
Target
Select Git revision
 
Commits (67)
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 1637 additions and 255 deletions
CHANGES.SlapOS.rst 0 → 100644
View file @ c0b64234
2.5.2+slapos14
--------------
- Unserialize 'None' to None in BuildoutSerializer.
2.5.2+slapos13
--------------
- Exit program with error code on error when running with ``-D`` (backport).
2.5.2+slapos12
--------------
- Optimize _install_and_load by using cache.
- Optimize _compute_part_signatures. Cache recipe signature and calculate once per recipe.
2.5.2+slapos011
---------------
- Use HTTPS for PyPI index (backport).
- Fix testing index when /bin/sh is dash (like on Debian).
2.5.2+slapos010
---------------
- Fix MissingOption when the serialised (!py!) value is None.
2.5.2+slapos009
---------------
- More fixes for Python 3
2.5.2+slapos008
---------------
- Also run deserializer on each get invocation.
- Fix syntax error on Python 3.
2.5.2+slapos007
---------------
- Introduce a workaround for the shebang line length limitation in
zc.buildout.easy_install.
2.5.2+slapos006
---------------
- Improve on the fly patches so that specified patches are
automatically applied on required eggs as well.
- Fix exception when easy_install is used without initialising a buildout object.
2.5.2+slapos005
---------------
Fix a regression introduced in 2.5.2+slapos004 with the review of:
- Add referred parts' hash strings in __buildout_signature__, that invokes
rebuild of a part when one of its (recursive) dependencies are modified.
2.5.2+slapos004
---------------
Rebase to refresh some patches and fix commits that were wrongly cherry-picked
from Nexedi 1.x branch. Changed:
- Give the same permission but write as owner to group and other.
- Make buildout.rmtree working with symlink as a path argument.
- Add referred parts' hash strings in __buildout_signature__, that invokes
rebuild of a part when one of its (recursive) dependencies are modified.
- Write .installed.cfg only once, in safe way and only if there's any change.
- Escape $$ character to $.
2.5.2+slapos003
---------------
- Expand sys.path when PYTHONEXTRAPATH is present for develop.
2.5.2+slapos002
---------------
- Fix a bug where slapos.libnetworkcache is lost when buildout script
is upgraded.
2.5.2+slapos001
---------------
- Rebase with 2.5.2
- In verbose mode, save .installed.cfg each time a part is installed.
2.5.1+slapos001
---------------
- Rebase with 2.5.1
2.5.0+slapos001
---------------
- Rebase with 2.5.0
- Fix a bug in $$ escape handling.
2.4.7+slapos001
---------------
- Rebase with 2.4.7.
- Support Python 3.
2.4.5+slapos001
---------------
- Support ${:_profile_base_location_}.
- Support on the fly patches in easy_install.
- Merge PYTHONEXTRAPATH value into PYTHONPATH in easy_install so that we can
use some eggs that are required to build an egg.
- Support network cache in download.py and easy_install.py
- Cache downloaded data in zc/buildout/buildout.py:_open() in memory
to accelerate remote extends.
- Give the same permission but write as owner to group and other.
- Make buildout.rmtree working with symlink as a path argument.
https://bugs.launchpad.net/zc.buildout/+bug/144228
- Respect specified versions of initial eggs in bootstrap.
- Keep develop-eggs directory in bootstrap.
- Add referred parts' hash strings in __buildout_signature__, that invokes
rebuild of a part when one of its (recursive) dependencies are modified.
- Allow assigning non-string values to section keys. Restricted to selected
python base types.
- Write .installed.cfg only once, in safe way and only if there's any change.
- Put only one [buildout] section in .installed.cfg.
- Escape $$ character to $.
- Compare Options with sorted result, if possible.
- Add '--dry-run' option.
- Add '--skip-signature-check' option.
not applied (still required?)
-----------------------------
- 83fc3a2 Always use build() in easy_install.py to install eggs.
(it's not required if the commit below is not applied, I guess)
- b7cd5ca Include '.postN' in generated egg's version so that version pinning with 'N.N.N.postN' works.
- 1761c65 Workaround M2Crypto bug of https redirection.
CHANGES.rst
View file @ c0b64234
Change History
**************
?.?.? (unreleased)
==================
- Use HTTPS for PyPI's index. PyPI redirects HTTP to HTTPS by default
now so using HTTPS directly avoids the potential for that redirect
being modified in flight.
2.5.2 (2016-06-07)
==================
......
setup.py
View file @ c0b64234
......@@ -12,7 +12,7 @@
#
##############################################################################
name = "zc.buildout"
version = '2.5.2'
version = '2.5.2+slapos014'
import os
from setuptools import setup
......
src/zc/buildout/bootstrap.txt
View file @ c0b64234
......@@ -70,7 +70,7 @@ Let's try with an unknown version::
...
X
...
No local packages or download links found for zc.buildout==UNKNOWN
No local packages or...download links found for zc.buildout==UNKNOWN
...
Now let's try with `2.0.0`, which happens to exist::
......
src/zc/buildout/buildout.py
View file @ c0b64234
This diff is collapsed.
src/zc/buildout/buildout.txt
View file @ c0b64234
......@@ -333,6 +333,10 @@ we'll see that the directory gets removed and recreated:
... path = mydata
... """)
>>> print_(system(buildout+' --dry-run'), end='')
Develop: '/sample-buildout/recipes'
Uninstalling data-dir.
Installing data-dir.
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Uninstalling data-dir.
......@@ -353,6 +357,10 @@ If any of the files or directories created by a recipe are removed,
the part will be reinstalled:
>>> rmdir(sample_buildout, 'mydata')
>>> print_(system(buildout+' --dry-run'), end='')
Develop: '/sample-buildout/recipes'
Uninstalling data-dir.
Installing data-dir.
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Uninstalling data-dir.
......@@ -804,12 +812,16 @@ COMMAND_LINE_VALUE).
DEFAULT_VALUE
bin-directory= bin
DEFAULT_VALUE
check-signature= true
DEFAULT_VALUE
develop= recipes
/sample-buildout/buildout.cfg
develop-eggs-directory= develop-eggs
DEFAULT_VALUE
directory= /sample-buildout
COMPUTED_VALUE
dry-run= false
DEFAULT_VALUE
eggs-directory= /sample-buildout/eggs
DEFAULT_VALUE
executable= ...
......@@ -1003,6 +1015,102 @@ _buildout_section_name_ to get the current section name.
my_name debug
recipe recipes:debug
It is possible to have access to profile base url from section by
using ${:_profile_base_location_}:
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... develop = recipes
... parts = data-dir debug
... log-level = INFO
...
... [debug]
... recipe = recipes:debug
... profile_base_location = ${:_profile_base_location_}
...
... [data-dir]
... recipe = recipes:mkdir
... path = mydata
... """)
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Uninstalling debug.
Updating data-dir.
Installing debug.
_profile_base_location_ /sample-buildout
profile_base_location /sample-buildout
recipe recipes:debug
Keep in mind that in case of sections spaning across multiple profiles,
the topmost value will be presented:
>>> write(sample_buildout, 'extended.cfg',
... """
... [debug]
... profile_base_location = ${:_profile_base_location_}
... """)
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... extends = extended.cfg
... develop = recipes
... parts = data-dir debug
... log-level = INFO
...
... [debug]
... recipe = recipes:debug
... profile_base_location = ${:_profile_base_location_}
...
... [data-dir]
... recipe = recipes:mkdir
... path = mydata
... """)
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Updating data-dir.
Updating debug.
_profile_base_location_ /sample-buildout
profile_base_location /sample-buildout
recipe recipes:debug
But of course, in case if accessing happens in extended profile's section,
this profile's location will be exposed:
>>> write(sample_buildout, 'extended.cfg',
... """
... [debug]
... profile_base_location = ${:_profile_base_location_}
... """)
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... extends = extended.cfg
... develop = recipes
... parts = data-dir debug
... log-level = INFO
...
... [debug]
... recipe = recipes:debug
...
... [data-dir]
... recipe = recipes:mkdir
... path = mydata
... """)
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Updating data-dir.
Updating debug.
_profile_base_location_ /sample-buildout
profile_base_location /sample-buildout
recipe recipes:debug
>>> remove(sample_buildout, 'extended.cfg')
Automatic part selection and ordering
-------------------------------------
......@@ -2433,7 +2541,7 @@ were created.
The .installed.cfg is only updated for the recipes that ran:
>>> cat(sample_buildout, '.installed.cfg')
... # doctest: +NORMALIZE_WHITESPACE
... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
[buildout]
installed_develop_eggs = /sample-buildout/develop-eggs/recipes.egg-link
parts = debug d1 d2 d3 d4
......@@ -2463,7 +2571,7 @@ The .installed.cfg is only updated for the recipes that ran:
<BLANKLINE>
[d4]
__buildout_installed__ = /sample-buildout/data2-extra
__buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
__buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== d2:...
path = /sample-buildout/data2-extra
recipe = recipes:mkdir
......@@ -2475,6 +2583,7 @@ Now, if we run the buildout without the install command:
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Uninstalling d4.
Uninstalling d2.
Uninstalling d1.
Uninstalling debug.
......@@ -2484,7 +2593,8 @@ Now, if we run the buildout without the install command:
Installing d2.
d2: Creating directory data2
Updating d3.
Updating d4.
Installing d4.
d4: Creating directory data2-extra
We see the output of the debug recipe and that data2 was created. We
also see that d1 and d2 have gone away:
......@@ -2537,9 +2647,9 @@ provide alternate locations, and even names for these directories.
Creating directory '/sample-alt/developbasket'.
Develop: '/sample-buildout/recipes'
Uninstalling d4.
Uninstalling d3.
Uninstalling d2.
Uninstalling debug.
Uninstalling d3.
>>> ls(alt)
d basket
......@@ -2647,8 +2757,10 @@ database is shown.
allow-hosts = *
allow-picked-versions = true
bin-directory = /sample-buildout/bin
check-signature = true
develop-eggs-directory = /sample-buildout/develop-eggs
directory = /sample-buildout
dry-run = false
eggs-directory = /sample-buildout/eggs
executable = python
find-links =
......@@ -2945,7 +3057,6 @@ or paths to use:
>>> remove('setup.cfg')
>>> print_(system(buildout + ' -csetup.cfg init demo other ./src'), end='')
Creating '/sample-bootstrapped/setup.cfg'.
Creating directory '/sample-bootstrapped/develop-eggs'.
Getting distribution for 'zc.recipe.egg>=2.0.0a3'.
Got zc.recipe.egg
Installing py.
......@@ -3004,7 +3115,6 @@ for us:
>>> remove('setup.cfg')
>>> print_(system(buildout + ' -csetup.cfg init demo other ./src'), end='')
Creating '/sample-bootstrapped/setup.cfg'.
Creating directory '/sample-bootstrapped/develop-eggs'.
Installing py.
Generated script '/sample-bootstrapped/bin/demo'.
Generated script '/sample-bootstrapped/bin/distutilsscript'.
......@@ -3024,9 +3134,9 @@ using the `index` option::
[buildout]
...
index = http://index.example.com/
index = https://index.example.com/
This index, or the default of http://pypi.python.org/simple/ if no
This index, or the default of https://pypi.python.org/simple/ if no
index is specified, will always be searched for distributions unless
running buildout with options that prevent searching for
distributions. The latest version of the distribution that meets the
......
src/zc/buildout/debugging.txt
View file @ c0b64234
......@@ -56,12 +56,13 @@ And create a buildout that uses it:
If we run the buildout, we'll get an error:
>>> print_(system(buildout), end='')
>>> print_(system(buildout, with_exit_code=True), end='')
Develop: '/sample-buildout/recipes'
Installing data-dir.
While:
Installing data-dir.
Error: Missing option: data-dir:directory
EXIT CODE: 1
If we want to debug the error, we can add the -D option. Here's we'll
......@@ -71,7 +72,7 @@ supply some input:
... up
... p sorted(self.options.keys())
... q
... """), end='')
... """, with_exit_code=True), end='')
Develop: '/sample-buildout/recipes'
Installing data-dir.
> /zc/buildout/buildout.py(925)__getitem__()
......@@ -85,6 +86,8 @@ supply some input:
File "/zc/buildout/buildout.py", line 1352, in main
getattr(buildout, command)(args)
File "/zc/buildout/buildout.py", line 383, in install
self._install_parts(install_args)
File buildout.py", line 791, in _install_parts
installed_files = self[part]._call(recipe.install)
File "/zc/buildout/buildout.py", line 961, in _call
return f()
......@@ -95,3 +98,4 @@ supply some input:
MissingOption: Missing option: data-dir:directory
<BLANKLINE>
Starting pdb:
EXIT CODE: 1
src/zc/buildout/download.py
View file @ c0b64234
......@@ -204,10 +204,29 @@ class Download(object):
handle, tmp_path = tempfile.mkstemp(prefix='buildout-')
os.close(handle)
try:
tmp_path, headers = urlretrieve(url, tmp_path)
if not check_md5sum(tmp_path, md5sum):
raise ChecksumError(
'MD5 checksum mismatch downloading %r' % url)
from .buildout import network_cache_parameter_dict as nc
if not download_network_cached(
nc.get('download-dir-url'),
nc.get('download-cache-url'),
tmp_path, url, self.logger,
nc.get('signature-certificate-list'), md5sum):
# Download from original url if not cached or md5sum doesn't match.
tmp_path, headers = urlretrieve(url, tmp_path)
if not check_md5sum(tmp_path, md5sum):
raise ChecksumError(
'MD5 checksum mismatch downloading %r' % url)
# Upload the file to network cache.
if nc.get('upload-cache-url') and nc.get('upload-dir-url'):
upload_network_cached(
nc.get('upload-dir-url'),
nc.get('upload-cache-url'), url, tmp_path, self.logger,
nc.get('signature-private-key-file'),
nc.get('shacache-ca-file'),
nc.get('shacache-cert-file'),
nc.get('shacache-key-file'),
nc.get('shadir-ca-file'),
nc.get('shadir-cert-file'),
nc.get('shadir-key-file'))
except IOError:
e = sys.exc_info()[1]
os.remove(tmp_path)
......@@ -276,6 +295,8 @@ def remove(path):
if os.path.exists(path):
os.remove(path)
from zc.buildout.networkcache import download_network_cached, \
upload_network_cached
def locate_at(source, dest):
if dest is None or realpath(dest) == realpath(source):
......
src/zc/buildout/easy_install.py
View file @ c0b64234
This diff is collapsed.
src/zc/buildout/easy_install.txt
View file @ c0b64234
......@@ -46,7 +46,7 @@ index
The URL of an index server, or almost any other valid URL. :)
If not specified, the Python Package Index,
http://pypi.python.org/simple/, is used. You can specify an
https://pypi.python.org/simple/, is used. You can specify an
alternate index with this option. If you use the links option and
if the links point to the needed distributions, then the index can
be anything and will be largely ignored. In the examples, here,
......@@ -1073,7 +1073,7 @@ index
The URL of an index server, or almost any other valid URL. :)
If not specified, the Python Package Index,
http://pypi.python.org/simple/, is used. You can specify an
https://pypi.python.org/simple/, is used. You can specify an
alternate index with this option. If you use the links option and
if the links point to the needed distributions, then the index can
be anything and will be largely ignored. In the examples, here,
......
src/zc/buildout/extends-cache.txt
View file @ c0b64234
......@@ -457,12 +457,30 @@ used:
... extends = %sbase.cfg
... bar = foo
... """ % server_url)
>>> write(server_data, 'baseC.cfg', """\
... [buildout]
... extends-cache = cache
... extends = baseB.cfg
... bar = foo
... """)
>>> write(server_data, 'baseD.cfg', """\
... [buildout]
... extends-cache = cache
... bar = foo
... """)
>>> mkdir(server_data, 'deeper')
>>> write(server_data, 'deeper', 'base.cfg', """\
... [buildout]
... extends-cache = cache
... extends = ../baseD.cfg
... bar = foo
... """)
>>> write('buildout.cfg', """\
... [buildout]
... extends-cache = cache
... newest = true
... extends = %sbaseA.cfg %sbaseB.cfg
... """ % (server_url, server_url))
... extends = %sbaseA.cfg %sbaseB.cfg %sbaseC.cfg %sdeeper/base.cfg
... """ % (server_url, server_url, server_url, server_url))
>>> print_(system(buildout + " -n"))
Unused options for buildout: 'bar' 'foo'.
......@@ -480,6 +498,10 @@ a better solution would re-use the logging already done by the utility.)
The URL http://localhost/baseA.cfg was downloaded.
The URL http://localhost/base.cfg was downloaded.
The URL http://localhost/baseB.cfg was downloaded.
The URL http://localhost/baseC.cfg was downloaded.
The URL http://localhost/baseB.cfg was downloaded.
The URL http://localhost/deeper/base.cfg was downloaded.
The URL http://localhost/baseD.cfg was downloaded.
Not upgrading because not running a local buildout command.
Unused options for buildout: 'bar' 'foo'.
......
src/zc/buildout/networkcache.py 0 → 100644
View file @ c0b64234
This diff is collapsed.
src/zc/buildout/repeatable.txt
View file @ c0b64234
......@@ -317,8 +317,8 @@ When everything is pinned, no output is generated:
recipe v2
The Python package index is case-insensitive. Both
http://pypi.python.org/simple/Django/ and
http://pypi.python.org/simple/dJaNgO/ work. And distributions aren't always
https://pypi.python.org/simple/Django/ and
https://pypi.python.org/simple/dJaNgO/ work. And distributions aren't always
naming themselves consistently case-wise. So all version names are normalized
and case differences won't impact the pinning:
......
src/zc/buildout/rmtree.py
View file @ c0b64234
......@@ -24,6 +24,8 @@ def rmtree (path):
read only file.
This tries to chmod the file to writeable and retries before giving up.
Also it tries to remove symlink itself if a symlink as passed as
path argument.
>>> from tempfile import mkdtemp
Let's make a directory ...
......@@ -39,6 +41,8 @@ def rmtree (path):
>>> foo = os.path.join (d, 'foo')
>>> _ = open (foo, 'w').write ('huhu')
>>> bar = os.path.join (d, 'bar')
>>> os.symlink(bar, bar)
and make it unwriteable
......@@ -50,12 +54,51 @@ def rmtree (path):
and now the directory is gone
>>> os.path.isdir (d)
0
Let's make a directory ...
>>> d = mkdtemp()
and make sure it is actually there
>>> os.path.isdir (d)
1
Now create a broken symlink ...
>>> foo = os.path.join (d, 'foo')
>>> os.symlink(foo + '.not_exist', foo)
rmtree should be able to remove it:
>>> rmtree (foo)
and now the directory is gone
>>> os.path.isdir (foo)
0
cleanup directory
>>> rmtree (d)
and now the directory is gone
>>> os.path.isdir (d)
0
"""
def retry_writeable (func, path, exc):
os.chmod (path, 384) # 0600
func (path)
if func is os.path.islink:
os.unlink(path)
elif func is os.lstat:
if not os.path.islink(path):
raise
os.unlink(path)
else:
os.chmod(path, 0o600)
func(path)
shutil.rmtree (path, onerror = retry_writeable)
......
src/zc/buildout/testing.py
View file @ c0b64234
......@@ -173,18 +173,14 @@ def wait_until(label, func, *args, **kw):
time.sleep(0.01)
raise ValueError('Timed out waiting for: '+label)
class TestOptions(zc.buildout.buildout.Options):
def initialize(self):
pass
class Buildout(zc.buildout.buildout.Buildout):
def __init__(self):
zc.buildout.buildout.Buildout.__init__(
self, '', [('buildout', 'directory', os.getcwd())])
Options = TestOptions
def initialize(self, *args):
pass
def buildoutSetUp(test):
......@@ -228,7 +224,7 @@ def buildoutSetUp(test):
register_teardown(lambda: rmtree(tmp))
zc.buildout.easy_install.default_index_url = 'file://'+tmp
os.environ['buildout-testing-index-url'] = (
os.environ['BUILDOUT_TESTING_INDEX_URL'] = (
zc.buildout.easy_install.default_index_url)
def tmpdir(name):
......
src/zc/buildout/tests.py
View file @ c0b64234
......@@ -1007,6 +1007,7 @@ Uninstall recipes need to be called when a part is removed too:
uninstalling
Installing demo.
installing
Unused options for demo: 'x'.
>>> write('buildout.cfg', '''
......@@ -1498,7 +1499,7 @@ some evil recipes that exit uncleanly:
>>> mkdir('recipes')
>>> write('recipes', 'recipes.py',
... '''
... import os
... import sys
...
... class Clean:
... def __init__(*_): pass
......@@ -1506,10 +1507,10 @@ some evil recipes that exit uncleanly:
... def update(_): pass
...
... class EvilInstall(Clean):
... def install(_): os._exit(1)
... def install(_): sys.exit(1)
...
... class EvilUpdate(Clean):
... def update(_): os._exit(1)
... def update(_): sys.exit(1)
... ''')
>>> write('recipes', 'setup.py',
......@@ -1603,10 +1604,10 @@ Now let's look at 3 cases:
>>> print_(system(buildout+' buildout:parts='), end='')
Develop: '/sample-buildout/recipes'
Uninstalling p2.
Uninstalling p1.
Uninstalling p4.
Uninstalling p3.
Uninstalling p2.
Uninstalling p1.
3. We exit while installing or updating after uninstalling:
......@@ -2078,6 +2079,80 @@ def dealing_with_extremely_insane_dependencies():
Error: Couldn't find a distribution for 'pack5'.
"""
def test_part_pulled_by_recipe():
"""
>>> mkdir(sample_buildout, 'recipes')
>>> write(sample_buildout, 'recipes', 'test.py',
... '''
... class Recipe:
...
... def __init__(self, buildout, name, options):
... self.x = buildout[options['x']][name]
...
... def install(self):
... print self.x
... return ()
...
... update = install
... ''')
>>> write(sample_buildout, 'recipes', 'setup.py',
... '''
... from setuptools import setup
... setup(
... name = "recipes",
... entry_points = {'zc.buildout': ['test = test:Recipe']},
... )
... ''')
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... develop = recipes
... parts = a
... [a]
... recipe = recipes:test
... x = b
... [b]
... <= a
... a = A
... b = B
... c = ${c:x}
... [c]
... x = c
... ''')
>>> os.chdir(sample_buildout)
>>> buildout = os.path.join(sample_buildout, 'bin', 'buildout')
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Installing b.
B
Installing a.
A
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Updating b.
B
Updating a.
A
>>> cat('.installed.cfg') # doctest: +ELLIPSIS
[buildout]
...
[b]
__buildout_installed__ =
__buildout_signature__ = recipes-... c:...
...
[a]
__buildout_installed__ =
__buildout_signature__ = recipes-... b:...
...
"""
def read_find_links_to_load_extensions():
r"""
We'll create a wacky buildout extension that just announces itself when used:
......@@ -2742,6 +2817,73 @@ def increment_on_command_line():
recipe='zc.buildout:debug'
"""
def bug_664539_simple_buildout():
r"""
>>> write('buildout.cfg', '''
... [buildout]
... parts = escape
...
... [escape]
... recipe = zc.buildout:debug
... foo = $${nonexistent:option}
... ''')
>>> print_(system(buildout), end='')
Installing escape.
foo='${nonexistent:option}'
recipe='zc.buildout:debug'
"""
def bug_664539_reference():
r"""
>>> write('buildout.cfg', '''
... [buildout]
... parts = escape
...
... [escape]
... recipe = zc.buildout:debug
... foo = ${:bar}
... bar = $${nonexistent:option}
... ''')
>>> print_(system(buildout), end='')
Installing escape.
bar='${nonexistent:option}'
foo='${nonexistent:option}'
recipe='zc.buildout:debug'
"""
def bug_664539_complex_buildout():
r"""
>>> write('buildout.cfg', '''
... [buildout]
... parts = escape
...
... [escape]
... recipe = zc.buildout:debug
... foo = ${level1:foo}
...
... [level1]
... recipe = zc.buildout:debug
... foo = ${level2:foo}
...
... [level2]
... recipe = zc.buildout:debug
... foo = $${nonexistent:option}
... ''')
>>> print_(system(buildout), end='')
Installing level2.
foo='${nonexistent:option}'
recipe='zc.buildout:debug'
Installing level1.
foo='${nonexistent:option}'
recipe='zc.buildout:debug'
Installing escape.
foo='${nonexistent:option}'
recipe='zc.buildout:debug'
"""
def test_constrained_requirement():
"""
zc.buildout.easy_install._constrained_requirement(constraint, requirement)
......@@ -3210,7 +3352,7 @@ def easy_install_SetUp(test):
def buildout_txt_setup(test):
zc.buildout.testing.buildoutSetUp(test)
mkdir = test.globs['mkdir']
eggs = os.environ['buildout-testing-index-url'][7:]
eggs = os.environ['BUILDOUT_TESTING_INDEX_URL'][7:]
test.globs['sample_eggs'] = eggs
create_sample_eggs(test)
......@@ -3230,7 +3372,7 @@ def buildout_txt_setup(test):
os.path.join(eggs, 'zc.recipe.egg'),
)
egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.+]+)-py(\d[.]\d).egg$'
).match
def makeNewRelease(project, ws, dest, version='99.99'):
dist = ws.find(pkg_resources.Requirement.parse(project))
......
zc.recipe.egg_/CHANGES.SlapOS.txt 0 → 100644
View file @ c0b64234
2.0.3+slapos003
---------------
- Improve on the fly patches so that specified patches are
automatically applied on required eggs as well.
2.0.3+slapos002
---------------
- Fix setup-eggs option in :develop.
2.0.3+slapos001
---------------
- Support environment in :develop as well.
- Support on the fly patches in zc.recipe.egg by ``EGGNAME-patches``,
``EGGNAME-patch-options``, ``EGGNAME-patch-binary`` (or
``patch-binary``) and ``EGGNAME-patch-revision`` options.
- Support on the fly patches in zc.recipe.egg:custom by ``patches``,
``patch-options``, ``patch-binary`` and ``patch-revision`` options.
(options ``EGGNAME-*`` are also supported as well).
- Add setup-eggs option in :custom and :develop.
zc.recipe.egg_/setup.py
View file @ c0b64234
......@@ -14,7 +14,7 @@
"""Setup for zc.recipe.egg package
"""
version = '2.0.4.dev0'
version = '2.0.3+slapos003'
import os
from setuptools import setup, find_packages
......
zc.recipe.egg_/src/zc/recipe/egg/README.rst
View file @ c0b64234
......@@ -9,6 +9,19 @@ eggs
requirement strings. Each string must be given on a separate
line.
patch-binary
The path to the patch executable.
EGGNAME-patches
A new-line separated list of patchs to apply when building.
EGGNAME-patch-options
Options to give to the patch program when applying patches.
EGGNAME-patch-revision
An integer to specify the revision (default is the number of
patches).
find-links
A list of URLs, files, or directories to search for distributions.
......
zc.recipe.egg_/src/zc/recipe/egg/api.rst
View file @ c0b64234
......@@ -97,17 +97,14 @@ of extra requirements to be included in the working set.
We can see that the options were augmented with additional data
computed by the egg recipe by looking at .installed.cfg:
>>> cat(sample_buildout, '.installed.cfg')
>>> cat(sample_buildout, '.installed.cfg') # doctest: +ELLIPSIS
[buildout]
installed_develop_eggs = /sample-buildout/develop-eggs/sample.egg-link
parts = sample-part
<BLANKLINE>
[sample-part]
__buildout_installed__ =
__buildout_signature__ = sample-6aWMvV2EJ9Ijq+bR8ugArQ==
zc.recipe.egg-cAsnudgkduAa/Fd+WJIM6Q==
setuptools-0.7-py2.4.egg
zc.buildout-+rYeCcmFuD1K/aB77XTj5A==
__buildout_signature__ = sample-... setuptools-...egg zc.buildout-... zc.recipe.egg-...
_b = /sample-buildout/bin
_d = /sample-buildout/develop-eggs
_e = /sample-buildout/eggs
......
zc.recipe.egg_/src/zc/recipe/egg/custom.py
View file @ c0b64234
......@@ -15,6 +15,7 @@
"""
import logging
import os
import re
import sys
import zc.buildout.easy_install
......@@ -28,17 +29,19 @@ class Base:
self.name, self.options = name, options
options['_d'] = buildout['buildout']['develop-eggs-directory']
self.build_ext = build_ext(buildout, options)
def update(self):
return self.install()
options['_e'] = buildout['buildout']['eggs-directory']
class Custom(Base):
environment_section = options.get('environment')
if environment_section:
self.environment = buildout[environment_section]
else:
self.environment = {}
environment_data = list(self.environment.items())
environment_data.sort()
options['_environment-data'] = repr(environment_data)
def __init__(self, buildout, name, options):
Base.__init__(self, buildout, name, options)
self.build_ext = build_ext(buildout, options)
links = options.get('find-links',
buildout['buildout'].get('find-links'))
......@@ -54,45 +57,20 @@ class Custom(Base):
options['index'] = index
self.index = index
environment_section = options.get('environment')
if environment_section:
self.environment = buildout[environment_section]
else:
self.environment = {}
environment_data = list(self.environment.items())
environment_data.sort()
options['_environment-data'] = repr(environment_data)
options['_e'] = buildout['buildout']['eggs-directory']
if buildout['buildout'].get('offline') == 'true':
self.install = lambda: ()
self.newest = buildout['buildout'].get('newest') == 'true'
def install(self):
options = self.options
distribution = options.get('egg')
if distribution is None:
distribution = options.get('eggs')
if distribution is None:
distribution = self.name
else:
logger.warn("The eggs option is deprecated. Use egg instead")
distribution = options.get('egg', options.get('eggs', self.name)
).strip()
def install(self):
self._set_environment()
try:
return zc.buildout.easy_install.build(
distribution, options['_d'], self.build_ext,
self.links, self.index, sys.executable,
[options['_e']], newest=self.newest,
)
self._install_setup_eggs()
return self._install()
finally:
self._restore_environment()
def update(self):
return self.install()
def _set_environment(self):
self._saved_environment = {}
......@@ -114,6 +92,77 @@ class Custom(Base):
except KeyError:
pass
def _install_setup_eggs(self):
options = self.options
setup_eggs = [
r.strip()
for r in options.get('setup-eggs', '').split('\n')
if r.strip()]
if setup_eggs:
ws = zc.buildout.easy_install.install(
setup_eggs, options['_e'],
links=self.links,
index=self.index,
executable=sys.executable,
path=[options['_d'], options['_e']],
newest=self.newest,
)
extra_path = os.pathsep.join(ws.entries)
self.environment['PYTHONEXTRAPATH'] = os.environ['PYTHONEXTRAPATH'] = extra_path
def _get_patch_dict(self, options, distribution):
patch_dict = {}
global_patch_binary = options.get('patch-binary', 'patch')
def get_option(egg, key, default):
return options.get('%s-%s' % (egg, key),
options.get(key, default))
egg = re.sub('[<>=].*', '', distribution)
patches = filter(lambda x:x,
map(lambda x:x.strip(),
get_option(egg, 'patches', '').splitlines()))
patches = list(patches)
if not patches:
return patch_dict
patch_options = get_option(egg, 'patch-options', '-p0').split()
patch_binary = get_option(egg, 'patch-binary', global_patch_binary)
patch_revision = int(get_option(egg, 'patch-revision', len(patches)))
patch_dict[egg] = {
'patches':patches,
'patch_options':patch_options,
'patch_binary':patch_binary,
'patch_revision':patch_revision,
}
return patch_dict
class Custom(Base):
def __init__(self, buildout, name, options):
Base.__init__(self, buildout, name, options)
if buildout['buildout'].get('offline') == 'true':
self._install = lambda: ()
def _install(self):
options = self.options
distribution = options.get('egg')
if distribution is None:
distribution = options.get('eggs')
if distribution is None:
distribution = self.name
else:
logger.warn("The eggs option is deprecated. Use egg instead")
distribution = options.get('egg', options.get('eggs', self.name)
).strip()
patch_dict = self._get_patch_dict(options, distribution)
return zc.buildout.easy_install.build(
distribution, options['_d'], self.build_ext,
self.links, self.index, sys.executable,
[options['_e']], newest=self.newest, patch_dict=patch_dict,
)
class Develop(Base):
......@@ -122,7 +171,7 @@ class Develop(Base):
options['setup'] = os.path.join(buildout['buildout']['directory'],
options['setup'])
def install(self):
def _install(self):
options = self.options
return zc.buildout.easy_install.develop(
options['setup'], options['_d'], self.build_ext)
......
zc.recipe.egg_/src/zc/recipe/egg/custom.rst
View file @ c0b64234
......@@ -20,6 +20,23 @@ rpath
A new-line separated list of directories to search for dynamic libraries
at run time.
setup-eggs
A new-line separated list of eggs that need to be installed
beforehand. It is useful to meet the `setup_requires` requirement.
patch-binary
The path to the patch executable.
patches
A new-line separated list of patchs to apply when building.
patch-options
Options to give to the patch program when applying patches.
patch-revision
An integer to specify the revision (default is the number of
patches).
define
A comma-separated list of names of C preprocessor variables to
define.
......@@ -426,8 +443,8 @@ Create a clean buildout.cfg w/o the checkenv recipe, and delete the recipe:
... """ % dict(server=link_server))
>>> print_(system(buildout), end='') # doctest: +ELLIPSIS
Develop: '/sample-buildout/recipes'
Uninstalling checkenv.
Uninstalling extdemo.
Uninstalling checkenv.
Installing extdemo...
>>> rmdir(sample_buildout, 'recipes')
......@@ -455,6 +472,10 @@ rpath
A new-line separated list of directories to search for dynamic libraries
at run time.
setup-eggs
A new-line separated list of eggs that need to be installed
beforehand. It is useful to meet the `setup_requires` requirement.
define
A comma-separated list of names of C preprocessor variables to
define.
......@@ -491,6 +512,10 @@ swig-cpp
swig-opts
List of SWIG command line options
environment
The name of a section with additional environment variables. The
environment variables are set before the egg is built.
To illustrate this, we'll use a directory containing the extdemo
example from the earlier section:
......
zc.recipe.egg_/src/zc/recipe/egg/egg.py
View file @ c0b64234
......@@ -51,6 +51,38 @@ class Eggs(object):
options['develop-eggs-directory'] = b_options['develop-eggs-directory']
options['_d'] = options['develop-eggs-directory'] # backward compat.
def _get_patch_dict(self, options, egg=None):
patch_dict = {}
global_patch_binary = options.get('patch-binary', 'patch')
if egg:
egg = re.sub('[<>=].*', '', egg)
egg_list = [egg]
else:
egg_list = [x[:-8] for x in options.keys() if x.endswith('-patches')]
def get_option(egg, key, default):
if len(egg_list) == 1:
return options.get('%s-%s' % (egg, key),
options.get(key, default))
else:
return options.get('%s-%s' % (egg, key), default)
for egg in egg_list:
patches = filter(lambda x:x,
map(lambda x:x.strip(),
get_option(egg, 'patches', '').splitlines()))
patches = list(patches)
if not patches:
continue
patch_options = get_option(egg, 'patch-options', '-p0').split()
patch_binary = get_option(egg, 'patch-binary', global_patch_binary)
patch_revision = int(get_option(egg, 'patch-revision', len(patches)))
patch_dict[egg] = {
'patches':patches,
'patch_options':patch_options,
'patch_binary':patch_binary,
'patch_revision':patch_revision,
}
return patch_dict
def working_set(self, extra=()):
"""Separate method to just get the working set
......@@ -75,13 +107,15 @@ class Eggs(object):
[options['develop-eggs-directory'], options['eggs-directory']]
)
else:
patch_dict = self._get_patch_dict(options)
ws = zc.buildout.easy_install.install(
distributions, options['eggs-directory'],
links=self.links,
index=self.index,
path=[options['develop-eggs-directory']],
newest=self.buildout['buildout'].get('newest') == 'true',
allow_hosts=self.allow_hosts)
allow_hosts=self.allow_hosts,
patch_dict=patch_dict)
return orig_distributions, ws
......
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7