Skip to content

S
slapos.recipe.cmmi
Commit 3de46334 authored by Julien Muchembled's avatar Julien Muchembled
Browse files
Options

Fix tests

parent a76ec7be
Hide whitespace changes
Inline Side-by-side
Showing with 81 additions and 98 deletions
slapos/recipe/cmmi/README.rst
View file @ 3de46334
......@@ -263,14 +263,17 @@ will add it to environmet variables as the following::
Example usage
=============
We'll use a simple tarball to demonstrate the recipe.
We'll use a few tarballs to demonstrate the recipe.
We'll modify one of them in-place but we don't want to alter the source tree.
>>> import os.path
>>> import os
>>> src = join(os.path.dirname(__file__), 'testdata')
>>> ls(src)
- Foo-Bar-0.0.0.tar.gz
- haproxy-1.4.8-dummy.tar.gz
- package-0.0.0.tar.gz
>>> package_path = join(tmpdir('testdata'), 'package-0.0.0.tar.gz')
>>> os.symlink(join(src, 'package-0.0.0.tar.gz'), package_path)
The package contains a dummy ``configure`` script that will simply
echo the options it was called with and create a ``Makefile`` that
......@@ -286,8 +289,8 @@ Let's create a buildout to build and install the package.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... """ % src)
... url = file://%s
... """ % package_path)
This will download, extract and build our demo package with the
default build options.
......@@ -309,9 +312,9 @@ Check option "promises"
...
... [packagex]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... promises = /usr/bin/myfoo
... """ % src)
... """ % package_path)
This will download, extract and build our demo package with the
default build options.
......@@ -414,8 +417,7 @@ and building that.
>>> checkout_dir = tmpdir('checkout')
>>> import setuptools.archive_util
>>> setuptools.archive_util.unpack_archive('%s/package-0.0.0.tar.gz' % src,
... checkout_dir)
>>> setuptools.archive_util.unpack_archive(package_path, checkout_dir)
>>> ls(checkout_dir)
d package-0.0.0
......@@ -466,7 +468,7 @@ with a new buildout and provide more options.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%(src)s/package-0.0.0.tar.gz
... url = file://%s
... md5sum = 6b94295c042a91ea3203857326bc9209
... prefix = /somewhere/else
... environment-section = build-environment
......@@ -481,7 +483,7 @@ with a new buildout and provide more options.
... patches =
... patches/configure.patch
... patches/Makefile.dist.patch
... """ % dict(src=src))
... """ % package_path)
This configuration uses custom configure options, an environment section,
per-part customization to the environment, custom prefix, multiple make
......@@ -565,11 +567,12 @@ and a new buildout to try it out
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%(src)s/package-0.0.0.tar.gz
... url = file://%(package_path)s
... pre-configure-hook = %(module)s:preconfigure
... pre-make-hook = %(module)s:premake
... post-make-hook = %(module)s:postmake
... """ % dict(src=src, module='%s/customhandlers.py' % hooks))
... """ % dict(package_path=package_path,
... module=join(hooks, 'customhandlers.py')))
>>> print(system(buildout))
Uninstalling package.
......@@ -600,12 +603,12 @@ Let's create a buildout to use these options.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-configure = echo "Configure part: ${:_buildout_section_name_}"
... pre-build = echo "OH OH OH" > a.txt
... pre-install = cat a.txt
... post-install = rm -f a.txt && echo "Finished."
... """ % src)
... """ % package_path)
This will run pre-configure, pre-build, pre-install, post-install as
shell command in the corresponding stage.
......@@ -637,7 +640,7 @@ The recipe can specify build options for each platform. For example,
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-configure = echo "Configure in common platform"
... post-install = echo "Finished."
...
......@@ -645,7 +648,7 @@ The recipe can specify build options for each platform. For example,
... pre-configure = echo "Configure in the CYGWIN platform"
... pre-install = echo "Installing in the CYGWIN"
... post-install = echo -n "CYGWIN " && ${package:post-install}
... """ % src)
... """ % package_path)
In the linux, the recipe gets the options from part 'package', there
are only ``pre-configure`` and ``post-install``. the output will be
......@@ -689,9 +692,9 @@ Let's see what happens when set prefix in the buildout section:
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-configure = mkdir -p "${buildout:prefix}"
... """ % src)
... """ % package_path)
>>> print(system(buildout))
Uninstalling package.
......@@ -718,10 +721,10 @@ example,
... [package]
... recipe = slapos.recipe.cmmi
... prefix = ${buildout:parts-directory}/package
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-configure = rm -rf "${buildout:prefix}"
... post-install = test -d "${buildout:prefix}" || echo "None"
... """ % src)
... """ % package_path)
>>> print(system(buildout))
Uninstalling package.
......@@ -748,13 +751,13 @@ prefix:
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/a.txt"
... [package-2]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/b.txt"; echo
... """ % (src, src))
... """ % (package_path, package_path))
>>> print(system(buildout))
Uninstalling package.
......@@ -787,9 +790,9 @@ as nothing it is removing):
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/a.txt"
... """ % src)
... """ % package_path)
>>> print(system(buildout))
Uninstalling package-2.
......@@ -832,17 +835,17 @@ value in the part section. For example::
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... configure-command = true
... make-binary = true
...
... [package-2]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... configure-command = true
... make-binary = true
... post-install = echo package magic prefix is ${package:prefix}
... """ % (src, src))
... """ % (package_path, package_path))
>>> print(system(buildout))
Uninstalling package.
......@@ -863,10 +866,10 @@ can display "prefix" value in the stdout.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... configure-command = ./configure
... pre-install = sed -i -e "s/installing package/installing package at \\$\\$prefix /g" Makefile
... """ % src)
... """ % package_path)
>>> print(system(buildout))
Uninstalling package-2.
......@@ -888,11 +891,11 @@ replaced with the recipe final prefix.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... configure-command = ./configure
... make-targets = install-lib prefix=%%(prefix)s
... pre-install = sed -i -e "s/installing package/installing package at \\$\\$prefix /g" Makefile
... """ % src)
... """ % package_path)
>>> print(system(buildout))
Uninstalling package.
......@@ -906,17 +909,12 @@ Install shared package
======================
Use option ``shared`` to install a shared package.
>>> import os
>>> _ = system('chmod -R u+w %(path)s && rm -rf %(path)s' % dict(
... path=join(os.path.dirname(__file__), 'shared')))
>>> shared_dir = join(os.path.dirname(__file__), 'shared')
>>> os.mkdir(shared_dir)
>>> _ = system('chmod -R u+w %(path)s && rm -rf %(path)s' % dict(
... path=join(os.path.dirname(__file__), 'another_shared_dir')))
>>> another_shared_dir = join(
... os.path.dirname(__file__), 'another_shared_dir')
>>> os.mkdir(another_shared_dir)
>>> import subprocess
>>> shared_dir = tmpdir('shared')
>>> another_shared_dir = tmpdir('another_shared_dir')
>>> __tear_downs.insert(0, lambda: subprocess.call(
... ('chmod', '-R', 'u+w', shared_dir, another_shared_dir)))
If no ``shared-part-list`` is set, and ``shared`` is True, ``shared`` feature
is not used.
......@@ -929,9 +927,9 @@ is not used.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... shared = True
... """% src)
... """ % package_path)
>>> print(system(buildout)) #doctest:+ELLIPSIS
Uninstalling package.
......@@ -946,12 +944,11 @@ debugging.
Also a shell script with the environment variable is created, so that
developer can try same build process as the recipe tried.
>>> _ = system('mv %s/package-0.0.0.tar.gz %s/package-0.0.0.tar.gz.bak' % (src, src))
>>> os.rename(package_path, package_path + '.bak')
>>> import tarfile
>>> from io import BytesIO
>>> import sys
>>> tarpath = os.path.join(src, 'package-0.0.0.tar.gz')
>>> with tarfile.open(tarpath, 'w:gz') as tar:
>>> with tarfile.open(package_path, 'w:gz') as tar:
... configure = b'invalid'
... info = tarfile.TarInfo('configure.off')
... info.size = len(configure)
......@@ -966,39 +963,39 @@ developer can try same build process as the recipe tried.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... shared = True
... environment =
... FOO=bar
... """ % (shared_dir, src))
... """ % (shared_dir, package_path))
>>> print(system(buildout)) #doctest:+ELLIPSIS
package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
package: shared at /shared/package/<MD5SUM:0>
Uninstalling package.
Installing package.
package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
package: [ENV] FOO = bar
package: Command 'set -e;./configure --prefix="/shared/package/FIRST_SHARED_PACKAGE_HASH"' returned non-zero exit status 127.
package: Compilation error. The package is left as is at /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__ where you can inspect what went wrong.
package: Command 'set -e;./configure --prefix="/shared/package/<MD5SUM:0>"' returned non-zero exit status 127.
package: Compilation error. The package is left as is at /shared/package/<MD5SUM:0>__compile__ where you can inspect what went wrong.
A shell script slapos.recipe.build.env.sh has been generated. You can source it in your shell to reproduce build environment.
/bin/sh: 1: ./configure: not found
While:
Installing package.
Error: System error
>>> import glob
>>> cat(glob.glob(os.path.join(shared_dir, 'package/**__compile__/slapos.recipe.build.env.sh'))[0])
>>> cat(glob.glob(join(shared_dir, 'package/**__compile__/slapos.recipe.build.env.sh'))[0])
export FOO=bar
...
Next time buildout runs, it detects that the build failed, remove the compile dir and retry.
>>> print(system(buildout)) #doctest:+ELLIPSIS
package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
package: shared at /shared/package/<MD5SUM:0>
Installing package.
package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
package: [ENV] FOO = bar
package: Removing already existing directory /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__
package: Command 'set -e;./configure --prefix="/shared/package/FIRST_SHARED_PACKAGE_HASH"' returned non-zero exit status 127.
package: Compilation error. The package is left as is at /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__ where you can inspect what went wrong.
package: Removing already existing directory /shared/package/<MD5SUM:0>__compile__
package: Command 'set -e;./configure --prefix="/shared/package/<MD5SUM:0>"' returned non-zero exit status 127.
package: Compilation error. The package is left as is at /shared/package/<MD5SUM:0>__compile__ where you can inspect what went wrong.
A shell script slapos.recipe.build.env.sh has been generated. You can source it in your shell to reproduce build environment.
/bin/sh: 1: ./configure: not found
While:
......@@ -1012,8 +1009,7 @@ shared part. The shared part was left in an inconsistent state that looked like
Let's simulate a scenario where buildout is terminated in the middle of a build.
For this, we need a package that takes a lot of time to install.
>>> tarpath = os.path.join(src, 'package-0.0.0.tar.gz')
>>> with tarfile.open(tarpath, 'w:gz') as tar:
>>> with tarfile.open(package_path, 'w:gz') as tar:
... configure = b'#!/bin/sh\necho configure started\nsleep 300;'
... info = tarfile.TarInfo('configure')
... info.size = len(configure)
......@@ -1023,7 +1019,6 @@ For this, we need a package that takes a lot of time to install.
We also need a bit more complex method to run buildout so that we send a termination signal in the
middle of build process.
>>> import subprocess
>>> import signal
>>> import time
>>> buildout_process = subprocess.Popen(buildout, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
......@@ -1048,14 +1043,14 @@ This was not the case in version 0.11 of this recipe. If installation of a share
not detected and the part was considered as installed.
>>> _ = system('mv %s/package-0.0.0.tar.gz.bak %s/package-0.0.0.tar.gz' % (src, src))
>>> os.rename(package_path + '.bak', package_path)
>>> print(system(buildout)) #doctest:+ELLIPSIS
package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
package: shared at /shared/package/<MD5SUM:0>
Installing package.
package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
package: [ENV] FOO = bar
package: Removing already existing directory /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__
configure --prefix=/shared/package/FIRST_SHARED_PACKAGE_HASH
package: Removing already existing directory /shared/package/<MD5SUM:0>__compile__
configure --prefix=/shared/package/<MD5SUM:0>
building package
installing package
......@@ -1086,15 +1081,15 @@ used instead of installing if one package has been installed.
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... shared = True
... environment =
... FOO=bar
... """ % (shared_dir, another_shared_dir, src))
... """ % (shared_dir, another_shared_dir, package_path))
>>> print(system(buildout)) #doctest:+ELLIPSIS
package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
package: shared at /shared/package/<MD5SUM:0>
Installing package.
package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
package: This shared package has been installed by other package
If options change, reinstall in different location:
......@@ -1107,17 +1102,17 @@ If options change, reinstall in different location:
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... url = file://%s
... shared =True
... change = True
... """ % (shared_dir, src))
... """ % (shared_dir, package_path))
>>> print(system(buildout)) #doctest:+ELLIPSIS
package: shared at /shared/package/ANOTHER_SHARED_PACKAGE_HASH
package: shared at /shared/package/<MD5SUM:1>
Uninstalling package.
Installing package.
package: Checking whether package is installed at shared path: /shared/package/ANOTHER_SHARED_PACKAGE_HASH
configure --prefix=/shared/package/ANOTHER_SHARED_PACKAGE_HASH
package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:1>
configure --prefix=/shared/package/<MD5SUM:1>
building package
installing package
......
slapos/recipe/cmmi/tests.py
View file @ 3de46334
......@@ -328,24 +328,15 @@ class NonInformativeTests(unittest.TestCase):
self.assertTrue(os.path.exists(build_directory))
def test_suite():
# Hash used in the test depend on the file:// url of the package, so
# we use re-normalizer to replace SHARED_PACKAGE_HASH with the actual
# hash for this path.
package_url = 'file://%s/package-0.0.0.tar.gz' % os.path.join(
os.path.dirname(__file__), 'testdata')
signature_FIRST_SHARED_PACKAGE_HASH = Signature('.slapos.recipe.cmmi.signature')
signature_FIRST_SHARED_PACKAGE_HASH.update('environment', 'FOO=bar')
signature_FIRST_SHARED_PACKAGE_HASH.update('recipe', 'slapos.recipe.cmmi')
signature_FIRST_SHARED_PACKAGE_HASH.update('shared', 'True')
signature_FIRST_SHARED_PACKAGE_HASH.update('url', package_url)
signature_ANOTHER_SHARED_PACKAGE_HASH = Signature('.slapos.recipe.cmmi.signature')
signature_ANOTHER_SHARED_PACKAGE_HASH.update('change', 'True')
signature_ANOTHER_SHARED_PACKAGE_HASH.update('recipe', 'slapos.recipe.cmmi')
signature_ANOTHER_SHARED_PACKAGE_HASH.update('shared', 'True')
signature_ANOTHER_SHARED_PACKAGE_HASH.update('url', package_url)
sums = []
def md5sum(m):
x = m.group(0)
try:
i = sums.index(x)
except ValueError:
i = len(sums)
sums.append(x)
return '<MD5SUM:%s>' % i
suite = unittest.TestSuite((
doctest.DocFileSuite(
'README.rst',
......@@ -361,10 +352,7 @@ def test_suite():
'--prefix=/shared/'),
(re.compile(r'\s/\S+\/shared\/'),
' /shared/'),
(re.compile('FIRST_SHARED_PACKAGE_HASH'),
signature_FIRST_SHARED_PACKAGE_HASH.hexdigest()),
(re.compile('ANOTHER_SHARED_PACKAGE_HASH'),
signature_ANOTHER_SHARED_PACKAGE_HASH.hexdigest()),
(re.compile('[0-9a-f]{32}'), md5sum),
# Normalize subprocess.CalledProcessError message, on python >= 3.6
# there's an extra . at the end.
(re.compile(r'Command (.*) returned non-zero exit status (\d+)[\.]{0,1}'),
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7