Skip to content

S
slapos.recipe.cmmi
Commit 3de46334 authored by Julien Muchembled's avatar Julien Muchembled
Browse files
Options

Fix tests

parent a76ec7be
Pipeline #9200 passed with stage
    Show whitespace changes
    Inline Side-by-side
    Showing with 81 additions and 98 deletions
    slapos/recipe/cmmi/README.rst
    View file @ 3de46334
    ......@@ -263,14 +263,17 @@ will add it to environmet variables as the following::
    Example usage
    =============
    We'll use a simple tarball to demonstrate the recipe.
    We'll use a few tarballs to demonstrate the recipe.
    We'll modify one of them in-place but we don't want to alter the source tree.
    >>> import os.path
    >>> import os
    >>> src = join(os.path.dirname(__file__), 'testdata')
    >>> ls(src)
    - Foo-Bar-0.0.0.tar.gz
    - haproxy-1.4.8-dummy.tar.gz
    - package-0.0.0.tar.gz
    >>> package_path = join(tmpdir('testdata'), 'package-0.0.0.tar.gz')
    >>> os.symlink(join(src, 'package-0.0.0.tar.gz'), package_path)
    The package contains a dummy ``configure`` script that will simply
    echo the options it was called with and create a ``Makefile`` that
    ......@@ -286,8 +289,8 @@ Let's create a buildout to build and install the package.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... """ % src)
    ... url = file://%s
    ... """ % package_path)
    This will download, extract and build our demo package with the
    default build options.
    ......@@ -309,9 +312,9 @@ Check option "promises"
    ...
    ... [packagex]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... promises = /usr/bin/myfoo
    ... """ % src)
    ... """ % package_path)
    This will download, extract and build our demo package with the
    default build options.
    ......@@ -414,8 +417,7 @@ and building that.
    >>> checkout_dir = tmpdir('checkout')
    >>> import setuptools.archive_util
    >>> setuptools.archive_util.unpack_archive('%s/package-0.0.0.tar.gz' % src,
    ... checkout_dir)
    >>> setuptools.archive_util.unpack_archive(package_path, checkout_dir)
    >>> ls(checkout_dir)
    d package-0.0.0
    ......@@ -466,7 +468,7 @@ with a new buildout and provide more options.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%(src)s/package-0.0.0.tar.gz
    ... url = file://%s
    ... md5sum = 6b94295c042a91ea3203857326bc9209
    ... prefix = /somewhere/else
    ... environment-section = build-environment
    ......@@ -481,7 +483,7 @@ with a new buildout and provide more options.
    ... patches =
    ... patches/configure.patch
    ... patches/Makefile.dist.patch
    ... """ % dict(src=src))
    ... """ % package_path)
    This configuration uses custom configure options, an environment section,
    per-part customization to the environment, custom prefix, multiple make
    ......@@ -565,11 +567,12 @@ and a new buildout to try it out
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%(src)s/package-0.0.0.tar.gz
    ... url = file://%(package_path)s
    ... pre-configure-hook = %(module)s:preconfigure
    ... pre-make-hook = %(module)s:premake
    ... post-make-hook = %(module)s:postmake
    ... """ % dict(src=src, module='%s/customhandlers.py' % hooks))
    ... """ % dict(package_path=package_path,
    ... module=join(hooks, 'customhandlers.py')))
    >>> print(system(buildout))
    Uninstalling package.
    ......@@ -600,12 +603,12 @@ Let's create a buildout to use these options.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-configure = echo "Configure part: ${:_buildout_section_name_}"
    ... pre-build = echo "OH OH OH" > a.txt
    ... pre-install = cat a.txt
    ... post-install = rm -f a.txt && echo "Finished."
    ... """ % src)
    ... """ % package_path)
    This will run pre-configure, pre-build, pre-install, post-install as
    shell command in the corresponding stage.
    ......@@ -637,7 +640,7 @@ The recipe can specify build options for each platform. For example,
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-configure = echo "Configure in common platform"
    ... post-install = echo "Finished."
    ...
    ......@@ -645,7 +648,7 @@ The recipe can specify build options for each platform. For example,
    ... pre-configure = echo "Configure in the CYGWIN platform"
    ... pre-install = echo "Installing in the CYGWIN"
    ... post-install = echo -n "CYGWIN " && ${package:post-install}
    ... """ % src)
    ... """ % package_path)
    In the linux, the recipe gets the options from part 'package', there
    are only ``pre-configure`` and ``post-install``. the output will be
    ......@@ -689,9 +692,9 @@ Let's see what happens when set prefix in the buildout section:
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-configure = mkdir -p "${buildout:prefix}"
    ... """ % src)
    ... """ % package_path)
    >>> print(system(buildout))
    Uninstalling package.
    ......@@ -718,10 +721,10 @@ example,
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... prefix = ${buildout:parts-directory}/package
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-configure = rm -rf "${buildout:prefix}"
    ... post-install = test -d "${buildout:prefix}" || echo "None"
    ... """ % src)
    ... """ % package_path)
    >>> print(system(buildout))
    Uninstalling package.
    ......@@ -748,13 +751,13 @@ prefix:
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/a.txt"
    ... [package-2]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/b.txt"; echo
    ... """ % (src, src))
    ... """ % (package_path, package_path))
    >>> print(system(buildout))
    Uninstalling package.
    ......@@ -787,9 +790,9 @@ as nothing it is removing):
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/a.txt"
    ... """ % src)
    ... """ % package_path)
    >>> print(system(buildout))
    Uninstalling package-2.
    ......@@ -832,17 +835,17 @@ value in the part section. For example::
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... configure-command = true
    ... make-binary = true
    ...
    ... [package-2]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... configure-command = true
    ... make-binary = true
    ... post-install = echo package magic prefix is ${package:prefix}
    ... """ % (src, src))
    ... """ % (package_path, package_path))
    >>> print(system(buildout))
    Uninstalling package.
    ......@@ -863,10 +866,10 @@ can display "prefix" value in the stdout.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... configure-command = ./configure
    ... pre-install = sed -i -e "s/installing package/installing package at \\$\\$prefix /g" Makefile
    ... """ % src)
    ... """ % package_path)
    >>> print(system(buildout))
    Uninstalling package-2.
    ......@@ -888,11 +891,11 @@ replaced with the recipe final prefix.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... configure-command = ./configure
    ... make-targets = install-lib prefix=%%(prefix)s
    ... pre-install = sed -i -e "s/installing package/installing package at \\$\\$prefix /g" Makefile
    ... """ % src)
    ... """ % package_path)
    >>> print(system(buildout))
    Uninstalling package.
    ......@@ -906,17 +909,12 @@ Install shared package
    ======================
    Use option ``shared`` to install a shared package.
    >>> import os
    >>> _ = system('chmod -R u+w %(path)s && rm -rf %(path)s' % dict(
    ... path=join(os.path.dirname(__file__), 'shared')))
    >>> shared_dir = join(os.path.dirname(__file__), 'shared')
    >>> os.mkdir(shared_dir)
    >>> _ = system('chmod -R u+w %(path)s && rm -rf %(path)s' % dict(
    ... path=join(os.path.dirname(__file__), 'another_shared_dir')))
    >>> another_shared_dir = join(
    ... os.path.dirname(__file__), 'another_shared_dir')
    >>> os.mkdir(another_shared_dir)
    >>> import subprocess
    >>> shared_dir = tmpdir('shared')
    >>> another_shared_dir = tmpdir('another_shared_dir')
    >>> __tear_downs.insert(0, lambda: subprocess.call(
    ... ('chmod', '-R', 'u+w', shared_dir, another_shared_dir)))
    If no ``shared-part-list`` is set, and ``shared`` is True, ``shared`` feature
    is not used.
    ......@@ -929,9 +927,9 @@ is not used.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... shared = True
    ... """% src)
    ... """ % package_path)
    >>> print(system(buildout)) #doctest:+ELLIPSIS
    Uninstalling package.
    ......@@ -946,12 +944,11 @@ debugging.
    Also a shell script with the environment variable is created, so that
    developer can try same build process as the recipe tried.
    >>> _ = system('mv %s/package-0.0.0.tar.gz %s/package-0.0.0.tar.gz.bak' % (src, src))
    >>> os.rename(package_path, package_path + '.bak')
    >>> import tarfile
    >>> from io import BytesIO
    >>> import sys
    >>> tarpath = os.path.join(src, 'package-0.0.0.tar.gz')
    >>> with tarfile.open(tarpath, 'w:gz') as tar:
    >>> with tarfile.open(package_path, 'w:gz') as tar:
    ... configure = b'invalid'
    ... info = tarfile.TarInfo('configure.off')
    ... info.size = len(configure)
    ......@@ -966,39 +963,39 @@ developer can try same build process as the recipe tried.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... shared = True
    ... environment =
    ... FOO=bar
    ... """ % (shared_dir, src))
    ... """ % (shared_dir, package_path))
    >>> print(system(buildout)) #doctest:+ELLIPSIS
    package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: shared at /shared/package/<MD5SUM:0>
    Uninstalling package.
    Installing package.
    package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
    package: [ENV] FOO = bar
    package: Command 'set -e;./configure --prefix="/shared/package/FIRST_SHARED_PACKAGE_HASH"' returned non-zero exit status 127.
    package: Compilation error. The package is left as is at /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__ where you can inspect what went wrong.
    package: Command 'set -e;./configure --prefix="/shared/package/<MD5SUM:0>"' returned non-zero exit status 127.
    package: Compilation error. The package is left as is at /shared/package/<MD5SUM:0>__compile__ where you can inspect what went wrong.
    A shell script slapos.recipe.build.env.sh has been generated. You can source it in your shell to reproduce build environment.
    /bin/sh: 1: ./configure: not found
    While:
    Installing package.
    Error: System error
    >>> import glob
    >>> cat(glob.glob(os.path.join(shared_dir, 'package/**__compile__/slapos.recipe.build.env.sh'))[0])
    >>> cat(glob.glob(join(shared_dir, 'package/**__compile__/slapos.recipe.build.env.sh'))[0])
    export FOO=bar
    ...
    Next time buildout runs, it detects that the build failed, remove the compile dir and retry.
    >>> print(system(buildout)) #doctest:+ELLIPSIS
    package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: shared at /shared/package/<MD5SUM:0>
    Installing package.
    package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
    package: [ENV] FOO = bar
    package: Removing already existing directory /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__
    package: Command 'set -e;./configure --prefix="/shared/package/FIRST_SHARED_PACKAGE_HASH"' returned non-zero exit status 127.
    package: Compilation error. The package is left as is at /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__ where you can inspect what went wrong.
    package: Removing already existing directory /shared/package/<MD5SUM:0>__compile__
    package: Command 'set -e;./configure --prefix="/shared/package/<MD5SUM:0>"' returned non-zero exit status 127.
    package: Compilation error. The package is left as is at /shared/package/<MD5SUM:0>__compile__ where you can inspect what went wrong.
    A shell script slapos.recipe.build.env.sh has been generated. You can source it in your shell to reproduce build environment.
    /bin/sh: 1: ./configure: not found
    While:
    ......@@ -1012,8 +1009,7 @@ shared part. The shared part was left in an inconsistent state that looked like
    Let's simulate a scenario where buildout is terminated in the middle of a build.
    For this, we need a package that takes a lot of time to install.
    >>> tarpath = os.path.join(src, 'package-0.0.0.tar.gz')
    >>> with tarfile.open(tarpath, 'w:gz') as tar:
    >>> with tarfile.open(package_path, 'w:gz') as tar:
    ... configure = b'#!/bin/sh\necho configure started\nsleep 300;'
    ... info = tarfile.TarInfo('configure')
    ... info.size = len(configure)
    ......@@ -1023,7 +1019,6 @@ For this, we need a package that takes a lot of time to install.
    We also need a bit more complex method to run buildout so that we send a termination signal in the
    middle of build process.
    >>> import subprocess
    >>> import signal
    >>> import time
    >>> buildout_process = subprocess.Popen(buildout, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    ......@@ -1048,14 +1043,14 @@ This was not the case in version 0.11 of this recipe. If installation of a share
    not detected and the part was considered as installed.
    >>> _ = system('mv %s/package-0.0.0.tar.gz.bak %s/package-0.0.0.tar.gz' % (src, src))
    >>> os.rename(package_path + '.bak', package_path)
    >>> print(system(buildout)) #doctest:+ELLIPSIS
    package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: shared at /shared/package/<MD5SUM:0>
    Installing package.
    package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
    package: [ENV] FOO = bar
    package: Removing already existing directory /shared/package/FIRST_SHARED_PACKAGE_HASH__compile__
    configure --prefix=/shared/package/FIRST_SHARED_PACKAGE_HASH
    package: Removing already existing directory /shared/package/<MD5SUM:0>__compile__
    configure --prefix=/shared/package/<MD5SUM:0>
    building package
    installing package
    ......@@ -1086,15 +1081,15 @@ used instead of installing if one package has been installed.
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... shared = True
    ... environment =
    ... FOO=bar
    ... """ % (shared_dir, another_shared_dir, src))
    ... """ % (shared_dir, another_shared_dir, package_path))
    >>> print(system(buildout)) #doctest:+ELLIPSIS
    package: shared at /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: shared at /shared/package/<MD5SUM:0>
    Installing package.
    package: Checking whether package is installed at shared path: /shared/package/FIRST_SHARED_PACKAGE_HASH
    package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:0>
    package: This shared package has been installed by other package
    If options change, reinstall in different location:
    ......@@ -1107,17 +1102,17 @@ If options change, reinstall in different location:
    ...
    ... [package]
    ... recipe = slapos.recipe.cmmi
    ... url = file://%s/package-0.0.0.tar.gz
    ... url = file://%s
    ... shared =True
    ... change = True
    ... """ % (shared_dir, src))
    ... """ % (shared_dir, package_path))
    >>> print(system(buildout)) #doctest:+ELLIPSIS
    package: shared at /shared/package/ANOTHER_SHARED_PACKAGE_HASH
    package: shared at /shared/package/<MD5SUM:1>
    Uninstalling package.
    Installing package.
    package: Checking whether package is installed at shared path: /shared/package/ANOTHER_SHARED_PACKAGE_HASH
    configure --prefix=/shared/package/ANOTHER_SHARED_PACKAGE_HASH
    package: Checking whether package is installed at shared path: /shared/package/<MD5SUM:1>
    configure --prefix=/shared/package/<MD5SUM:1>
    building package
    installing package
    ......
    slapos/recipe/cmmi/tests.py
    View file @ 3de46334
    ......@@ -328,24 +328,15 @@ class NonInformativeTests(unittest.TestCase):
    self.assertTrue(os.path.exists(build_directory))
    def test_suite():
    # Hash used in the test depend on the file:// url of the package, so
    # we use re-normalizer to replace SHARED_PACKAGE_HASH with the actual
    # hash for this path.
    package_url = 'file://%s/package-0.0.0.tar.gz' % os.path.join(
    os.path.dirname(__file__), 'testdata')
    signature_FIRST_SHARED_PACKAGE_HASH = Signature('.slapos.recipe.cmmi.signature')
    signature_FIRST_SHARED_PACKAGE_HASH.update('environment', 'FOO=bar')
    signature_FIRST_SHARED_PACKAGE_HASH.update('recipe', 'slapos.recipe.cmmi')
    signature_FIRST_SHARED_PACKAGE_HASH.update('shared', 'True')
    signature_FIRST_SHARED_PACKAGE_HASH.update('url', package_url)
    signature_ANOTHER_SHARED_PACKAGE_HASH = Signature('.slapos.recipe.cmmi.signature')
    signature_ANOTHER_SHARED_PACKAGE_HASH.update('change', 'True')
    signature_ANOTHER_SHARED_PACKAGE_HASH.update('recipe', 'slapos.recipe.cmmi')
    signature_ANOTHER_SHARED_PACKAGE_HASH.update('shared', 'True')
    signature_ANOTHER_SHARED_PACKAGE_HASH.update('url', package_url)
    sums = []
    def md5sum(m):
    x = m.group(0)
    try:
    i = sums.index(x)
    except ValueError:
    i = len(sums)
    sums.append(x)
    return '<MD5SUM:%s>' % i
    suite = unittest.TestSuite((
    doctest.DocFileSuite(
    'README.rst',
    ......@@ -361,10 +352,7 @@ def test_suite():
    '--prefix=/shared/'),
    (re.compile(r'\s/\S+\/shared\/'),
    ' /shared/'),
    (re.compile('FIRST_SHARED_PACKAGE_HASH'),
    signature_FIRST_SHARED_PACKAGE_HASH.hexdigest()),
    (re.compile('ANOTHER_SHARED_PACKAGE_HASH'),
    signature_ANOTHER_SHARED_PACKAGE_HASH.hexdigest()),
    (re.compile('[0-9a-f]{32}'), md5sum),
    # Normalize subprocess.CalledProcessError message, on python >= 3.6
    # there's an extra . at the end.
    (re.compile(r'Command (.*) returned non-zero exit status (\d+)[\.]{0,1}'),
    ......
    Markdown is supported
    0% or
    You are about to add 0 people to the discussion. Proceed with caution.
    Finish editing this message first!
    Please register or to comment
    GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7