1. 07 Jan, 2016 2 commits
    • re6st-node: reimplement in Python the part to build files to send to OBS · d181e4f5
      Makefile was so horrible and unreliable.
      I tried https://waf.io/ which is nice but not suitable for this
      (see the example 'wscript' at the end).
      Functional improvements:
      - better detection of what needs to be rebuilt or not
      - reproducible tarballs, except for the re6stnet egg
        (and the main tarball if the egg is rebuilt)
      - fewer temporary files
      And support for OSC is back.
      import os, shutil, subprocess, urllib
      from waflib import Node, Utils
      PREFIX = "opt/re6st"
      BOOTSTRAP_URL = "http://downloads.buildout.org/1/bootstrap.py"
      repo_dict = dict(
      def configure(ctx):
          for name, url in repo_dict.iteritems():
              if ctx.path.find_node(name) is None:
                  ctx.exec_command(("git", "clone", url), stdout=None, stderr=None)
      def cfg(task):
          o, = task.outputs
          bld = task.generator.bld
          o.write(task.inputs[0].read() % dict(
              ROOT="${buildout:directory}/" + bld.bldnode.path_from(o.parent),
      def bootstrap(task):
          b = task.outputs[0]
          d = b.parent.parent
          bootstrap = urllib.urlopen(BOOTSTRAP_URL).read()
          for cache in "download-cache", "extends-cache":
              cache = d.make_node(cache).abspath()
              if os.path.exists(cache):
          cwd = d.abspath()
          task.exec_command(("python2", "-S"), input=bootstrap, cwd=cwd)
          task.exec_command((b.abspath(), "buildout:parts=python"), cwd=cwd)
      def sdist(task):
          r, p = task.inputs
          d = p.find_node("../../download-cache/dist")
          for x in d.ant_glob("re6stnet-*", quiet=True):
          task.exec_command((p.abspath(), "setup.py", "sdist", "-d", d.abspath()),
      def build(bld):
          b = bld.bldnode.make_node(PREFIX)
          buildout_cfg = b.make_node("buildout.cfg")
          bld(source="buildout.cfg.in", target=buildout_cfg, rule=cfg)
          tg = bld(source=(buildout_cfg, "slapos"), rule=bootstrap,
                   target=map(b.make_node, ("bin/buildout", "bin/python")))
          buildout, python = tg.target
          r = bld.path.find_node("re6stnet")
          tg = bld(source=(r, python), rule=sdist, update_outputs=True,
          bld(name="buildout", source=(buildout_cfg, tg.target),
              rule=lambda task: task.exec_command((buildout.abspath(),),
                  stdout=None, stderr=None, cwd=b.abspath()))
      def h_file(fname, h_file=Utils.h_file):
          if os.path.isdir(fname):
              m = Utils.md5(fname)
              n = len(fname) + 1
              for dirpath, dirs, files in os.walk(fname):
                  if dirpath.endswith(("/.git", ".egg-info")):
                      del dirs[:]
                  m.update(dirpath[n:] + '\0')
                  for fname in files:
                      m.update("%s\0%s\0" % (h_file(os.path.join(dirpath, fname)),
              return m.digest()
          return h_file(fname)
      Utils.h_file = h_file
      def find_resource(self, lst):
          if isinstance(lst, str):
              lst = [x for x in Node.split_path(lst) if x and x != '.']
          return self.get_bld().search_node(lst) \
              or self.get_src().find_node(lst)
      Node.Node.find_resource = find_resource
      Julien Muchembled committed
    • Rewrite of re6st-node packaging · f9b61bf9
      - With a Makefile instead of a shell script to make build files,
        it's easier to only redo the necessary parts.
      - Do not bootstrap buildout in advance. This caused build failures on old
        dists when this was done with Python 2.7 (parts/buildout/site.py imported
        a module that does not exist on Python 2.6). Now everything can be prepared
        on a recent dist.
      - Minimize list of build-deps. We don't need setuptools anymore.
        We also don't rely on system chrpath anymore because SLE_12 does not have it.
      - Refactoring with upstream packaging.
      - Debian: fix upgrade path from 're6stnet' package.
      - Package re6st manpages and logrotate configuration.
      - systemd integration on RPM-based dists.
      - Fix shebang of OpenVPN/NetworkManager hooks.
      - Do never ship *.py[co] files in packages. Remove them on uninstallation
        (not done yet for Arch).
      - chrpath: Suse dists use RUNPATH instead of PATH
      - Stop stripping ELF files.
      - Greatly reduce size of packages by removing eggs/parts that were only useful
        during the build, like Perl.
      TODO: integrate with osc working copy ?
      For the record, here is an example to strip ELF files:
      	cd $(DESTDIR)/$(TARGET) \
      	&& s() { chmod u+w $$x && strip -v -R .comment -R .note "$$@" $$x; } \
      	&& x=`find -type f -executable ! -name '*.so*' |xargs -r file | \
      		sed -n 's/:.*ELF.*\(executable\|shared\).*//p'` && s \
      	&& x=`find -type f -name '*.so*' |xargs -r file | \
      		sed -n 's/:.*ELF.*shared.*//p'` && s --strip-unneeded
      Julien Muchembled committed