See nexedi/slapos.recipe.build@f458eb8b

The expected usage of zc.buildout.download.Download() is that the caller is
supposed to remove the temporary files (cf [1] [2])

[1]: https://github.com/buildout/buildout/blob/3ed7c06bf7be594dcb3c906c2517ee438f059251/src/zc/buildout/download.txt#L55
[2]: https://github.com/buildout/buildout/blob/3da22a7dacb9a4d1f771e7763e014214c31d935b/src/zc/buildout/download.py#L171-L176

See merge request nexedi/slapos.recipe.template!5

setuptools 42.0.0 prints a warning:

   WARNING: The easy_install command is deprecated and will be removed in a future version

use a test normalizer to ignore this warning in doctest.

Local variable `template` is used for different things, it is bad. Use different variable name for different thing.

compile function takes a lot of time, thus this change saves time when the same source is used many times in apache frontend SR.

__init__ should only check options. By doing the actual work in install/update,
it is now possible to use a template that is produced by another part.

'mode' was implemented like 'umask' but it has never been used this way,
and the 'umask' option is deprecated:
- remove support for unused 'umask' option
- change 'mode' to actually apply the wanted mode, regardless the current umask
- the default mode changes: if 'mode' is not given, the resulting file is only
  executable (actually 0777 & ~umask) if the content starts with '#!',
  else its mode is: 0666 & ~umask

This is an incompatible change. For example, you would now write:

  import easy_install zc.buildout.easy_install

instead of:

  import zc zc.buildout.easy_install

To be used for zope's inituser file, for example.

UTF-8 encoding for template imports was hardcoded from the beginning -
from 320f29b2 (Add jinja2 "import" directive support.)

Since in the previous patch we added option for input/output encoding,
it makes sense to also use it, instead of hardcoding utf-8.

/reviewed-by @vpelletier  (on nexedi/slapos.recipe.template!1)

Currently slapos.recipe.template:jinja2 fails if source contains
non-ascii input:

    <BLANKLINE>
    An internal error occurred due to a bug in either zc.buildout or in a
    recipe being used:
    Traceback (most recent call last):
      File ".../site-packages/zc/buildout/buildout.py", line 1992, in main
        getattr(buildout, command)(args)
      File ".../site-packages/zc/buildout/buildout.py", line 550, in install
        [self[part]['recipe'] for part in install_parts]
      File ".../site-packages/zc/buildout/buildout.py", line 1135, in __getitem__
        options._initialize()
      File ".../site-packages/zc/buildout/buildout.py", line 1239, in _initialize
        self.initialize()
      File ".../site-packages/zc/buildout/buildout.py", line 1248, in initialize
        self.recipe = recipe_class(buildout, name, self)
      File ".../slapos.recipe.template/slapos/recipe/template/jinja2_template.py", line 208, in __init__
        env.compile(source, filename=template),
      File ".../site-packages/jinja2/environment.py", line 551, in compile
        source = self._parse(source, name, filename)
      File ".../site-packages/jinja2/environment.py", line 470, in _parse
        return Parser(self, source, name, encode_filename(filename)).parse()
      File ".../site-packages/jinja2/parser.py", line 31, in __init__
        self.stream = environment._tokenize(source, name, filename, state)
      File ".../site-packages/jinja2/environment.py", line 501, in _tokenize
        source = self.preprocess(source, name, filename)
      File ".../site-packages/jinja2/environment.py", line 495, in preprocess
        self.iter_extensions(), text_type(source))
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 219: ordinal not in range(128)

it also fails writing-out non-ascii input (which could originate even
programmatically even if source was purely ascii):

    <BLANKLINE>
    An internal error occurred due to a bug in either zc.buildout or in a
    recipe being used:
    Traceback (most recent call last):
      File ".../site-packages/zc/buildout/buildout.py", line 1992, in main
        getattr(buildout, command)(args)
      File ".../site-packages/zc/buildout/buildout.py", line 666, in install
        installed_files = self[part]._call(recipe.install)
      File ".../site-packages/zc/buildout/buildout.py", line 1407, in _call
        return f()
      File ".../slapos/slapos.recipe.template/slapos/recipe/template/jinja2_template.py", line 227, in install
        out.write(self.template.render(**self.context))
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 206-211: ordinal not in range(128)

Fix it.

@vpelletier suggested to add `encoding` option so it is flexible and
'utf-8' is not hardcoded.

/reviewed-by @vpelletier  (on !1)