README.rst 22.5 KB
Newer Older
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
1 2 3 4 5 6
=====================
 slapos.recipe.build
=====================

.. contents::

7 8
Default
-------
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
9

Julien Muchembled's avatar
Julien Muchembled committed
10 11 12 13 14
The default recipe can be used to execute ad-hoc Python code at
init/install/update phases. `install` must create the path pointed to by
`location` (default is ${buildout:parts-directory}/${:_buildout_section_name_})
and any other file system change is not tracked by buildout. `install` defaults
to `update`, in which case `location` is ignored.
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
15

Julien Muchembled's avatar
Julien Muchembled committed
16
Example that installs software::
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
17 18 19 20 21 22 23 24 25

  [buildout]
  parts =
    script

  [script]
  recipe = slapos.recipe.build
  slapos_promise =
    directory:include
Julien Muchembled's avatar
Julien Muchembled committed
26 27 28 29
    file:share/man/man1/foo.1
    statlib:lib/libfoo.a
    statlib:lib/libfoo.la
    dynlib:bin/foo linked:libbar.so.1,libc.so.6,libfoo.so.1 rpath:${bar:location}/lib,!/lib
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
30 31
  x86 = http://host/path/x86.zip [md5sum]
  x86-64 =  http://host/path/x64.zip [md5sum]
Julien Muchembled's avatar
Julien Muchembled committed
32 33 34 35 36 37 38
  install =
    url, md5sum = options[guessPlatform()].split()
    extract_dir = self.extract(self.download(url, md5sum))
    self.copyTree(guessworkdir(extract_dir), location)
    ${:update}
  update =
    ...
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
39

40 41 42 43 44
Using the init option::

  [section-one]
  recipe = slapos.recipe.build
  init =
Julien Muchembled's avatar
Julien Muchembled committed
45 46
    import platform
    options['foo'] = platform.uname()[4]
47 48 49 50

  [section-two]
  bar = ${section-one:foo}

51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108

A simplified example::

  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = section-two
  ...
  ... [section-one]
  ... recipe = slapos.recipe.build
  ... init =
  ...   options['foo'] = 'foo from section-one'
  ...
  ... [section-two]
  ... recipe = slapos.recipe.build
  ... bar = ${section-one:foo}
  ... install =
  ...   import os
  ...   os.mkdir(options['location'])
  ...   print('Installed section-two with option %s.' % options['bar'])
  ... """)

  >>> print(system(buildout))
  Installing section-one.
  Installing section-two.
  Installed section-two with option foo from section-one.
  ...

  >>> ls(sample_buildout, 'parts')
  d section-two

In case of error, a proper traceback is displayed and nothing is installed

  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = section-two
  ...
  ... [section-two]
  ... recipe = slapos.recipe.build
  ... install =
  ...   import os
  ...   os.mkdir(options['location'])
  ...   print(1 / 0.) # this is an error !
  ... """)

  >>> print(system(buildout))
  Uninstalling section-two.
  Uninstalling section-one.
  Installing section-two.
  ...
  While:
    Installing section-two.
  <BLANKLINE>
  An internal error occurred due to a bug in either zc.buildout or in a
  recipe being used:
  Traceback (most recent call last):
  ...
109 110
    File "section-two", line 3, in <module>
      print(1 / 0.) # this is an error !
111 112 113 114 115 116
  ZeroDivisionError: float division by zero

  >>> ls(sample_buildout, 'parts')
  <BLANKLINE>


Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151
Pure download
~~~~~~~~~~~~~

::

  [buildout]
  parts =
    download

  [download]
  recipe = slapos.recipe.build:download
  url = https://some.url/file

Such profile will download https://some.url/file and put it in
buildout:parts-directory/download/download

filename parameter can be used to change destination named filename.

destination parameter allows to put explicit destination.

md5sum parameter allows pass md5sum.

mode (octal, so for rw-r--r-- use 0644) allows to set mode

Exposes target attribute which is path to downloaded file.

Notes
-----

This recipe suffers from buildout download utility issue, which will do not
try to redownload resource with wrong md5sum.

==============================
 slapos.recipe.build:gitclone
==============================
Antoine Catton's avatar
Antoine Catton committed
152

153
Checkout a git repository and its submodules by default.
154 155
Supports slapos.libnetworkcache if present, and if boolean 'use-cache' option
is true.
Antoine Catton's avatar
Antoine Catton committed
156 157

Examples
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
158
--------
Antoine Catton's avatar
Antoine Catton committed
159

160 161
Those examples use slapos.recipe.build repository as an example.

Antoine Catton's avatar
Antoine Catton committed
162
Simple clone
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
163
~~~~~~~~~~~~
Antoine Catton's avatar
Antoine Catton committed
164

165
Only `repository` parameter is required. For each buildout run,
Antoine Catton's avatar
Antoine Catton committed
166 167
the recipe will pick up the latest commit on the remote master branch::

168 169 170 171 172 173 174
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
175
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
176
  ... use-cache = true
177
  ... """)
Antoine Catton's avatar
Antoine Catton committed
178 179

This will clone the git repository in `parts/git-clone` directory.
180 181
Then let's run the buildout::

Bryton Lacquement's avatar
Bryton Lacquement committed
182
  >>> print(system(buildout))
183
  Installing git-clone.
184
  Cloning into '/sample-buildout/parts/git-clone'...
185 186 187 188 189

Let's take a look at the buildout parts directory now::

  >>> ls(sample_buildout, 'parts')
  d git-clone
Antoine Catton's avatar
Antoine Catton committed
190

191 192
When updating, it will do a "git fetch; git reset @{upstream}"::

Bryton Lacquement's avatar
Bryton Lacquement committed
193
  >>> print(system(buildout))
194 195 196 197
  Updating git-clone.
  Fetching origin
  HEAD is now at ...

Antoine Catton's avatar
Antoine Catton committed
198
Specific branch
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
199
~~~~~~~~~~~~~~~
Antoine Catton's avatar
Antoine Catton committed
200 201 202 203

You can specify a specific branch using `branch` option. For each
run it will take the latest commit on this remote branch::

204 205 206 207 208 209 210
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
211 212
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
  ... branch = build_remove_downloaded_files
213
  ... """)
Antoine Catton's avatar
Antoine Catton committed
214

215 216
Then let's run the buildout::

Bryton Lacquement's avatar
Bryton Lacquement committed
217
  >>> print(system(buildout))
218
  Uninstalling git-clone.
219
  Running uninstall recipe.
220
  Installing git-clone.
221
  Cloning into '/sample-buildout/parts/git-clone'...
222 223 224 225 226 227 228

Let's take a look at the buildout parts directory now::

  >>> ls(sample_buildout, 'parts')
  d git-clone

And let's see that current branch is "build"::
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
229

230 231
  >>> import subprocess
  >>> cd('parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
232
  >>> print(subprocess.check_output(['git', 'branch'], universal_newlines=True))
233
  * build_remove_downloaded_files
Antoine Catton's avatar
Antoine Catton committed
234

235 236 237
When updating, it will do a "git fetch; git reset build"::

  >>> cd(sample_buildout)
Bryton Lacquement's avatar
Bryton Lacquement committed
238
  >>> print(system(buildout))
239 240 241 242
  Updating git-clone.
  Fetching origin
  HEAD is now at ...

Antoine Catton's avatar
Antoine Catton committed
243
Specific revision
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
244
~~~~~~~~~~~~~~~~~
Antoine Catton's avatar
Antoine Catton committed
245 246

You can specify a specific commit hash or tag using `revision` option.
247
This option has priority over the "branch" option::
Antoine Catton's avatar
Antoine Catton committed
248

249 250 251 252 253 254 255 256
  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
257
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
258 259
  ... revision = 2566127
  ... """)
Antoine Catton's avatar
Antoine Catton committed
260

261 262
Then let's run the buildout::

Bryton Lacquement's avatar
Bryton Lacquement committed
263
  >>> print(system(buildout))
264
  Uninstalling git-clone.
265
  Running uninstall recipe.
266
  Installing git-clone.
267
  Cloning into '/sample-buildout/parts/git-clone'...
268
  HEAD is now at 2566127 ...
269 270 271 272 273 274

Let's take a look at the buildout parts directory now::

  >>> ls(sample_buildout, 'parts')
  d git-clone

275
And let's see that current revision is "2566127"::
276 277 278

  >>> import subprocess
  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
279
  >>> print(subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD'], universal_newlines=True))
280
  2566127
Antoine Catton's avatar
Antoine Catton committed
281

282
When updating, it shouldn't do anything as revision is mentioned::
283 284

  >>> cd(sample_buildout)
Bryton Lacquement's avatar
Bryton Lacquement committed
285
  >>> print(system(buildout))
286
  Updating git-clone.
Rafael Monnerat's avatar
Rafael Monnerat committed
287
  ...
288

289
Empty revision/branch
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
290
~~~~~~~~~~~~~~~~~~~~~
291 292 293 294 295 296 297 298 299 300 301 302 303

Specifying an empty revision or an empty branch will make buildout
ignore those values as if it was not present at all (allowing to easily
extend an existing section specifying a branch)::

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone-with-branch]
  ... recipe = slapos.recipe.build:gitclone
304
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
305 306 307 308 309 310 311 312
  ... revision = 2566127
  ...
  ... [git-clone]
  ... <= git-clone-with-branch
  ... revision =
  ... branch = master
  ... """)

Bryton Lacquement's avatar
Bryton Lacquement committed
313
  >>> print(system(buildout))
314 315 316 317 318 319
  Uninstalling git-clone.
  Running uninstall recipe.
  Installing git-clone.
  Cloning into '/sample-buildout/parts/git-clone'...

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
320
  >>> print(system('git branch'))
321
  * master
322

323
Revision/branch priority
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
324
~~~~~~~~~~~~~~~~~~~~~~~~
325 326 327 328 329 330 331 332 333 334 335 336

If both revision and branch parameters are set, revision parameters is used
and branch parameter is ignored::

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
337
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
338 339 340 341
  ... branch = mybranch
  ... revision = 2566127
  ... """)

Bryton Lacquement's avatar
Bryton Lacquement committed
342
  >>> print(system(buildout))
343 344 345 346 347 348 349 350
  Uninstalling git-clone.
  Running uninstall recipe.
  Installing git-clone.
  Warning: "branch" parameter with value "mybranch" is ignored. Checking out to revision 2566127...
  Cloning into '/sample-buildout/parts/git-clone'...
  HEAD is now at 2566127 ...

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
351
  >>> print(system('git branch'))
352 353
  * master

354
Setup a "develop" repository
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
355
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
356

357
If you need to setup a repository that will be manually altered over time for
358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376
development purposes, you need to make sure buildout will NOT alter it and NOT
erase your local modifications by specifying the "develop" flag::

  [buildout]
  parts = git-clone

  [git-clone]
  recipe = slapos.recipe.build:gitclone
  repository = https://example.net/example.git/
  develop = true

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
377
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
378 379
  ... develop = true
  ... """)
Antoine Catton's avatar
Antoine Catton committed
380

Bryton Lacquement's avatar
Bryton Lacquement committed
381
  >>> print(system(buildout))
382 383 384 385 386 387 388 389 390
  Uninstalling git-clone.
  Running uninstall recipe.
  Installing git-clone.
  Cloning into '/sample-buildout/parts/git-clone'...

Buildout will then keep local modifications, instead of resetting the
repository::

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
391
  >>> print(system('echo foo > setup.py'))
392 393

  >>> cd(sample_buildout)
Bryton Lacquement's avatar
Bryton Lacquement committed
394
  >>> print(system(buildout))
395
  Updating git-clone.
Rafael Monnerat's avatar
Rafael Monnerat committed
396 397 398
  ...
  <BLANKLINE>

399 400

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
401
  >>> print(system('cat setup.py'))
402 403
  foo

404 405 406
Then, when update occurs, nothing is done::

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
407
  >>> print(system('echo kept > local_change'))
408

Bryton Lacquement's avatar
Bryton Lacquement committed
409
  >>> print(system('git remote add broken http://git.erp5.org/repos/nowhere'))
410 411 412
  ...

  >>> cd(sample_buildout)
Bryton Lacquement's avatar
Bryton Lacquement committed
413
  >>> print(system(buildout))
414
  Updating git-clone.
Rafael Monnerat's avatar
Rafael Monnerat committed
415
  ...
416

417
  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement's avatar
Bryton Lacquement committed
418
  >>> print(system('cat local_change'))
419 420
  kept

421 422 423 424 425 426 427 428 429 430
In case of uninstall, buildout will keep the repository directory::

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
431
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
432 433 434 435 436
  ... develop = true
  ... # Triggers uninstall/install because of section signature change
  ... foo = bar
  ... """)

Bryton Lacquement's avatar
Bryton Lacquement committed
437
  >>> print(system(buildout))
438 439 440 441 442
  Uninstalling git-clone.
  Running uninstall recipe.
  You have uncommited changes in /sample-buildout/parts/git-clone. This folder will be left as is.
  Installing git-clone.
  destination directory already exists.
Rafael Monnerat's avatar
Rafael Monnerat committed
443 444
  ...
  <BLANKLINE>
Antoine Catton's avatar
Antoine Catton committed
445

446 447
Specific git binary
~~~~~~~~~~~~~~~~~~~
448

449 450 451
The default git command is `git`, if for a any reason you don't
have git in your path, you can specify git binary path with `git-command`
option.
452

453
Ignore SSL certificate
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
454
~~~~~~~~~~~~~~~~~~~~~~
455 456 457 458 459 460 461 462 463 464 465 466 467 468

By default, when remote server use SSL protocol git checks if the SSL
certificate of the remote server is valid before executing commands.
You can force git to ignore this check using `ignore-ssl-certificate`
boolean option::

  [buildout]
  parts = git-clone

  [git-clone]
  recipe = slapos.recipe.build:gitclone
  repository = https://example.net/example.git/
  ignore-ssl-certificate = true

469 470 471 472 473 474 475 476 477 478 479 480 481 482 483
Ignore cloning submodules
~~~~~~~~~~~~~~~~~~~~~~~~~

By default, cloning the repository will clone its submodules also. You can force
git to ignore cloning submodules by defining `ignore-cloning-submodules` boolean
option to 'true'::

  [buildout]
  parts = git-clone

  [git-clone]
  recipe = slapos.recipe.build:gitclone
  repository = https://lab.nexedi.com/tiwariayush/test_erp5
  ignore-cloning-submodules = true

484 485 486 487 488 489 490 491 492 493 494 495
Other options
~~~~~~~~~~~~~

shared
    Clone with ``--shared`` option if true. See ``git-clone`` command.

sparse-checkout
    The value of the `sparse-checkout` option is written to the
    ``$GITDIR/info/sparse-checkout`` file, which is used to populate the working
    directory sparsely. See the `SPARSE CHECKOUT` section of ``git-read-tree``
    command. This feature is disabled if the value is empty or unset.

Antoine Catton's avatar
Antoine Catton committed
496
Full example
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
497
~~~~~~~~~~~~
Antoine Catton's avatar
Antoine Catton committed
498 499 500 501 502 503 504 505 506 507 508 509

::

  [buildout]
  parts = git-clone

  [git-binary]
  recipe = hexagonit.recipe.cmmi
  url = http://git-core.googlecode.com/files/git-1.7.12.tar.gz

  [git-clone]
  recipe = slapos.recipe.build:gitclone
510
  repository = http://example.net/example.git/
Antoine Catton's avatar
Antoine Catton committed
511
  git-command = ${git-binary:location}/bin/git
512
  revision = 0123456789abcdef
Antoine Catton's avatar
Antoine Catton committed
513

514

Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
515 516 517
=========================
 slapos.recipe.build:npm
=========================
518 519 520 521

Downloads and installs node.js packages using Node Package Manager (NPM).

Examples
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
522
--------
523 524

Basic example
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
525
~~~~~~~~~~~~~
526 527 528 529 530 531 532 533 534 535 536

Here is example to install one or several modules::

  [buildout]
  parts = node-package

  [node-package]
  recipe = slapos.recipe.build:npm
  modules =
    colors
    express
537

538 539 540 541 542 543
  # Optional argument specifying perl buildout part, if existing.
  # If specified, recipe will use the perl installed by buildout.
  # If not specified, will take the globally available perl executable.
  node = node-0.6

Specific version
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
544
~~~~~~~~~~~~~~~~
Cédric de Saint Martin's avatar
Cédric de Saint Martin committed
545
::
546 547 548 549 550

  [buildout]
  parts = node-package

  [node-package]
Kazuhiko Shiozaki's avatar
Kazuhiko Shiozaki committed
551
  recipe = slapos.recipe.build:npm
552
  modules =
553
    express@1.0.2
554
  node = node-0.6
555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623

==========================
 slapos.recipe.build:vm.*
==========================

This is a set of recipes to build Virtual Machine images and execute commands
inside them. They rely on QEMU and OpenSSH: executables are found via the PATH
environment variable. They do nothing on update.

Common options
--------------

location
    Folder where the recipe stores any produced file.
    Default: ${buildout:parts-directory}/<section_name>

environment
    Extra environment for the spawn executables. It can either be the name of a
    section or a list of variables (1 per line, in the form ``key=value``).
    Values are expanded with current environment using Python %-dict formatting.

mem
    Python expression evaluating to an integer that specifies the
    RAM size in MB for the VM.

smp
    Number of CPUs for the VM. Default: 1

Example
~~~~~~~

::

  [vm-run-environment]
  PATH = ${openssh:location}/bin:${qemu:location}/bin:%(PATH)s

  [vm-run-base]
  recipe = slapos.recipe.build:vm.run
  environment = vm-run-environment
  mem = 256 * (${:smp} + 1)
  smp = 4

slapos.recipe.build:vm.install-debian
-------------------------------------

Install Debian from an ISO image. Additional required binaries:

- ``7z`` (from 7zip), to extract kernel/initrd from the ISO;
- ``file``, which is used to test that the VM image is bootable.

Currently, it only produces `raw` images, in `discard` mode (see ``-drive``
QEMU option): combined the use of ``discard`` mount option, this minimizes
the used space on disk.

Options
~~~~~~~

location
    Produced files: ``<dist>.img`` (1 for each token of `dists`), ``passwd``
    and optionally ``ssh.key``

arch
    QEMU architecture (the recipe runs the ``qemu-system-<arch>`` executable).
    It is also used to select the ISO in the sections refered by `dists`.
    Default to host architecture.

dists
    List of VMs to build: each token refers to a buildout section name that
    describes the ISOs to use. See `ISO sections`_ below.
624
    Tokens can't contain `'.'` characters.
625 626 627 628 629 630

size
    Size of the VM image. This must be an integer, optionally followed by a
    IEC or SI suffix.

mem
631
    Default: 384
632

633
[<dist>/]preseed.<preseed>
634 635
    Set the <preseed> value for the installation. The recipe has many default
    preseed values: you can see the list in the ``InstallDebianRecipe.preseed``
636 637 638 639
    class attribute (file ``slapos/recipe/vm.py``). Aliases are recognized
    (but the recipe includes a mapping that may be out-of-date.).
    Any value except ``passwd/*`` can optionally be prefixed so that they only
    apply for a particular VM.
640

641
[<dist>/]debconf.<owner>
642 643
    List of debconf value for <owner> (usually a package name),
    each line with 2 whitespace-separated parts: <key> <value>.
644
    Like for preseed.* values, they can be specific to <dist>.
645 646 647 648

late-command
    Shell commands to execute at the end of the installation. They are run
    inside the target system. This is a reliable alternative to the
649 650
    ``preseed.preseed/late_command`` option. The ``DIST`` shell variable is
    set to the VM being built.
651 652

packages
653
    Extra packages to install.
654
    Like for `late-command`, do not use ``preseed.pkgsel/include``.
655 656 657
    If you want to install packages only for some specific <dist>, you can do
    it in ``late-command``, by testing ``$DIST`` and using
    ``apt-get install -y``.
658

659 660 661 662 663 664 665 666
vm.run
    Boolean value that is `true` by default, to configure the VM for use with
    the `slapos.recipe.build:vm.run`_ recipe:

    - make sure that the `ssh` and `sudo` packages are installed
    - an SSH key is automatically created with ``ssh-keygen``, and it can be
      used to connect as `root`

667 668 669 670 671 672 673 674 675 676 677 678 679 680
ISO sections
~~~~~~~~~~~~

<arch>.iso
    Name of the section that provides the ISO image, for example by downloading
    it. This section must define 2 options: `location` is the folder
    containing the ISO, and `filename` is the file name of the ISO.

<arch>.kernel
    Path to kernel image inside the ISO.

<arch>.initrd
    Path to initrd image inside the ISO.

681 682 683
User setup
~~~~~~~~~~

684 685
By default, there's no normal user created. Another rule is that a random
password is automatically generated if there is no password specified.
686

687
You have nothing to do if you only plan to use the VM with `vm.run`.
688 689 690 691 692 693

For more information about the ``passwd/*`` preseed values, you can look at
the ``user-setup-udeb`` package at
https://anonscm.debian.org/cgit/d-i/user-setup.git/tree/
and in particular the ``user-setup-ask`` and ``user-setup-apply`` scripts.

694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716
Example
~~~~~~~

::

  [vm-install-environment]
  # vm-run-environment refers to the section in common options
  PATH = ${file:location}/bin:${p7zip:location}/bin:${vm-run-environment:PATH}

  [vm-debian]
  recipe = slapos.recipe.build:vm.install-debian
  environment = vm-install-environment
  dists = debian-jessie debian-stretch
  size = 2Gi
  late-command =
  # rdnssd causes too much trouble with QEMU 2.7, because the latter acts as
  # a DNS proxy on both IPv4 and IPv6 without translating queries to what the
  # host supports.
    dpkg -P rdnssd
  debconf.debconf =
    debconf/frontend noninteractive
    debconf/priority critical
  # minimal size
717
  preseed.apt-setup/enable-source-repositories = false
718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740
  preseed.recommends = false
  preseed.tasks =

  [debian-jessie]
  x86_64.iso = debian-amd64-netinst.iso
  x86_64.kernel = install.amd/vmlinuz
  x86_64.initrd = install.amd/initrd.gz

  [debian-stretch]
  <= debian-jessie
  x86_64.iso = debian-amd64-testing-netinst.iso

  [debian-amd64-netinst.iso]
  ...

slapos.recipe.build:vm.run
--------------------------

Execute shell commands inside a VM, in snapshot mode (the VM image is not
modified).

``${buildout:directory}`` is always mounted as `/mnt/buildout` inside the VM.

741 742 743 744 745 746
Mount points use the 9p file-system. Make sure that:

- QEMU is built with --enable-virtfs;
- the VM runs a kernel that is recent enough (Debian Squeeze kernel 2.6.32 is
  known to fail, and you'd have to use the one from squeeze-backports).

747 748 749 750 751 752 753 754 755 756 757 758 759 760 761
Options
~~~~~~~

location
    Folder where to store any produce file. Inside the guest, it is pointed to
    by the PARTDIR environment variable. It is also used as temporary storage
    for changes to the VM image.

vm
    Folder containing the VM images and the `ssh.key`` file. See the `location`
    option of the `vm.install-*` recipes.

dist
    VM image to use inside the `vm` folder.

762 763 764
drives
    Extra drives. Each line is passed with -drive

765 766 767 768 769 770 771 772 773 774 775 776 777
commands
    List of <command> options, each one being a shell script to execute via
    SSH. They are processed in sequence. This is usually only required if you
    want to reboot the VM. Default: command

mount.<name>
    Extra mount point. The value is a host folder that is mounted as
    ``/mnt/<name>``.

stop-ssh
    Tell `reboot` function how to stop SSH (see Helpers_).
    Default: systemctl stop ssh

778
user
779 780 781 782 783 784 785 786 787
    Execute commands with this user. The value can be ``root``. By default,
    it is empty and it means that:

    - a ``slapos`` user is created with the same uid/gid than the user using
      this recipe on the host, which can help accessing mount points;
    - sudo must be installed and the created user is allowed to become root
      without password.

    In any case, SSH connects as root.
788

789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826
wait-ssh
    Time to wait for (re)boot. The recipe fails if it can't connect to the SSH
    server after this number of seconds. Default: 60

Helpers
~~~~~~~

Before commands are executed, all `mount.<name>` are mounted
and a few helpers are set to make scripting easier.

set -e
    This is done before anything else, to make buildout abort if any untested
    command fails.

reboot
    Function to safely reboot the guest. The next command in `commands` will be
    executed once the SSH server is back.

map <host_path>
    Function to map a folder inside ``${buildout:directory}``.

PARTDIR
    Folder where to store any produced file. Inside the guest, it actually
    maps to `location` on the host. This is useful because you can't write
    ``PARTDIR=`map ${:location}``` if you don't explicitly set `location`.

Example
~~~~~~~

::

  [vm-run-base]
  # extends above example in common options
  vm = ${vm-debian:location}
  dist = debian-jessie

  [vm-debian]
  # extends above example in vm.install-debian
827
  packages += build-essential devscripts equivs git
828 829 830 831 832 833 834 835 836 837 838

  [userhosts-repository]
  recipe = slapos.recipe.build:gitclone
  repository = https://lab.nexedi.com/nexedi/userhosts.git
  # we don't need a working directory on the host
  sparse-checkout = /.gitignore

  [build-userhosts-map]
  <= vm-run-base
  repository = `map ${userhosts-repository:location}`
  command =
839 840 841
    git clone -s ${:repository} userhosts
    cd userhosts
    mk-build-deps -irs sudo -t 'apt-get -y'
842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862
    dpkg-buildpackage -uc -b -jauto
    cd ..
    mv *.changes *.deb $PARTDIR

  # Alternate way, which is required if [userhosts-repository] is extended
  # in such way that the repository is outside ${buildout:directory}.
  [build-userhosts-mount]
  <= build-userhosts-map
  mount.userhosts = ${userhosts-repository:location}
  repository = /mnt/userhosts

  [test-reboot]
  <= vm-run-base
  commands = hello world
  hello =
    uptime -s
    echo Hello ...
    reboot
  world =
    uptime -s
    echo ... world!