Blame view

README.rst 22 KB
Kazuhiko Shiozaki committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
=====================
 slapos.recipe.build
=====================

.. contents::

Examples
--------

Recipe to build the software.

Example buildout::

  [buildout]
  parts =
    script

  [script]
  # default way with using script
  recipe = slapos.recipe.build
  url_0 = http://host/path/file.tar.gz
  md5sum = 9631070eac74f92a812d4785a84d1b4e
  script =
    import os
    os.chdir(%(work_directory)s)
    unpack(%(url_0), strip_path=True)
    execute('make')
    execute('make install DEST=%(location)s')
  slapos_promise =
    directory:bin
    dynlib:bin/file linked:libz.so.1,libc.so.6,libmagic.so.1 rpath:${zlib:location}/lib,!/lib
    directory:include
    file:include/magic.h
    directory:lib
    statlib:lib/libmagic.a
    statlib:lib/libmagic.la
    dynlib:lib/libmagic.so linked:libz.so.1,libc.so.6 rpath:${zlib:location}/lib
    dynlib:lib/libmagic.so.1 linked:libz.so.1,libc.so.6 rpath:${zlib:location}/lib
    dynlib:lib/libmagic.so.1.0.0 linked:libz.so.1,libc.so.6 rpath:${zlib:location}/lib
    directory:share
    directory:share/man
    directory:share/man/man1
    file:share/man/man1/file.1
    directory:share/man/man3
    file:share/man/man3/libmagic.3
    directory:share/man/man4
    file:share/man/man4/magic.4
    directory:share/man/man5
    directory:share/misc
    file:share/misc/magic.mgc

  [multiarchitecture]
  recipe = slapos.recipe.build
  slapos_promise =
    ...
  x86 = http://host/path/x86.zip [md5sum]
  x86-64 =  http://host/path/x64.zip [md5sum]
  script =
    if not self.options.get('url'): self.options['url'], self.options['md5sum'] = self.options[guessPlatform()].split(' ')
    extract_dir = self.extract(self.download(self.options['url'], self.options.get('md5sum')))
    workdir = guessworkdir(extract_dir)
    self.copyTree(workdir, "%(location)s")

You can remove formatting by using option “format = no” (default is “yes”)

For example::

  [escaping]
  recipe = slapos.recipe.build
  example = foobar's one
  script =
Bryton Lacquement committed
72
    print('%%s' %% self.options['example'])
Kazuhiko Shiozaki committed
73 74 75 76 77 78 79 80
    # will print “foobar's one”

  [no-escaping]
  recipe = slapos.recipe.build
  example = foobar's one
  foo = bar
  format = no
  script =
Bryton Lacquement committed
81
    print('%s' % self.options['example'])
Kazuhiko Shiozaki committed
82
    # will print “foobar's one”
Bryton Lacquement committed
83
    print('%(foo)s')
Kazuhiko Shiozaki committed
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122
    # will print “%(foo)s”

Pure download
~~~~~~~~~~~~~

Note: deprecated entry-point.

::

  [buildout]
  parts =
    download

  [download]
  recipe = slapos.recipe.build:download
  url = https://some.url/file

Such profile will download https://some.url/file and put it in
buildout:parts-directory/download/download

filename parameter can be used to change destination named filename.

destination parameter allows to put explicit destination.

md5sum parameter allows pass md5sum.

mode (octal, so for rw-r--r-- use 0644) allows to set mode

Exposes target attribute which is path to downloaded file.

Notes
-----

This recipe suffers from buildout download utility issue, which will do not
try to redownload resource with wrong md5sum.

==============================
 slapos.recipe.build:gitclone
==============================
Antoine Catton committed
123

Ayush Tiwari committed
124
Checkout a git repository and its submodules by default.
Julien Muchembled committed
125 126
Supports slapos.libnetworkcache if present, and if boolean 'use-cache' option
is true.
Antoine Catton committed
127 128

Examples
Kazuhiko Shiozaki committed
129
--------
Antoine Catton committed
130

131 132
Those examples use slapos.recipe.build repository as an example.

Antoine Catton committed
133
Simple clone
Kazuhiko Shiozaki committed
134
~~~~~~~~~~~~
Antoine Catton committed
135

136
Only `repository` parameter is required. For each buildout run,
Antoine Catton committed
137 138
the recipe will pick up the latest commit on the remote master branch::

139 140 141 142 143 144 145
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
146
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
Julien Muchembled committed
147
  ... use-cache = true
148
  ... """)
Antoine Catton committed
149 150

This will clone the git repository in `parts/git-clone` directory.
151 152
Then let's run the buildout::

Bryton Lacquement committed
153
  >>> print(system(buildout))
154
  Installing git-clone.
155
  Cloning into '/sample-buildout/parts/git-clone'...
156 157 158 159 160

Let's take a look at the buildout parts directory now::

  >>> ls(sample_buildout, 'parts')
  d git-clone
Antoine Catton committed
161

Cédric de Saint Martin committed
162 163
When updating, it will do a "git fetch; git reset @{upstream}"::

Bryton Lacquement committed
164
  >>> print(system(buildout))
Cédric de Saint Martin committed
165 166 167 168
  Updating git-clone.
  Fetching origin
  HEAD is now at ...

Antoine Catton committed
169
Specific branch
Kazuhiko Shiozaki committed
170
~~~~~~~~~~~~~~~
Antoine Catton committed
171 172 173 174

You can specify a specific branch using `branch` option. For each
run it will take the latest commit on this remote branch::

175 176 177 178 179 180 181
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
182 183
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
  ... branch = build_remove_downloaded_files
184
  ... """)
Antoine Catton committed
185

186 187
Then let's run the buildout::

Bryton Lacquement committed
188
  >>> print(system(buildout))
189
  Uninstalling git-clone.
190
  Running uninstall recipe.
191
  Installing git-clone.
192
  Cloning into '/sample-buildout/parts/git-clone'...
193 194 195 196 197 198 199

Let's take a look at the buildout parts directory now::

  >>> ls(sample_buildout, 'parts')
  d git-clone

And let's see that current branch is "build"::
Kazuhiko Shiozaki committed
200

201 202
  >>> import subprocess
  >>> cd('parts', 'git-clone')
Bryton Lacquement committed
203
  >>> print(subprocess.check_output(['git', 'branch'], universal_newlines=True))
Ayush Tiwari committed
204
  * build_remove_downloaded_files
Antoine Catton committed
205

Cédric de Saint Martin committed
206 207 208
When updating, it will do a "git fetch; git reset build"::

  >>> cd(sample_buildout)
Bryton Lacquement committed
209
  >>> print(system(buildout))
Cédric de Saint Martin committed
210 211 212 213
  Updating git-clone.
  Fetching origin
  HEAD is now at ...

Antoine Catton committed
214
Specific revision
Kazuhiko Shiozaki committed
215
~~~~~~~~~~~~~~~~~
Antoine Catton committed
216 217

You can specify a specific commit hash or tag using `revision` option.
218
This option has priority over the "branch" option::
Antoine Catton committed
219

220 221 222 223 224 225 226 227
  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
228
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
229 230
  ... revision = 2566127
  ... """)
Antoine Catton committed
231

232 233
Then let's run the buildout::

Bryton Lacquement committed
234
  >>> print(system(buildout))
235
  Uninstalling git-clone.
236
  Running uninstall recipe.
237
  Installing git-clone.
238
  Cloning into '/sample-buildout/parts/git-clone'...
Ayush Tiwari committed
239
  HEAD is now at 2566127 ...
240 241 242 243 244 245

Let's take a look at the buildout parts directory now::

  >>> ls(sample_buildout, 'parts')
  d git-clone

246
And let's see that current revision is "2566127"::
247 248 249

  >>> import subprocess
  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
250
  >>> print(subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD'], universal_newlines=True))
251
  2566127
Antoine Catton committed
252

Ayush Tiwari committed
253
When updating, it shouldn't do anything as revision is mentioned::
Cédric de Saint Martin committed
254 255

  >>> cd(sample_buildout)
Bryton Lacquement committed
256
  >>> print(system(buildout))
Cédric de Saint Martin committed
257
  Updating git-clone.
Rafael Monnerat committed
258
  ...
259

260
Empty revision/branch
Kazuhiko Shiozaki committed
261
~~~~~~~~~~~~~~~~~~~~~
262 263 264 265 266 267 268 269 270 271 272 273 274

Specifying an empty revision or an empty branch will make buildout
ignore those values as if it was not present at all (allowing to easily
extend an existing section specifying a branch)::

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone-with-branch]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
275
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
276 277 278 279 280 281 282 283
  ... revision = 2566127
  ...
  ... [git-clone]
  ... <= git-clone-with-branch
  ... revision =
  ... branch = master
  ... """)

Bryton Lacquement committed
284
  >>> print(system(buildout))
285 286 287 288 289 290
  Uninstalling git-clone.
  Running uninstall recipe.
  Installing git-clone.
  Cloning into '/sample-buildout/parts/git-clone'...

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
291
  >>> print(system('git branch'))
292
  * master
Cédric de Saint Martin committed
293

294
Revision/branch priority
Kazuhiko Shiozaki committed
295
~~~~~~~~~~~~~~~~~~~~~~~~
296 297 298 299 300 301 302 303 304 305 306 307

If both revision and branch parameters are set, revision parameters is used
and branch parameter is ignored::

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
308
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
309 310 311 312
  ... branch = mybranch
  ... revision = 2566127
  ... """)

Bryton Lacquement committed
313
  >>> print(system(buildout))
314 315 316 317 318 319 320 321
  Uninstalling git-clone.
  Running uninstall recipe.
  Installing git-clone.
  Warning: "branch" parameter with value "mybranch" is ignored. Checking out to revision 2566127...
  Cloning into '/sample-buildout/parts/git-clone'...
  HEAD is now at 2566127 ...

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
322
  >>> print(system('git branch'))
323 324
  * master

325
Setup a "develop" repository
Kazuhiko Shiozaki committed
326
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
327

Ayush Tiwari committed
328
If you need to setup a repository that will be manually altered over time for
329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347
development purposes, you need to make sure buildout will NOT alter it and NOT
erase your local modifications by specifying the "develop" flag::

  [buildout]
  parts = git-clone

  [git-clone]
  recipe = slapos.recipe.build:gitclone
  repository = https://example.net/example.git/
  develop = true

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
348
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
349 350
  ... develop = true
  ... """)
Antoine Catton committed
351

Bryton Lacquement committed
352
  >>> print(system(buildout))
353 354 355 356 357 358 359 360 361
  Uninstalling git-clone.
  Running uninstall recipe.
  Installing git-clone.
  Cloning into '/sample-buildout/parts/git-clone'...

Buildout will then keep local modifications, instead of resetting the
repository::

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
362
  >>> print(system('echo foo > setup.py'))
363 364

  >>> cd(sample_buildout)
Bryton Lacquement committed
365
  >>> print(system(buildout))
366
  Updating git-clone.
Rafael Monnerat committed
367 368 369
  ...
  <BLANKLINE>

370 371

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
372
  >>> print(system('cat setup.py'))
373 374
  foo

Cédric de Saint Martin committed
375 376 377
Then, when update occurs, nothing is done::

  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
378
  >>> print(system('echo kept > local_change'))
Cédric de Saint Martin committed
379

Bryton Lacquement committed
380
  >>> print(system('git remote add broken http://git.erp5.org/repos/nowhere'))
Cédric de Saint Martin committed
381 382 383
  ...

  >>> cd(sample_buildout)
Bryton Lacquement committed
384
  >>> print(system(buildout))
Cédric de Saint Martin committed
385
  Updating git-clone.
Rafael Monnerat committed
386
  ...
Ayush Tiwari committed
387

Cédric de Saint Martin committed
388
  >>> cd(sample_buildout, 'parts', 'git-clone')
Bryton Lacquement committed
389
  >>> print(system('cat local_change'))
Cédric de Saint Martin committed
390 391
  kept

392 393 394 395 396 397 398 399 400 401
In case of uninstall, buildout will keep the repository directory::

  >>> cd(sample_buildout)
  >>> write(sample_buildout, 'buildout.cfg',
  ... """
  ... [buildout]
  ... parts = git-clone
  ...
  ... [git-clone]
  ... recipe = slapos.recipe.build:gitclone
Ayush Tiwari committed
402
  ... repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
403 404 405 406 407
  ... develop = true
  ... # Triggers uninstall/install because of section signature change
  ... foo = bar
  ... """)

Bryton Lacquement committed
408
  >>> print(system(buildout))
409 410 411 412 413
  Uninstalling git-clone.
  Running uninstall recipe.
  You have uncommited changes in /sample-buildout/parts/git-clone. This folder will be left as is.
  Installing git-clone.
  destination directory already exists.
Rafael Monnerat committed
414 415
  ...
  <BLANKLINE>
Antoine Catton committed
416

Cédric de Saint Martin committed
417 418
Specific git binary
~~~~~~~~~~~~~~~~~~~
Jérome Perrin committed
419

Cédric de Saint Martin committed
420 421 422
The default git command is `git`, if for a any reason you don't
have git in your path, you can specify git binary path with `git-command`
option.
Jérome Perrin committed
423

Benjamin Blanc committed
424
Ignore SSL certificate
Kazuhiko Shiozaki committed
425
~~~~~~~~~~~~~~~~~~~~~~
Benjamin Blanc committed
426 427 428 429 430 431 432 433 434 435 436 437 438 439

By default, when remote server use SSL protocol git checks if the SSL
certificate of the remote server is valid before executing commands.
You can force git to ignore this check using `ignore-ssl-certificate`
boolean option::

  [buildout]
  parts = git-clone

  [git-clone]
  recipe = slapos.recipe.build:gitclone
  repository = https://example.net/example.git/
  ignore-ssl-certificate = true

Ayush Tiwari committed
440 441 442 443 444 445 446 447 448 449 450 451 452 453 454
Ignore cloning submodules
~~~~~~~~~~~~~~~~~~~~~~~~~

By default, cloning the repository will clone its submodules also. You can force
git to ignore cloning submodules by defining `ignore-cloning-submodules` boolean
option to 'true'::

  [buildout]
  parts = git-clone

  [git-clone]
  recipe = slapos.recipe.build:gitclone
  repository = https://lab.nexedi.com/tiwariayush/test_erp5
  ignore-cloning-submodules = true

Julien Muchembled committed
455 456 457 458 459 460 461 462 463 464 465 466
Other options
~~~~~~~~~~~~~

shared
    Clone with ``--shared`` option if true. See ``git-clone`` command.

sparse-checkout
    The value of the `sparse-checkout` option is written to the
    ``$GITDIR/info/sparse-checkout`` file, which is used to populate the working
    directory sparsely. See the `SPARSE CHECKOUT` section of ``git-read-tree``
    command. This feature is disabled if the value is empty or unset.

Antoine Catton committed
467
Full example
Kazuhiko Shiozaki committed
468
~~~~~~~~~~~~
Antoine Catton committed
469 470 471 472 473 474 475 476 477 478 479 480

::

  [buildout]
  parts = git-clone

  [git-binary]
  recipe = hexagonit.recipe.cmmi
  url = http://git-core.googlecode.com/files/git-1.7.12.tar.gz

  [git-clone]
  recipe = slapos.recipe.build:gitclone
481
  repository = http://example.net/example.git/
Antoine Catton committed
482
  git-command = ${git-binary:location}/bin/git
483
  revision = 0123456789abcdef
Antoine Catton committed
484

485

Kazuhiko Shiozaki committed
486 487 488
=========================
 slapos.recipe.build:npm
=========================
Cédric de Saint Martin committed
489 490 491 492

Downloads and installs node.js packages using Node Package Manager (NPM).

Examples
Kazuhiko Shiozaki committed
493
--------
Cédric de Saint Martin committed
494 495

Basic example
Kazuhiko Shiozaki committed
496
~~~~~~~~~~~~~
Cédric de Saint Martin committed
497 498 499 500 501 502 503 504 505 506 507

Here is example to install one or several modules::

  [buildout]
  parts = node-package

  [node-package]
  recipe = slapos.recipe.build:npm
  modules =
    colors
    express
508

Cédric de Saint Martin committed
509 510 511 512 513 514
  # Optional argument specifying perl buildout part, if existing.
  # If specified, recipe will use the perl installed by buildout.
  # If not specified, will take the globally available perl executable.
  node = node-0.6

Specific version
Kazuhiko Shiozaki committed
515
~~~~~~~~~~~~~~~~
Cédric de Saint Martin committed
516
::
Cédric de Saint Martin committed
517 518 519 520 521

  [buildout]
  parts = node-package

  [node-package]
Kazuhiko Shiozaki committed
522
  recipe = slapos.recipe.build:npm
Cédric de Saint Martin committed
523
  modules =
Cédric de Saint Martin committed
524
    express@1.0.2
Cédric de Saint Martin committed
525
  node = node-0.6
Julien Muchembled committed
526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594

==========================
 slapos.recipe.build:vm.*
==========================

This is a set of recipes to build Virtual Machine images and execute commands
inside them. They rely on QEMU and OpenSSH: executables are found via the PATH
environment variable. They do nothing on update.

Common options
--------------

location
    Folder where the recipe stores any produced file.
    Default: ${buildout:parts-directory}/<section_name>

environment
    Extra environment for the spawn executables. It can either be the name of a
    section or a list of variables (1 per line, in the form ``key=value``).
    Values are expanded with current environment using Python %-dict formatting.

mem
    Python expression evaluating to an integer that specifies the
    RAM size in MB for the VM.

smp
    Number of CPUs for the VM. Default: 1

Example
~~~~~~~

::

  [vm-run-environment]
  PATH = ${openssh:location}/bin:${qemu:location}/bin:%(PATH)s

  [vm-run-base]
  recipe = slapos.recipe.build:vm.run
  environment = vm-run-environment
  mem = 256 * (${:smp} + 1)
  smp = 4

slapos.recipe.build:vm.install-debian
-------------------------------------

Install Debian from an ISO image. Additional required binaries:

- ``7z`` (from 7zip), to extract kernel/initrd from the ISO;
- ``file``, which is used to test that the VM image is bootable.

Currently, it only produces `raw` images, in `discard` mode (see ``-drive``
QEMU option): combined the use of ``discard`` mount option, this minimizes
the used space on disk.

Options
~~~~~~~

location
    Produced files: ``<dist>.img`` (1 for each token of `dists`), ``passwd``
    and optionally ``ssh.key``

arch
    QEMU architecture (the recipe runs the ``qemu-system-<arch>`` executable).
    It is also used to select the ISO in the sections refered by `dists`.
    Default to host architecture.

dists
    List of VMs to build: each token refers to a buildout section name that
    describes the ISOs to use. See `ISO sections`_ below.
Julien Muchembled committed
595
    Tokens can't contain `'.'` characters.
Julien Muchembled committed
596 597 598 599 600 601 602 603

size
    Size of the VM image. This must be an integer, optionally followed by a
    IEC or SI suffix.

mem
    Default: 256

Julien Muchembled committed
604
[<dist>/]preseed.<preseed>
Julien Muchembled committed
605 606
    Set the <preseed> value for the installation. The recipe has many default
    preseed values: you can see the list in the ``InstallDebianRecipe.preseed``
Julien Muchembled committed
607 608 609 610
    class attribute (file ``slapos/recipe/vm.py``). Aliases are recognized
    (but the recipe includes a mapping that may be out-of-date.).
    Any value except ``passwd/*`` can optionally be prefixed so that they only
    apply for a particular VM.
Julien Muchembled committed
611

Julien Muchembled committed
612
[<dist>/]debconf.<owner>
Julien Muchembled committed
613 614
    List of debconf value for <owner> (usually a package name),
    each line with 2 whitespace-separated parts: <key> <value>.
Julien Muchembled committed
615
    Like for preseed.* values, they can be specific to <dist>.
Julien Muchembled committed
616 617 618 619

late-command
    Shell commands to execute at the end of the installation. They are run
    inside the target system. This is a reliable alternative to the
Julien Muchembled committed
620 621
    ``preseed.preseed/late_command`` option. The ``DIST`` shell variable is
    set to the VM being built.
Julien Muchembled committed
622 623

packages
Julien Muchembled committed
624
    Extra packages to install.
Julien Muchembled committed
625
    Like for `late-command`, do not use ``preseed.pkgsel/include``.
Julien Muchembled committed
626 627 628
    If you want to install packages only for some specific <dist>, you can do
    it in ``late-command``, by testing ``$DIST`` and using
    ``apt-get install -y``.
Julien Muchembled committed
629

Julien Muchembled committed
630 631 632 633 634 635 636 637
vm.run
    Boolean value that is `true` by default, to configure the VM for use with
    the `slapos.recipe.build:vm.run`_ recipe:

    - make sure that the `ssh` and `sudo` packages are installed
    - an SSH key is automatically created with ``ssh-keygen``, and it can be
      used to connect as `root`

Julien Muchembled committed
638 639 640 641 642 643 644 645 646 647 648 649 650 651
ISO sections
~~~~~~~~~~~~

<arch>.iso
    Name of the section that provides the ISO image, for example by downloading
    it. This section must define 2 options: `location` is the folder
    containing the ISO, and `filename` is the file name of the ISO.

<arch>.kernel
    Path to kernel image inside the ISO.

<arch>.initrd
    Path to initrd image inside the ISO.

Julien Muchembled committed
652 653 654
User setup
~~~~~~~~~~

Julien Muchembled committed
655 656
By default, there's no normal user created. Another rule is that a random
password is automatically generated if there is no password specified.
Julien Muchembled committed
657

Julien Muchembled committed
658
You have nothing to do if you only plan to use the VM with `vm.run`.
Julien Muchembled committed
659 660 661 662 663 664

For more information about the ``passwd/*`` preseed values, you can look at
the ``user-setup-udeb`` package at
https://anonscm.debian.org/cgit/d-i/user-setup.git/tree/
and in particular the ``user-setup-ask`` and ``user-setup-apply`` scripts.

Julien Muchembled committed
665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687
Example
~~~~~~~

::

  [vm-install-environment]
  # vm-run-environment refers to the section in common options
  PATH = ${file:location}/bin:${p7zip:location}/bin:${vm-run-environment:PATH}

  [vm-debian]
  recipe = slapos.recipe.build:vm.install-debian
  environment = vm-install-environment
  dists = debian-jessie debian-stretch
  size = 2Gi
  late-command =
  # rdnssd causes too much trouble with QEMU 2.7, because the latter acts as
  # a DNS proxy on both IPv4 and IPv6 without translating queries to what the
  # host supports.
    dpkg -P rdnssd
  debconf.debconf =
    debconf/frontend noninteractive
    debconf/priority critical
  # minimal size
Julien Muchembled committed
688
  preseed.apt-setup/enable-source-repositories = false
Julien Muchembled committed
689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711
  preseed.recommends = false
  preseed.tasks =

  [debian-jessie]
  x86_64.iso = debian-amd64-netinst.iso
  x86_64.kernel = install.amd/vmlinuz
  x86_64.initrd = install.amd/initrd.gz

  [debian-stretch]
  <= debian-jessie
  x86_64.iso = debian-amd64-testing-netinst.iso

  [debian-amd64-netinst.iso]
  ...

slapos.recipe.build:vm.run
--------------------------

Execute shell commands inside a VM, in snapshot mode (the VM image is not
modified).

``${buildout:directory}`` is always mounted as `/mnt/buildout` inside the VM.

Julien Muchembled committed
712 713 714 715 716 717
Mount points use the 9p file-system. Make sure that:

- QEMU is built with --enable-virtfs;
- the VM runs a kernel that is recent enough (Debian Squeeze kernel 2.6.32 is
  known to fail, and you'd have to use the one from squeeze-backports).

Julien Muchembled committed
718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745
Options
~~~~~~~

location
    Folder where to store any produce file. Inside the guest, it is pointed to
    by the PARTDIR environment variable. It is also used as temporary storage
    for changes to the VM image.

vm
    Folder containing the VM images and the `ssh.key`` file. See the `location`
    option of the `vm.install-*` recipes.

dist
    VM image to use inside the `vm` folder.

commands
    List of <command> options, each one being a shell script to execute via
    SSH. They are processed in sequence. This is usually only required if you
    want to reboot the VM. Default: command

mount.<name>
    Extra mount point. The value is a host folder that is mounted as
    ``/mnt/<name>``.

stop-ssh
    Tell `reboot` function how to stop SSH (see Helpers_).
    Default: systemctl stop ssh

Julien Muchembled committed
746
user
Julien Muchembled committed
747 748 749 750 751 752 753 754 755
    Execute commands with this user. The value can be ``root``. By default,
    it is empty and it means that:

    - a ``slapos`` user is created with the same uid/gid than the user using
      this recipe on the host, which can help accessing mount points;
    - sudo must be installed and the created user is allowed to become root
      without password.

    In any case, SSH connects as root.
Julien Muchembled committed
756

Julien Muchembled committed
757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794
wait-ssh
    Time to wait for (re)boot. The recipe fails if it can't connect to the SSH
    server after this number of seconds. Default: 60

Helpers
~~~~~~~

Before commands are executed, all `mount.<name>` are mounted
and a few helpers are set to make scripting easier.

set -e
    This is done before anything else, to make buildout abort if any untested
    command fails.

reboot
    Function to safely reboot the guest. The next command in `commands` will be
    executed once the SSH server is back.

map <host_path>
    Function to map a folder inside ``${buildout:directory}``.

PARTDIR
    Folder where to store any produced file. Inside the guest, it actually
    maps to `location` on the host. This is useful because you can't write
    ``PARTDIR=`map ${:location}``` if you don't explicitly set `location`.

Example
~~~~~~~

::

  [vm-run-base]
  # extends above example in common options
  vm = ${vm-debian:location}
  dist = debian-jessie

  [vm-debian]
  # extends above example in vm.install-debian
Julien Muchembled committed
795
  packages += build-essential devscripts equivs git
Julien Muchembled committed
796 797 798 799 800 801 802 803 804 805 806

  [userhosts-repository]
  recipe = slapos.recipe.build:gitclone
  repository = https://lab.nexedi.com/nexedi/userhosts.git
  # we don't need a working directory on the host
  sparse-checkout = /.gitignore

  [build-userhosts-map]
  <= vm-run-base
  repository = `map ${userhosts-repository:location}`
  command =
Julien Muchembled committed
807 808 809
    git clone -s ${:repository} userhosts
    cd userhosts
    mk-build-deps -irs sudo -t 'apt-get -y'
Julien Muchembled committed
810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830
    dpkg-buildpackage -uc -b -jauto
    cd ..
    mv *.changes *.deb $PARTDIR

  # Alternate way, which is required if [userhosts-repository] is extended
  # in such way that the repository is outside ${buildout:directory}.
  [build-userhosts-mount]
  <= build-userhosts-map
  mount.userhosts = ${userhosts-repository:location}
  repository = /mnt/userhosts

  [test-reboot]
  <= vm-run-base
  commands = hello world
  hello =
    uptime -s
    echo Hello ...
    reboot
  world =
    uptime -s
    echo ... world!