KVM instantiation recipe.

slapos/slapos.recipe.kvm/CHANGES.txt

+Changelog
+=========
+1.2 (unreleased)
+----------------
+
+1.0.14 (2011-02-01)
+-------------------
+
+- Remove cpu parameter which is not supported
+
+1.0.13 (2011-02-01)
+-------------------
+
+- KVM size parameters are provided by the instance profile
+
+1.0.12 (2011-02-01)
+-------------------
+
+- Generate random mac address
+
+1.0.11 (2011-01-18)
+-------------------
+
+- Updated recipe to the new BaseSlapRecipe.py
+  [Davide Tammaro]
+
+1.0.10 (2011-01-10)
+-------------------
+
+- Remove KVM dhcp server
+
+1.0.9 (2010-12-28)
+------------------
+
+- Wait the QMP command before starting
+
+1.0.8 (2010-12-28)
+------------------
+
+- Wait for KVM to be started
+
+1.0.7 (2010-11-30)
+------------------
+
+  - Add controller template
+
+1.0.6 (2010-11-29)
+------------------
+
+  - Protect vnc session with a password
+
+1.0.5 (2010-11-24)
+------------------
+
+  - Write on disk image
+  - Add bios menu
+  - Create a disk image detected as virtual disk
+  - Use latest slapos.librecipe API
+  - Use nbd to access the cdrom image
+
+1.0.3 (2010-10-26)
+------------------
+  - Add MANIFEST.in
+  - Remove ssh feature, with is not compatible with the slapos design
+  - Simplify runner generation
+  - Adding python gzip version instead of linux gzip one
+
+1.0.2 (2010-10-25)
+------------------
+- Slapify kvm installation [Charles Villette]
+  - Adding tests to configuration options
+  - Adding memory limit support
+  - Adding configuration wrapper
+  - Adding new configuration options for kvm __init__
+  - Adding non-hardcoded qemu path for kvm launch
+  - Changing names to slapos.recipe instead of kvm.recipe
+
+1.0.1 (2010-05-10)
+-------------------
+
+- Reimplemented ssh.py to use SSHClient.
+  [Rafael Monnerat]
+
+1.0 (2010-05-04)
+----------------
+
+- Initial Release
+  [Rafael Monnerat]

slapos/slapos.recipe.kvm/MANIFEST.in

+include CHANGES.txt
+recursive-include src/slapos/recipe/kvm *.in

slapos/slapos.recipe.kvm/README.txt

+Introduction
+============
+
+The erp5.recipe.kvm aims to integrate KVM setups and buildout. This recipe is 
+able to download one remote image and setup a KVM environment to use it. 
+
+This recipe is also capable to reuse images or partitions already present on
+disk to create the setup. 
+
+Examples
+========
+
+The follow examples lists different kind of configurations.
+
+
+KVM with Remote and gzipped Image
+----------------------------------
+
+    [kvm-testing-with-remote-gzip-image]
+    image = http://URL/public.mdv2010.0_x86_64.qcow2.img.gz
+
+    # md5 checks are optional
+    md5_download = adcff8adcff8adcff8adcff8
+    md5_image = 1a4e371a4e371a4e371a4e371a4e37
+
+    gzip = true
+    
+    # Use -hda instead -drive arg
+    # Default is drive (see Options bellow)
+    image_type = hda
+
+    ### Common Configuration bellow. ###
+    
+    # VNC is optional
+    kvm_vnc = <SOME-IP>:<VNC-DISPLAY>
+
+    # Graphic is optional
+    kvm_graphic = std
+
+    
+    # Define list of redirections.
+    kvm_redir =
+      tcp:13480::80
+    
+    kvm_net =
+      user,hostname=publicmandriva
+      nic,model=ne2k_pci
+
+    # This automatically create a redirection for 13456 -> 22
+    ssh_port = 13456
+    ssh_key_path = /path/to/mykey.key
+
+KVM with Remote and raw Image
+----------------------------------
+
+    [kvm-testing-with-remote-raw-image]
+    image = http://URL/public.mdv2010.0_x86_64.qcow2.img
+    md5_download = 1a4e371a4e371a4e371a4e371a4e37
+    md5_image = 1a4e371a4e371a4e371a4e371a4e37
+
+    gzip = false
+
+    ### The Rest Same configuration as previous ###
+
+KVM with direct local Image file
+----------------------------------
+
+This does not copy and/or download the image.
+
+    [kvm-testing-with-local-image]
+
+    file = /home/you/public.mdv2010.0_x86_64.qcow2.img
+    md5_image = 1a4e371a4e371a4e371a4e371a4e37
+
+    ### The Rest Same configuration as previous ###
+
+KVM with a linux partition
+----------------------------------
+
+This does not copy and/or download the image.
+
+    [kvm-testing-a-linux-partition]
+
+    file = /dev/sdb
+
+    ### The Rest Same configuration as previous ###
+
+
+Options
+=======
+
+
+location
+
+  When define, it does not use buildout parts directory to allocate the image.
+
+image
+
+  URL to a remote image. 
+
+file 
+
+  Use file makes recipe ignore image option. You can define a path to a image
+  or partion to use. 
+
+image_type 
+
+  You can define how the KVM will use the image with "-hdx" or "-drive". By
+  default it uses drive and the result is:
+
+    "kvm -drive file=IMAGE,if=virtio,cache=none,boot=on ..."
+
+  if you use image_type=hda:
+     
+    "kvm -hda IMAGE ..." 
+
+gzip
+
+  If true means the image is compressed (gzipped), and the recipe will
+  uncompress the image before copy it.
+
+md5_download
+
+  When defined, this values is used to verify the file downloaded.
+
+md5_image
+
+  When defined, this values is used to verify the image generated, don't use it
+  when a partition is used in file parameter.
+
+kvm_vnc
+
+  Define the ip-address:display used by KVM to start the VNC server. If not
+  defined, no VNC port is created.
+
+kvm_redir
+
+  Define port redirections to the buildout.
+
+kvm_graphic
+
+  If defined it adds the "-vga value" at the KVM command.
+
+kvm_net
+
+  Define the net definitions, each value defines one "-net" in kvm command.
+  Example:
+  
+    kvm_net =
+      user,hostname=publicmandriva
+      nic,model=ne2k_pci
+
+  It generates:
+
+    "kvm -net user,hostname=publicmandriva -net  nic,model=ne2k_pci ..."
+
+kvm_snapshot 
+
+  Use "-snapshot" when run a KVM. This not write the changes direct into the
+  image. Default value is False.
+
+ssh_port
+
+  If defined creates a new redirection for port 22 and creates few script to
+  connect to the instance.
+
+ssh_hostname
+
+  By default it uses localhost. You don't need to define this.
+
+ssh_key_path
+
+  Path to the ssh key used to connect without password to the image running.
+  
+ssh_user
+
+  Define the server that will be used to connect to the instance. 
+
+kvm_bin_directory
+
+  Place where the scripts will be created. By default it uses bin-directory from
+  buildout.
+
+kvm_run_directory
+
+   Place where the pid file will be created, by default it uses var-directory
+   from buildout.
+
+
+Generated Commands
+====================
+
+Few scripts are generated to you manage your KVM instance. The scripts names are
+created with the followed standard:
+
+   KVM-PARTS-NAME-ctl
+
+Commands usage
+---------------
+   
+KVM-PARTS-NAME-ctl (start|stop|status|restart)
+  
+  This script is used to manage the KVM instance.
+
+
+KVM-PARTS-NAME-sendfile REMOTEFILE LOCALFILE
+
+  Copy the local file to a remote place.
+
+KVM-PARTS-NAME-getfile REMOTEFILE LOCALFILE
+
+  Copy the remote file to a local place.
+
+KVM-PARTS-NAME-runscript COMMAND
+
+  Run a command into remote KVM computer.

slapos/slapos.recipe.kvm/setup.py

+from setuptools import setup, find_packages
+
+name = "slapos.recipe.kvm"
+version = '1.2-dev'
+
+def read(name):
+  return open(name).read()
+
+long_description=( read('README.txt')
+                   + '\n' +
+                   read('CHANGES.txt')
+                 )
+
+setup(
+    name = name,
+    version = version,
+    description = "ZC Buildout recipe for create a kvm setup for a image",
+    long_description=long_description,
+    license = "GPLv3",
+    keywords = "buildout kvm",
+    classifiers=[
+        "Framework :: Buildout :: Recipe",
+        "Programming Language :: Python",
+    ],
+    packages = find_packages('src'),
+    package_dir = {'': 'src'},
+    include_package_data=True,
+    install_requires = [
+      'zc.recipe.egg',
+      'setuptools',
+      'slapos.lib.recipe',
+      ],
+    namespace_packages = ['slapos', 'slapos.recipe'],
+    entry_points = {'zc.buildout': ['default = %s:Recipe' % name]},
+    )

slapos/slapos.recipe.kvm/src/slapos/__init__.py

+# See http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages
+try:
+    __import__('pkg_resources').declare_namespace(__name__)
+except ImportError:
+    from pkgutil import extend_path
+    __path__ = extend_path(__path__, __name__) 

slapos/slapos.recipe.kvm/src/slapos/recipe/__init__.py

+# See http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages
+try:
+    __import__('pkg_resources').declare_namespace(__name__)
+except ImportError:
+    from pkgutil import extend_path
+    __path__ = extend_path(__path__, __name__) 

slapos/slapos.recipe.kvm/src/slapos/recipe/kvm/__init__.py

+##############################################################################
+#
+# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
+#
+# WARNING: This program as such is intended to be used by professional
+# programmers who take the whole responsibility of assessing all potential
+# consequences resulting from its eventual inadequacies and bugs
+# End users who are looking for a ready-to-use solution with commercial
+# guarantees and support are strongly adviced to contract a Free Software
+# Service Company
+#
+# This program is Free Software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+#
+##############################################################################
+import os
+import sys
+from slapos.lib.recipe.BaseSlapRecipe import BaseSlapRecipe
+import subprocess
+import binascii
+import random
+
+import pkg_resources
+
+
+class Recipe(BaseSlapRecipe):
+
+  def _install(self):
+
+    #Get the IP list
+    connection_found = False
+    ip = self.getGlobalIPv6Address()
+    for tap, dummy in self.parameter_dict['ip_list']:
+      # Get an ip associated to a tap interface
+      if tap:
+        connection_found = True
+    if not connection_found:
+      raise NotImplementedError("Do not support ip without tap interface")
+
+    # Disk path
+    disk_path = os.path.join(self.data_root_directory, 'virtual.qcow2')
+    socket_path = os.path.join(self.var_directory, 'qmp_socket')
+    # XXX Weak password
+    vnc_passwd = binascii.hexlify(os.urandom(4))
+
+    #XXX pid_file path, database_path and xml path
+    pid_file_path = os.path.join(self.run_directory, 'pid_file')
+    database_path = os.path.join(self.data_root_directory, 'slapmonitor_database')
+    #xml_path = os.path.join(self.var_directory, 'slapreport.xml' )
+
+    # Create disk if needed
+    if not os.path.exists(disk_path):
+      retcode = subprocess.call(["%s create -f qcow2 %s %iG" % (
+          self.options['qemu_img_path'], disk_path,
+          int(self.options['disk_size']))], shell=True)
+      if retcode != 0:
+        raise OSError, "Disk creation failed!"
+
+    # Instanciate KVM
+    kvm_config = {}
+    # Options nbd_ip and nbd_port are provided by slapos master
+    kvm_config.update(self.options)
+    #raise NotImplementedError("%s" % self.parameter_dict)
+    kvm_config['vnc_ip'] = ip
+    kvm_config['tap_interface'] = tap
+    kvm_config['nbd_ip'] = self.parameter_dict['nbd_ip']
+    kvm_config['nbd_port'] = self.parameter_dict['nbd_port']
+    #XXX
+    kvm_config['pid_file'] = pid_file_path 
+    kvm_config['image'] = disk_path
+    # First octet has to represent a locally administered address
+    octet_list = [254] + [random.randint(0x00, 0xff) for x in range(5)]
+    kvm_config['mac_address'] = ':'.join(['%02x' % x for x in octet_list])
+    kvm_config['qmp_socket'] = socket_path
+    kvm_config['hostname'] = "slaposkvm"
+
+    kvm_wrapper_template_location = pkg_resources.resource_filename(
+                                             __name__, os.path.join(
+                                             'template', 'kvm_run.in'))
+    kvm_runner_path = self.createRunningWrapper("kvm",
+          self.substituteTemplate(kvm_wrapper_template_location, kvm_config))
+
+
+    # Instanciate KVM controller
+    controller_config = {}
+    # Options nbd_ip and nbd_port are provided by slapos master
+    controller_config.update(self.options)
+    controller_config['qmp_socket'] = socket_path
+    controller_config['vnc_passwd'] = vnc_passwd
+    controller_config['python_path'] = sys.executable
+
+    controller_wrapper_template_location = pkg_resources.resource_filename(
+                                             __name__, os.path.join(
+                                             'template', 'kvm_controller_run.in'))
+    controller_runner_path = self.createRunningWrapper("kvm_controller",
+          self.substituteTemplate(controller_wrapper_template_location, controller_config))
+
+    #XXX Instanciate Slapmonitor
+    slapmonitor_config={}
+    slapmonitor_config.update(self.options)
+    slapmonitor_config['database_path'] = database_path 
+    slapmonitor_config['pid_file'] = pid_file_path
+    slapmonitor_config['python_path'] = sys.executable
+    slapmonitor_wrapper_template_location = pkg_resources.resource_filename(
+                                             __name__, os.path.join(
+                                             'template', 'slapmonitor_run.in'))
+    slapmonitor_runner_path = self.createRunningWrapper("slapmonitor",
+          self.substituteTemplate(slapmonitor_wrapper_template_location, slapmonitor_config))
+
+
+    #XXX Instanciate Slapreport
+    slapreport_config={}
+    slapreport_config.update(self.options)
+    slapreport_config['database_path'] = database_path 
+    slapreport_config['python_path'] = sys.executable
+    slapreport_wrapper_template_location = pkg_resources.resource_filename(
+                                             __name__, os.path.join(
+                                             'template', 'slapreport_run.in'))
+    slapreport_runner_path = self.createReportRunningWrapper(self.substituteTemplate(
+                      slapreport_wrapper_template_location, slapreport_config))
+    
+
+
+
+    self.computer_partition.setConnectionDict(dict(
+      vnc_connection_string="vnc://[%s]:1" % ip,
+      vnc_password=vnc_passwd,
+    ))
+
+    return [kvm_runner_path, controller_runner_path]
+

slapos/slapos.recipe.kvm/src/slapos/recipe/kvm/template/kvm_controller_run.in

+#!%(python_path)s
+# BEWARE: This file is operated by slapgrid
+# BEWARE: It will be overwritten automatically
+
+# Echo client program
+import socket
+import time
+
+# Connect to KVM qmp socket
+so = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+connected = False
+while not connected:
+  try:
+    so.connect('%(qmp_socket)s')
+  except socket.error:
+    time.sleep(1)
+  else:
+    connected = True
+data = so.recv(1024)
+
+# Enable qmp
+so.send('{ "execute": "qmp_capabilities" }')
+data = so.recv(1024)
+
+# Set VNC password
+so.send('{ "execute": "change", ' \
+        '"arguments": { "device": "vnc", "target": "password", ' \
+        '               "arg": "%(vnc_passwd)s" } }')
+data = so.recv(1024)
+
+# Finish
+so.close()

slapos/slapos.recipe.kvm/src/slapos/recipe/kvm/template/kvm_run.in

+#!/bin/sh
+# BEWARE: This file is operated by slapgrid
+# BEWARE: It will be overwritten automatically
+
+# TODO: -net nic,model=virtio, but OS installer has to provide the virtio_net
+# module
+exec %(qemu_path)s \
+  -net nic,macaddr=%(mac_address)s \
+  -net tap,ifname=%(tap_interface)s,script=no,downscript=no \
+  -smp %(smp_count)s \
+  -m %(ram_size)s \
+  -cdrom nbd:[%(nbd_ip)s]:%(nbd_port)s \
+  -drive file=%(image)s,if=virtio,boot=on \
+  -vnc [%(vnc_ip)s]:1,ipv6,tls,password \
+  -boot menu=on \
+  -qmp unix:%(qmp_socket)s,server \
+  -pidfile %(pid_file)s

slapos/slapos.recipe.kvm/src/slapos/recipe/kvm/template/slapmonitor_run.in

+#!/bin/sh
+# BEWARE: This file is operated by slapgrid
+# BEWARE: It will be overwritten automatically
+exec %(python_path)s %(slapmonitor_path)s %(pid_file)s %(database_path)s

slapos/slapos.recipe.kvm/src/slapos/recipe/kvm/template/slapreport_run.in

+#!/bin/sh
+# BEWARE: This file is operated by slapgrid
+# BEWARE: It will be overwritten automatically
+exec %(python_path)s %(slapreport_path)s $1 %(database_path)s