Merge Antoine Catton work of mariadb backup.

Squashed commit of the following:

commit e55a243673fdd5bccb281b070f0e97832bba696d
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 14:41:57 2011 +0200

    Minor: removed unused variable "returncode"

commit 5a863d2e034782e2f58bbab01a0eb57d581a0b30
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 14:40:00 2011 +0200

    Use subprocess.check_call instead of checking return code of
    subprocess.call()

commit 91fc5e8d4893eec41cc651ef2510c4feaab85eee
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 14:33:24 2011 +0200

    Minor: removing useless commented code in davstorage software.cfg

commit 753fc3084429aefe9381f2e0f438c4303cadf6e4
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 14:27:48 2011 +0200

    Minor: removing rebootstrap of the software release

    Another new python for mariadb is useless.

commit 62b3d3959834921005e7b23f1588644448e6d4d5
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 14:23:52 2011 +0200

    Partially revert 3728936e6a1b2728bcdefc70298bb4adb69a777e in order to
    clean the git history.

commit 05a458a71311298b2620cebb3122e6a8fe54c622
Merge: 87f61f7 a33be315
Author: Łukasz Nowak <luke@nexedi.com>
Date:   Mon Sep 19 13:35:17 2011 +0200

    Merge branch 'master' into mariadb-backup

commit 87f61f793058b9a2e5584b31691c47bc720bcca8
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 13:28:33 2011 +0200

    Minor: removing trailing spaces

commit 7bf410ecb9e3f1966c579a327ed4d5df80193046
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 13:08:53 2011 +0200

    Minor: Adding import-script.sh I forgot to commit

commit 7435626070c300c4e4e579c632ccf4700dc7aab8
Merge: 0510486 13d6e1d3
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 19 11:55:33 2011 +0200

    Merge branch 'master' into mariadb-backup

    Conflicts:
    	slapos/recipe/mysql/__init__.py

commit 05104863cd426d4977ceaed6d2509764dd46b4bd
Merge: 1b43ef4 4299234e
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Sep 8 17:24:44 2011 +0200

    Merge branch 'master' into mariadb-backup

commit 1b43ef420bc23be6cec791b690ad38b1468cbcc2
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Sep 8 15:56:35 2011 +0200

    Trim the branch to just add davstorage backup

commit 9f61de06d0da2c42281703409b55e94005125d3c
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Sep 8 15:37:43 2011 +0200

    Switch MySQL Backup to duplicity with webdav remote server

commit 2fa9ae925a4629424bd728ec059ea59e9c2a63be
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Sep 8 15:34:40 2011 +0200

    Minor: fix the buildout of duplicity to work

commit 51e512e5ecfc9e23823e8ba02f168eb34ac7fca4
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Sep 8 14:11:07 2011 +0200

    Adding duplicity software release

commit 094abfd41ba87f340c52dfd540f09b8f5b2fbb2c
Merge: f59c5c5 150c49e
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 5 14:04:03 2011 +0200

    Merge branch 'webdav' into mariadb-failover

commit 150c49e3cd12e3b8f935c864ccc292c082049289
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Sep 5 13:15:40 2011 +0200

    Working webdav storage

commit 159054aec04f13ed0e5687a8466bd4803a8c1350
Author: Antoine Catton <acatton@tiolive.com>
Date:   Fri Sep 2 17:25:08 2011 +0200

    Link with the right dbm

commit 7bd8c5ad00c53a7830dd45093656907978d26c21
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Sep 1 09:35:31 2011 +0200

    Enable dav-fs module on apache compilation

commit f59c5c549360ac8d5e16b3c510305502a41880f6
Author: Antoine Catton <acatton@tiolive.com>
Date:   Wed Aug 31 15:13:11 2011 +0200

    Adding pull_backup_server on mariadb

commit 3728936e6a1b2728bcdefc70298bb4adb69a777e
Author: Antoine Catton <acatton@tiolive.com>
Date:   Wed Aug 31 15:12:35 2011 +0200

    Switch mariadb to dropbear

commit dbe4efeda490d8f201d39ba1d1f2cac2e69c38f3
Author: Antoine Catton <acatton@tiolive.com>
Date:   Wed Aug 31 15:03:09 2011 +0200

    Linking dropbear with slapos zlib

commit f6c9b477a9ba4fd67ccc0b19e2a1a8adfa767b11
Merge: bb80ae9 1a85f89
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 30 14:05:24 2011 +0200

    Merging from pull-backup-server branch

commit bb80ae9bd341cfdf2663f7cff85b2a2cffc2cb32
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 30 13:58:41 2011 +0200

    Switch from OpenSSH client to Dropbear client

commit d0cf9dfc9f34f7ab3bc747018bcb47f10afa8170
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 30 13:57:21 2011 +0200

    Minor: Dropbear buildout fine tuning

commit 5f50ae5c79cce438ee0eabbede9590c797156dad
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 30 09:08:41 2011 +0200

    Add ipv6 patch for dropbear

    Took here :
    <https://lists.openwrt.org/pipermail/openwrt-devel/2009-May/004299.html>

commit 80a2e24a8c6c54bd04d22b1b96710faa1c354bc2
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Aug 29 17:54:36 2011 +0200

    Updating the pull_backup_server buildout

commit 8736902307cde71ae6f95e257b3e89fef7d721db
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Aug 29 17:51:19 2011 +0200

    Replacing openssh buildout by dropbear one.

    Adding my own dropbear patch which was submitted to
    dropbear developpers.

commit 1a85f89df53a6c770371935f24448d5c12c1f75c
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Aug 25 11:54:11 2011 +0200

    Minor: use internal writeFile function instead open(..., 'w')

commit cc1be9293072f9b95772dcc666cbbc2bf77b6bf2
Author: Antoine Catton <acatton@tiolive.com>
Date:   Thu Aug 25 09:40:57 2011 +0200

    Adding rdiff_backup installation and configuration

commit 38882163cb12d965ba0f1e3c87de9775ffe11ed0
Author: Antoine Catton <acatton@tiolive.com>
Date:   Wed Aug 24 09:48:07 2011 +0200

    Minor: adding ssh compression

commit eb5bff2affcc6a4b6e5d332a2c59c9663839c6a0
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 17:35:57 2011 +0200

    Minor: fix adding ssh_config template

commit 07fcca2381cb9167779aa18434b0d8e1514d6403
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 17:32:24 2011 +0200

    Minor: fix of error due to ssh_conf to self.ssh_conf renaming

commit c3f7c4944e960b463166e000422888bec13556a0
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 16:47:32 2011 +0200

    Adding ssh_config file generation

commit 4c6ebe52eab924eca18df4950a2bce6f10513971
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 16:43:52 2011 +0200

    Minor: adding consistency to known_hosts file creation

commit c814412504d1b9682c180e40507a861ec125ebb9
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 16:40:50 2011 +0200

    Minor: put the parameter dict reading at the begining

commit 956db5d669d25bfbb0f0675a1d06d1c05a4056ad
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 16:39:09 2011 +0200

    Minor: save sshd_conf options

commit 2535efd9dc003474b31480b529a1ecf2b057e2c0
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 15:11:15 2011 +0200

    Minor: adding template for known_hosts file

commit 02f865391d3a95cbe87f7461696dec339ffe32e2
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 12:58:55 2011 +0200

    Add known_hosts file generation

commit 411052f4452c273674a6e3d5497807e6f6041948
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 12:57:40 2011 +0200

    Minor: strip the ssh key value.

commit 669996d528a3177a601b60ad0767b8e5c6791455
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 12:56:35 2011 +0200

    Minor: dirty fix to avoid "unused option" message

commit 8dda614bcb389ae57398c020eafcdabf77354281
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 12:55:08 2011 +0200

    Minor: save cron.d path

commit ca95838a0512a9065fa08f51152791ca918fadb0
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 11:18:55 2011 +0200

    Minor: code condensation

commit 1e251053ab1a3eb7acd2bf5af42c075f56a1fbdb
Author: Antoine Catton <acatton@tiolive.com>
Date:   Tue Aug 23 10:38:46 2011 +0200

    Pull Backup Server recipe first draft

commit 9bf0841e55078e5f5d4ea822f05cbb0b2840f6d2
Author: Antoine Catton <acatton@tiolive.com>
Date:   Mon Aug 22 13:18:08 2011 +0200

    Minor: code cleaning on mysql recipe

commit 91a5ec43e7835acc283a88733b0edcc146013b5b
Author: Antoine Catton <acatton@tiolive.com>
Date:   Fri Aug 19 15:47:01 2011 +0200

    Minor: adding sshd_binary option to mariadb

commit ba53f1ed65021273f0c5797830ec164963391249
Author: Antoine Catton <acatton@tiolive.com>
Date:   Fri Aug 19 15:12:44 2011 +0200

    Adding openssh part to mariadb software release

commit 4fa4540c300e62322ff0eddd0609a02ad5d20f26
Author: Antoine Catton <acatton@tiolive.com>
Date:   Fri Aug 19 15:11:47 2011 +0200

    Adding openssh part

commit 15c4b3203aabefde58dcfcf43fafb3174290a671
Author: Antoine Catton <acatton@tiolive.com>
Date:   Fri Aug 19 13:25:42 2011 +0200

    Minor: clean up white spaces in mysql recipe

component/apache/buildout.cfg

@@ -42,14 +42,14 @@ configure-options = --enable-authn-alias
                     --enable-proxy-scgi
                     --enable-so
                     --enable-dav
+                    --enable-dav-fs
                     --disable-ssl
                     --with-included-apr
                     --with-z=${zlib:location}
                     --with-expat=${libexpat:location}
                     --with-pcre=${pcre:location}
                     --with-sqlite3=${sqlite3:location}
-                    --with-dbm=gdbm
-                    --with-gdm=${gdbm:location}
+                    --with-gdbm=${gdbm:location}
                     --without-ssl
                     --without-lber
                     --without-ldap
@@ -92,6 +92,7 @@ configure-options = --disable-static
                     --enable-proxy-http
                     --enable-proxy-scgi
                     --enable-dav
+                    --enable-dav-fs
                     --enable-so
                     --enable-ssl
                     --with-included-apr
@@ -100,8 +101,7 @@ configure-options = --disable-static
                     --with-expat=${libexpat:location}
                     --with-pcre=${pcre:location}
                     --with-sqlite3=${sqlite3:location}
-                    --with-dbm=gdbm
-                    --with-gdm=${gdbm:location}
+                    --with-gdbm=${gdbm:location}
                     --without-lber
                     --without-ldap
                     --without-ndbm

component/duplicity/buildout.cfg

+[buildout]
+extends =
+  ../librsync/buildout.cfg
+  ../patch/buildout.cfg
+parts =
+  duplicity
+
+[duplicity]
+recipe = slapos.recipe.build
+url = http://code.launchpad.net/duplicity/0.6-series/0.6.15/+download/duplicity-0.6.15.tar.gz
+md5sum = 88f3c990f41fde86cd7d5af5a1bc7b81
+patch = ${:_profile_base_location_}/duplicity-ipv6.patch 32c44816a9a59401e233ef622bf50223
+script =
+  import os
+  import tempfile
+  import sys
+  workingdir = guessworkdir(self.extract(self.download(%(url)r, %(md5sum)r)))
+  os.chdir(workingdir)
+  self.applyPatchList('${:patch}', patch_binary='${patch:location}/bin/patch', patch_options='-p0', cwd=workingdir)
+  call([sys.executable, os.path.join(workingdir, 'setup.py'),
+       'build_ext',
+       '--include-dirs', os.path.join('${librsync:location}', 'include'),
+       '--library-dirs', os.path.join('${librsync:location}', 'lib'),
+       '--rpath', os.path.join('${librsync:location}', 'lib')])
+  call([sys.executable, os.path.join(workingdir, 'setup.py'),
+        'install',
+        '--prefix', %(location)r])
+  os.chdir(os.path.join(%(location)r, 'bin'))
+  paths = [os.path.join(%(location)r, 'lib', 'python%%s.%%s' %% sys.version_info[:2], 'site-packages')]
+  from zc.buildout.easy_install import scripts
+  scripts([('duplicity', '__builtin__', 'execfile')], [], sys.executable, '${buildout:bin-directory}', arguments='%%r' %% os.path.abspath('duplicity'), extra_paths=paths)
+  scripts([('rdiffdir', '__builtin__', 'execfile')], [], sys.executable, '${buildout:bin-directory}', arguments='%%r' %% os.path.abspath('rdiffdir'), extra_paths=paths)

component/duplicity/duplicity-ipv6.patch

+=== modified file 'src/urlparse_2_5.py'
+--- src/urlparse_2_5.py	2011-03-06 15:12:33 +0000
++++ src/urlparse_2_5.py	2011-08-31 14:21:06 +0000
+@@ -109,18 +109,19 @@
+     password = property(get_password)
+ 
+     def get_hostname(self):
+-        netloc = self.netloc
+-        if "@" in netloc:
+-            netloc = _rsplit(netloc, "@", 1)[1]
+-        if ":" in netloc:
+-            netloc = netloc.split(":", 1)[0]
+-        return netloc.lower() or None
++        netloc = self.netloc.split('@')[-1]
++        if '[' in netloc and ']' in netloc:
++            return netloc.split(']')[0][1:].lower()
++        elif ':' in netloc:
++            return netloc.split(':')[0].lower()
++        elif netloc == '':
++            return None
++        else:
++            return netloc.lower()
+     hostname = property(get_hostname)
+ 
+     def get_port(self):
+-        netloc = self.netloc
+-        if "@" in netloc:
+-            netloc = _rsplit(netloc, "@", 1)[1]
++        netloc = self.netloc.split('@')[-1].split(']')[-1]
+         if ":" in netloc:
+             port = netloc.split(":", 1)[1]
+             return int(port, 10)
+

setup.py

@@ -40,6 +40,7 @@ setup(name=name,
       entry_points={
         'zc.buildout': [
           'download = slapos.recipe.download:Recipe',
+          'davstorage = slapos.recipe.davstorage:Recipe',
           'erp5 = slapos.recipe.erp5:Recipe',
           'erp5testnode = slapos.recipe.erp5testnode:Recipe',
           'helloworld = slapos.recipe.helloworld:Recipe',

slapos/recipe/davstorage/__init__.py

+##############################################################################
+#
+# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
+#
+# WARNING: This program as such is intended to be used by professional
+# programmers who take the whole responsibility of assessing all potential
+# consequences resulting from its eventual inadequacies and bugs
+# End users who are looking for a ready-to-use solution with commercial
+# guarantees and support are strongly adviced to contract a Free Software
+# Service Company
+#
+# This program is Free Software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+#
+##############################################################################
+from slapos.recipe.librecipe import BaseSlapRecipe
+import os
+import subprocess
+import pkg_resources
+import zc.buildout
+import zc.recipe.egg
+import sys
+
+class Recipe(BaseSlapRecipe):
+  def getTemplateFilename(self, template_name):
+    return pkg_resources.resource_filename(__name__,
+        'template/%s' % template_name)
+
+  def _install(self):
+    self.path_list = []
+    self.requirements, self.ws = self.egg.working_set()
+    document_root = self.createDataDirectory('www')
+    self.apache_config = self.installApache(document_root)
+    self.setConnectionDict(
+      dict(url='https://[%s]:%s/' % (self.apache_config['ip'],
+                                     self.apache_config['port']),
+           user=self.apache_config['user'],
+           password=self.apache_config['password']),
+    )
+    return self.path_list
+
+  def installApache(self, document_root, ip=None, port=None):
+    if ip is None:
+      ip=self.getGlobalIPv6Address()
+    if port is None:
+      port = '9080'
+
+    htpasswd_config = self.createHtpasswd()
+    ssl_config = self.createCertificate(size=2048)
+
+    apache_config = dict(
+      pid_file=os.path.join(self.run_directory, 'httpd.pid'),
+      lock_file=os.path.join(self.run_directory, 'httpd.lock'),
+      davlock_db=os.path.join(self.run_directory, 'davdb.lock'),
+      ip=ip,
+      port=port,
+      error_log=os.path.join(self.log_directory, 'httpd-error.log'),
+      access_log=os.path.join(self.log_directory, 'httpd-access.log'),
+      document_root=document_root,
+      modules_dir=self.options['apache_modules_dir'],
+      mime_types=self.options['apache_mime_file'],
+      server_root=self.work_directory,
+      email_address='admin@vifib.net',
+      htpasswd_file=htpasswd_config['htpasswd_file'],
+      ssl_certificate=ssl_config['certificate'],
+      ssl_key=ssl_config['key'],
+    )
+    httpd_config_file = self.createConfigurationFile('httpd.conf',
+      self.substituteTemplate(self.getTemplateFilename('httpd.conf.in'),
+                              apache_config))
+    self.path_list.append(httpd_config_file)
+    apache_runner = zc.buildout.easy_install.scripts(
+      [('httpd', 'slapos.recipe.librecipe.execute', 'execute')],
+      self.ws, sys.executable, self.wrapper_directory,
+      arguments=[self.options['apache_binary'],
+                 '-f', httpd_config_file,
+                 '-DFOREGROUND',
+                ]
+    )[0]
+    self.path_list.append(apache_runner)
+    return dict(ip=apache_config['ip'],
+                port=apache_config['port'],
+                user=htpasswd_config['user'],
+                password=htpasswd_config['password']
+               )
+
+  def createHtpasswd(self):
+    htpasswd = self.createConfigurationFile('htpasswd', '')
+    self.path_list.append(htpasswd)
+    password = self.generatePassword()
+    user = 'user'
+    subprocess.check_call([self.options['apache_htpasswd'],
+                           '-bc', htpasswd,
+                           user, password
+                          ])
+    return dict(htpasswd_file=htpasswd,
+                user=user,
+                password=password)
+
+  def createCertificate(self, size=1024, subject='/C=FR/L=Marcq-en-Baroeul/O=Nexedi'):
+    key_file = os.path.join(self.etc_directory, 'httpd.key')
+    self.path_list.append(key_file)
+
+    certificate_file = os.path.join(self.etc_directory, 'httpd.crt')
+    self.path_list.append(certificate_file)
+
+    subprocess.check_call([self.options['openssl_binary'],
+                           'req', '-x509', '-nodes',
+                           '-newkey', 'rsa:%s' % size,
+                           '-subj', str(subject),
+                           '-out', certificate_file,
+                           '-keyout', key_file
+                          ])
+    return dict(key=key_file,
+                certificate=certificate_file)

slapos/recipe/davstorage/template/httpd.conf.in

+ServerRoot "%(server_root)s"
+
+Listen [%(ip)s]:%(port)s
+
+# Needed modules
+LoadModule authn_file_module "%(modules_dir)s/mod_authn_file.so"
+LoadModule authz_host_module "%(modules_dir)s/mod_authz_host.so"
+LoadModule authz_user_module "%(modules_dir)s/mod_authz_user.so"
+LoadModule auth_basic_module "%(modules_dir)s/mod_auth_basic.so"
+LoadModule auth_digest_module "%(modules_dir)s/mod_auth_digest.so"
+LoadModule log_config_module "%(modules_dir)s/mod_log_config.so"
+LoadModule headers_module "%(modules_dir)s/mod_headers.so"
+LoadModule setenvif_module "%(modules_dir)s/mod_setenvif.so"
+LoadModule ssl_module "%(modules_dir)s/mod_ssl.so"
+LoadModule mime_module "%(modules_dir)s/mod_mime.so"
+LoadModule dav_module "%(modules_dir)s/mod_dav.so"
+LoadModule dav_fs_module "%(modules_dir)s/mod_dav_fs.so"
+LoadModule dir_module "%(modules_dir)s/mod_dir.so"
+
+ServerAdmin %(email_address)s
+
+# Quiet Server header (if not, Apache give its life history)
+# It's safer
+ServerTokens ProductOnly
+
+DocumentRoot "%(document_root)s"
+PidFile "%(pid_file)s"
+LockFile "%(lock_file)s"
+DavLockDB "%(davlock_db)s"
+
+<Directory />
+    Options FollowSymLinks
+    AllowOverride None
+    Order deny,allow
+    Deny from all
+</Directory>
+
+<Directory %(document_root)s>
+    Options Indexes MultiViews
+    AllowOverride None
+    Order allow,deny
+    Allow from all
+
+    Dav On
+    # Security Rules to avoid DDoS Attacks
+    DavDepthInfinity Off
+    LimitXMLRequestBody 0
+
+    # Cross-Origin Resources Sharing
+    Header always set Access-Control-Max-Age "0"
+    Header always set Access-Control-Allow-Origin "*"
+    Header always set Access-Control-Allow-Methods "OPTIONS, GET, HEAD, POST, PUT, DELETE, PROPFIND"
+    Header always set Access-Control-Allow-Headers "Content-Type, X-Requested-With, X-HTTP-Method-Override, Accept, Authorization, Depth"
+    SetEnvIf Origin "(.+)" ORIGIN=$1
+    Header always set Access-Control-Allow-Origin %%{ORIGIN}e
+
+    AuthType Basic
+    AuthName "WebDAV Storage"
+    AuthUserFile "%(htpasswd_file)s"
+    <LimitExcept OPTIONS>
+        Require valid-user
+    </LimitExcept>
+
+</Directory>
+
+ErrorLog "%(error_log)s"
+LogLevel warn
+
+LogFormat "%%h %%l %%u %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\"" combined
+LogFormat "%%h %%l %%u %%t \"%%r\" %%>s %%b" common
+CustomLog "%(access_log)s" common
+
+DefaultType text/plain
+TypesConfig "%(mime_types)s"
+AddType application/x-compress .Z
+AddType application/x-gzip .gz .tgz
+
+SSLRandomSeed startup builtin
+SSLRandomSeed connect builtin
+SSLEngine on
+SSLCertificateFile "%(ssl_certificate)s"
+SSLCertificateKeyFile "%(ssl_key)s"

slapos/recipe/mysql/__init__.py

@@ -31,6 +31,8 @@ import pkg_resources
 import sys
 import zc.buildout
 import ConfigParser
+import re
+import urlparse
 
 class Recipe(BaseSlapRecipe):
   def getTemplateFilename(self, template_name):
@@ -44,17 +46,18 @@ class Recipe(BaseSlapRecipe):
     # self.cron_d is a directory, where cron jobs can be registered
     self.cron_d = self.installCrond()
     self.logrotate_d, self.logrotate_backup = self.installLogrotate()
-    
+
     mysql_conf = self.installMysqlServer(self.getLocalIPv4Address(), 45678)
-      
+    self.mysql_backup_directory = mysql_conf['backup_directory']
+
     ca_conf = self.installCertificateAuthority()
     key, certificate = self.requestCertificate('MySQL')
-    
+
     stunnel_conf = self.installStunnel(self.getGlobalIPv6Address(),
         self.getLocalIPv4Address(), 12345, mysql_conf['tcp_port'],
         certificate, key, ca_conf['ca_crl'],
         ca_conf['certificate_authority_path'])
-    
+
     self.linkBinary()
     self.setConnectionDict(dict(
       stunnel_ip = stunnel_conf['public_ip'],
@@ -107,7 +110,7 @@ class Recipe(BaseSlapRecipe):
       )[0]
     self.path_list.append(wrapper)
     return cron_d
-  
+
   def installLogrotate(self):
     """Installs logortate main configuration file and registers its to cron"""
     logrotate_d = os.path.abspath(os.path.join(self.etc_directory,
@@ -232,7 +235,7 @@ class Recipe(BaseSlapRecipe):
     self.path_list.append(wrapper)
     return stunnel_conf
 
-    
+
   def installMysqlServer(self, ip, port, database='db', user='user',
       template_filename=None, mysql_conf=None):
     if mysql_conf is None:
@@ -267,7 +270,6 @@ class Recipe(BaseSlapRecipe):
           mysql_conf))
 
     mysql_script_list = []
-    
     mysql_script_list.append(pkg_resources.resource_string(__name__,
                    'template/initmysql.sql.in') % {
                       'mysql_database': mysql_conf['mysql_database'],
@@ -325,7 +327,32 @@ class Recipe(BaseSlapRecipe):
         incremental_backup])[0]
     self.path_list.append(backup_controller)
     mysql_backup_cron = os.path.join(self.cron_d, 'mysql_backup')
-    open(mysql_backup_cron, 'w').write('0 0 * * * ' + backup_controller)
+    open(mysql_backup_cron, 'w').write('0 0 * * * %r' % str(backup_controller))
     self.path_list.append(mysql_backup_cron)
+    mysql_conf.update(backup_directory=incremental_backup)
     # The return could be more explicit database, user ...
+    remote_url = self.installWebDAVBackup()
+    remote_backup_cron = os.path.join(self.cron_d, 'remote_backup')
+    with open(remote_backup_cron, 'w') as file_:
+      file_.write('1 0 * * * %s' % ' '.join([
+        '%r' % str(self.options['duplicity_binary']),
+        '--no-encryption',
+        '%r' % str(backup_directory), '%r' % str(remote_url),
+      ]))
     return mysql_conf
+
+  def installWebDAVBackup(self):
+    computer_partition = self.request(
+      # XXX: Hardcoded url
+      'http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/webdav:/software/davstorage/software.cfg'
+      'davstorage',
+      'mysql_backup',
+    )
+    url = re.sub('^http', 'webdav', computer_partition.getConnectionParameter('url'))
+    url = list(urlparse.urlparse(url))
+    url[1] = '%(user)s:%(password)s@%(netloc)s' % {
+      'user': computer_partition.getConnectionParameter('user'),
+      'password': computer_partition.getConnectionParameter('password'),
+      'netloc': url[1],
+    }
+    return urlparse.urlunparse(url)

software/davstorage/instance.cfg

+[buildout]
+parts =
+  instance
+
+eggs-directory = ${buildout:eggs-directory}
+develop-eggs-directory = ${buildout:develop-eggs-directory}
+
+[instance]
+recipe = ${instance-recipe:egg}:${instance-recipe:module}
+dcrond_binary = ${dcron:location}/sbin/crond
+logrotate_binary = ${logrotate:location}/usr/sbin/logrotate
+apache_binary = ${apache:location}/bin/httpd
+apache_modules_dir = ${apache:location}/modules/
+apache_mime_file = ${apache:location}/conf/mime.types
+apache_htpasswd = ${apache:location}/bin/htpasswd
+openssl_binary = ${openssl:location}/bin/openssl

software/davstorage/software.cfg

+[buildout]
+
+find-links +=
+    http://www.nexedi.org/static/packages/source/slapos.buildout/
+
+extends =
+  ../../component/apache/buildout.cfg
+  ../../component/dcron/buildout.cfg
+   ../../component/logrotate/buildout.cfg
+  ../../stack/shacache-client.cfg
+  ../../component/python-2.7/buildout.cfg
+
+# Use only quite well working sites.
+allow-hosts =
+  *.nexedi.org
+  *.python.org
+  *.sourceforge.net
+  dist.repoze.org
+  effbot.org
+  github.com
+  peak.telecommunity.com
+  psutil.googlecode.com
+  www.dabeaz.com
+
+
+parts +=
+  template
+  apache
+  eggs
+  instance-recipe-egg
+
+
+unzip= true
+
+[eggs]
+recipe = zc.recipe.egg
+eggs =
+
+[instance-recipe]
+egg = slapos.cookbook
+module = davstorage
+
+[instance-recipe-egg]
+recipe = zc.recipe.egg
+python = python2.7
+eggs = ${instance-recipe:egg}
+
+[template]
+# Default template for the instance.
+recipe = slapos.recipe.template
+url = ${:_profile_base_location_}/instance.cfg
+md5sum = 51b6213889573ae7b1dec0bd65384432
+output = ${buildout:directory}/template.cfg
+mode = 0644
+
+[versions]
+slapos.libnetworkcache = 0.3
+zc.buildout = 1.5.3-dev-SlapOS-005

software/mariadb/import-script.sh

+#!%(bash_binary)s
+
+# Didn't we already restore the database ?
+[ -f %(lock_file)r ] && exit 127
+touch %(lock_file)r
+
+dbname=db
+# Wait for MySQL to be started
+while ! %(mysql_binary)r --socket=%(mysql_socket)r -u root -e "use $dbname;"
+do
+    sleep 5
+done
+
+# Restore dump
+%(duplicity_binary)r restore --no-encryption %(parameter_remote_backup)r %(local_directory)r
+zcat %(dump_name)r | %(mysql_binary)r --socket=%(mysql_socket)r -D $dbname -u root

software/mariadb/instance.cfg

@@ -18,3 +18,4 @@ openssl_binary = ${openssl:location}/bin/openssl
 perl_binary = ${perl:location}/bin/perl
 rdiff_backup_binary = ${buildout:bin-directory}/rdiff-backup
 stunnel_binary = ${stunnel:location}/bin/stunnel
+duplicity_binary = ${buildout:bin-directory}/duplicity

software/mariadb/software.cfg

@@ -2,7 +2,6 @@
 
 extensions =
   slapos.zcbworkarounds
-  slapos.rebootstrap
 
 find-links +=
     http://www.nexedi.org/static/packages/source/slapos.buildout/
@@ -18,6 +17,7 @@ extends =
   ../../component/rdiff-backup/buildout.cfg
   ../../component/lxml-python/buildout.cfg
   ../../stack/shacache-client.cfg
+  ../../component/duplicity/buildout.cfg
 
 # Use only quite well working sites.
 allow-hosts =
@@ -70,7 +70,7 @@ eggs =
 # Default template for the instance.
 recipe = slapos.recipe.template
 url = ${:_profile_base_location_}/instance.cfg
-md5sum = 69c32a67c5640d36ee042d2cfc35843d
+md5sum = ae477237183657c372ce47ad303d53b2
 output = ${buildout:directory}/template.cfg
 mode = 0644