Commit 2fcbb362 authored by Priscila Manhaes's avatar Priscila Manhaes

merge to master

parents b162017f f7fdb589
*pyc *pyc
.installed.cfg
bin/
build build
develop-eggs/
dist dist
downloads/
eggs/
parts/
slapos.cookbook.egg-info slapos.cookbook.egg-info
SlapOS repository rules
=======================
Short version for impatient
---------------------------
'master' branch is always stable. One shall never work in this branch. One
shall expect reversion or removal of his commits in 'master' branch, if they
are not because of merging another branch.
Everyone has to create branch, basing on 'master' or any other branch. All
work shall be done in such branch. It is advised to review it before merging
into 'master'. It is very good habit to tag such merges.
A bit more background
---------------------
SlapOS repository 'master' branch is supposed to be stable. No new work is
accepted on this branch. Each time someone uses (clones, extends) 'master'
branch it is supposed to work (as far as it was tested).
Tags on 'master' branch are supposed to be repeatable software definitions
forever. All versions and external links shall be setup in a way, that they
wouldn't change in day, month, year, etc.
Everyone can make his own branch to introduce new software to SlapOS. There
are no rules about those branches, as each software is specific.
It is good idea to ask for review before merging changes into 'master' branch.
Peer review can detect issues not detected by testing, development or self
analysis.
After some software is stabilised and no more work is planned it is acceptable
to remove such abandoned branch in order to minimise amount of branches.
0.6 (unreleased) Changes
================ =======
0.16 (unreleased)
-----------------
* No changes yet. * No changes yet.
0.15 (2011-07-13)
-----------------
* Encrypt connection by default. [Vivien Alger]
0.14 (2011-07-13)
-----------------
* Provide new way to instantiate kvm. [Cedric de Saint Martin, Vivien Alger]
0.13 (2011-07-13)
-----------------
* Implement generic execute_wait wrapper, which allows to wait for some files
to appear before starting service depending on it. [Łukasz Nowak]
0.12 (2011-07-11)
-----------------
* Fix slaprunner, phpmyadmin software releases, added
wordpress software release. [Cedric de Saint Martin]
0.11 (2011-07-07)
-----------------
* Enable test suite runner for vifib.
0.10 (2011-07-01)
-----------------
* Add PHPMyAdmin software release used in SlapOS tutorials
[Cedric de Saint Martin]
* Add slaprunner software release [Cedric de Saint Martin]
0.9 (2011-06-24)
----------------
* mysql recipe : Changing slapos.recipe.erp5.execute to
slapos.recipe.librecipe.execute [Cedric de Saint Martin]
0.8 (2011-06-15)
----------------
* Add MySQL and MariaDB standalone software release and recipe
[Cedric de Saint Martin]
* Fixed slapos.recipe.erp5testnode instantiation [Sebastien Robin]
0.7 (2011-06-14)
----------------
* Fix slapos.recipe.erp5 package by providing site.zcml in it. [Łukasz Nowak]
* Improve slapos.recipe.erp5testnode partition instantiation error reporting
[Sebastien Robin]
0.6 (2011-06-13)
----------------
* Fixed slapos.recipe.erp5 instantiation. [Łukasz Nowak]
0.5 (2011-06-13) 0.5 (2011-06-13)
================ ----------------
* Implement zabbix agent instantiation. [Łukasz Nowak] * Implement zabbix agent instantiation. [Łukasz Nowak]
* Drop dependency on Zope2. [Łukasz Nowak] * Drop dependency on Zope2. [Łukasz Nowak]
* Share more in slapos.recipe.librecipe module. [Łukasz Nowak] * Share more in slapos.recipe.librecipe module. [Łukasz Nowak]
0.4 (2011-06-09) 0.4 (2011-06-09)
================ ----------------
* Remove reference to slapos.tool.networkcache as it was removed from pypi. [Łukasz Nowak] * Remove reference to slapos.tool.networkcache as it was removed from pypi. [Łukasz Nowak]
* Add Kumofs standalone software release and recipe [Cedric de Saint Martin] * Add Kumofs standalone software release and recipe [Cedric de Saint Martin]
* Add Memcached standalone software release and recipe [Cedric de Saint Martin] * Add Memcached standalone software release and recipe [Cedric de Saint Martin]
0.3 (2011-06-09) 0.3 (2011-06-09)
================ ----------------
* Moved out template and build to separate distributions [Łukasz Nowak] * Moved out template and build to separate distributions [Łukasz Nowak]
* Depend on slapos.core instead of depracated slapos.slap [Romain Courteaud] * Depend on slapos.core instead of depracated slapos.slap [Romain Courteaud]
...@@ -26,11 +88,11 @@ ...@@ -26,11 +88,11 @@
* Allow to control full environment in erp5 module [Łukasz Nowak] * Allow to control full environment in erp5 module [Łukasz Nowak]
0.2 (2011-05-30) 0.2 (2011-05-30)
================ ----------------
* Allow to pass zope_environment in erp5 entry point [Łukasz Nowak] * Allow to pass zope_environment in erp5 entry point [Łukasz Nowak]
0.1 (2011-05-27) 0.1 (2011-05-27)
================ ----------------
* All slapos.recipe.* became slapos.cookbook:* [Łukasz Nowak] * All slapos.recipe.* became slapos.cookbook:* [Łukasz Nowak]
include CHANGES.txt include CHANGES.txt
include slapos/recipe/erp5/template/site.zcml
recursive-include slapos/recipe *.in recursive-include slapos/recipe *.in
recursive-include slapos/recipe README.*.txt recursive-include slapos/recipe README.*.txt
slapos.cookbook slapos.cookbook
=============== ===============
Cookbook of SlapOS recipes.
[buildout]
parts = apache-php
extends =
../apache/buildout.cfg
[apache-php]
# Note: Shall react on each build of apache and reinstall itself
recipe = hexagonit.recipe.cmmi
url = http://fr2.php.net/get/php-5.3.6.tar.bz2/from/this/mirror
md5sum = 2286f5a82a6e8397955a0025c1c2ad98
configure-options =
--with-apxs2=${apache:location}/bin/apxs
--with-libxml-dir=${libxml2:location}
--with-mysql=${mariadb:location}
--with-zlib-dir=${zlib:location}
--with-mcrypt=${libmcrypt:location}
--enable-mbstring
--enable-session
--disable-all
environment =
PKG_CONFIG_PATH=${libxml2:location}/lib/pkgconfig
PATH=${libxml2:location}/bin:%(PATH)s
LDFLAGS =-L${libtool:location}/lib -Wl,-rpath -Wl,${libtool:location}/lib -L${mariadb:location}/lib -Wl,-rpath -Wl,${mariadb:location}/lib -L${zlib:location}/lib -Wl,-rpath -Wl,${zlib:location}/lib -L${libmcrypt:location}/lib -Wl,-rpath -Wl,${libmcrypt:location}/lib
[apache-php-xmlrpc-gd]
# Note: Shall react on each build of apache and reinstall itself
recipe = hexagonit.recipe.cmmi
url = http://fr2.php.net/get/php-5.3.6.tar.bz2/from/this/mirror
md5sum = 2286f5a82a6e8397955a0025c1c2ad98
configure-options =
--with-apxs2=${apache:location}/bin/apxs
--with-libxml-dir=${libxml2:location}
--with-gd
--with-zlib-dir=${zlib:location}
--with-mcrypt=${libmcrypt:location}
--with-xmlrpc=${xml-rpc:location}
--enable-mbstring
--enable-session
--disable-all
environment =
PKG_CONFIG_PATH=${libxml2:location}/lib/pkgconfig
PATH=${libxml2:location}/bin:%(PATH)s
LDFLAGS =-L${xml-rpc:location}/lib -Wl,-rpath -Wl,${xml-rpc:location}/lib -L${zlib:location}/lib -Wl,-rpath -Wl,${zlib:location}/lib -L${libmcrypt:location}/lib -Wl,-rpath -Wl,${libmcrypt:location}/lib
[libmcrypt]
recipe = hexagonit.recipe.cmmi
url = http://sourceforge.net/projects/mcrypt/files/Libmcrypt/2.5.8/libmcrypt-2.5.8.tar.bz2/download
md5sum = c4f491dd411a09e9de3b8702ea6f73eb
[xml-rpc]
recipe = hexagonit.recipe.cmmi
url = http://downloads.sourceforge.net/project/phpxmlrpc/phpxmlrpc/2.2.2/xmlrpc-2.2.2.tar.gz
md5sum = 59a644c636c6d98267d0c99b406ae9e8
\ No newline at end of file
...@@ -13,6 +13,55 @@ extends = ...@@ -13,6 +13,55 @@ extends =
../sqlite3/buildout.cfg ../sqlite3/buildout.cfg
../zlib/buildout.cfg ../zlib/buildout.cfg
[apache-no-ssl]
# inspired on http://old.aclark.net/team/aclark/blog/a-lamp-buildout-for-wordpress-and-other-php-apps/
recipe = hexagonit.recipe.cmmi
url = http://mir2.ovh.net/ftp.apache.org/dist//httpd/httpd-2.2.19.tar.bz2
md5sum = 832f96a6ec4b8fc7cf49b9efd4e89060
configure-options = --enable-authn-alias
--enable-bucketeer
--enable-cache
--enable-case-filter
--enable-case-filter-in
--enable-cgid
--enable-charset-lite
--enable-disk-cache
--enable-echo
--enable-exception-hook
--enable-mods-shared=all
--enable-optional-fn-export
--enable-optional-fn-import
--enable-optional-hook-export
--enable-optional-hook-import
--enable-proxy
--enable-proxy-ajp
--enable-proxy-balancer
--enable-proxy-connect
--enable-proxy-ftp
--enable-proxy-http
--enable-proxy-scgi
--enable-so
--disable-ssl
--with-included-apr
--with-z=${zlib:location}
--with-expat=${libexpat:location}
--with-pcre=${pcre:location}
--with-sqlite3=${sqlite3:location}
--with-dbm=gdbm
--with-gdm=${gdbm:location}
--without-ssl
--without-lber
--without-ldap
--without-ndbm
--without-berkeley-db
--without-pgsql
--without-mysql
--without-sqlite2
--without-oracle
--without-freedts
--without-odbc
--without-iconv
[apache] [apache]
# inspired on http://old.aclark.net/team/aclark/blog/a-lamp-buildout-for-wordpress-and-other-php-apps/ # inspired on http://old.aclark.net/team/aclark/blog/a-lamp-buildout-for-wordpress-and-other-php-apps/
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
......
...@@ -6,7 +6,7 @@ parts = ...@@ -6,7 +6,7 @@ parts =
[bison] [bison]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://ftp.gnu.org/gnu/bison/bison-2.4.3.tar.gz url = http://ftp.gnu.org/gnu/bison/bison-2.5.tar.bz2
md5sum = ea45c778b36bdc7a720096819e292a73 md5sum = 9dba20116b13fc61a0846b0058fbe004
environment = environment =
M4=${m4:location}/bin/m4 M4=${m4:location}/bin/m4
...@@ -4,8 +4,8 @@ parts = ...@@ -4,8 +4,8 @@ parts =
[coreutils] [coreutils]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.gz url = http://ftp.gnu.org/gnu/coreutils/coreutils-8.12.tar.gz
md5sum = 36909ae68840d73a800120cf74af794a md5sum = fce7999953a67243d00d75cc86dbcaa6
configure-options = configure-options =
--prefix=${buildout:parts-directory}/${:_buildout_section_name_} --enable-install-program=tr,basename,uname,cat,cp,ls --prefix=${buildout:parts-directory}/${:_buildout_section_name_} --enable-install-program=tr,basename,uname,cat,cp,ls
environment = environment =
......
[buildout]
extends =
../mysql-tritonn-5.0/buildout.cfg
../python-2.4/buildout.cfg
../lxml-python/buildout.cfg
../mysql-python/buildout.cfg
../subversion/buildout.cfg
../pysvn-python/buildout.cfg
../python-ldap-python/buildout.cfg
../glib/buildout.cfg
parts =
cmf15
itools
mysql-python
products-other
products-deps
products-erp5
[cmf15]
recipe = plone.recipe.distros
urls =
http://www.zope.org/Products/CMF/CMF-1.5.4/CMF-1.5.4.tar.gz
nested-packages =
CMF-1.5.4.tar.gz
version-suffix-packages =
CMF-1.5.4.tar.gz
[itools]
# use a custom build for itools, to add lib64 to the include path
recipe = zc.recipe.egg:custom
python = python2.4
egg = itools
find-links =
http://download.hforge.org/itools/0.20/
include-dirs =
${glib:location}/include/glib-2.0
${glib:location}/lib/glib-2.0/include
library-dirs =
${glib:location}/lib
rpath =
${glib:location}/lib
[products-deps]
recipe = plone.recipe.distros
urls =
http://www.zope.org/Members/shh/ExtFile/1.4.4/ExtFile-1.4.4.tar.gz
http://www.zope.org/Members/NIP/ZMailIn/1.0.1/ZMailIn-1-0-1.tgz
http://www.zope.org/Members/NIP/ZMailIn/1.0.0/CMFMailIn-1.0.0
http://www.zope.org/Products/PluggableAuthService/PluggableAuthService-1.1b2/PluggableAuthService-1.1b2.tar.gz
http://download.hforge.org/localizer/Localizer-1.2.3.tar.gz
version-suffix-packages =
Localizer-1.2.3.tar.gz
[products-ldap]
recipe = plone.recipe.distros
urls =
http://www.dataflake.org/software/ldapmultiplugins/ldapmultiplugins_1.1/LDAPMultiPlugins-1_1.tgz
http://www.dataflake.org/software/ldapuserfolder/ldapuserfolder_2.6/LDAPUserFolder-2_6.tgz
[products-other]
# Recipe infrae.subversion is using svn command under the hood, but there is
# no way to pass --trust-server-cert --non-interactive --no-auth-cache, so in 2.12 falvour
# it is better to evaluate usage of provided subversion command
recipe = plone.recipe.command
svn_param =--trust-server-cert --non-interactive --no-auth-cache --quiet
# dircty hack to support PluginRegistry/utils.py:17 assumption that products
# are in Products folder
# XXX: Zelenium was eggfied for recent zope versions (2.12) and are available at Bazaar.
# some better alternative should be used in future.
location = ${buildout:parts-directory}/${:_buildout_section_name_}
destination = ${:location}/Products
stop-on-error = true
update-command = ${:command}
command = ${subversion:location}/bin/svn checkout ${:svn_param} http://svn.plone.org/svn/collective/DCWorkflowGraph/tags/release-0_3/ ${:destination}/DCWorkflowGraph &&
${subversion:location}/bin/svn checkout ${:svn_param} svn://svn.zope.org/repos/main/Zelenium/trunk/@110603 ${:destination}/Zelenium &&
${subversion:location}/bin/svn checkout ${:svn_param} svn://svn.zope.org/repos/main/PluginRegistry/tags/1.0 ${:destination}/PluginRegistry &&
${subversion:location}/bin/svn checkout ${:svn_param} http://svn.plone.org/svn/archetypes/MimetypesRegistry/tags/Archetypes-1.4.0-final ${:destination}/MimetypesRegistry
[eggs]
recipe = zc.recipe.egg
python = python2.4
eggs =
${itools:egg}
${mysql-python:egg}
${lxml-python:egg}
${pysvn-python:egg}
${python-ldap-python:egg}
PyXML
ClientForm
SOAPpy
cElementTree
chardet
ctypes
elementtree
erp5.recipe.mysqldatabase
erp5diff
ipdb
mechanize
numpy
ordereddict
pycrypto
paramiko
ply
python-ldap
python-magic
python-memcached
pytz
simplejson
threadframe
timerserver
urlnorm
uuid
xml_marshaller
xupdate_processor
feedparser
extra-paths =
${zope-2.8:location}/lib/python
# shut down script generation. Other parts can generate scripts as needed by
# reusing ${eggs:eggs}
# parameterizing the version of the generated python interpreter name by the
# python section version causes dependency between this egg section and the
# installation of python, which we don't want on an instance
interpreter = python2.4
scripts =
python=${:interpreter}
ipython=i${:interpreter}
[mysql-python]
python = python2.4
[lxml-python]
python = python2.4
[python-ldap-python]
python = python2.4
[pysvn-python]
python = python2.4
[omelette]
# XXX don't use this part until this omelette bug is fixed:
# https://bugs.launchpad.net/collective.buildout/+bug/553005
recipe = collective.recipe.omelette
eggs = ${eggs:eggs}
packages =
${itools:lib} .
[precache-eggs]
python = python2.4
eggs +=
plone.recipe.zope2zeoserver
...@@ -10,8 +10,8 @@ parts = ...@@ -10,8 +10,8 @@ parts =
[freetype] [freetype]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://download.savannah.gnu.org/releases/freetype/freetype-2.4.4.tar.gz url = http://download.savannah.gnu.org/releases/freetype/freetype-2.4.5.tar.bz2
md5sum = 9273efacffb683483e58a9e113efae9f md5sum = 90428a6d8ec4876cd1eb94858c2a59b0
configure-options = configure-options =
--disable-static --disable-static
environment = environment =
......
...@@ -10,8 +10,8 @@ parts = ...@@ -10,8 +10,8 @@ parts =
[gmp] [gmp]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = ftp://ftp.gmplib.org/pub/gmp-4.3.2/gmp-4.3.2.tar.bz2 url = ftp://ftp.gmplib.org/pub/gmp-5.0.2/gmp-5.0.2.tar.bz2
md5sum = dd60683d7057917e34630b4a787932e8 md5sum = 0bbaedc82fb30315b06b1588b9077cd3
# GMP does not correctly detect achitecture so it have to be given # GMP does not correctly detect achitecture so it have to be given
# as hexagonit.recipe.cmmi is using shell expansion in subproceses # as hexagonit.recipe.cmmi is using shell expansion in subproceses
# backticks are working # backticks are working
...@@ -22,8 +22,8 @@ environment = ...@@ -22,8 +22,8 @@ environment =
[mpfr] [mpfr]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://www.mpfr.org/mpfr-3.0.0/mpfr-3.0.0.tar.bz2 url = http://www.mpfr.org/mpfr-3.0.1/mpfr-3.0.1.tar.bz2
md5sum = f45bac3584922c8004a10060ab1a8f9f md5sum = bfbecb2eacb6d48432ead5cfc3f7390a
configure-options = configure-options =
--with-gmp=${gmp:location} --with-gmp=${gmp:location}
environment = environment =
...@@ -50,15 +50,15 @@ filename = ecj.jar ...@@ -50,15 +50,15 @@ filename = ecj.jar
[gcc-download] [gcc-download]
recipe = hexagonit.recipe.download recipe = hexagonit.recipe.download
url = http://www.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.5.2/gcc-4.5.2.tar.bz2 url = http://www.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.5.3/gcc-4.5.3.tar.bz2
md5sum = d6559145853fbaaa0fd7556ed93bce9a md5sum = 8e0b5c12212e185f3e4383106bfa9cc6
strip-top-level-dir = True strip-top-level-dir = True
destination = ${gcc-java-source:location} destination = ${gcc-java-source:location}
[gcc-java-download] [gcc-java-download]
recipe = hexagonit.recipe.download recipe = hexagonit.recipe.download
url = http://www.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.5.2/gcc-java-4.5.2.tar.bz2 url = http://www.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.5.3/gcc-java-4.5.3.tar.bz2
md5sum = fe2b647bace18dc7867a4192def46e2c md5sum = 08e045fdbdc22ac9af3aec3b8d16dbab
strip-top-level-dir = True strip-top-level-dir = True
destination = ${gcc-java-source:location} destination = ${gcc-java-source:location}
ignore-existing = true ignore-existing = true
......
...@@ -13,8 +13,8 @@ parts = ...@@ -13,8 +13,8 @@ parts =
[git] [git]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://kernel.org/pub/software/scm/git/git-1.7.3.4.tar.bz2 url = http://kernel.org/pub/software/scm/git/git-1.7.4.5.tar.bz2
md5sum = 3a2602016f98c529cda7b9fad1a6e216 md5sum = 2fa6c4c847ed87523cf55de54af457eb
configure-options = configure-options =
--with-curl=${curl:location} --with-curl=${curl:location}
--with-openssl=${openssl:location} --with-openssl=${openssl:location}
......
[buildout]
parts =
gzip
[gzip]
recipe = hexagonit.recipe.cmmi
url = ftp://ftp.gnu.org/pub/gnu/gzip/gzip-1.4.tar.gz
md5sum = e381b8506210c794278f5527cba0e765
...@@ -10,7 +10,7 @@ versions = versions ...@@ -10,7 +10,7 @@ versions = versions
[versions] [versions]
# special version of z3c.recipe.openoffice with architecture autodetection # special version of z3c.recipe.openoffice with architecture autodetection
z3c.recipe.openoffice = 0.3.1dev7 z3c.recipe.openoffice = 0.3.1dev8
[libreoffice-bin] [libreoffice-bin]
recipe = z3c.recipe.openoffice recipe = z3c.recipe.openoffice
...@@ -19,6 +19,6 @@ install-pyuno-egg = no ...@@ -19,6 +19,6 @@ install-pyuno-egg = no
hack-openoffice-python = no hack-openoffice-python = no
flavour = libreoffice flavour = libreoffice
# here, two %s are used, first one is for directory name (eg. x86_64), and second one is for filename (eg. x86-64). # here, two %s are used, first one is for directory name (eg. x86_64), and second one is for filename (eg. x86-64).
download-url = http://download.documentfoundation.org/libreoffice/old/stable/3.3.2/rpm/x86_64/LibO-SDK_3.3_Linux_x86-64_install-rpm_en-US.tar.gz # base-url = http://download.documentfoundation.org/libreoffice/stable/3.3.2/rpm/%s/LibO_3.3.2_Linux_%s_install-rpm_en-US.tar.gz
cpio = ${cpio:location}/bin/cpio cpio = ${cpio:location}/bin/cpio
...@@ -3,10 +3,20 @@ extends = ...@@ -3,10 +3,20 @@ extends =
../popt/buildout.cfg ../popt/buildout.cfg
parts = logrotate parts = logrotate
[logrotate-3.7.9-O_CLOEXEC.optional.patch]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename}
download-only = true
md5sum = 6beac248c978b767d4bccc1b7eebe6bd
filename = ${:_buildout_section_name_}
[logrotate] [logrotate]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = https://fedorahosted.org/releases/l/o/logrotate/logrotate-3.7.9.tar.gz url = https://fedorahosted.org/releases/l/o/logrotate/logrotate-3.7.9.tar.gz
md5sum = eeba9dbca62a9210236f4b83195e4ea5 md5sum = eeba9dbca62a9210236f4b83195e4ea5
patch-options = -p1
patches =
${logrotate-3.7.9-O_CLOEXEC.optional.patch:location}/${logrotate-3.7.9-O_CLOEXEC.optional.patch:filename}
configure-command = true configure-command = true
make-options = PREFIX=${buildout:parts-directory}/${:_buildout_section_name_} make-options = PREFIX=${buildout:parts-directory}/${:_buildout_section_name_}
environment = environment =
......
diff --git a/config.c b/config.c
index e6d5d1d..dd004a9 100644
--- a/config.c
+++ b/config.c
@@ -519,7 +519,11 @@ static int readConfigFile(const char *configFile, struct logInfo *defConfig)
length arrays -- of course, if we aren't run setuid it doesn't
matter much */
+#ifdef O_CLOEXEC
fd = open(configFile, O_RDONLY | O_CLOEXEC);
+#else
+ fd = open(configFile, O_RDONLY);
+#endif
if (fd < 0) {
message(MESS_ERROR, "failed to open config file %s: %s\n",
configFile, strerror(errno));
...@@ -13,6 +13,18 @@ PATH = ${libxslt:location}/bin:%(PATH)s ...@@ -13,6 +13,18 @@ PATH = ${libxslt:location}/bin:%(PATH)s
[lxml-python] [lxml-python]
recipe = zc.recipe.egg:custom recipe = zc.recipe.egg:custom
egg = lxml egg = lxml
# Note: Workaround lxml.de issues blocking buildout runs
# Empty index makes setuptools NOT trying to find any meta information about
# lxml...
index =
# ...so it is wise to tell where lxml can be found
find-links =
http://pypi.python.org/pypi/lxml/2.2.8
http://pypi.python.org/pypi/lxml/2.3
# Note: Whenever someone is going to remove it, one shall check, that buildout
# can run in newest mode, without any locally downloaded cache
rpath = rpath =
${libxml2:location}/lib/ ${libxml2:location}/lib/
${libxslt:location}/lib/ ${libxslt:location}/lib/
......
...@@ -4,5 +4,5 @@ parts = ...@@ -4,5 +4,5 @@ parts =
[m4] [m4]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://ftp.gnu.org/gnu/m4/m4-1.4.15.tar.gz url = http://ftp.gnu.org/gnu/m4/m4-1.4.16.tar.bz2
md5sum = 5649a2e593b6c639deae9e72ede777dd md5sum = 8a7cef47fecab6272eb86a6be6363b2f
[buildout]
parts =
noVNC
[noVNC]
recipe = hexagonit.recipe.download
url = https://github.com/kanaka/noVNC/tarball/master
strip-top-level-dir = true
[buildout] [buildout]
parts = poppler parts = poppler
extends = extends =
../curl/buildout.cfg
../fontconfig/buildout.cfg ../fontconfig/buildout.cfg
../freetype/buildout.cfg ../freetype/buildout.cfg
../jbigkit/buildout.cfg
../libjpeg/buildout.cfg ../libjpeg/buildout.cfg
../libpng/buildout.cfg ../libpng/buildout.cfg
../libtiff/buildout.cfg ../libtiff/buildout.cfg
...@@ -17,8 +17,18 @@ extends = ...@@ -17,8 +17,18 @@ extends =
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
md5sum = 8d7276d1943078c76aabe9f2ec52d50b md5sum = 8d7276d1943078c76aabe9f2ec52d50b
url = http://poppler.freedesktop.org/poppler-0.17.1.tar.gz url = http://poppler.freedesktop.org/poppler-0.17.1.tar.gz
configure-options =
--disable-cairo-output
--disable-cms
--disable-curl
--disable-gtk-doc-html
--disable-gtk-test
--disable-poppler-cpp
--disable-poppler-glib
--disable-poppler-qt4
--enable-zlib
environment = environment =
PATH=${pkgconfig:location}/bin:%(PATH)s PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${fontconfig:location}/lib/pkgconfig:${freetype:location}/lib/pkgconfig:${curl:location}/lib/pkgconfig:${libpng:location}/lib/pkgconfig PKG_CONFIG_PATH=${fontconfig:location}/lib/pkgconfig:${freetype:location}/lib/pkgconfig:${libpng:location}/lib/pkgconfig
CPPFLAGS=-I${libjpeg:location}/include -I${libpng:location}/include -I${libtiff:location}/include -I${zlib:location}/include CPPFLAGS=-I${libjpeg:location}/include -I${libpng:location}/include -I${libtiff:location}/include -I${zlib:location}/include
LDFLAGS=-L${libjpeg:location}/lib -Wl,-rpath=${libjpeg:location}/lib -L${libtiff:location}/lib -Wl,-rpath=${libtiff:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib LDFLAGS=-L${jbigkit:location}/lib -Wl,-rpath=${jbigkit:location}/lib -L${libjpeg:location}/lib -Wl,-rpath=${libjpeg:location}/lib -L${libtiff:location}/lib -Wl,-rpath=${libtiff:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib
[buildout] [buildout]
extends = extends =
../../stack/shacache-client.cfg
../lxml-python/buildout.cfg ../lxml-python/buildout.cfg
../python-2.7/buildout.cfg ../python-2.7/buildout.cfg
...@@ -11,6 +12,20 @@ find-links = ...@@ -11,6 +12,20 @@ find-links =
versions = versions versions = versions
allow-hosts =
*.googlecode.com
*.nexedi.org
*.python.org
alastairs-place.net
code.google.com
github.com
peak.telecommunity.com
# separate from system python
include-site-packages = false
exec-sitecustomize = false
allowed-eggs-from-site-packages =
[lxml-python] [lxml-python]
python = python2.7 python = python2.7
...@@ -18,43 +33,42 @@ python = python2.7 ...@@ -18,43 +33,42 @@ python = python2.7
recipe = z3c.recipe.scripts recipe = z3c.recipe.scripts
python = python2.7 python = python2.7
eggs = eggs =
slapos.libnetworkcache
zc.buildout
${lxml-python:egg} ${lxml-python:egg}
slapos.core slapos.core
[versions] [versions]
zc.buildout = 1.5.3-dev-SlapOS-001 zc.buildout = 1.5.3-dev-SlapOS-005
Jinja2 = 2.5.5 Jinja2 = 2.5.5
Werkzeug = 0.6.2 Werkzeug = 0.6.2
hexagonit.recipe.cmmi = 1.5.0 hexagonit.recipe.cmmi = 1.5.0
lxml = 2.3 lxml = 2.3
meld3 = 0.6.7 meld3 = 0.6.7
setuptools = 0.6c12dev-r88795 netaddr = 0.7.5
slapos.core = 0.2 setuptools = 0.6c12dev-r88846
slapos.core = 0.9
slapos.libnetworkcache = 0.2
xml-marshaller = 0.9.7 xml-marshaller = 0.9.7
z3c.recipe.scripts = 1.0.1 z3c.recipe.scripts = 1.0.1
zc.recipe.egg = 1.3.2 zc.recipe.egg = 1.3.2
# Required by: # Required by:
# slapos.core==0.2 # slapos.core==0.9
Flask = 0.6.1 Flask = 0.7.2
# Required by: # Required by:
# hexagonit.recipe.cmmi==1.5.0 # hexagonit.recipe.cmmi==1.5.0
hexagonit.recipe.download = 1.5.0 hexagonit.recipe.download = 1.5.0
# Required by: # Required by:
# slapos.core==0.2 # slapos.core==0.9
netaddr = 0.7.5
# Required by:
# slapos.core==0.2
netifaces = 0.5 netifaces = 0.5
# Required by: # Required by:
# slapos.core==0.2 # slapos.core==0.9
supervisor = 3.0a10 supervisor = 3.0a10
# Required by: # Required by:
# slapos.core==0.2 # slapos.core==0.9
zope.interface = 3.6.3 zope.interface = 3.6.4
# Sphinx - Open Source Search Server
# http://sphinxsearch.com/
[buildout] [buildout]
parts = sphinx parts = sphinx
extends = extends =
...@@ -5,13 +8,6 @@ extends = ...@@ -5,13 +8,6 @@ extends =
../mariadb/buildout.cfg ../mariadb/buildout.cfg
../zlib/buildout.cfg ../zlib/buildout.cfg
[sphinx-1.10-fix_nosigpipe.patch]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename}
md5sum = b606b2a267ebfbd009bb1e72e7a29462
download-only = true
filename = sphinx-1.10-fix_nosigpipe.patch
[sphinx-1.10-beta-snowball.patch] [sphinx-1.10-beta-snowball.patch]
recipe = hexagonit.recipe.download recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename} url = ${:_profile_base_location_}/${:filename}
...@@ -22,8 +18,8 @@ filename = sphinx-1.10-beta-snowball.patch ...@@ -22,8 +18,8 @@ filename = sphinx-1.10-beta-snowball.patch
[sphinx] [sphinx]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = http://www.sphinxsearch.com/files/sphinx-1.10-beta.tar.gz url = http://sphinxsearch.com/files/sphinx-2.0.1-beta.tar.gz
md5sum = 5b52ce9e93a73c66d37bc3a2402f14fa md5sum = 95c217d81d0b7a4ff73d5297318c3481
configure-options = configure-options =
--with-mysql --with-mysql
--with-mysql-includes=${mariadb:location}/include/mysql --with-mysql-includes=${mariadb:location}/include/mysql
...@@ -34,7 +30,6 @@ configure-options = ...@@ -34,7 +30,6 @@ configure-options =
--without-unixodbc --without-unixodbc
patch-options = -p1 patch-options = -p1
patches = patches =
${sphinx-1.10-fix_nosigpipe.patch:location}/${sphinx-1.10-fix_nosigpipe.patch:filename}
${sphinx-1.10-beta-snowball.patch:location}/${sphinx-1.10-beta-snowball.patch:filename} ${sphinx-1.10-beta-snowball.patch:location}/${sphinx-1.10-beta-snowball.patch:filename}
environment = environment =
CPPFLAGS=-I${zlib:location}/include -I${libexpat:location}/include CPPFLAGS=-I${zlib:location}/include -I${libexpat:location}/include
......
...@@ -17,8 +17,8 @@ filename = stunnel-4-hooks.py ...@@ -17,8 +17,8 @@ filename = stunnel-4-hooks.py
[stunnel-4] [stunnel-4]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
url = ftp://ftp.stunnel.org/stunnel/obsolete/4.x/stunnel-4.36.tar.gz url = ftp://ftp.stunnel.org/stunnel/stunnel-4.39.tar.gz
md5sum = 600a09b03798424842b24548ca1e4235 md5sum = 853739119a8364daea750154af6d7e79
pre-configure-hook = ${stunnel-4-hook-download:location}/${stunnel-4-hook-download:filename}:pre_configure_hook pre-configure-hook = ${stunnel-4-hook-download:location}/${stunnel-4-hook-download:filename}:pre_configure_hook
configure-options = configure-options =
--enable-ipv6 --enable-ipv6
......
...@@ -9,6 +9,18 @@ extends = ...@@ -9,6 +9,18 @@ extends =
parts = parts =
w3m w3m
versions = versions
find-links =
http://www.nexedi.org/static/packages/source/slapos.buildout/
[w3m-w3m.gcc.forward.compat.patch]
recipe = hexagonit.recipe.download
url =${:_profile_base_location_}/${:filename}
filename = w3m.gcc.forward.compat.patch
download-only = true
md5sum = 75422a6f7f671b3a6d9add6724cc0945
[w3m] [w3m]
recipe = hexagonit.recipe.cmmi recipe = hexagonit.recipe.cmmi
md5sum = 1b845a983a50b8dec0169ac48479eacc md5sum = 1b845a983a50b8dec0169ac48479eacc
...@@ -27,8 +39,18 @@ configure-options = ...@@ -27,8 +39,18 @@ configure-options =
--disable-external-uri-loader --disable-external-uri-loader
--disable-w3mmailer --disable-w3mmailer
patch-options =
-p1
patches =
${w3m-w3m.gcc.forward.compat.patch:location}/${w3m-w3m.gcc.forward.compat.patch:filename}
environment = environment =
PATH=${pkgconfig:location}/bin:%(PATH)s PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${openssl:location}/lib/pkgconfig PKG_CONFIG_PATH=${openssl:location}/lib/pkgconfig
CPPFLAGS=-I${ncurses:location}/include/ -I${zlib:location}/include/ CPPFLAGS=-I${ncurses:location}/include/ -I${zlib:location}/include/
LDFLAGS=-Wl,--as-needed -L${garbage-collector:location}/lib -Wl,-rpath=${garbage-collector:location}/lib -L${ncurses:location}/lib -Wl,-rpath=${ncurses:location}/lib -L${openssl:location}/lib -Wl,-rpath=${openssl:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib LDFLAGS=-Wl,--as-needed -L${garbage-collector:location}/lib -Wl,-rpath=${garbage-collector:location}/lib -L${ncurses:location}/lib -Wl,-rpath=${ncurses:location}/lib -L${openssl:location}/lib -Wl,-rpath=${openssl:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib
[versions]
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-005
diff --git a/istream.c b/istream.c
index 8967280..d4c788a 100644
--- a/istream.c
+++ b/istream.c
@@ -22,8 +22,8 @@
static void basic_close(int *handle);
static int basic_read(int *handle, char *buf, int len);
-static void file_close(struct file_handle *handle);
-static int file_read(struct file_handle *handle, char *buf, int len);
+static void file_close(struct w3m_file_handle *handle);
+static int file_read(struct w3m_file_handle *handle, char *buf, int len);
static int str_read(Str handle, char *buf, int len);
@@ -114,7 +114,7 @@ newFileStream(FILE * f, void (*closep) ())
stream = New(union input_stream);
init_base_stream(&stream->base, STREAM_BUF_SIZE);
stream->file.type = IST_FILE;
- stream->file.handle = New(struct file_handle);
+ stream->file.handle = New(struct w3m_file_handle);
stream->file.handle->f = f;
if (closep)
stream->file.handle->close = closep;
@@ -658,13 +658,13 @@ basic_read(int *handle, char *buf, int len)
}
static void
-file_close(struct file_handle *handle)
+file_close(struct w3m_file_handle *handle)
{
handle->close(handle->f);
}
static int
-file_read(struct file_handle *handle, char *buf, int len)
+file_read(struct w3m_file_handle *handle, char *buf, int len)
{
return fread(buf, 1, len, handle->f);
}
diff --git a/istream.h b/istream.h
index a220d8b..6d9736d 100644
--- a/istream.h
+++ b/istream.h
@@ -20,7 +20,7 @@ struct stream_buffer {
typedef struct stream_buffer *StreamBuffer;
-struct file_handle {
+struct w3m_file_handle {
FILE *f;
void (*close) ();
};
@@ -53,7 +53,7 @@ struct base_stream {
struct file_stream {
struct stream_buffer stream;
- struct file_handle *handle;
+ struct w3m_file_handle *handle;
char type;
char iseos;
int (*read) ();
...@@ -2,7 +2,7 @@ from setuptools import setup, find_packages ...@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
import glob import glob
import os import os
version = '0.6-dev' version = '0.16-dev'
name = 'slapos.cookbook' name = 'slapos.cookbook'
long_description = open("README.txt").read() + "\n" + \ long_description = open("README.txt").read() + "\n" + \
open("CHANGES.txt").read() + "\n" open("CHANGES.txt").read() + "\n"
...@@ -49,8 +49,13 @@ setup(name=name, ...@@ -49,8 +49,13 @@ setup(name=name,
'libcloud = slapos.recipe.libcloud:Recipe', 'libcloud = slapos.recipe.libcloud:Recipe',
'libcloudrequest = slapos.recipe.libcloudrequest:Recipe', 'libcloudrequest = slapos.recipe.libcloudrequest:Recipe',
'memcached = slapos.recipe.memcached:Recipe', 'memcached = slapos.recipe.memcached:Recipe',
'mysql = slapos.recipe.mysql:Recipe',
'nbdserver = slapos.recipe.nbdserver:Recipe', 'nbdserver = slapos.recipe.nbdserver:Recipe',
'nosqltestbed = slapos.recipe.nosqltestbed:NoSQLTestBed', 'nosqltestbed = slapos.recipe.nosqltestbed:NoSQLTestBed',
'osoeslaptraining = slapos.recipe.osoeslaptraining:Request',
'osoeslaptraining.request = slapos.recipe.osoeslaptraining:Request',
'osoeslaptraining.static = slapos.recipe.osoeslaptraining:Static',
'osoeslaptraining.simple = slapos.recipe.osoeslaptraining:Simple',
'proactive = slapos.recipe.proactive:Recipe', 'proactive = slapos.recipe.proactive:Recipe',
'sheepdogtestbed = slapos.recipe.sheepdogtestbed:SheepDogTestBed', 'sheepdogtestbed = slapos.recipe.sheepdogtestbed:SheepDogTestBed',
'siptester = slapos.recipe.siptester:SipTesterRecipe', 'siptester = slapos.recipe.siptester:SipTesterRecipe',
......
mysql
=========
Instantiates MySQL instance.
...@@ -34,6 +34,9 @@ import sys ...@@ -34,6 +34,9 @@ import sys
import zc.buildout import zc.buildout
import zc.recipe.egg import zc.recipe.egg
import ConfigParser import ConfigParser
import re
_isurl = re.compile('([a-zA-Z0-9+.-]+)://').match
# based on Zope2.utilities.mkzopeinstance.write_inituser # based on Zope2.utilities.mkzopeinstance.write_inituser
def Zope2InitUser(path, username, password): def Zope2InitUser(path, username, password):
...@@ -68,20 +71,27 @@ class Recipe(BaseSlapRecipe): ...@@ -68,20 +71,27 @@ class Recipe(BaseSlapRecipe):
self.getLocalIPv4Address(), 23000, 23060) self.getLocalIPv4Address(), 23000, 23060)
mysql_conf = self.installMysqlServer(self.getLocalIPv4Address(), 45678) mysql_conf = self.installMysqlServer(self.getLocalIPv4Address(), 45678)
user, password = self.installERP5() user, password = self.installERP5()
zodb_dir = os.path.join(self.data_root_directory, 'zodb')
self._createDirectory(zodb_dir) if self.parameter_dict.get("slap_software_type", "").lower() == "cluster":
zodb_root_path = os.path.join(zodb_dir, 'root.fs') # Site access is done by HAProxy
zope_access = self.installZope(ip=self.getLocalIPv4Address(), zope_access, site_access = self.installZopeCluster()
port=12000 + 1, name='zope_%s' % 1, else:
zodb_configuration_string=self.substituteTemplate( zope_access = self.installZopeStandalone()
self.getTemplateFilename('zope-zodb-snippet.conf.in'), site_access = zope_access
dict(zodb_root_path=zodb_root_path)), with_timerservice=True)
key, certificate = self.requestCertificate('Login Based Access') key, certificate = self.requestCertificate('Login Based Access')
apache_conf = dict( apache_conf = dict(
apache_login=self.installBackendApache(ip=self.getGlobalIPv6Address(), apache_login=self.installBackendApache(ip=self.getGlobalIPv6Address(),
port=13000, backend=zope_access, key=key, certificate=certificate)) port=13000, backend=site_access, key=key, certificate=certificate))
self.installERP5Site(user, password, zope_access, mysql_conf,
conversion_server_conf, memcached_conf, kumo_conf, self.site_id) default_bt5_list = []
if self.parameter_dict.get("flavour", "default") == 'configurator':
default_bt5_list = self.options.get("configurator_bt5_list", '').split()
self.installERP5Site(user, password, zope_access, mysql_conf,
conversion_server_conf, memcached_conf, kumo_conf,
self.site_id, default_bt5_list)
self.installTestRunner(ca_conf, mysql_conf, conversion_server_conf, self.installTestRunner(ca_conf, mysql_conf, conversion_server_conf,
memcached_conf, kumo_conf) memcached_conf, kumo_conf)
self.installTestSuiteRunner(ca_conf, mysql_conf, conversion_server_conf, self.installTestSuiteRunner(ca_conf, mysql_conf, conversion_server_conf,
...@@ -96,6 +106,87 @@ class Recipe(BaseSlapRecipe): ...@@ -96,6 +106,87 @@ class Recipe(BaseSlapRecipe):
)) ))
return self.path_list return self.path_list
def installZopeStandalone(self):
""" Install a single Zope instance without ZEO Server.
"""
zodb_dir = os.path.join(self.data_root_directory, 'zodb')
self._createDirectory(zodb_dir)
zodb_root_path = os.path.join(zodb_dir, 'root.fs')
thread_amount_per_zope = int(self.options.get(
'single_zope_thread_amount', 4))
return self.installZope(ip=self.getLocalIPv4Address(),
port=12000 + 1, name='zope_%s' % 1,
zodb_configuration_string=self.substituteTemplate(
self.getTemplateFilename('zope-zodb-snippet.conf.in'),
dict(zodb_root_path=zodb_root_path)), with_timerservice=True,
thread_amount=thread_amount_per_zope)
def installZopeCluster(self):
""" Install ERP5 using ZEO Cluster
"""
site_check_path = '/%s/getId' % self.site_id
thread_amount_per_zope = int(self.options.get(
'cluster_zope_thread_amount', 1))
activity_node_amount = 2
user_node_amount = 2
ip = self.getLocalIPv4Address()
storage_dict = self._requestZeoFileStorage('Zeo Server 1', 'main')
zeo_conf = self.installZeo(ip)
tidstorage_config = dict(host=ip, port='6001')
mount_point = '/'
check_path = '/erp5/account_module'
known_tid_storage_identifier_dict = {}
known_tid_storage_identifier_dict[
(((storage_dict['ip'],storage_dict['port']),), storage_dict['storage_name'])
] = (zeo_conf[storage_dict['storage_name']]['path'], check_path or mount_point)
zodb_configuration_string = self.substituteTemplate(
self.getTemplateFilename('zope-zeo-snippet.conf.in'), dict(
storage_name=storage_dict['storage_name'],
address='%s:%s' % (storage_dict['ip'], storage_dict['port']),
mount_point=mount_point
))
zope_port = 12000
# One Distribution Node (Single Thread Always)
zope_port += 1
self.installZope(ip, zope_port, 'zope_distribution', with_timerservice=True,
zodb_configuration_string=zodb_configuration_string,
tidstorage_config=tidstorage_config)
# Activity Nodes (Single Thread Always)
for i in range(activity_node_amount):
zope_port += 1
self.installZope(ip, zope_port, 'zope_activity_%s' % i,
with_timerservice=True,
zodb_configuration_string=zodb_configuration_string,
tidstorage_config=tidstorage_config)
# Four Web Page Nodes (Human access)
login_url_list = []
for i in range(user_node_amount):
zope_port += 1
login_url_list.append(self.installZope(ip, zope_port,
'zope_login_%s' % i, with_timerservice=False,
zodb_configuration_string=zodb_configuration_string,
tidstorage_config=tidstorage_config,
thread_amount=thread_amount_per_zope))
login_haproxy = self.installHaproxy(ip, 15001, 'login',
site_check_path, login_url_list)
self.installTidStorage(tidstorage_config['host'],
tidstorage_config['port'],
known_tid_storage_identifier_dict, 'http://' + login_haproxy)
return login_url_list[-1], login_haproxy
def _requestZeoFileStorage(self, server_name, storage_name): def _requestZeoFileStorage(self, server_name, storage_name):
"""Local, slap.request compatible, call to ask for filestorage on Zeo """Local, slap.request compatible, call to ask for filestorage on Zeo
...@@ -478,10 +569,50 @@ class Recipe(BaseSlapRecipe): ...@@ -478,10 +569,50 @@ class Recipe(BaseSlapRecipe):
self._createDirectory(os.path.join(self.erp5_directory, directory)) self._createDirectory(os.path.join(self.erp5_directory, directory))
self._createDirectory(os.path.join(self.erp5_directory, 'etc', self._createDirectory(os.path.join(self.erp5_directory, 'etc',
'package-includes')) 'package-includes'))
# Symlink to BT5 repositories defined in instance config.
# Those paths will eventually end up in the ZODB, and having symlinks
# inside the XXX makes it possible to reuse such ZODB with another software
# release[ version].
# Note: this path cannot be used for development, it's really just a
# read-only repository.
repository_path = os.path.join(self.var_directory, "bt5_repository")
if not os.path.isdir(repository_path):
os.mkdir(repository_path)
self.path_list.append(repository_path)
self.bt5_repository_list = []
append = self.bt5_repository_list.append
for repository in self.options.get('bt5_repository_list', '').split():
repository = repository.strip()
if not repository:
continue
if _isurl(repository) and not repository.startswith("file://"):
# XXX: assume it's a valid URL
append(repository)
continue
if repository.startswith('file://'):
repository = repository.replace('file://', '', '')
if os.path.isabs(repository):
repo_id = hashlib.sha1(repository).hexdigest()
link = os.path.join(repository_path, repo_id)
if os.path.lexists(link):
if not os.path.islink(link):
raise zc.buildout.UserError(
'Target link already %r exists but it is not link' % link)
os.unlink(link)
os.symlink(repository, link)
self.logger.debug('Created link %r -> %r' % (link, repository_path))
# Always provide a URL-Type
append("file://" + link)
return user, password return user, password
def installERP5Site(self, user, password, zope_access, mysql_conf, def installERP5Site(self, user, password, zope_access, mysql_conf,
conversion_server_conf=None, memcached_conf=None, kumo_conf=None, erp5_site_id='erp5'): conversion_server_conf=None, memcached_conf=None, kumo_conf=None,
erp5_site_id='erp5', default_bt5_list=[]):
""" Create a script controlled by supervisor, which creates a erp5 """ Create a script controlled by supervisor, which creates a erp5
site on current available zope and mysql environment""" site on current available zope and mysql environment"""
conversion_server = None conversion_server = None
...@@ -494,19 +625,18 @@ class Recipe(BaseSlapRecipe): ...@@ -494,19 +625,18 @@ class Recipe(BaseSlapRecipe):
kumo_conf = {} kumo_conf = {}
# XXX Conversion server and memcache server coordinates are not relevant # XXX Conversion server and memcache server coordinates are not relevant
# for pure site creation. # for pure site creation.
https_connection_url = "http://%s:%s@%s/" % (user, password, zope_access)
mysql_connection_string = "%(mysql_database)s@%(ip)s:%(tcp_port)s %(mysql_user)s %(mysql_password)s" % mysql_conf mysql_connection_string = "%(mysql_database)s@%(ip)s:%(tcp_port)s %(mysql_user)s %(mysql_password)s" % mysql_conf
# XXX URL list vs. repository + list of bt5 names? bt5_list = self.parameter_dict.get("bt5_list", "").split() or default_bt5_list
bt5_list = self.parameter_dict.get("bt5_list", "").split() bt5_repository_list = self.parameter_dict.get("bt5_repository_list", "").split() \
bt5_repository_list = self.parameter_dict.get("bt5_repository_list", "").split() or getattr(self, 'bt5_repository_list', [])
self.path_list.extend(zc.buildout.easy_install.scripts([('erp5_update', self.path_list.extend(zc.buildout.easy_install.scripts([('erp5_update',
__name__ + '.erp5', 'updateERP5')], self.ws, __name__ + '.erp5', 'updateERP5')], self.ws,
sys.executable, self.wrapper_directory, sys.executable, self.wrapper_directory,
arguments=[erp5_site_id, arguments=[erp5_site_id,
mysql_connection_string, mysql_connection_string,
https_connection_url, [user, password, zope_access],
memcached_conf.get('memcached_url'), memcached_conf.get('memcached_url'),
conversion_server, conversion_server,
kumo_conf.get("kumo_address"), kumo_conf.get("kumo_address"),
......
...@@ -24,84 +24,356 @@ ...@@ -24,84 +24,356 @@
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
# #
############################################################################## ##############################################################################
import time
import urllib import httplib
import xmlrpclib import json
import base64
import socket import socket
import time
def updateERP5(args):
# FIXME Use a dict
site_id = args[0]
mysql_string = args[1]
base_url = args[2]
memcached_provider = args[3]
conversion_server = args[4]
kumo_provider = args[5]
bt5_list = args[6]
bt5_repository_list = []
if len(args) > 7:
bt5_repository_list = args[7]
if len(bt5_list) > 0 and len(bt5_repository_list) == 0: class ERP5Updater(object):
bt5_repository_list = ["http://www.erp5.org/dists/snapshot/bt5"]
erp5_catalog_storage = "erp5_mysql_innodb_catalog" erp5_catalog_storage = "erp5_mysql_innodb_catalog"
business_template_setup_finished = 0 header_dict = {}
external_service_assertion = 1
update_script_id = "ERP5Site_assertExternalServiceList" sleeping_time = 120
sleep = 120
while True: def __init__(self, user, password, host,
site_id, mysql_url, memcached_address,
conversion_server_address, persistent_cache_address,
bt5_list, bt5_repository_list):
authentication_string = '%s:%s' % (user, password)
base64string = base64.encodestring(authentication_string).strip()
self.header_dict['Authorization'] = 'Basic %s' % base64string
self.host = host
self.site_id = site_id
self.business_template_repository_list = bt5_repository_list
self.business_template_list = bt5_list
self.memcached_address = memcached_address
self.persintent_cached_address = persistent_cache_address
self.mysql_url = mysql_url
host, port = conversion_server_address.split(":")
self.conversion_server_address = host
self.conversion_server_port = int(port)
def log(self, level, message):
date = time.strftime("%a, %d %b %Y %H:%M:%S +0000")
print "%s - %s : %s" % (date, level, message)
def getConnectionToZope(self):
return httplib.HTTPConnection(self.host)
def GET(self, path):
zope_connection = self.getConnectionToZope()
self.log("INFO", "GET ZOPE: %s, PATH: %s" % (self.host, path))
try: try:
proxy = xmlrpclib.ServerProxy(base_url) zope_connection.request('GET', path, headers=self.header_dict)
print "Adding site at %r" % base_url result = zope_connection.getresponse()
if proxy.isERP5SitePresent() == False: data = result.read()
url = '%s/manage_addProduct/ERP5/manage_addERP5Site' % base_url status = result.status
result = urllib.urlopen(url, urllib.urlencode({
"id": site_id,
"erp5_catalog_storage": erp5_catalog_storage,
"erp5_sql_connection_string": mysql_string,
"cmf_activity_sql_connection_string": mysql_string, }))
print "ERP5 Site creation output: %s" % result.read()
if not business_template_setup_finished:
if proxy.isERP5SitePresent() == True:
print "Start to set initial business template setup."
# Update URL to ERP5 Site
erp5 = xmlrpclib.ServerProxy("%s/%s" % (base_url, site_id),
allow_none=1)
repository_list = erp5.portal_templates.getRepositoryList()
if len(bt5_repository_list) > 0 and \
set(bt5_repository_list) != set(repository_list):
erp5.portal_templates.\
updateRepositoryBusinessTemplateList(bt5_repository_list, None)
installed_bt5_list =\
erp5.portal_templates.getInstalledBusinessTemplateTitleList()
for bt5 in bt5_list:
if bt5 not in installed_bt5_list:
erp5.portal_templates.\
installBusinessTemplatesFromRepositories([bt5])
repository_set = set(erp5.portal_templates.getRepositoryList())
installed_bt5_list = erp5.portal_templates.\
getInstalledBusinessTemplateTitleList()
if (set(repository_set) == set(bt5_repository_list)) and \
len([i for i in bt5_list if i not in installed_bt5_list]) == 0:
print "Repositories updated and business templates installed."
business_template_setup_finished = 1
if external_service_assertion:
url = "%s/%s/%s" % (base_url, site_id, update_script_id)
result = urllib.urlopen(url, urllib.urlencode({
"memcached" : memcached_provider,
"kumo" : kumo_provider,
"conversion_server" : conversion_server,})).read()
external_service_assertion = not (result == "True")
except IOError:
print "Unable to create the ERP5 Site!"
except socket.error, e: except socket.error, e:
print "Unable to connect to ZOPE! %s" % e data = "Unable to connect to %s (socket.error: %s)" % (self.host, e)
except xmlrpclib.Fault, e: self.log("ERROR", data)
print "XMLRPC Fault: %s" % e status = 0
time.sleep(sleep) finally:
zope_connection.close()
return status, data
def POST(self, path, post_dict):
zope_connection = self.getConnectionToZope()
param_list = []
for item in post_dict:
value = post_dict[item]
if isinstance(value, type([])):
for v in value:
param_list.append("%s:list=%s" % (item, v))
else:
param_list.append("%s=%s" % (item, value))
params = "&".join(param_list)
self.log("INFO", "GET ZOPE: %s, PATH: %s, PARAMS: %s" % \
(self.host, path, params))
try:
zope_connection.request('POST', path, params, headers=self.header_dict)
result = zope_connection.getresponse()
data = result.read()
status = result.status
except socket.error, e:
data = "Unable to connect to %s (socket.error: %s)" % (self.host, e)
self.log("ERROR", data)
status = 0
finally:
zope_connection.close()
return status, data
def isERP5Present(self):
status, data = self.GET("/isERP5SitePresent")
self.log("DEBUG", "isERP5Present : %s, %s" % (status, data))
return status == 200 and not (data == "False")
def loadSystemSignatureDict(self):
base_path = \
"/%s/portal_introspections/getSystemSignatureAsJSON" % self.site_id
status, data = self.GET(base_path)
if status == 200:
data_dict = json.loads(data)
self.system_signature_dict = data_dict
elif status == 404:
# Unable to find portal_introspection, this means erp5_base is not
# installed yet.
self.system_signature_dict = None
self.log("DEBUG", "Unable to find portal_introspection, this means " + \
"erp5_base is not installed yet.")
else:
self.log("ERROR", "Unable to get SystemSignature to %s (status: %s, data: %s)" % \
(self.host, status, data))
def getSystemSignatureDict(self, item=None, default=None):
system_signature_dict = getattr(self, 'system_signature_dict', None)
if system_signature_dict is not None and item is not None:
return system_signature_dict.get(item, default)
return system_signature_dict
def getMissingBusinessTemplateRepositoryList(self):
found_list = self.getSystemSignatureDict(
"business_template_repository_list", [])
return [i for i in self.business_template_repository_list
if i not in found_list]
def getMissingBusinessTemplateList(self):
bt5_dict = self.getSystemSignatureDict("business_template_dict", [])
found_bt5_list = bt5_dict.keys()
return [bt for bt in self.business_template_list\
if bt not in found_bt5_list]
def isBusinessTemplateUpdated(self):
return len(self.getMissingBusinessTemplateList()) == 0
def isBusinessTemplateRepositoryUpdated(self):
return len(self.getMissingBusinessTemplateRepositoryList()) == 0
def updateBusinessTemplateList(self):
""" Update Business Template Configuration, including the repositories
"""
if not self.isBusinessTemplateUpdated():
# Before update the business templates, it is required to make
# sure the repositories are updated.
if not self.isBusinessTemplateRepositoryUpdated():
# Require to update Business template Repository
repository_list = self.getSystemSignatureDict(
"business_template_repository_list", [])
repository_list.extend(self.getMissingBusinessTemplateRepositoryList())
self._setRepositoryList(repository_list)
# Require to update Business template
for bt in self.getMissingBusinessTemplateList():
self._installBusinessTemplateList([bt])
return True
return False
def _setRepositoryList(self, repository_list):
""" Set repository list on portal_templates """
set_path = "/%s/portal_templates/updateRepositoryBusinessTemplateList" % self.site_id
self.POST(set_path, {"repository_list": repository_list})
def _installBusinessTemplateList(self, name_list, update_catalog=False):
""" Install a Business Template on Remote ERP5 setup """
set_path = "/%s/portal_templates/installBusinessTemplatesFromRepositories" % self.site_id
self.POST(set_path, {"template_list": name_list,
"only_newer": 1,
"update_catalog": int(update_catalog)})
def _createActiveSystemPreference(self):
""" Assert that at least one enabled System Preference is present on
the erp5 instance.
"""
self.log("INFO", "Try to create New System Preference into ERP5!")
path = "/%s/portal_preferences/createActiveSystemPreference" % self.site_id
status, data = self.POST(path, {})
if status != 200:
self.log("ERROR", "Unable to create System Preference, an error ocurred %s." % data)
def updateConversionServer(self):
""" Update Conversion server Configuration """
external_connection_dict = self.getSystemSignatureDict("external_connection_dict")
host_key = None
port_key = None
for external_connection in external_connection_dict:
# Search for Configurated value for Conversion Server
if external_connection.endswith("getPreferredOoodocServerAddress"):
host_key = external_connection
elif external_connection.endswith("getPreferredOoodocServerPortNumber"):
port_key = external_connection
if None not in [host_key, port_key]:
break
if None in [host_key, port_key]:
self.log("ERROR", "Unable to find the Active System Preference to Update!")
self._createActiveSystemPreference()
return True
is_updated = self._assertAndUpdateDocument(host_key, self.conversion_server_address,
"setPreferredOoodocServerAddress")
is_updated = is_updated or self._assertAndUpdateDocument(port_key,
self.conversion_server_port,
"setPreferredOoodocServerPortNumber")
return is_updated
def updateMemcached(self):
# Assert Memcached configuration
self._assertAndUpdateDocument(
"portal_memcached/default_memcached_plugin/getUrlString",
self.memcached_address,
"setUrlString")
# Assert Persistent cache configuration (Kumofs)
self._assertAndUpdateDocument(
"portal_memcached/persistent_memcached_plugin/getUrlString",
self.persintent_cached_address,
"setUrlString")
def _assertAndUpdateDocument(self, key, expected_value, update_method):
external_connection_dict = self.getSystemSignatureDict("external_connection_dict")
# Assert Memcached configuration
found_address = external_connection_dict.get(key)
if found_address != expected_value:
document_path = "/".join(key.split("/")[:-1])
self.log("INFO",
"Document require update at %s (Found: %s, Expected: %s)" % \
(document_path, found_address, expected_value))
set_path = "/%s/%s/%s" % (self.site_id, document_path, update_method)
self.POST(set_path, {"value": expected_value})
return True
return False
def updateMysql(self):
""" This API is not implemented yet, because it is not needed to
update Mysql Connection on ERP5 Sites.
"""
pass
def updatePortalActivities(self):
""" This API is not implemented yet, because it is not needed for
a single instance configuration. This method should define which
instances will handle activities, which one will distribute
activities
"""
pass
def updateERP5Site(self):
if not self.isERP5Present():
url = '/manage_addProduct/ERP5/manage_addERP5Site'
self.POST(url, {
"id": self.site_id,
"erp5_catalog_storage": self.erp5_catalog_storage,
"erp5_sql_connection_string": self.mysql_url,
"cmf_activity_sql_connection_string": self.mysql_url})
return True
return False
def _hasActivityPresent(self):
activity_dict = self.getSystemSignatureDict("activity_dict")
if activity_dict["total"] > 0:
return True
def _hasFailureActivity(self):
activity_dict = self.getSystemSignatureDict("activity_dict")
if activity_dict["failure"] > 0:
return True
def _updatePreRequiredBusinessTemplateList(self):
""" Update only the first part of bt5."""
# This list contains the minimal set of bt5 required to install
# portal_introspections. Move portal_introspection to erp5_core
# can remove this set.
pre_required_business_template_list = [i for i in self.business_template_list\
if i.startswith("erp5_full_text") or i == "erp5_base"]
if len(self.business_template_repository_list) > 0 and \
len(pre_required_business_template_list):
pre_required_business_template_list.insert(0, "erp5_core_proxy_field_legacy")
self._setRepositoryList(self.business_template_repository_list)
time.sleep(30)
for bt in pre_required_business_template_list:
update_catalog = bt.endswith("_catalog")
self._installBusinessTemplateList([bt], update_catalog)
else:
self.log("ERROR", "Unable to install erp5_base, it is not on your " +\
"requested business templates list. Once it is installed " +\
"setup will continue")
def run(self):
""" Keep running until kill"""
while 1:
time.sleep(30)
if not self.updateERP5Site():
self.loadSystemSignatureDict()
if self.getSystemSignatureDict() is None:
self.log("INFO", "The erp5_base is not installed yet, trying to " +\
"install it before continue.")
self._updatePreRequiredBusinessTemplateList()
time.sleep(60)
continue
if self._hasActivityPresent():
self.log("DEBUG", "Waiting for activities on ERP5...")
if self._hasFailureActivity():
self.log("ERROR", "Update progress found " +\
"Failure activities and it will not progress until " +\
" activites issue be solved")
continue
if self.updateBusinessTemplateList():
continue
self.updateMemcached()
if self.updateConversionServer():
# If update Conversion Server adds a bit more delay to continue
# To wait for activiies.
time.sleep(60)
continue
time.sleep(self.sleeping_time)
def updateERP5(argument_list):
site_id = argument_list[0]
mysql_url = argument_list[1]
user, password, host = argument_list[2]
memcached_address = argument_list[3]
conversion_server_address = argument_list[4]
persistent_cache_provider = argument_list[5]
bt5_list = argument_list[6]
bt5_repository_list = []
if len(argument_list) > 7:
bt5_repository_list = argument_list[7]
if len(bt5_list) > 0 and len(bt5_repository_list) == 0:
bt5_repository_list = ["http://www.erp5.org/dists/snapshot/bt5"]
erp5_upgrader = ERP5Updater(
user=user,
password=password,
host=host,
site_id=site_id,
mysql_url=mysql_url,
memcached_address=memcached_address,
conversion_server_address=conversion_server_address,
persistent_cache_address=persistent_cache_provider,
bt5_list=bt5_list,
bt5_repository_list=bt5_repository_list)
erp5_upgrader.run()
...@@ -28,6 +28,9 @@ plugin-load = ha_innodb_plugin.so ...@@ -28,6 +28,9 @@ plugin-load = ha_innodb_plugin.so
#innodb_log_file_size = 256M #innodb_log_file_size = 256M
#innodb_log_buffer_size = 8M #innodb_log_buffer_size = 8M
# very important to allow parallel indexing
innodb_locks_unsafe_for_binlog = 1
# Some dangerous settings you may want to uncomment if you only want # Some dangerous settings you may want to uncomment if you only want
# performance or less disk access. Useful for unit tests. # performance or less disk access. Useful for unit tests.
#innodb_flush_log_at_trx_commit = 0 #innodb_flush_log_at_trx_commit = 0
......
...@@ -46,45 +46,49 @@ class SlapOSControler(object): ...@@ -46,45 +46,49 @@ class SlapOSControler(object):
'reference': config['computer_id'], 'reference': config['computer_id'],
'software_root': config['software_root']})) 'software_root': config['software_root']}))
def runSoftwareRelease(self, config, environment, process_group_pid_set=None): def runSoftwareRelease(self, config, environment, process_group_pid_set=None,
stdout=None, stderr=None):
print "SlapOSControler.runSoftwareRelease" print "SlapOSControler.runSoftwareRelease"
while True: cpu_count = os.sysconf("SC_NPROCESSORS_ONLN")
cpu_count = os.sysconf("SC_NPROCESSORS_ONLN") os.putenv('MAKEFLAGS', '-j%s' % cpu_count)
os.putenv('MAKEFLAGS', '-j%s' % cpu_count) os.environ['PATH'] = environment['PATH']
os.environ['PATH'] = environment['PATH'] slapgrid = subprocess.Popen([config['slapgrid_software_binary'], '-v', '-c',
stdout = open(os.path.join( #'--buildout-parameter',"'-U -N' -o",
config['instance_root'],'.runSoftwareRelease_out'), config['slapos_config']],
'w+') stdout=stdout, stderr=stderr,
stderr = open(os.path.join( close_fds=True, preexec_fn=os.setsid)
config['instance_root'],'.runSoftwareRelease_err'), process_group_pid_set.add(slapgrid.pid)
'w+') slapgrid.wait()
slapgrid = subprocess.Popen([config['slapgrid_software_binary'], '-v', '-c', stdout.seek(0)
#'--buildout-parameter',"'-U -N' -o", stderr.seek(0)
config['slapos_config']], process_group_pid_set.remove(slapgrid.pid)
stdout=stdout, stderr=stderr, status_dict = {'status_code':slapgrid.returncode,
close_fds=True, preexec_fn=os.setsid) 'stdout':stdout.read(),
process_group_pid_set.add(slapgrid.pid) 'stderr':stderr.read()}
slapgrid.wait() stdout.close()
stdout.seek(0) stderr.close()
stderr.seek(0) return status_dict
process_group_pid_set.remove(slapgrid.pid)
status_dict = {'status_code':slapgrid.returncode,
'stdout':stdout.read(),
'stderr':stderr.read()}
stdout.close()
stderr.close()
return status_dict
def runComputerPartition(self, config, process_group_pid_set=None): def runComputerPartition(self, config, environment,
process_group_pid_set=None,
stdout=None, stderr=None):
print "SlapOSControler.runSoftwareRelease" print "SlapOSControler.runSoftwareRelease"
slap = slapos.slap.slap() slap = slapos.slap.slap()
slap.registerOpenOrder().request(self.software_profile, slap.registerOpenOrder().request(self.software_profile,
partition_reference='testing partition', partition_reference='testing partition',
partition_parameter_kw=config['instance_dict']) partition_parameter_kw=config['instance_dict'])
slapgrid = subprocess.Popen([config['slapgrid_partition_binary'], slapgrid = subprocess.Popen([config['slapgrid_partition_binary'],
config['slapos_config'], '-c', '-v'], close_fds=True, preexec_fn=os.setsid) config['slapos_config'], '-c', '-v'],
stdout=stdout, stderr=stderr,
close_fds=True, preexec_fn=os.setsid)
process_group_pid_set.add(slapgrid.pid) process_group_pid_set.add(slapgrid.pid)
slapgrid.wait() slapgrid.wait()
stdout.seek(0)
stderr.seek(0)
process_group_pid_set.remove(slapgrid.pid) process_group_pid_set.remove(slapgrid.pid)
if slapgrid.returncode != 0: status_dict = {'status_code':slapgrid.returncode,
raise ValueError('Slapgrid instance failed') 'stdout':stdout.read(),
'stderr':stderr.read()}
stdout.close()
stderr.close()
return status_dict
...@@ -98,7 +98,7 @@ class Recipe(BaseSlapRecipe): ...@@ -98,7 +98,7 @@ class Recipe(BaseSlapRecipe):
partition_reference=CONFIG['partition_reference'], partition_reference=CONFIG['partition_reference'],
environment=dict(PATH=os.environ['PATH']), environment=dict(PATH=os.environ['PATH']),
vcs_authentication_list=eval(self.parameter_dict.get( vcs_authentication_list=eval(self.parameter_dict.get(
'vcs_authentication_list')), 'vcs_authentication_list', 'None')),
) )
])) ]))
......
...@@ -55,6 +55,16 @@ def safeRpcCall(function, *args): ...@@ -55,6 +55,16 @@ def safeRpcCall(function, *args):
time.sleep(retry) time.sleep(retry)
retry += retry >> 1 retry += retry >> 1
def getInputOutputFileList(config, command_name):
stdout = open(os.path.join(
config['instance_root'],'.%s_out' % command_name),
'w+')
stdout.write("%s\n" % command_name)
stderr = open(os.path.join(
config['instance_root'],'.%s_err' % command_name),
'w+')
return (stdout, stderr)
slapos_controler = None slapos_controler = None
def run(args): def run(args):
...@@ -168,42 +178,25 @@ branch = %(branch)s ...@@ -168,42 +178,25 @@ branch = %(branch)s
revision=repository_revision.split('-')[1]) revision=repository_revision.split('-')[1])
updater.checkout() updater.checkout()
# Now prepare the installation of SlapOS # Now prepare the installation of SlapOS and create instance
slapos_controler = SlapOSControler(config, slapos_controler = SlapOSControler(config,
process_group_pid_set=process_group_pid_set) process_group_pid_set=process_group_pid_set)
# this should be always true later, but it is too slow for now for method_name in ("runSoftwareRelease", "runComputerPartition"):
status_dict = slapos_controler.runSoftwareRelease(config, stdout, stderr = getInputOutputFileList(config, method_name)
environment=config['environment'], slapos_method = getattr(slapos_controler, method_name)
process_group_pid_set=process_group_pid_set, status_dict = slapos_method(config,
) environment=config['environment'],
process_group_pid_set=process_group_pid_set,
stdout=stdout, stderr=stderr
)
if status_dict['status_code'] != 0:
break
if status_dict['status_code'] != 0: if status_dict['status_code'] != 0:
safeRpcCall(master.reportTaskFailure, safeRpcCall(master.reportTaskFailure,
test_result_path, status_dict, config['test_node_title']) test_result_path, status_dict, config['test_node_title'])
retry_software = True retry_software = True
continue continue
# create instances, it should take some seconds only
slapos_controler.runComputerPartition(config,
process_group_pid_set=process_group_pid_set)
# update repositories downloaded by buildout. Later we should get
# from master a list of repositories
# The section below seems useless since we override buildout
# configuration to use local checkouts
#repository_path_list = glob(os.path.join(config['software_root'],
#'*', 'parts', 'git_repository', '*'))
#assert len(repository_path_list) >= 0
#for repository_path in repository_path_list:
#updater = Updater(repository_path, git_binary=config['git_binary'])
#updater.checkout()
#if os.path.split(repository_path)[-1] == repository_name:
## redo checkout with good revision, the previous one is used
## to pull last code
#updater = Updater(repository_path, git_binary=config['git_binary'],
#revision=revision)
#updater.checkout()
partition_path = os.path.join(config['instance_root'], partition_path = os.path.join(config['instance_root'],
config['partition_reference']) config['partition_reference'])
run_test_suite_path = os.path.join(partition_path, 'bin', run_test_suite_path = os.path.join(partition_path, 'bin',
...@@ -249,5 +242,4 @@ branch = %(branch)s ...@@ -249,5 +242,4 @@ branch = %(branch)s
if os.path.exists(supervisord_pid_file): if os.path.exists(supervisord_pid_file):
os.kill(int(open(supervisord_pid_file).read().strip()), signal.SIGTERM) os.kill(int(open(supervisord_pid_file).read().strip()), signal.SIGTERM)
except: except:
pass pass
\ No newline at end of file
...@@ -220,9 +220,10 @@ class Recipe(BaseSlapRecipe): ...@@ -220,9 +220,10 @@ class Recipe(BaseSlapRecipe):
self.substituteTemplate(template_filename, self.substituteTemplate(template_filename,
stunnel_conf)) stunnel_conf))
wrapper = zc.buildout.easy_install.scripts([('stunnel', wrapper = zc.buildout.easy_install.scripts([('stunnel',
'slapos.recipe.librecipe.execute', 'execute')], self.ws, sys.executable, 'slapos.recipe.librecipe.execute', 'execute_wait')], self.ws,
self.wrapper_directory, arguments=[ sys.executable, self.wrapper_directory, arguments=[
self.options['stunnel_binary'].strip(), stunnel_conf_path] [self.options['stunnel_binary'].strip(), stunnel_conf_path],
[ca_certificate, key]]
)[0] )[0]
self.path_list.append(wrapper) self.path_list.append(wrapper)
...@@ -267,4 +268,4 @@ class Recipe(BaseSlapRecipe): ...@@ -267,4 +268,4 @@ class Recipe(BaseSlapRecipe):
config['kumo_gateway_port']), config['kumo_gateway_port']),
kumo_gateway_ip=config['kumo_gateway_ip'], kumo_gateway_ip=config['kumo_gateway_ip'],
kumo_gateway_port=config['kumo_gateway_port'], kumo_gateway_port=config['kumo_gateway_port'],
) )
\ No newline at end of file
...@@ -30,113 +30,291 @@ from slapos.recipe.librecipe import BaseSlapRecipe ...@@ -30,113 +30,291 @@ from slapos.recipe.librecipe import BaseSlapRecipe
import subprocess import subprocess
import binascii import binascii
import random import random
import zc.buildout
import pkg_resources import pkg_resources
import ConfigParser
import hashlib
class Recipe(BaseSlapRecipe): class Recipe(BaseSlapRecipe):
def _install(self): def _install(self):
"""
Set the connection dictionnary for the computer partition and create a list
of paths to the different wrappers
Parameters : none
Returns : List path_list
"""
self.path_list = []
self.requirements, self.ws = self.egg.working_set()
self.cron_d = self.installCrond()
self.ca_conf = self.installCertificateAuthority()
self.key_path, self.certificate_path = self.requestCertificate('noVNC')
kvm_conf = self.installKvm(vnc_ip = self.getLocalIPv4Address())
vnc_port = 5900 + kvm_conf['vnc_display']
noVNC_conf = self.installNoVnc(source_ip = self.getGlobalIPv6Address(),
source_port = 6080,
target_ip = kvm_conf['vnc_ip'],
target_port = vnc_port,
python_path = kvm_conf['python_path'])
self.linkBinary()
self.computer_partition.setConnectionDict(dict(
url = "https://[%s]:%s/vnc.html?host=[%s]&port=%s&encrypt=1" % (noVNC_conf['source_ip'],
noVNC_conf['source_port'],
noVNC_conf['source_ip'],
noVNC_conf['source_port']
),
password = kvm_conf['vnc_passwd']))
return self.path_list
#Get the IP list def installKvm(self, vnc_ip):
"""
Create kvm configuration dictionnary and instanciate a wrapper for kvm and
kvm controller
Parameters : IP the vnc server is listening on
Returns : Dictionnary kvm_conf
"""
kvm_conf = dict(vnc_ip = vnc_ip)
connection_found = False connection_found = False
ip = self.getGlobalIPv6Address() for tap_interface, dummy in self.parameter_dict['ip_list']:
for tap, dummy in self.parameter_dict['ip_list']:
# Get an ip associated to a tap interface # Get an ip associated to a tap interface
if tap: if tap_interface:
connection_found = True connection_found = True
if not connection_found: if not connection_found:
raise NotImplementedError("Do not support ip without tap interface") raise NotImplementedError("Do not support ip without tap interface")
kvm_conf['tap_interface'] = tap_interface
# Disk path # Disk path
disk_path = os.path.join(self.data_root_directory, 'virtual.qcow2') kvm_conf['disk_path'] = os.path.join(self.data_root_directory,
socket_path = os.path.join(self.var_directory, 'qmp_socket') 'virtual.qcow2')
kvm_conf['socket_path'] = os.path.join(self.var_directory, 'qmp_socket')
# XXX Weak password # XXX Weak password
vnc_passwd = binascii.hexlify(os.urandom(4)) ##XXX -Vivien: add an option to generate one password for all instances
# and/or to input it yourself
kvm_conf['vnc_passwd'] = binascii.hexlify(os.urandom(4))
#XXX pid_file path, database_path and xml path #XXX pid_file path, database_path, path to python binary and xml path
pid_file_path = os.path.join(self.run_directory, 'pid_file') kvm_conf['pid_file_path'] = os.path.join(self.run_directory, 'pid_file')
database_path = os.path.join(self.data_root_directory, 'slapmonitor_database') kvm_conf['database_path'] = os.path.join(self.data_root_directory,
'slapmonitor_database')
kvm_conf['python_path'] = sys.executable
kvm_conf['qemu_path'] = self.options['qemu_path']
#xml_path = os.path.join(self.var_directory, 'slapreport.xml' ) #xml_path = os.path.join(self.var_directory, 'slapreport.xml' )
# Create disk if needed # Create disk if needed
if not os.path.exists(disk_path): if not os.path.exists(kvm_conf['disk_path']):
retcode = subprocess.call(["%s create -f qcow2 %s %iG" % ( retcode = subprocess.call(["%s create -f qcow2 %s %iG" % (
self.options['qemu_img_path'], disk_path, self.options['qemu_img_path'], kvm_conf['disk_path'],
int(self.options['disk_size']))], shell=True) int(self.options['disk_size']))], shell=True)
if retcode != 0: if retcode != 0:
raise OSError, "Disk creation failed!" raise OSError, "Disk creation failed!"
# Instanciate KVM
kvm_config = {}
# Options nbd_ip and nbd_port are provided by slapos master # Options nbd_ip and nbd_port are provided by slapos master
kvm_config.update(self.options) kvm_conf['nbd_ip'] = self.parameter_dict['nbd_ip']
#raise NotImplementedError("%s" % self.parameter_dict) kvm_conf['nbd_port'] = self.parameter_dict['nbd_port']
kvm_config['vnc_ip'] = ip
kvm_config['tap_interface'] = tap
kvm_config['nbd_ip'] = self.parameter_dict['nbd_ip']
kvm_config['nbd_port'] = self.parameter_dict['nbd_port']
#XXX
kvm_config['pid_file'] = pid_file_path
kvm_config['image'] = disk_path
# First octet has to represent a locally administered address # First octet has to represent a locally administered address
octet_list = [254] + [random.randint(0x00, 0xff) for x in range(5)] octet_list = [254] + [random.randint(0x00, 0xff) for x in range(5)]
kvm_config['mac_address'] = ':'.join(['%02x' % x for x in octet_list]) kvm_conf['mac_address'] = ':'.join(['%02x' % x for x in octet_list])
kvm_config['qmp_socket'] = socket_path
kvm_config['hostname'] = "slaposkvm"
kvm_wrapper_template_location = pkg_resources.resource_filename( kvm_conf['hostname'] = "slaposkvm"
__name__, os.path.join( kvm_conf['smp_count'] = self.options['smp_count']
'template', 'kvm_run.in')) kvm_conf['ram_size'] = self.options['ram_size']
kvm_runner_path = self.createRunningWrapper("kvm",
self.substituteTemplate(kvm_wrapper_template_location, kvm_config))
kvm_conf['vnc_display'] = 1
# Instanciate KVM
kvm_template_location = pkg_resources.resource_filename(
__name__, os.path.join(
'template', 'kvm_run.in'))
kvm_runner_path = self.createRunningWrapper("kvm",
self.substituteTemplate(kvm_template_location,
kvm_conf))
self.path_list.append(kvm_runner_path)
# Instanciate KVM controller # Instanciate KVM controller
controller_config = {} kvm_controller_template_location = pkg_resources.resource_filename(
# Options nbd_ip and nbd_port are provided by slapos master __name__, os.path.join(
controller_config.update(self.options) 'template',
controller_config['qmp_socket'] = socket_path 'kvm_controller_run.in' ))
controller_config['vnc_passwd'] = vnc_passwd
controller_config['python_path'] = sys.executable
controller_wrapper_template_location = pkg_resources.resource_filename(
__name__, os.path.join(
'template', 'kvm_controller_run.in'))
controller_runner_path = self.createRunningWrapper("kvm_controller",
self.substituteTemplate(controller_wrapper_template_location, controller_config))
#XXX Instanciate Slapmonitor
slapmonitor_config={}
slapmonitor_config.update(self.options)
slapmonitor_config['database_path'] = database_path
slapmonitor_config['pid_file'] = pid_file_path
slapmonitor_config['python_path'] = sys.executable
slapmonitor_wrapper_template_location = pkg_resources.resource_filename(
__name__, os.path.join(
'template', 'slapmonitor_run.in'))
slapmonitor_runner_path = self.createRunningWrapper("slapmonitor",
self.substituteTemplate(slapmonitor_wrapper_template_location, slapmonitor_config))
#XXX Instanciate Slapreport
slapreport_config={}
slapreport_config.update(self.options)
slapreport_config['database_path'] = database_path
slapreport_config['python_path'] = sys.executable
slapreport_wrapper_template_location = pkg_resources.resource_filename(
__name__, os.path.join(
'template', 'slapreport_run.in'))
slapreport_runner_path = self.createReportRunningWrapper(self.substituteTemplate(
slapreport_wrapper_template_location, slapreport_config))
kvm_controller_runner_path = self.createRunningWrapper("kvm_controller",
self.substituteTemplate(kvm_controller_template_location,
kvm_conf))
self.path_list.append(kvm_controller_runner_path)
# Instanciate Slapmonitor
##slapmonitor_runner_path = self.instanciate_wrapper("slapmonitor",
# [database_path, pid_file_path, python_path])
# Instanciate Slapreport
##slapreport_runner_path = self.instanciate_wrapper("slapreport",
# [database_path, python_path])
return kvm_conf
def installNoVnc(self, source_ip, source_port, target_ip, target_port,
python_path):
"""
Create noVNC configuration dictionnary and instanciate Websockify proxy
Parameters : IP of the proxy, port on which is situated the proxy,
IP of the vnc server, port on which is situated the vnc server,
path to python binary
self.computer_partition.setConnectionDict(dict( Returns : noVNC configuration dictionnary
vnc_connection_string="vnc://[%s]:1" % ip, """
vnc_password=vnc_passwd,
)) noVNC_conf = {}
noVNC_conf['source_ip'] = source_ip
noVNC_conf['source_port'] = source_port
# Instanciate Websockify
websockify_runner_path = zc.buildout.easy_install.scripts([('websockify',
'slapos.recipe.librecipe.execute', 'execute_wait')], self.ws,
sys.executable, self.wrapper_directory, arguments=[
[python_path.strip(),
self.options['websockify_path'],
'--web',
self.options['noVNC_location'],
'--key=%s' % (self.key_path),
'--cert=%s' % (self.certificate_path),
'--ssl-only',
'%s:%s' % (source_ip, source_port),
'%s:%s' % (target_ip, target_port)],
[self.certificate_path, self.key_path]]
)[0]
self.path_list.append(websockify_runner_path)
return noVNC_conf
return [kvm_runner_path, controller_runner_path] def linkBinary(self):
"""Links binaries to instance's bin directory for easier exposal"""
for linkline in self.options.get('link_binary_list', '').splitlines():
if not linkline:
continue
target = linkline.split()
if len(target) == 1:
target = target[0]
path, linkname = os.path.split(target)
else:
linkname = target[1]
target = target[0]
link = os.path.join(self.bin_directory, linkname)
if os.path.lexists(link):
if not os.path.islink(link):
raise zc.buildout.UserError(
'Target link already %r exists but it is not link' % link)
os.unlink(link)
os.symlink(target, link)
self.logger.debug('Created link %r -> %r' % (link, target))
self.path_list.append(link)
def installCertificateAuthority(self, ca_country_code='XX',
ca_email='xx@example.com', ca_state='State', ca_city='City',
ca_company='Company'):
backup_path = self.createBackupDirectory('ca')
self.ca_dir = os.path.join(self.data_root_directory, 'ca')
self._createDirectory(self.ca_dir)
self.ca_request_dir = os.path.join(self.ca_dir, 'requests')
self._createDirectory(self.ca_request_dir)
config = dict(ca_dir=self.ca_dir, request_dir=self.ca_request_dir)
self.ca_private = os.path.join(self.ca_dir, 'private')
self.ca_certs = os.path.join(self.ca_dir, 'certs')
self.ca_crl = os.path.join(self.ca_dir, 'crl')
self.ca_newcerts = os.path.join(self.ca_dir, 'newcerts')
self.ca_key_ext = '.key'
self.ca_crt_ext = '.crt'
for d in [self.ca_private, self.ca_crl, self.ca_newcerts, self.ca_certs]:
self._createDirectory(d)
for f in ['crlnumber', 'serial']:
if not os.path.exists(os.path.join(self.ca_dir, f)):
open(os.path.join(self.ca_dir, f), 'w').write('01')
if not os.path.exists(os.path.join(self.ca_dir, 'index.txt')):
open(os.path.join(self.ca_dir, 'index.txt'), 'w').write('')
openssl_configuration = os.path.join(self.ca_dir, 'openssl.cnf')
config.update(
working_directory=self.ca_dir,
country_code=ca_country_code,
state=ca_state,
city=ca_city,
company=ca_company,
email_address=ca_email,
)
self._writeFile(openssl_configuration, pkg_resources.resource_string(
__name__, 'template/openssl.cnf.ca.in') % config)
self.path_list.extend(zc.buildout.easy_install.scripts([
('certificate_authority',
__name__ + '.certificate_authority', 'runCertificateAuthority')],
self.ws, sys.executable, self.wrapper_directory, arguments=[dict(
openssl_configuration=openssl_configuration,
openssl_binary=self.options['openssl_binary'],
certificate=os.path.join(self.ca_dir, 'cacert.pem'),
key=os.path.join(self.ca_private, 'cakey.pem'),
crl=os.path.join(self.ca_crl),
request_dir=self.ca_request_dir
)]))
# configure backup
backup_cron = os.path.join(self.cron_d, 'ca_rdiff_backup')
open(backup_cron, 'w').write(
'''0 0 * * * %(rdiff_backup)s %(source)s %(destination)s'''%dict(
rdiff_backup=self.options['rdiff_backup_binary'],
source=self.ca_dir,
destination=backup_path))
self.path_list.append(backup_cron)
return dict(
ca_certificate=os.path.join(config['ca_dir'], 'cacert.pem'),
ca_crl=os.path.join(config['ca_dir'], 'crl'),
certificate_authority_path=config['ca_dir']
)
def requestCertificate(self, name):
hash = hashlib.sha512(name).hexdigest()
key = os.path.join(self.ca_private, hash + self.ca_key_ext)
certificate = os.path.join(self.ca_certs, hash + self.ca_crt_ext)
parser = ConfigParser.RawConfigParser()
parser.add_section('certificate')
parser.set('certificate', 'name', name)
parser.set('certificate', 'key_file', key)
parser.set('certificate', 'certificate_file', certificate)
parser.write(open(os.path.join(self.ca_request_dir, hash), 'w'))
return key, certificate
def installCrond(self):
timestamps = self.createDataDirectory('cronstamps')
cron_output = os.path.join(self.log_directory, 'cron-output')
self._createDirectory(cron_output)
catcher = zc.buildout.easy_install.scripts([('catchcron',
__name__ + '.catdatefile', 'catdatefile')], self.ws, sys.executable,
self.bin_directory, arguments=[cron_output])[0]
self.path_list.append(catcher)
cron_d = os.path.join(self.etc_directory, 'cron.d')
crontabs = os.path.join(self.etc_directory, 'crontabs')
self._createDirectory(cron_d)
self._createDirectory(crontabs)
# Use execute from erp5.
wrapper = zc.buildout.easy_install.scripts([('crond',
'slapos.recipe.librecipe.execute', 'execute')], self.ws, sys.executable,
self.wrapper_directory, arguments=[
self.options['dcrond_binary'].strip(), '-s', cron_d, '-c', crontabs,
'-t', timestamps, '-f', '-l', '5', '-M', catcher]
)[0]
self.path_list.append(wrapper)
return cron_d
import os
import subprocess
import time
import ConfigParser
def popenCommunicate(command_list, input=None):
subprocess_kw = dict(stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
if input is not None:
subprocess_kw.update(stdin=subprocess.PIPE)
popen = subprocess.Popen(command_list, **subprocess_kw)
result = popen.communicate(input)[0]
if popen.returncode is None:
popen.kill()
if popen.returncode != 0:
raise ValueError('Issue during calling %r, result was:\n%s' % (
command_list, result))
return result
class CertificateAuthority:
def __init__(self, key, certificate, openssl_binary,
openssl_configuration, request_dir):
self.key = key
self.certificate = certificate
self.openssl_binary = openssl_binary
self.openssl_configuration = openssl_configuration
self.request_dir = request_dir
def checkAuthority(self):
file_list = [ self.key, self.certificate ]
ca_ready = True
for f in file_list:
if not os.path.exists(f):
ca_ready = False
break
if ca_ready:
return
for f in file_list:
if os.path.exists(f):
os.unlink(f)
try:
# no CA, let us create new one
popenCommunicate([self.openssl_binary, 'req', '-nodes', '-config',
self.openssl_configuration, '-new', '-x509', '-extensions',
'v3_ca', '-keyout', self.key, '-out', self.certificate,
'-days', '10950'], 'Automatic Certificate Authority\n')
except:
try:
for f in file_list:
if os.path.exists(f):
os.unlink(f)
except:
# do not raise during cleanup
pass
raise
def _checkCertificate(self, common_name, key, certificate):
file_list = [key, certificate]
ready = True
for f in file_list:
if not os.path.exists(f):
ready = False
break
if ready:
return False
for f in file_list:
if os.path.exists(f):
os.unlink(f)
csr = certificate + '.csr'
try:
popenCommunicate([self.openssl_binary, 'req', '-config',
self.openssl_configuration, '-nodes', '-new', '-keyout',
key, '-out', csr, '-days', '3650'],
common_name + '\n')
try:
popenCommunicate([self.openssl_binary, 'ca', '-batch', '-config',
self.openssl_configuration, '-out', certificate,
'-infiles', csr])
finally:
if os.path.exists(csr):
os.unlink(csr)
except:
try:
for f in file_list:
if os.path.exists(f):
os.unlink(f)
except:
# do not raise during cleanup
pass
raise
else:
return True
def checkRequestDir(self):
for request_file in os.listdir(self.request_dir):
parser = ConfigParser.RawConfigParser()
parser.readfp(open(os.path.join(self.request_dir, request_file), 'r'))
if self._checkCertificate(parser.get('certificate', 'name'),
parser.get('certificate', 'key_file'), parser.get('certificate',
'certificate_file')):
print 'Created certificate %r' % parser.get('certificate', 'name')
def runCertificateAuthority(args):
ca_conf = args[0]
ca = CertificateAuthority(ca_conf['key'], ca_conf['certificate'],
ca_conf['openssl_binary'], ca_conf['openssl_configuration'],
ca_conf['request_dir'])
while True:
ca.checkAuthority()
ca.checkRequestDir()
time.sleep(60)
...@@ -11,7 +11,7 @@ so = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) ...@@ -11,7 +11,7 @@ so = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
connected = False connected = False
while not connected: while not connected:
try: try:
so.connect('%(qmp_socket)s') so.connect('%(socket_path)s')
except socket.error: except socket.error:
time.sleep(1) time.sleep(1)
else: else:
......
...@@ -10,8 +10,8 @@ exec %(qemu_path)s \ ...@@ -10,8 +10,8 @@ exec %(qemu_path)s \
-smp %(smp_count)s \ -smp %(smp_count)s \
-m %(ram_size)s \ -m %(ram_size)s \
-cdrom nbd:[%(nbd_ip)s]:%(nbd_port)s \ -cdrom nbd:[%(nbd_ip)s]:%(nbd_port)s \
-drive file=%(image)s,if=virtio,boot=on \ -drive file=%(disk_path)s,if=virtio,boot=on \
-vnc [%(vnc_ip)s]:1,ipv6,tls,password \ -vnc %(vnc_ip)s:1,ipv4,password \
-boot menu=on \ -boot menu=on \
-qmp unix:%(qmp_socket)s,server \ -qmp unix:%(socket_path)s,server \
-pidfile %(pid_file)s -pidfile %(pid_file_path)s
#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#
# This definition stops the following lines choking if HOME isn't
# defined.
HOME = .
RANDFILE = $ENV::HOME/.rnd
# Extra OBJECT IDENTIFIER info:
#oid_file = $ENV::HOME/.oid
oid_section = new_oids
# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)
[ new_oids ]
# We can add new OIDs in here for use by 'ca', 'req' and 'ts'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6
# Policies used by the TSA examples.
tsa_policy1 = 1.2.3.4.1
tsa_policy2 = 1.2.3.4.5.6
tsa_policy3 = 1.2.3.4.5.7
####################################################################
[ ca ]
default_ca = CA_default # The default ca section
####################################################################
[ CA_default ]
dir = %(working_directory)s # Where everything is kept
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
#unique_subject = no # Set to 'no' to allow creation of
# several ctificates with same subject.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crlnumber = $dir/crlnumber # the current crl number
# must be commented out to leave a V1 CRL
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem # The private key
RANDFILE = $dir/private/.rand # private random number file
x509_extensions = usr_cert # The extentions to add to the cert
# Comment out the following two lines for the "traditional"
# (and highly broken) format.
name_opt = ca_default # Subject Name options
cert_opt = ca_default # Certificate field options
# Extension copying option: use with caution.
# copy_extensions = copy
# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crlnumber must also be commented out to leave a V1 CRL.
# crl_extensions = crl_ext
default_days = 3650 # how long to certify for
default_crl_days = 30 # how long before next CRL
default_md = default # use public key default MD
preserve = no # keep passed DN ordering
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy = policy_match
# For the CA policy
[ policy_match ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
####################################################################
[ req ]
default_bits = 2048
default_md = sha1
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
#attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
# Passwords for private keys if not present they will be prompted for
# input_password = secret
# output_password = secret
# This sets a mask for permitted string types. There are several options.
# default: PrintableString, T61String, BMPString.
# pkix : PrintableString, BMPString (PKIX recommendation before 2004)
# utf8only: only UTF8Strings (PKIX recommendation after 2004).
# nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings).
# MASK:XXXX a literal mask value.
# WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings.
string_mask = utf8only
# req_extensions = v3_req # The extensions to add to a certificate request
[ req_distinguished_name ]
countryName = Country Name (2 letter code)
countryName_value = %(country_code)s
countryName_min = 2
countryName_max = 2
stateOrProvinceName = State or Province Name (full name)
stateOrProvinceName_value = %(state)s
localityName = Locality Name (eg, city)
localityName_value = %(city)s
0.organizationName = Organization Name (eg, company)
0.organizationName_value = %(company)s
# we can do this but it is not needed normally :-)
#1.organizationName = Second Organization Name (eg, company)
#1.organizationName_default = World Wide Web Pty Ltd
commonName = Common Name (eg, your name or your server\'s hostname)
commonName_max = 64
emailAddress = Email Address
emailAddress_value = %(email_address)s
emailAddress_max = 64
# SET-ex3 = SET extension number 3
#[ req_attributes ]
#challengePassword = A challenge password
#challengePassword_min = 4
#challengePassword_max = 20
#
#unstructuredName = An optional company name
[ usr_cert ]
# These extensions are added when 'ca' signs a request.
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment = "OpenSSL Generated Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
# This is required for TSA certificates.
# extendedKeyUsage = critical,timeStamping
[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
[ v3_ca ]
# Extensions for a typical CA
# PKIX recommendation.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer
# This is what PKIX recommends but some broken software chokes on critical
# extensions.
#basicConstraints = critical,CA:true
# So we do this instead.
basicConstraints = CA:true
# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as an test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign
# Some might want this also
# nsCertType = sslCA, emailCA
# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy
# DER hex encoding of an extension: beware experts only!
# obj=DER:02:03
# Where 'obj' is a standard or added object
# You can even override a supported extension:
# basicConstraints= critical, DER:30:03:01:01:FF
[ crl_ext ]
# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.
# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always
[ proxy_cert_ext ]
# These extensions should be added when creating a proxy certificate
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment = "OpenSSL Generated Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
# This really needs to be in place for it to be a proxy certificate.
proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo
####################################################################
[ tsa ]
default_tsa = tsa_config1 # the default TSA section
[ tsa_config1 ]
# These are used by the TSA reply generation only.
dir = /etc/pki/tls # TSA root directory
serial = $dir/tsaserial # The current serial number (mandatory)
crypto_device = builtin # OpenSSL engine to use for signing
signer_cert = $dir/tsacert.pem # The TSA signing certificate
# (optional)
certs = $dir/cacert.pem # Certificate chain to include in reply
# (optional)
signer_key = $dir/private/tsakey.pem # The TSA private key (optional)
default_policy = tsa_policy1 # Policy if request did not specify it
# (optional)
other_policies = tsa_policy2, tsa_policy3 # acceptable policies (optional)
digests = md5, sha1 # Acceptable message digests (mandatory)
accuracy = secs:1, millisecs:500, microsecs:100 # (optional)
clock_precision_digits = 0 # number of digits after dot. (optional)
ordering = yes # Is ordering defined for timestamps?
# (optional, default: no)
tsa_name = yes # Must the TSA name be included in the reply?
# (optional, default: no)
ess_cert_id_chain = no # Must the ESS cert id chain be included?
# (optional, default: no)
#!/bin/sh #!/bin/sh
# BEWARE: This file is operated by slapgrid # BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically # BEWARE: It will be overwritten automatically
exec %(python_path)s %(slapmonitor_path)s %(pid_file)s %(database_path)s exec %(python_path)s %(slapmonitor_path)s %(pid_file_path)s %(database_path)s
...@@ -10,6 +10,23 @@ def execute(args): ...@@ -10,6 +10,23 @@ def execute(args):
# Note: Candidate for slapos.lib.recipe # Note: Candidate for slapos.lib.recipe
os.execv(args[0], args + sys.argv[1:]) os.execv(args[0], args + sys.argv[1:])
def execute_wait(args):
"""Execution but after all files in args[1] exists"""
exec_list = list(args[0])
file_list = list(args[1])
sleep = 60
while True:
ready = True
for f in file_list:
if not os.path.exists(f):
print 'File %r does not exists, sleeping for %s' % (f, sleep)
ready = False
if ready:
break
time.sleep(sleep)
os.execv(exec_list[0], exec_list + sys.argv[1:])
child_pg = None child_pg = None
......
...@@ -220,9 +220,10 @@ class Recipe(BaseSlapRecipe): ...@@ -220,9 +220,10 @@ class Recipe(BaseSlapRecipe):
self.substituteTemplate(template_filename, self.substituteTemplate(template_filename,
stunnel_conf)) stunnel_conf))
wrapper = zc.buildout.easy_install.scripts([('stunnel', wrapper = zc.buildout.easy_install.scripts([('stunnel',
'slapos.recipe.librecipe.execute', 'execute')], self.ws, sys.executable, 'slapos.recipe.librecipe.execute', 'execute_wait')], self.ws,
self.wrapper_directory, arguments=[ sys.executable, self.wrapper_directory, arguments=[
self.options['stunnel_binary'].strip(), stunnel_conf_path] [self.options['stunnel_binary'].strip(), stunnel_conf_path],
[ca_certificate, key]]
)[0] )[0]
self.path_list.append(wrapper) self.path_list.append(wrapper)
return stunnel_conf return stunnel_conf
......
##############################################################################
#
# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from slapos.recipe.librecipe import BaseSlapRecipe
import hashlib
import os
import pkg_resources
import sys
import zc.buildout
import ConfigParser
class Recipe(BaseSlapRecipe):
def getTemplateFilename(self, template_name):
return pkg_resources.resource_filename(__name__,
'template/%s' % template_name)
def _install(self):
self.path_list = []
self.requirements, self.ws = self.egg.working_set()
# self.cron_d is a directory, where cron jobs can be registered
self.cron_d = self.installCrond()
self.logrotate_d, self.logrotate_backup = self.installLogrotate()
mysql_conf = self.installMysqlServer(self.getLocalIPv4Address(), 45678)
ca_conf = self.installCertificateAuthority()
key, certificate = self.requestCertificate('MySQL')
stunnel_conf = self.installStunnel(self.getGlobalIPv6Address(),
self.getLocalIPv4Address(), 12345, mysql_conf['tcp_port'],
certificate, key, ca_conf['ca_crl'],
ca_conf['certificate_authority_path'])
self.linkBinary()
self.setConnectionDict(dict(
stunnel_ip = stunnel_conf['public_ip'],
stunnel_port = stunnel_conf['public_port'],
mysql_database = mysql_conf['mysql_database'],
mysql_user = mysql_conf['mysql_user'],
mysql_password = mysql_conf['mysql_password'],
))
return self.path_list
def linkBinary(self):
"""Links binaries to instance's bin directory for easier exposal"""
for linkline in self.options.get('link_binary_list', '').splitlines():
if not linkline:
continue
target = linkline.split()
if len(target) == 1:
target = target[0]
path, linkname = os.path.split(target)
else:
linkname = target[1]
target = target[0]
link = os.path.join(self.bin_directory, linkname)
if os.path.lexists(link):
if not os.path.islink(link):
raise zc.buildout.UserError(
'Target link already %r exists but it is not link' % link)
os.unlink(link)
os.symlink(target, link)
self.logger.debug('Created link %r -> %r' % (link, target))
self.path_list.append(link)
def installCrond(self):
timestamps = self.createDataDirectory('cronstamps')
cron_output = os.path.join(self.log_directory, 'cron-output')
self._createDirectory(cron_output)
catcher = zc.buildout.easy_install.scripts([('catchcron',
__name__ + '.catdatefile', 'catdatefile')], self.ws, sys.executable,
self.bin_directory, arguments=[cron_output])[0]
self.path_list.append(catcher)
cron_d = os.path.join(self.etc_directory, 'cron.d')
crontabs = os.path.join(self.etc_directory, 'crontabs')
self._createDirectory(cron_d)
self._createDirectory(crontabs)
wrapper = zc.buildout.easy_install.scripts([('crond',
'slapos.recipe.librecipe.execute', 'execute')], self.ws, sys.executable,
self.wrapper_directory, arguments=[
self.options['dcrond_binary'].strip(), '-s', cron_d, '-c', crontabs,
'-t', timestamps, '-f', '-l', '5', '-M', catcher]
)[0]
self.path_list.append(wrapper)
return cron_d
def installLogrotate(self):
"""Installs logortate main configuration file and registers its to cron"""
logrotate_d = os.path.abspath(os.path.join(self.etc_directory,
'logrotate.d'))
self._createDirectory(logrotate_d)
logrotate_backup = self.createBackupDirectory('logrotate')
logrotate_conf = self.createConfigurationFile("logrotate.conf",
"include %s" % logrotate_d)
logrotate_cron = os.path.join(self.cron_d, 'logrotate')
state_file = os.path.join(self.data_root_directory, 'logrotate.status')
open(logrotate_cron, 'w').write('0 0 * * * %s -s %s %s' %
(self.options['logrotate_binary'], state_file, logrotate_conf))
self.path_list.extend([logrotate_d, logrotate_conf, logrotate_cron])
return logrotate_d, logrotate_backup
def registerLogRotation(self, name, log_file_list, postrotate_script):
"""Register new log rotation requirement"""
open(os.path.join(self.logrotate_d, name), 'w').write(
self.substituteTemplate(self.getTemplateFilename(
'logrotate_entry.in'),
dict(file_list=' '.join(['"'+q+'"' for q in log_file_list]),
postrotate=postrotate_script, olddir=self.logrotate_backup)))
def installCertificateAuthority(self, ca_country_code='XX',
ca_email='xx@example.com', ca_state='State', ca_city='City',
ca_company='Company'):
backup_path = self.createBackupDirectory('ca')
self.ca_dir = os.path.join(self.data_root_directory, 'ca')
self._createDirectory(self.ca_dir)
self.ca_request_dir = os.path.join(self.ca_dir, 'requests')
self._createDirectory(self.ca_request_dir)
config = dict(ca_dir=self.ca_dir, request_dir=self.ca_request_dir)
self.ca_private = os.path.join(self.ca_dir, 'private')
self.ca_certs = os.path.join(self.ca_dir, 'certs')
self.ca_crl = os.path.join(self.ca_dir, 'crl')
self.ca_newcerts = os.path.join(self.ca_dir, 'newcerts')
self.ca_key_ext = '.key'
self.ca_crt_ext = '.crt'
for d in [self.ca_private, self.ca_crl, self.ca_newcerts, self.ca_certs]:
self._createDirectory(d)
for f in ['crlnumber', 'serial']:
if not os.path.exists(os.path.join(self.ca_dir, f)):
open(os.path.join(self.ca_dir, f), 'w').write('01')
if not os.path.exists(os.path.join(self.ca_dir, 'index.txt')):
open(os.path.join(self.ca_dir, 'index.txt'), 'w').write('')
openssl_configuration = os.path.join(self.ca_dir, 'openssl.cnf')
config.update(
working_directory=self.ca_dir,
country_code=ca_country_code,
state=ca_state,
city=ca_city,
company=ca_company,
email_address=ca_email,
)
self._writeFile(openssl_configuration, pkg_resources.resource_string(
__name__, 'template/openssl.cnf.ca.in') % config)
self.path_list.extend(zc.buildout.easy_install.scripts([
('certificate_authority',
__name__ + '.certificate_authority', 'runCertificateAuthority')],
self.ws, sys.executable, self.wrapper_directory, arguments=[dict(
openssl_configuration=openssl_configuration,
openssl_binary=self.options['openssl_binary'],
certificate=os.path.join(self.ca_dir, 'cacert.pem'),
key=os.path.join(self.ca_private, 'cakey.pem'),
crl=os.path.join(self.ca_crl),
request_dir=self.ca_request_dir
)]))
# configure backup
backup_cron = os.path.join(self.cron_d, 'ca_rdiff_backup')
open(backup_cron, 'w').write(
'''0 0 * * * %(rdiff_backup)s %(source)s %(destination)s'''%dict(
rdiff_backup=self.options['rdiff_backup_binary'],
source=self.ca_dir,
destination=backup_path))
self.path_list.append(backup_cron)
return dict(
ca_certificate=os.path.join(config['ca_dir'], 'cacert.pem'),
ca_crl=os.path.join(config['ca_dir'], 'crl'),
certificate_authority_path=config['ca_dir']
)
def requestCertificate(self, name):
hash = hashlib.sha512(name).hexdigest()
key = os.path.join(self.ca_private, hash + self.ca_key_ext)
certificate = os.path.join(self.ca_certs, hash + self.ca_crt_ext)
parser = ConfigParser.RawConfigParser()
parser.add_section('certificate')
parser.set('certificate', 'name', name)
parser.set('certificate', 'key_file', key)
parser.set('certificate', 'certificate_file', certificate)
parser.write(open(os.path.join(self.ca_request_dir, hash), 'w'))
return key, certificate
def installStunnel(self, public_ip, private_ip, public_port, private_port,
ca_certificate, key, ca_crl, ca_path):
"""Installs stunnel"""
template_filename = self.getTemplateFilename('stunnel.conf.in')
log = os.path.join(self.log_directory, 'stunnel.log')
pid_file = os.path.join(self.run_directory, 'stunnel.pid')
stunnel_conf = dict(
public_ip=public_ip,
private_ip=private_ip,
public_port=public_port,
pid_file=pid_file,
log=log,
cert = ca_certificate,
key = key,
ca_crl = ca_crl,
ca_path = ca_path,
private_port = private_port,
)
stunnel_conf_path = self.createConfigurationFile("stunnel.conf",
self.substituteTemplate(template_filename,
stunnel_conf))
wrapper = zc.buildout.easy_install.scripts([('stunnel',
'slapos.recipe.librecipe.execute', 'execute_wait')], self.ws,
sys.executable, self.wrapper_directory, arguments=[
[self.options['stunnel_binary'].strip(), stunnel_conf_path],
[ca_certificate, key]]
)[0]
self.path_list.append(wrapper)
return stunnel_conf
def installMysqlServer(self, ip, port, database='db', user='user',
template_filename=None, mysql_conf=None):
if mysql_conf is None:
mysql_conf = {}
backup_directory = self.createBackupDirectory('mysql')
if template_filename is None:
template_filename = self.getTemplateFilename('my.cnf.in')
error_log = os.path.join(self.log_directory, 'mysqld.log')
slow_query_log = os.path.join(self.log_directory, 'mysql-slow.log')
mysql_conf.update(
ip=ip,
data_directory=os.path.join(self.data_root_directory,
'mysql'),
tcp_port=port,
pid_file=os.path.join(self.run_directory, 'mysqld.pid'),
socket=os.path.join(self.run_directory, 'mysqld.sock'),
error_log=error_log,
slow_query_log=slow_query_log,
mysql_database=database,
mysql_user=user,
mysql_password=self.generatePassword(),
)
self.registerLogRotation('mysql', [error_log, slow_query_log],
'%(mysql_binary)s --no-defaults -B --user=root '
'--socket=%(mysql_socket)s -e "FLUSH LOGS"' % dict(
mysql_binary=self.options['mysql_binary'],
mysql_socket=mysql_conf['socket']))
self._createDirectory(mysql_conf['data_directory'])
mysql_conf_path = self.createConfigurationFile("my.cnf",
self.substituteTemplate(template_filename,
mysql_conf))
mysql_script_list = []
for x_database, x_user, x_password in \
[(mysql_conf['mysql_database'],
mysql_conf['mysql_user'],
mysql_conf['mysql_password']),
]:
mysql_script_list.append(pkg_resources.resource_string(__name__,
'template/initmysql.sql.in') % {
'mysql_database': x_database,
'mysql_user': x_user,
'mysql_password': x_password})
mysql_script_list.append('EXIT')
mysql_script = '\n'.join(mysql_script_list)
self.path_list.extend(zc.buildout.easy_install.scripts([('mysql_update',
__name__ + '.mysql', 'updateMysql')], self.ws,
sys.executable, self.wrapper_directory, arguments=[dict(
mysql_script=mysql_script,
mysql_binary=self.options['mysql_binary'].strip(),
mysql_upgrade_binary=self.options['mysql_upgrade_binary'].strip(),
socket=mysql_conf['socket'],
)]))
self.path_list.extend(zc.buildout.easy_install.scripts([('mysqld',
__name__ + '.mysql', 'runMysql')], self.ws,
sys.executable, self.wrapper_directory, arguments=[dict(
mysql_install_binary=self.options['mysql_install_binary'].strip(),
mysqld_binary=self.options['mysqld_binary'].strip(),
data_directory=mysql_conf['data_directory'].strip(),
mysql_binary=self.options['mysql_binary'].strip(),
socket=mysql_conf['socket'].strip(),
configuration_file=mysql_conf_path,
)]))
self.path_list.extend([mysql_conf_path])
# backup configuration
backup_directory = self.createBackupDirectory('mysql')
full_backup = os.path.join(backup_directory, 'full')
incremental_backup = os.path.join(backup_directory, 'incremental')
self._createDirectory(full_backup)
self._createDirectory(incremental_backup)
innobackupex_argument_list = [self.options['perl_binary'],
self.options['innobackupex_binary'],
'--defaults-file=%s' % mysql_conf_path,
'--socket=%s' %mysql_conf['socket'].strip(), '--user=root']
environment = dict(PATH='%s' % self.bin_directory)
innobackupex_incremental = zc.buildout.easy_install.scripts([(
'innobackupex_incremental', 'slapos.recipe.librecipe.execute', 'executee')],
self.ws, sys.executable, self.bin_directory, arguments=[
innobackupex_argument_list + ['--incremental'],
environment])[0]
self.path_list.append(innobackupex_incremental)
innobackupex_full = zc.buildout.easy_install.scripts([('innobackupex_full',
'slapos.recipe.librecipe.execute', 'executee')], self.ws,
sys.executable, self.bin_directory, arguments=[
innobackupex_argument_list,
environment])[0]
self.path_list.append(innobackupex_full)
backup_controller = zc.buildout.easy_install.scripts([
('innobackupex_controller', __name__ + '.innobackupex', 'controller')],
self.ws, sys.executable, self.bin_directory,
arguments=[innobackupex_incremental, innobackupex_full, full_backup,
incremental_backup])[0]
self.path_list.append(backup_controller)
mysql_backup_cron = os.path.join(self.cron_d, 'mysql_backup')
open(mysql_backup_cron, 'w').write('0 0 * * * ' + backup_controller)
self.path_list.append(mysql_backup_cron)
# The return could be more explicit database, user ...
return mysql_conf
import os
import subprocess
import time
import ConfigParser
def popenCommunicate(command_list, input=None):
subprocess_kw = dict(stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
if input is not None:
subprocess_kw.update(stdin=subprocess.PIPE)
popen = subprocess.Popen(command_list, **subprocess_kw)
result = popen.communicate(input)[0]
if popen.returncode is None:
popen.kill()
if popen.returncode != 0:
raise ValueError('Issue during calling %r, result was:\n%s' % (
command_list, result))
return result
class CertificateAuthority:
def __init__(self, key, certificate, openssl_binary,
openssl_configuration, request_dir):
self.key = key
self.certificate = certificate
self.openssl_binary = openssl_binary
self.openssl_configuration = openssl_configuration
self.request_dir = request_dir
def checkAuthority(self):
file_list = [ self.key, self.certificate ]
ca_ready = True
for f in file_list:
if not os.path.exists(f):
ca_ready = False
break
if ca_ready:
return
for f in file_list:
if os.path.exists(f):
os.unlink(f)
try:
# no CA, let us create new one
popenCommunicate([self.openssl_binary, 'req', '-nodes', '-config',
self.openssl_configuration, '-new', '-x509', '-extensions',
'v3_ca', '-keyout', self.key, '-out', self.certificate,
'-days', '10950'], 'Automatic Certificate Authority\n')
except:
try:
for f in file_list:
if os.path.exists(f):
os.unlink(f)
except:
# do not raise during cleanup
pass
raise
def _checkCertificate(self, common_name, key, certificate):
file_list = [key, certificate]
ready = True
for f in file_list:
if not os.path.exists(f):
ready = False
break
if ready:
return False
for f in file_list:
if os.path.exists(f):
os.unlink(f)
csr = certificate + '.csr'
try:
popenCommunicate([self.openssl_binary, 'req', '-config',
self.openssl_configuration, '-nodes', '-new', '-keyout',
key, '-out', csr, '-days', '3650'],
common_name + '\n')
try:
popenCommunicate([self.openssl_binary, 'ca', '-batch', '-config',
self.openssl_configuration, '-out', certificate,
'-infiles', csr])
finally:
if os.path.exists(csr):
os.unlink(csr)
except:
try:
for f in file_list:
if os.path.exists(f):
os.unlink(f)
except:
# do not raise during cleanup
pass
raise
else:
return True
def checkRequestDir(self):
for request_file in os.listdir(self.request_dir):
parser = ConfigParser.RawConfigParser()
parser.readfp(open(os.path.join(self.request_dir, request_file), 'r'))
if self._checkCertificate(parser.get('certificate', 'name'),
parser.get('certificate', 'key_file'), parser.get('certificate',
'certificate_file')):
print 'Created certificate %r' % parser.get('certificate', 'name')
def runCertificateAuthority(args):
ca_conf = args[0]
ca = CertificateAuthority(ca_conf['key'], ca_conf['certificate'],
ca_conf['openssl_binary'], ca_conf['openssl_configuration'],
ca_conf['request_dir'])
while True:
ca.checkAuthority()
ca.checkRequestDir()
time.sleep(60)
import os
import subprocess
import time
import sys
def runMysql(args):
sleep = 60
conf = args[0]
mysqld_wrapper_list = [conf['mysqld_binary'], '--defaults-file=%s' %
conf['configuration_file']]
# we trust mysql_install that if mysql directory is available mysql was
# correctly initalised
if not os.path.isdir(os.path.join(conf['data_directory'], 'mysql')):
while True:
# XXX: Protect with proper root password
# XXX: Follow http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html
popen = subprocess.Popen([conf['mysql_install_binary'],
'--skip-name-resolve', '--no-defaults', '--datadir=%s' %
conf['data_directory']],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = popen.communicate()[0]
if popen.returncode is None or popen.returncode != 0:
print "Failed to initialise server.\nThe error was: %s" % result
print "Waiting for %ss and retrying" % sleep
time.sleep(sleep)
else:
print "Mysql properly initialised"
break
else:
print "MySQL already initialised"
print "Starting %r" % mysqld_wrapper_list[0]
sys.stdout.flush()
sys.stderr.flush()
os.execl(mysqld_wrapper_list[0], *mysqld_wrapper_list)
def updateMysql(args):
conf = args[0]
sleep = 30
is_succeed = False
while True:
if not is_succeed:
mysql_upgrade_list = [conf['mysql_upgrade_binary'], '--no-defaults', '--user=root', '--socket=%s' % conf['socket']]
mysql_upgrade = subprocess.Popen(mysql_upgrade_list, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = mysql_upgrade.communicate()[0]
if mysql_upgrade.returncode is None:
mysql_upgrade.kill()
if mysql_upgrade.returncode != 0 and not 'is already upgraded' in result:
print "Command %r failed with result:\n%s" % (mysql_upgrade_list, result)
print 'Sleeping for %ss and retrying' % sleep
else:
if mysql_upgrade.returncode == 0:
print "MySQL database upgraded with result:\n%s" % result
else:
print "No need to upgrade MySQL database"
mysql_list = [conf['mysql_binary'].strip(), '--no-defaults', '-B', '--user=root', '--socket=%s' % conf['socket']]
mysql = subprocess.Popen(mysql_list, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = mysql.communicate(conf['mysql_script'])[0]
if mysql.returncode is None:
mysql.kill()
if mysql.returncode != 0:
print 'Command %r failed with:\n%s' % (mysql_list, result)
print 'Sleeping for %ss and retrying' % sleep
else:
is_succeed = True
print 'SlapOS initialisation script succesfully applied on database.'
sys.stdout.flush()
sys.stderr.flush()
time.sleep(sleep)
CREATE DATABASE IF NOT EXISTS %(mysql_database)s;
GRANT ALL PRIVILEGES ON %(mysql_database)s.* TO %(mysql_user)s@'%%' IDENTIFIED BY '%(mysql_password)s';
#GRANT ALL PRIVILEGES ON %(mysql_database)s.* TO %(mysql_user)s@* IDENTIFIED BY '%(mysql_password)s';
%(file_list)s {
daily
dateext
rotate 30
compress
notifempty
sharedscripts
create
postrotate
%(postrotate)s
endscript
olddir %(olddir)s
}
# ERP5 buildout my.cnf template based on my-huge.cnf shipped with mysql
# The MySQL server
[mysqld]
# ERP5 by default requires InnoDB storage. MySQL by default fallbacks to using
# different engine, like MyISAM. Such behaviour generates problems only, when
# tables requested as InnoDB are silently created with MyISAM engine.
#
# Loud fail is really required in such case.
sql-mode="NO_ENGINE_SUBSTITUTION"
skip-show-database
port = %(tcp_port)s
bind-address = %(ip)s
socket = %(socket)s
datadir = %(data_directory)s
pid-file = %(pid_file)s
log-error = %(error_log)s
#log-slow-file = %(slow_query_log)s
long_query_time = 5
max_allowed_packet = 128M
query_cache_size = 32M
plugin-load = ha_innodb_plugin.so
# The following are important to configure and depend a lot on to the size of
# your database and the available resources.
#innodb_buffer_pool_size = 4G
#innodb_log_file_size = 256M
#innodb_log_buffer_size = 8M
# Some dangerous settings you may want to uncomment if you only want
# performance or less disk access. Useful for unit tests.
#innodb_flush_log_at_trx_commit = 0
#innodb_flush_method = nosync
#innodb_doublewrite = 0
#sync_frm = 0
# Uncomment the following if you need binary logging, which is recommended
# on production instances (either for replication or incremental backups).
#log-bin=mysql-bin
# Force utf8 usage
collation_server = utf8_unicode_ci
character_set_server = utf8
skip-character-set-client-handshake
[mysql]
no-auto-rehash
socket = %(socket)s
[mysqlhotcopy]
interactive-timeout
#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#
# This definition stops the following lines choking if HOME isn't
# defined.
HOME = .
RANDFILE = $ENV::HOME/.rnd
# Extra OBJECT IDENTIFIER info:
#oid_file = $ENV::HOME/.oid
oid_section = new_oids
# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)
[ new_oids ]
# We can add new OIDs in here for use by 'ca', 'req' and 'ts'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6
# Policies used by the TSA examples.
tsa_policy1 = 1.2.3.4.1
tsa_policy2 = 1.2.3.4.5.6
tsa_policy3 = 1.2.3.4.5.7
####################################################################
[ ca ]
default_ca = CA_default # The default ca section
####################################################################
[ CA_default ]
dir = %(working_directory)s # Where everything is kept
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
#unique_subject = no # Set to 'no' to allow creation of
# several ctificates with same subject.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crlnumber = $dir/crlnumber # the current crl number
# must be commented out to leave a V1 CRL
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem # The private key
RANDFILE = $dir/private/.rand # private random number file
x509_extensions = usr_cert # The extentions to add to the cert
# Comment out the following two lines for the "traditional"
# (and highly broken) format.
name_opt = ca_default # Subject Name options
cert_opt = ca_default # Certificate field options
# Extension copying option: use with caution.
# copy_extensions = copy
# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crlnumber must also be commented out to leave a V1 CRL.
# crl_extensions = crl_ext
default_days = 3650 # how long to certify for
default_crl_days= 30 # how long before next CRL
default_md = default # use public key default MD
preserve = no # keep passed DN ordering
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy = policy_match
# For the CA policy
[ policy_match ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
####################################################################
[ req ]
default_bits = 2048
default_md = sha1
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
#attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
# Passwords for private keys if not present they will be prompted for
# input_password = secret
# output_password = secret
# This sets a mask for permitted string types. There are several options.
# default: PrintableString, T61String, BMPString.
# pkix : PrintableString, BMPString (PKIX recommendation before 2004)
# utf8only: only UTF8Strings (PKIX recommendation after 2004).
# nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings).
# MASK:XXXX a literal mask value.
# WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings.
string_mask = utf8only
# req_extensions = v3_req # The extensions to add to a certificate request
[ req_distinguished_name ]
countryName = Country Name (2 letter code)
countryName_value = %(country_code)s
countryName_min = 2
countryName_max = 2
stateOrProvinceName = State or Province Name (full name)
stateOrProvinceName_value = %(state)s
localityName = Locality Name (eg, city)
localityName_value = %(city)s
0.organizationName = Organization Name (eg, company)
0.organizationName_value = %(company)s
# we can do this but it is not needed normally :-)
#1.organizationName = Second Organization Name (eg, company)
#1.organizationName_default = World Wide Web Pty Ltd
commonName = Common Name (eg, your name or your server\'s hostname)
commonName_max = 64
emailAddress = Email Address
emailAddress_value = %(email_address)s
emailAddress_max = 64
# SET-ex3 = SET extension number 3
#[ req_attributes ]
#challengePassword = A challenge password
#challengePassword_min = 4
#challengePassword_max = 20
#
#unstructuredName = An optional company name
[ usr_cert ]
# These extensions are added when 'ca' signs a request.
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment = "OpenSSL Generated Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
# This is required for TSA certificates.
# extendedKeyUsage = critical,timeStamping
[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
[ v3_ca ]
# Extensions for a typical CA
# PKIX recommendation.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer
# This is what PKIX recommends but some broken software chokes on critical
# extensions.
#basicConstraints = critical,CA:true
# So we do this instead.
basicConstraints = CA:true
# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as an test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign
# Some might want this also
# nsCertType = sslCA, emailCA
# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy
# DER hex encoding of an extension: beware experts only!
# obj=DER:02:03
# Where 'obj' is a standard or added object
# You can even override a supported extension:
# basicConstraints= critical, DER:30:03:01:01:FF
[ crl_ext ]
# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.
# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always
[ proxy_cert_ext ]
# These extensions should be added when creating a proxy certificate
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment = "OpenSSL Generated Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
# This really needs to be in place for it to be a proxy certificate.
proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo
####################################################################
[ tsa ]
default_tsa = tsa_config1 # the default TSA section
[ tsa_config1 ]
# These are used by the TSA reply generation only.
dir = /etc/pki/tls # TSA root directory
serial = $dir/tsaserial # The current serial number (mandatory)
crypto_device = builtin # OpenSSL engine to use for signing
signer_cert = $dir/tsacert.pem # The TSA signing certificate
# (optional)
certs = $dir/cacert.pem # Certificate chain to include in reply
# (optional)
signer_key = $dir/private/tsakey.pem # The TSA private key (optional)
default_policy = tsa_policy1 # Policy if request did not specify it
# (optional)
other_policies = tsa_policy2, tsa_policy3 # acceptable policies (optional)
digests = md5, sha1 # Acceptable message digests (mandatory)
accuracy = secs:1, millisecs:500, microsecs:100 # (optional)
clock_precision_digits = 0 # number of digits after dot. (optional)
ordering = yes # Is ordering defined for timestamps?
# (optional, default: no)
tsa_name = yes # Must the TSA name be included in the reply?
# (optional, default: no)
ess_cert_id_chain = no # Must the ESS cert id chain be included?
# (optional, default: no)
foreground = yes
output = %(log)s
pid = %(pid_file)s
syslog = no
CApath = %(ca_path)s
key = %(key)s
CRLpath = %(ca_crl)s
cert = %(cert)s
[service]
accept = %(public_ip)s:%(public_port)s
connect = %(private_ip)s:%(private_port)s
##############################################################################
#
# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from slapos.recipe.librecipe import BaseSlapRecipe
import os
import shutil
import pkg_resources
import zc.buildout
import sys
import zc.recipe.egg
class BaseRecipe(BaseSlapRecipe):
def installMysqlServer(self, ip=None, port=None):
if ip is None:
ip = self.getLocalIPv4Address()
if port is None:
port = '3306'
mysql_conf = dict(
ip=ip,
data_directory=os.path.join(self.data_root_directory,
'mysql'),
tcp_port=port,
pid_file=os.path.join(self.run_directory, 'mysqld.pid'),
socket=os.path.join(self.run_directory, 'mysqld.sock'),
error_log=os.path.join(self.log_directory, 'mysqld.log'),
slow_query_log=os.path.join(self.log_directory,
'mysql-slow.log'),
database='appdb',
user='appuser',
password=self.generatePassword(),
)
self._createDirectory(mysql_conf['data_directory'])
mysql_conf_path = self.createConfigurationFile("my.cnf",
self.substituteTemplate(pkg_resources.resource_filename(__name__, 'template/my.cnf.in'),
mysql_conf))
mysql_script = pkg_resources.resource_string(__name__,
'template/mysqlinit.sql.in') % mysql_conf
self.path_list.extend(zc.buildout.easy_install.scripts([('mysql_update',
__name__ + '.mysql', 'updateMysql')], self.ws,
sys.executable, self.wrapper_directory, arguments=[dict(
mysql_script=mysql_script,
mysql_binary=self.options['mysql_binary'].strip(),
mysql_upgrade_binary=self.options['mysql_upgrade_binary'].strip(),
socket=mysql_conf['socket'],
)]))
self.path_list.extend(zc.buildout.easy_install.scripts([('mysqld',
__name__ + '.mysql', 'runMysql')], self.ws,
sys.executable, self.wrapper_directory, arguments=[dict(
mysql_install_binary=self.options['mysql_install_binary'].strip(),
mysqld_binary=self.options['mysqld_binary'].strip(),
data_directory=mysql_conf['data_directory'].strip(),
mysql_binary=self.options['mysql_binary'].strip(),
socket=mysql_conf['socket'].strip(),
configuration_file=mysql_conf_path,
)]))
self.path_list.extend([mysql_conf_path])
return dict(
mysql_host=mysql_conf['ip'],
mysql_port=mysql_conf['tcp_port'],
mysql_user=mysql_conf['user'],
mysql_password=mysql_conf['password'],
mysql_database=mysql_conf['database'],
)
def createHtdocs(self, source, document_root):
source = self.options['source'].strip()
document_root = self.createDataDirectory('htdocs')
for p in os.listdir(document_root):
path = os.path.join(document_root, p)
if os.path.isdir(path):
shutil.rmtree(path)
else:
os.unlink(path)
for p in os.listdir(source):
path = os.path.join(source, p)
if os.path.isdir(path):
shutil.copytree(path, os.path.join(document_root, p))
else:
shutil.copy2(path, os.path.join(document_root, p))
def installApache(self, document_root, ip=None, port=None):
if ip is None:
ip=self.getGlobalIPv6Address()
if port is None:
port = '9080'
apache_config = dict(
pid_file=os.path.join(self.run_directory, 'httpd.pid'),
lock_file=os.path.join(self.run_directory, 'httpd.lock'),
ip=ip,
port=port,
error_log=os.path.join(self.log_directory, 'httpd-error.log'),
access_log=os.path.join(self.log_directory, 'httpd-access.log'),
document_root=document_root,
php_ini_dir=self.etc_directory
)
config_file = self.createConfigurationFile('httpd.conf',
self.substituteTemplate(pkg_resources.resource_filename(__name__,
'template/apache.in'), apache_config))
self.path_list.append(config_file)
self.path_list.append(self.createConfigurationFile('php.ini',
self.substituteTemplate(pkg_resources.resource_filename(__name__,
'template/php.ini.in'), {})))
self.path_list.extend(zc.buildout.easy_install.scripts([(
'httpd',
__name__ + '.apache', 'runApache')], self.ws,
sys.executable, self.wrapper_directory, arguments=[
dict(
required_path_list=[],
binary=self.options['httpd_binary'],
config=config_file
)
]))
return 'http://[%s]:%s' % (ip, port)
def createConfiguration(self, template, document_root, destination, d):
directory = os.path.dirname(destination)
file = os.path.basename(destination)
path = document_root
if directory:
path = os.path.join(document_root, directory)
if not os.path.exists(path):
os.makedirs(path)
destination = os.path.join(path, file)
open(destination, 'w').write(open(template, 'r').read() % d)
class Static(BaseRecipe):
def _install(self):
self.path_list = []
self.requirements, self.ws = self.egg.working_set()
document_root = self.createDataDirectory('htdocs')
self.createHtdocs(self.options['source'].strip(), document_root)
url = self.installApache(document_root)
self.setConnectionDict(dict(url = url))
return self.path_list
class Simple(BaseRecipe):
def _install(self):
self.path_list = []
self.requirements, self.ws = self.egg.working_set()
document_root = self.createDataDirectory('htdocs')
self.createHtdocs(self.options['source'].strip(), document_root)
mysql_conf = self.installMysqlServer()
url = self.installApache(document_root)
self.setConnectionDict(dict(
url=url,
**mysql_conf
))
self.createConfiguration(self.options['template'], document_root,
self.options['configuration'], mysql_conf)
return self.path_list
class Request(BaseRecipe):
def _install(self):
self.path_list = []
self.requirements, self.ws = self.egg.working_set()
software_type = self.parameter_dict['slap_software_type']
if software_type == 'RootSoftwareInstance':
document_root = self.createDataDirectory('htdocs')
self.createHtdocs(self.options['source'].strip(), document_root)
mysql = self.request(self.software_release_url, 'MySQL Server', 'mysql')
mysql_conf = dict(
mysql_host=mysql.getConnectionParameter('mysql_host'),
mysql_port=mysql.getConnectionParameter('mysql_port'),
mysql_user=mysql.getConnectionParameter('mysql_user'),
mysql_password=mysql.getConnectionParameter('mysql_password'),
mysql_database=mysql.getConnectionParameter('mysql_database'),
)
url = self.installApache(document_root)
self.setConnectionDict(dict(
url=url,
))
self.createConfiguration(self.options['template'], document_root,
self.options['configuration'], mysql_conf)
elif software_type == 'MySQL Server':
mysql_conf = self.installMysqlServer()
self.setConnectionDict(dict(
**mysql_conf
))
else:
raise zc.buildout.UserError('Uknown software type %r' % software_type)
return self.path_list
import os
import sys
import time
def runApache(args):
sleep = 60
conf = args[0]
while True:
ready = True
for f in conf.get('required_path_list', []):
if not os.path.exists(f):
print 'File %r does not exists, sleeping for %s' % (f, sleep)
ready = False
if ready:
break
time.sleep(sleep)
apache_wrapper_list = [conf['binary'], '-f', conf['config'], '-DFOREGROUND']
apache_wrapper_list.extend(sys.argv[1:])
sys.stdout.flush()
sys.stderr.flush()
os.execl(apache_wrapper_list[0], *apache_wrapper_list)
import os
import subprocess
import time
import sys
def runMysql(args):
sleep = 60
conf = args[0]
mysqld_wrapper_list = [conf['mysqld_binary'], '--defaults-file=%s' %
conf['configuration_file']]
# we trust mysql_install that if mysql directory is available mysql was
# correctly initalised
if not os.path.isdir(os.path.join(conf['data_directory'], 'mysql')):
while True:
# XXX: Protect with proper root password
popen = subprocess.Popen([conf['mysql_install_binary'],
'--skip-name-resolve', '--no-defaults', '--datadir=%s' %
conf['data_directory']],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = popen.communicate()[0]
if popen.returncode is None or popen.returncode != 0:
print "Failed to initialise server.\nThe error was: %s" % result
print "Waiting for %ss and retrying" % sleep
time.sleep(sleep)
else:
print "Mysql properly initialised"
break
else:
print "MySQL already initialised"
print "Starting %r" % mysqld_wrapper_list[0]
sys.stdout.flush()
sys.stderr.flush()
os.execl(mysqld_wrapper_list[0], *mysqld_wrapper_list)
def updateMysql(args):
conf = args[0]
sleep = 30
is_succeed = False
while True:
if not is_succeed:
mysql_upgrade_list = [conf['mysql_upgrade_binary'], '--no-defaults', '--user=root', '--socket=%s' % conf['socket']]
mysql_upgrade = subprocess.Popen(mysql_upgrade_list, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = mysql_upgrade.communicate()[0]
if mysql_upgrade.returncode is None:
mysql_upgrade.kill()
if mysql_upgrade.returncode != 0 and not 'is already upgraded' in result:
print "Command %r failed with result:\n%s" % (mysql_upgrade_list, result)
print 'Sleeping for %ss and retrying' % sleep
else:
if mysql_upgrade.returncode == 0:
print "MySQL database upgraded with result:\n%s" % result
else:
print "No need to upgrade MySQL database"
mysql_script = conf.get('mysql_script')
if mysql_script:
mysql_list = [conf['mysql_binary'].strip(), '--no-defaults', '-B', '--user=root', '--socket=%s' % conf['socket']]
mysql = subprocess.Popen(mysql_list, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = mysql.communicate(conf['mysql_script'])[0]
if mysql.returncode is None:
mysql.kill()
if mysql.returncode != 0:
print 'Command %r failed with:\n%s' % (mysql_list, result)
print 'Sleeping for %ss and retrying' % sleep
else:
is_succeed = True
print 'SlapOS initialisation script succesfully applied on database.'
sys.stdout.flush()
sys.stderr.flush()
time.sleep(sleep)
# Apache static configuration
# Automatically generated
# Basic server configuration
PidFile "%(pid_file)s"
LockFile "%(lock_file)s"
Listen %(ip)s:%(port)s
PHPINIDir %(php_ini_dir)s
ServerAdmin someone@email
DefaultType text/plain
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType application/x-httpd-php .php .phtml .php5 .php4
AddType application/x-httpd-php-source .phps
# Log configuration
ErrorLog "%(error_log)s"
LogLevel warn
LogFormat "%%h %%{REMOTE_USER}i %%l %%u %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\"" combined
LogFormat "%%h %%{REMOTE_USER}i %%l %%u %%t \"%%r\" %%>s %%b" common
CustomLog "%(access_log)s" common
# Directory protection
<Directory />
Options FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
</Directory>
<Directory %(document_root)s>
Options FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all
</Directory>
DocumentRoot %(document_root)s
DirectoryIndex index.html index.php
# List of modules
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule headers_module modules/mod_headers.so
LoadModule dir_module modules/mod_dir.so
LoadModule php5_module modules/libphp5.so
# ERP5 buildout my.cnf template based on my-huge.cnf shipped with mysql
# The MySQL server
[mysqld]
# ERP5 by default requires InnoDB storage. MySQL by default fallbacks to using
# different engine, like MyISAM. Such behaviour generates problems only, when
# tables requested as InnoDB are silently created with MyISAM engine.
#
# Loud fail is really required in such case.
sql-mode="NO_ENGINE_SUBSTITUTION"
skip-show-database
port = %(tcp_port)s
bind-address = %(ip)s
socket = %(socket)s
datadir = %(data_directory)s
pid-file = %(pid_file)s
log-error = %(error_log)s
log-slow-file = %(slow_query_log)s
long_query_time = 5
max_allowed_packet = 128M
query_cache_size = 32M
plugin-load = ha_innodb_plugin.so
# The following are important to configure and depend a lot on to the size of
# your database and the available resources.
#innodb_buffer_pool_size = 4G
#innodb_log_file_size = 256M
#innodb_log_buffer_size = 8M
# Some dangerous settings you may want to uncomment if you only want
# performance or less disk access. Useful for unit tests.
#innodb_flush_log_at_trx_commit = 0
#innodb_flush_method = nosync
#innodb_doublewrite = 0
#sync_frm = 0
# Uncomment the following if you need binary logging, which is recommended
# on production instances (either for replication or incremental backups).
#log-bin=mysql-bin
# Force utf8 usage
collation_server = utf8_unicode_ci
character_set_server = utf8
skip-character-set-client-handshake
[mysql]
no-auto-rehash
socket = %(socket)s
[mysqlhotcopy]
interactive-timeout
CREATE DATABASE IF NOT EXISTS %(database)s;
GRANT ALL PRIVILEGES ON %(database)s.* TO %(user)s@localhost IDENTIFIED BY %(password)r;
GRANT ALL PRIVILEGES ON %(database)s.* TO %(user)s@'%%' IDENTIFIED BY %(password)r;
GRANT SHOW DATABASES ON *.* TO %(user)s@localhost IDENTIFIED BY %(password)r;
GRANT SHOW DATABASES ON *.* TO %(user)s@'%%' IDENTIFIED BY %(password)r;
FLUSH PRIVILEGES;
EXIT
[PHP]
engine = On
safe_mode = Off
expose_php = On
error_reporting = E_ALL | E_STRICT
display_errors = On
display_startup_errors = On
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
...@@ -68,9 +68,11 @@ class Recipe(BaseSlapRecipe): ...@@ -68,9 +68,11 @@ class Recipe(BaseSlapRecipe):
self.substituteTemplate(pkg_resources.resource_filename(__name__, self.substituteTemplate(pkg_resources.resource_filename(__name__,
'template/slapos.cfg.in'), configuration)) 'template/slapos.cfg.in'), configuration))
self.path_list.append(config_file) self.path_list.append(config_file)
execute_arguments = dict(path = os.environ['PATH'],
launch_args = [self.options['slaprunner'].strip(), config_file])
self.path_list.extend(zc.buildout.easy_install.scripts([('slaprunner', self.path_list.extend(zc.buildout.easy_install.scripts([('slaprunner',
'slapos.recipe.slaprunner.execute', 'execute')], self.ws, sys.executable, 'slapos.recipe.slaprunner.execute', 'execute')], self.ws, sys.executable,
self.wrapper_directory, arguments=[self.options['slaprunner'].strip(), self.wrapper_directory, arguments=execute_arguments))
config_file]))
self.setConnectionDict(dict(url='http://[%s]:%s' % (ipv6, runner_port))) self.setConnectionDict(dict(url='http://[%s]:%s' % (ipv6, runner_port)))
return self.path_list return self.path_list
...@@ -3,4 +3,6 @@ import os ...@@ -3,4 +3,6 @@ import os
def execute(args): def execute(args):
"""Portable execution with process replacement""" """Portable execution with process replacement"""
os.execv(args[0], args) if args.get("path", None):
os.environ['PATH'] = args["path"]
os.execv(args["launch_args"][0], args["launch_args"])
...@@ -122,7 +122,7 @@ SSLCARevocationPath %(ca_crl)s""" ...@@ -122,7 +122,7 @@ SSLCARevocationPath %(ca_crl)s"""
zodb_configuration_string = '\n'.join(zodb_configuration_list) zodb_configuration_string = '\n'.join(zodb_configuration_list)
zope_port = 12000 zope_port = 12000
# One Distribution Node # One Distribution Node
zope_port +=1 zope_port += 1
self.installZope(ip, zope_port, 'zope_distribution', with_timerservice=True, self.installZope(ip, zope_port, 'zope_distribution', with_timerservice=True,
zodb_configuration_string=zodb_configuration_string, zodb_configuration_string=zodb_configuration_string,
tidstorage_config=tidstorage_config) tidstorage_config=tidstorage_config)
...@@ -224,6 +224,8 @@ SSLCARevocationPath %(ca_crl)s""" ...@@ -224,6 +224,8 @@ SSLCARevocationPath %(ca_crl)s"""
kumo_conf = self.installKumo(self.getLocalIPv4Address()) kumo_conf = self.installKumo(self.getLocalIPv4Address())
self.installTestRunner(ca_conf, mysql_conf, conversion_server_conf, self.installTestRunner(ca_conf, mysql_conf, conversion_server_conf,
memcached_conf, kumo_conf) memcached_conf, kumo_conf)
self.installTestSuiteRunner(ca_conf, mysql_conf, conversion_server_conf,
memcached_conf, kumo_conf)
self.linkBinary() self.linkBinary()
self.setConnectionDict(dict( self.setConnectionDict(dict(
development_zope='http://%s:%s/' % (ip, zope_port), development_zope='http://%s:%s/' % (ip, zope_port),
......
{
"name":"Parameter",
"title": "Default parameter schema for ERP5 Software Release.",
"type": "object",
"additionalProperties": false,
"properties": {
"flavour" : {
"type" : "string",
"title" : "Sofware Release Flavour",
"optional": true,
"default" : "default",
"enum" : ["default", "configurator"]
},
"bt5_list": {
"type":"array",
"title" : "Business Template List",
"items" : {"type": "string"},
"optional" : true
},
"bt5_repository_list": {
"type" : "array",
"title" : "Business Template Repository List",
"items" : {"type": "string"},
"optional" : true
}
}
}
...@@ -66,3 +66,7 @@ link_binary_list = ...@@ -66,3 +66,7 @@ link_binary_list =
products = ${products:list} products = ${products:list}
environment = environment =
LD_LIBRARY_PATH = ${file:location}/lib:${zlib:location}/lib:${freetype:location}/lib:${libXext:location}/lib:${libXau:location}/lib:${libX11:location}/lib:${libXdmcp:location}/lib:${libxcb:location}/lib LD_LIBRARY_PATH = ${file:location}/lib:${zlib:location}/lib:${freetype:location}/lib:${libXext:location}/lib:${libXau:location}/lib:${libX11:location}/lib:${libXdmcp:location}/lib:${libxcb:location}/lib
bt5_repository_list = ${bt5-repository:list}
configurator_bt5_list = erp5_core_proxy_field_legacy erp5_full_text_myisam_catalog erp5_base erp5_workflow erp5_configurator erp5_configurator_standard erp5_configurator_maxma_demo erp5_configurator_ung
[buildout] [buildout]
extends = extends =
../../stack/shacache-client.cfg
../../stack/erp5.cfg ../../stack/erp5.cfg
versions = versions versions = versions
...@@ -7,6 +8,7 @@ versions = versions ...@@ -7,6 +8,7 @@ versions = versions
parts += parts +=
# Create instance template # Create instance template
template template
validator
# XXX: Workaround of SlapOS limitation # XXX: Workaround of SlapOS limitation
# Unzippig of eggs is required, as SlapOS do not yet provide nicely working # Unzippig of eggs is required, as SlapOS do not yet provide nicely working
...@@ -23,6 +25,133 @@ module = erp5 ...@@ -23,6 +25,133 @@ module = erp5
# Default template for erp5 instance. # Default template for erp5 instance.
recipe = slapos.recipe.template recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg url = ${:_profile_base_location_}/instance.cfg
md5sum = 292646776e6b7b9decf4291886a66603 md5sum = 65d9b269e204ba49ac5ff11e891a4b84
output = ${buildout:directory}/template.cfg output = ${buildout:directory}/template.cfg
mode = 0644 mode = 0644
[validator]
# Default json schema for instance parameters.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/configuration.json
md5sum = cbe1d75339c6cb20e1aef818797face1
output = ${buildout:directory}/validator.json
mode = 0644
[versions]
MySQL-python = 1.2.3
Paste = 1.7.5.1
PasteScript = 1.7.3
Products.CMFActionIcons = 2.1.3
Products.CMFCalendar = 2.2.2
Products.CMFCore = 2.2.4
Products.CMFDefault = 2.2.2
Products.CMFTopic = 2.2.1
Products.CMFUid = 2.2.1
Products.DCWorkflowGraph = 0.4nxd001
Products.ExternalEditor = 1.1.0
Products.GenericSetup = 1.6.3
Products.MimetypesRegistry = 2.0.2
Products.PluggableAuthService = 1.7.5
Products.PluginRegistry = 1.3b1
Products.TIDStorage = 5.4.7.dev-r45842
Products.Zelenium = 1.0.3
StructuredText = 2.11.1
Werkzeug = 0.6.2
buildout-versions = 1.6
cElementTree = 1.0.5-20051216
chardet = 1.0.1
cloudooo = 1.2.3
cloudooo.handler.ffmpeg = 0.1
cloudooo.handler.imagemagick = 0.1
cloudooo.handler.ooo = 0.2
cloudooo.handler.pdf = 0.1
csp-eventlet = 0.6.0
elementtree = 1.2.7-20070827-preview
erp5.conflictresolver = 0.3
erp5.recipe.cmmiforcei686 = 0.1.3
erp5diff = 0.8.1.3
eventlet = 0.9.16
feedparser = 5.0.1
five.localsitemanager = 2.0.5
greenlet = 0.3.1
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
http-parser = 0.6.2
ipdb = 0.4
meld3 = 0.6.7
ordereddict = 1.1
paramiko = 1.7.7.1
plone.recipe.command = 1.1
ply = 3.4
psutil = 0.2.1
pycrypto = 2.3
python-ldap = 2.4.0
python-memcached = 1.45
restkit = 3.3.0
rtjp-eventlet = 0.3.2
slapos.cookbook = 0.9
slapos.recipe.template = 1.1
threadframe = 0.2
timerserver = 2.0.2
urlnorm = 1.1.2
uuid = 1.30
validictory = 0.7.1
xupdate-processor = 0.4
# Required by:
# slapos.core==0.8
Flask = 0.7.1
# Required by:
# PasteScript==1.7.3
# cloudooo==1.2.3
PasteDeploy = 1.5.0
# Required by:
# cloudooo==1.2.3
WSGIUtils = 0.7
# Required by:
# cloudooo==1.2.3
# slapos.core==0.8
argparse = 1.1
# Required by:
# slapos.recipe.template==1.1
collective.recipe.template = 1.8
# Required by:
# SOAPpy==0.12.0nxd001
fpconst = 0.7.2
# Required by:
# ipdb==0.4
ipython = 0.10.2
# Required by:
# slapos.cookbook==0.9
netaddr = 0.7.5
# Required by:
# slapos.core==0.8
netifaces = 0.4
# Required by:
# cloudooo==1.2.3
python-magic = 0.4.0.1
# Required by:
# zc.buildout==1.5.3-dev-SlapOS-005
setuptools = 0.6c12dev-r88846
# Required by:
# slapos.cookbook==0.9
slapos.core = 0.8
# Required by:
# slapos.core==0.8
supervisor = 3.0a10
# Required by:
# slapos.cookbook==0.9
xml-marshaller = 0.9.7
...@@ -8,6 +8,11 @@ find-links = http://www.nexedi.org/static/packages/source/slapos.buildout/ ...@@ -8,6 +8,11 @@ find-links = http://www.nexedi.org/static/packages/source/slapos.buildout/
http://dist.repoze.org http://dist.repoze.org
http://www.nexedi.org/static/packages/source/ http://www.nexedi.org/static/packages/source/
# Separate from site eggs
allowed-eggs-from-site-packages =
include-site-packages = false
exec-sitecustomize = false
versions = versions versions = versions
rebootstrap-section = python2.6 rebootstrap-section = python2.6
...@@ -38,6 +43,10 @@ arguments = sys.argv[1:] + ["bootstrap"] ...@@ -38,6 +43,10 @@ arguments = sys.argv[1:] + ["bootstrap"]
section = python2.6 section = python2.6
version = 1 version = 1
[versions]
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-001
[eggs] [eggs]
recipe = zc.recipe.egg recipe = zc.recipe.egg
eggs = eggs =
...@@ -50,429 +59,3 @@ recipe = slapos.recipe.template ...@@ -50,429 +59,3 @@ recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg output = ${buildout:directory}/template.cfg
mode = 0644 mode = 0644
[versions]
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-002
[versions]
AccessControl = 2.13.4
Jinja2 = 2.5.5
Products.OFSP = 2.13.2
Werkzeug = 0.6.2
hexagonit.recipe.cmmi = 1.5.0
lxml = 2.3
manuel = 1.5.0
mechanize = 0.2.5
meld3 = 0.6.7
mr.developer = 1.17
slapos.cookbook = 0.4
slapos.core = 0.2
slapos.recipe.template = 1.1
zope.i18n = 3.7.4
zope.tales = 3.5.1
#Required by:
#Products.ExternalMethod 2.13.0
#Products.PythonScripts 2.13.0
#DocumentTemplate 2.13.1
#Products.ZCTextIndex 2.13.2
#Products.BTreeFolder2 2.13.3
#Zope2 2.13.7
#Products.ZCatalog 2.13.14
Acquisition = 2.13.7nxd001
#Required by:
#Products.PythonScripts 2.13.0
#Zope2 2.13.7
#Products.ZCatalog 2.13.14
DateTime = 3.0b1
#Required by:
#Products.MIMETools 2.13.0
#Products.PythonScripts 2.13.0
#Zope2 2.13.7
#Products.ZCatalog 2.13.14
DocumentTemplate = 2.13.1
#Required by:
#MultiMapping 2.13.0
#Products.ExternalMethod 2.13.0
#Record 2.13.0
#Missing 2.13.1
#Persistence 2.13.2
#Zope2 2.13.7
#Products.ZCatalog 2.13.14
ExtensionClass = 2.13.2
#Required by:
#slapos.core 0.2
Flask = 0.6.1
#Required by:
#Zope2 2.13.7
#Products.ZCatalog 2.13.14
Missing = 2.13.1
#Required by:
#Zope2 2.13.7
MultiMapping = 2.13.0
#Required by:
#Products.ExternalMethod 2.13.0
#Products.OFSP 2.13.2
#Products.ZCTextIndex 2.13.2
#Products.BTreeFolder2 2.13.3
#Zope2 2.13.7
#Products.ZCatalog 2.13.14
Persistence = 2.13.2
#Required by:
#Zope2 2.13.7
Products.BTreeFolder2 = 2.13.3
#Required by:
#Zope2 2.13.7
Products.ExternalMethod = 2.13.0
#Required by:
#Zope2 2.13.7
Products.MIMETools = 2.13.0
#Required by:
#Zope2 2.13.7
Products.MailHost = 2.13.1
#Required by:
#Zope2 2.13.7
Products.PythonScripts = 2.13.0
#Required by:
#Zope2 2.13.7
Products.StandardCacheManagers = 2.13.0
#Required by:
#Zope2 2.13.7
Products.ZCTextIndex = 2.13.2
#Required by:
#Zope2 2.13.7
Products.ZCatalog = 2.13.14
#Required by:
#slapos.cookbook 0.4
PyXML = 0.8.5
#Required by:
#Zope2 2.13.7
Record = 2.13.0
#Required by:
#Products.PythonScripts 2.13.0
#Zope2 2.13.7
RestrictedPython = 3.6.0
#Required by:
#zdaemon 2.0.4
#zLOG 2.11.1
#Zope2 2.13.7
#ZODB3 3.10.3
ZConfig = 2.9.0
#Required by:
#Products.ExternalMethod 2.13.0
#Products.BTreeFolder2 2.13.3
#Zope2 2.13.7
#zope.container 3.12.0
ZODB3 = 3.10.3
#Required by:
#slapos.cookbook 0.4
Zope2 = 2.13.7
#Required by:
#Zope2 2.13.7
ZopeUndo = 2.12.0
#Required by:
#mr.developer 1.17
#slapos.core 0.2
argparse = 1.2.1
#Required by:
#slapos.recipe.template 1.1
collective.recipe.template = 1.8
#Required by:
#Zope2 2.13.7
docutils = 0.7
#Required by:
#hexagonit.recipe.cmmi 1.5.0
hexagonit.recipe.download = 1.5.0
#Required by:
#Zope2 2.13.7
initgroups = 2.13.0
#Required by:
#slapos.core 0.2
#slapos.cookbook 0.4
netaddr = 0.7.5
#Required by:
#slapos.core 0.2
netifaces = 0.5
#Required by:
#Zope2 2.13.7
#zope.i18n 3.7.4
#zope.testbrowser 4.0.2
pytz = 2011g
#Required by:
#zc.buildout 1.5.3-dev-SlapOS-001
setuptools = 0.6c12dev-r88846
#Required by:
#slapos.core 0.2
supervisor = 3.0a10
#Required by:
#Zope2 2.13.7
tempstorage = 2.12.1
#Required by:
#Products.StandardCacheManagers 2.13.0
#Zope2 2.13.7
#zope.sendmail 3.7.4
transaction = 1.1.1
#Required by:
#slapos.cookbook 0.4
xml-marshaller = 0.9.7
#Required by:
#Products.PythonScripts 2.13.0
#Zope2 2.13.7
zExceptions = 2.13.0
#Required by:
#Zope2 2.13.7
zLOG = 2.11.1
#Required by:
#ZODB3 3.10.3
zc.lockfile = 1.0.0
#Required by:
#slapos.cookbook 0.4
zc.recipe.egg = 1.3.2
#Required by:
#Zope2 2.13.7
zdaemon = 2.0.4
#Required by:
#zope.site 3.9.2
zope.annotation = 3.5.0
#Required by:
#zope.container 3.12.0
zope.broken = 3.6.0
#Required by:
#Zope2 2.13.7
#zope.browsermenu 3.9.1
#zope.publisher 3.12.6
zope.browser = 1.3
#Required by:
#Zope2 2.13.7
zope.browsermenu = 3.9.1
#Required by:
#Zope2 2.13.7
zope.browserpage = 3.12.2
#Required by:
#Zope2 2.13.7
#zope.ptresource 3.9.0
zope.browserresource = 3.12.0
#Required by:
#Products.StandardCacheManagers 2.13.0
#Zope2 2.13.7
#zope.pagetemplate 3.5.2
#zope.lifecycleevent 3.7.0
#zope.contentprovider 3.7.2
#zope.viewlet 3.7.2
#zope.i18n 3.7.4
#zope.security 3.8.2
#zope.container 3.12.0
#zope.publisher 3.12.6
#zope.traversing 3.14.0
zope.component = 3.10.0
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
#zope.sendmail 3.7.4
#zope.publisher 3.12.6
zope.configuration = 3.7.4
#Required by:
#Products.BTreeFolder2 2.13.3
#Zope2 2.13.7
#zope.site 3.9.2
zope.container = 3.12.0
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
zope.contentprovider = 3.7.2
#Required by:
#Zope2 2.13.7
zope.contenttype = 3.5.3
#Required by:
#Zope2 2.13.7
zope.deferredimport = 3.5.3
#Required by:
#Products.ZCatalog 2.13.14
#zope.container 3.12.0
zope.dottedname = 3.4.6
#Required by:
#Products.BTreeFolder2 2.13.3
#Zope2 2.13.7
#zope.lifecycleevent 3.7.0
#zope.viewlet 3.7.2
#zope.schema 3.8.0
#zope.site 3.9.2
#zope.publisher 3.12.6
zope.event = 3.5.0-1
#Required by:
#Zope2 2.13.7
#zope.testing 3.10.2
#zope.publisher 3.12.6
zope.exceptions = 3.6.1
#Required by:
#zope.container 3.12.0
zope.filerepresentation = 3.6.0
#Required by:
#Zope2 2.13.7
#zope.size 3.4.1
#zope.pagetemplate 3.5.2
#zope.tal 3.5.2
#zope.viewlet 3.7.2
#zope.sendmail 3.7.4
#zope.security 3.8.2
#zope.traversing 3.14.0
zope.i18nmessageid = 3.5.3
#Required by:
#slapos.core 0.2
#Zope2 2.13.7
zope.interface = 3.6.3
#Required by:
#Products.BTreeFolder2 2.13.3
#Zope2 2.13.7
#zope.site 3.9.2
zope.lifecycleevent = 3.7.0
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
#zope.security 3.8.2
#zope.publisher 3.12.6
zope.location = 3.9.0
#Required by:
#Zope2 2.13.7
#zope.ptresource 3.9.0
zope.pagetemplate = 3.5.2
#Required by:
#Zope2 2.13.7
zope.processlifetime = 1.0
#Required by:
#Zope2 2.13.7
#zope.publisher 3.12.6
#zope.traversing 3.14.0
zope.proxy = 3.6.1
#Required by:
#Zope2 2.13.7
zope.ptresource = 3.9.0
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
#zope.traversing 3.14.0
zope.publisher = 3.12.6
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
#zope.sendmail 3.7.4
#zope.security 3.8.2
#zope.testbrowser 4.0.2
zope.schema = 3.8.0
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
#zope.site 3.9.2
#zope.traversing 3.14.0
zope.security = 3.8.2
#Required by:
#Zope2 2.13.7
zope.sendmail = 3.7.4
#Required by:
#Zope2 2.13.7
zope.sequencesort = 3.4.0
#Required by:
#Zope2 2.13.7
zope.site = 3.9.2
#Required by:
#Zope2 2.13.7
zope.size = 3.4.1
#Required by:
#Zope2 2.13.7
zope.structuredtext = 3.5.1
#Required by:
#Zope2 2.13.7
#zope.tales 3.5.1
zope.tal = 3.5.2
#Required by:
#Zope2 2.13.7
zope.testbrowser = 4.0.2
#Required by:
#Zope2 2.13.7
zope.testing = 3.10.2
#Required by:
#Zope2 2.13.7
#zope.viewlet 3.7.2
zope.traversing = 3.14.0
#Required by:
#Zope2 2.13.7
zope.viewlet = 3.7.2
[buildout] [buildout]
extensions =
slapos.zcbworkarounds
slapos.rebootstrap
find-links += find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/ http://www.nexedi.org/static/packages/source/slapos.buildout/
...@@ -15,6 +10,19 @@ extends = ...@@ -15,6 +10,19 @@ extends =
../../component/stunnel/buildout.cfg ../../component/stunnel/buildout.cfg
../../component/rdiff-backup/buildout.cfg ../../component/rdiff-backup/buildout.cfg
../../component/lxml-python/buildout.cfg ../../component/lxml-python/buildout.cfg
../../stack/shacache-client.cfg
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
versions = versions versions = versions
...@@ -31,11 +39,6 @@ parts += ...@@ -31,11 +39,6 @@ parts +=
# development / fast switching environment for whole software # development / fast switching environment for whole software
unzip = true unzip = true
[rebootstrap]
# Default first version of rebootstrapped python
version = 2
section = python2.7
[instance-recipe] [instance-recipe]
egg = slapos.cookbook egg = slapos.cookbook
module = kumofs module = kumofs
...@@ -60,13 +63,13 @@ output = ${buildout:directory}/template.cfg ...@@ -60,13 +63,13 @@ output = ${buildout:directory}/template.cfg
mode = 0644 mode = 0644
[versions] [versions]
slapos.cookbook = 0.4 slapos.cookbook = 0.13
erp5.recipe.cmmiforcei686 = 0.1.1 erp5.recipe.cmmiforcei686 = 0.1.1
hexagonit.recipe.cmmi = 1.5.0 hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0 hexagonit.recipe.download = 1.5.0
# Required by slapos.cookbook==0.4 # Required by slapos.cookbook==0.13
slapos.core = 0.2 slapos.core = 0.2
collective.recipe.template = 1.8 collective.recipe.template = 1.8
netaddr = 0.7.5 netaddr = 0.7.5
...@@ -74,4 +77,4 @@ xml-marshaller = 0.9.7 ...@@ -74,4 +77,4 @@ xml-marshaller = 0.9.7
setuptools = 0.6c12dev-r88795 setuptools = 0.6c12dev-r88795
# Use SlapOS patched zc.buildout # Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-001 zc.buildout = 1.5.3-dev-SlapOS-005
[buildout]
parts =
kvminstance
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
[kvminstance]
recipe = slapos.cookbook:kvm
qemu_path = ${kvm:location}/bin/qemu-system-x86_64
qemu_img_path = ${kvm:location}/bin/qemu-img
#slapmonitor_path = ${buildout:bin-directory}/slapmonitor
#slapreport_path = ${buildout:bin-directory}/slapreport
websockify_path = ${noVNC:location}/utils/wsproxy.py
noVNC_location = ${noVNC:location}
openssl_binary = ${openssl:location}/bin/openssl
rdiff_backup_binary = ${buildout:bin-directory}/rdiff-backup
dcrond_binary = ${dcron:location}/sbin/crond
smp_count = 1
ram_size = 1024
disk_size = 10
[buildout]
extends =
../../stack/kvm.cfg
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = d899f2111aab18ad25776f35ed49a91b
output = ${buildout:directory}/template.cfg
mode = 0644
[kvmsource]
command =
(${git:location}/bin/git clone --quiet http://git.erp5.org/repos/slapos.kvm.git ${:location} && cd ${:location} && git reset --hard 94ee45cc02e69798cac8209d2296fd1751125018) || (rm -fr ${:location} ; exit 1)
update-command =
[versions]
Jinja2 = 2.5.5
Werkzeug = 0.6.2
hexagonit.recipe.cmmi = 1.5.0
lxml = 2.3
meld3 = 0.6.7
plone.recipe.command = 1.1
slapos.cookbook = 0.15
slapos.recipe.template = 1.1
z3c.recipe.scripts = 1.0.1
# Required by:
# slapos.core==0.9
Flask = 0.7.2
# Required by:
# slapos.cookbook==0.15
PyXML = 0.8.4
# Required by:
# slapos.recipe.template==1.1
collective.recipe.template = 1.8
# Required by:
# hexagonit.recipe.cmmi==1.5.0
hexagonit.recipe.download = 1.5.0
# Required by:
# slapos.cookbook==0.15
netaddr = 0.7.5
# Required by:
# slapos.core==0.9
netifaces = 0.5
# Required by:
# slapos.cookbook==0.15
# slapos.core==0.9
# zc.buildout==1.5.3-dev-SlapOS-005
# zc.recipe.egg==1.3.2
setuptools = 0.6c12dev-r88846
# Required by:
# slapos.cookbook==0.15
slapos.core = 0.9
# Required by:
# slapos.core==0.9
supervisor = 3.0a10
# Required by:
# slapos.cookbook==0.15
xml-marshaller = 0.9.7
# Required by:
# slapos.cookbook==0.15
zc.recipe.egg = 1.3.2
# Required by:
# slapos.core==0.9
zope.interface = 3.6.4
[buildout]
parts =
instance
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
[instance]
recipe = ${instance-recipe:egg}:${instance-recipe:module}
dcrond_binary = ${dcron:location}/sbin/crond
innobackupex_binary = ${xtrabackup:location}/bin/innobackupex
logrotate_binary = ${logrotate:location}/usr/sbin/logrotate
mysql_binary = ${mariadb:location}/bin/mysql
mysql_install_binary = ${mariadb:location}/bin/mysql_install_db
mysql_upgrade_binary = ${mariadb:location}/bin/mysql_upgrade
mysqld_binary = ${mariadb:location}/libexec/mysqld
openssl_binary = ${openssl:location}/bin/openssl
perl_binary = ${perl:location}/bin/perl
rdiff_backup_binary = ${buildout:bin-directory}/rdiff-backup
stunnel_binary = ${stunnel:location}/bin/stunnel
[buildout]
extensions =
slapos.zcbworkarounds
slapos.rebootstrap
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
extends =
../../component/mariadb/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/stunnel/buildout.cfg
../../component/python-2.7/buildout.cfg
../../component/perl/buildout.cfg
../../component/xtrabackup/buildout.cfg
../../component/rdiff-backup/buildout.cfg
../../component/lxml-python/buildout.cfg
../../stack/shacache-client.cfg
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
versions = versions
parts +=
# Create instance template
#TODO : list here all parts.
template
libxslt
eggs
instance-recipe-egg
# XXX: Workaround of SlapOS limitation
# Unzippig of eggs is required, as SlapOS do not yet provide nicely working
# development / fast switching environment for whole software
unzip = true
[rebootstrap]
# Default first version of rebootstrapped python
version = 2
section = python2.7
[instance-recipe]
egg = slapos.cookbook
module = mysql
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[eggs]
recipe = zc.recipe.egg
python = python2.7
eggs =
${lxml-python:egg}
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = 69c32a67c5640d36ee042d2cfc35843d
output = ${buildout:directory}/template.cfg
mode = 0644
[versions]
slapos.cookbook = 0.9
# Required by slapos.cookbook==0.9
slapos.core = 0.4
collective.recipe.template = 1.8
netaddr = 0.7.5
xml-marshaller = 0.9.7
setuptools = 0.6c12dev-r88795
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
plone.recipe.command = 1.1
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-005
...@@ -14,7 +14,20 @@ extends = ...@@ -14,7 +14,20 @@ extends =
../../component/stunnel/buildout.cfg ../../component/stunnel/buildout.cfg
../../component/rdiff-backup/buildout.cfg ../../component/rdiff-backup/buildout.cfg
../../component/lxml-python/buildout.cfg ../../component/lxml-python/buildout.cfg
../../stack/shacache-client.cfg
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
versions = versions versions = versions
parts = parts =
...@@ -59,9 +72,9 @@ output = ${buildout:directory}/template.cfg ...@@ -59,9 +72,9 @@ output = ${buildout:directory}/template.cfg
mode = 0644 mode = 0644
[versions] [versions]
slapos.cookbook = 0.4 slapos.cookbook = 0.7
# Required by slapos.cookbook==0.4 # Required by slapos.cookbook==0.7
slapos.core = 0.2 slapos.core = 0.2
collective.recipe.template = 1.8 collective.recipe.template = 1.8
netaddr = 0.7.5 netaddr = 0.7.5
...@@ -73,4 +86,4 @@ hexagonit.recipe.download = 1.5.0 ...@@ -73,4 +86,4 @@ hexagonit.recipe.download = 1.5.0
plone.recipe.command = 1.1 plone.recipe.command = 1.1
# Use SlapOS patched zc.buildout # Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-001 zc.buildout = 1.5.3-dev-SlapOS-005
[buildout]
parts =
instance
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
[instance]
recipe = ${instance-recipe:egg}:${instance-recipe:module}
dcrond_binary = ${dcron:location}/sbin/crond
innobackupex_binary = ${xtrabackup:location}/bin/innobackupex
logrotate_binary = ${logrotate:location}/usr/sbin/logrotate
mysql_binary = ${mysql-5.1:location}/bin/mysql
mysql_install_binary = ${mysql-5.1:location}/bin/mysql_install_db
mysql_upgrade_binary = ${mysql-5.1:location}/bin/mysql_upgrade
mysqld_binary = ${mysql-5.1:location}/libexec/mysqld
openssl_binary = ${openssl:location}/bin/openssl
perl_binary = ${perl:location}/bin/perl
rdiff_backup_binary = ${buildout:bin-directory}/rdiff-backup
stunnel_binary = ${stunnel:location}/bin/stunnel
[buildout]
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
extends =
../../component/mysql-5.1/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/stunnel/buildout.cfg
../../component/python-2.7/buildout.cfg
../../component/perl/buildout.cfg
../../component/xtrabackup/buildout.cfg
../../component/rdiff-backup/buildout.cfg
../../component/lxml-python/buildout.cfg
../../stack/shacache-client.cfg
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
versions = versions
parts +=
#TODO : list here all parts.
# Create instance template
template
libxslt
eggs
instance-recipe-egg
rdiff-backup
# XXX: Workaround of SlapOS limitation
# Unzippig of eggs is required, as SlapOS do not yet provide nicely working
# development / fast switching environment for whole software
unzip = true
[instance-recipe]
egg = slapos.cookbook
module = mysql
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[eggs]
recipe = zc.recipe.egg
python = python2.7
eggs =
${lxml-python:egg}
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = 2764597a6e4fe243cdf6e37b6535e767
output = ${buildout:directory}/template.cfg
mode = 0644
[versions]
slapos.cookbook = 0.13
# Required by slapos.cookbook==0.13
slapos.core = 0.4
collective.recipe.template = 1.8
netaddr = 0.7.5
xml-marshaller = 0.9.7
setuptools = 0.6c12dev-r88795
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
plone.recipe.command = 1.1
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-005
[buildout]
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
parts = instance
[instance]
recipe = ${instance-recipe:egg}:${instance-recipe:module}
source = ${application:location}
template = ${application-template:location}/${application-template:filename}
configuration = ${application-configuration:location}
httpd_binary = ${apache:location}/bin/httpd
mysql_binary = ${mariadb:location}/bin/mysql
mysql_install_binary = ${mariadb:location}/bin/mysql_install_db
mysql_upgrade_binary = ${mariadb:location}/bin/mysql_upgrade
mysqld_binary = ${mariadb:location}/libexec/mysqld
<?php
$cfg['blowfish_secret'] = ''; /* YOU MUST FILL IN THIS FOR COOKIE AUTH! */
$i = 0;
$i++;
/* Server parameters */
$cfg['Servers'][$i]['host'] = '%(mysql_host)s';
$cfg['Servers'][$i]['port'] = '%(mysql_port)s';
/* Authentication type */
$cfg['Servers'][$i]['auth_type'] = 'cookie';
$cfg['Servers'][$i]['user'] = '';
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['compress'] = false;
/* Select mysqli if your server has it */
$cfg['Servers'][$i]['extension'] = 'mysql';
$cfg['Servers'][$i]['AllowNoPassword'] = false;
/* rajk - for blobstreaming */
$cfg['Servers'][$i]['bs_garbage_threshold'] = 50;
$cfg['Servers'][$i]['bs_repository_threshold'] = '32M';
$cfg['Servers'][$i]['bs_temp_blob_timeout'] = 600;
$cfg['Servers'][$i]['bs_temp_log_threshold'] = '32M';
$cfg['UploadDir'] = '';
$cfg['SaveDir'] = '';
?>
\ No newline at end of file
[buildout]
versions = versions
parts =
template
apache-php
mariadb
eggs
instance-recipe-egg
extends =
../../stack/lamp.cfg
../../stack/shacache-client.cfg
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = efdb8509f40c86b1b73924fc1ce92f13
output = ${buildout:directory}/template.cfg
mode = 0644
[application]
recipe = hexagonit.recipe.download
url = http://downloads.sourceforge.net/project/phpmyadmin/phpMyAdmin/3.3.10/phpMyAdmin-3.3.10-all-languages.tar.bz2?r=http%3A%2F%2Fwww.phpmyadmin.net%2Fhome_page%2Fdownloads.php&ts=1300959842&use_mirror=sunet
#md5sum = Student may put here md5sum of this file, this is good idea
[application-template]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/phpmyadmin.inc.php.in
md5sum = caab45c34c75661c214f4628ff545bb4
download-only = True
filename = template.in
mode = 0644
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[application-configuration]
location = config.inc.php
[instance-recipe]
egg = slapos.cookbook
module = osoeslaptraining.simple
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[versions]
slapos.cookbook = 0.12
# Required by slapos.cookbook==0.12
slapos.core = 0.8
collective.recipe.template = 1.8
netaddr = 0.7.5
xml-marshaller = 0.9.7
setuptools = 0.6c12dev-r88795
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
plone.recipe.command = 1.1
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-005
[buildout]
parts =
instance
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
[instance]
recipe = ${instance-recipe:egg}:${instance-recipe:module}
slaprunner = ${buildout:directory}/bin/slaprunner
slapgrid_sr = ${buildout:directory}/bin/slapgrid-sr
slapgrid_cp = ${buildout:directory}/bin/slapgrid-cp
slapproxy = ${buildout:directory}/bin/slapproxy
supervisor = ${buildout:directory}/bin/slapgrid-supervisorctl
[buildout]
extends =
../../stack/flask.cfg
../../stack/shacache-client.cfg
parts =
template
eggs
instance-recipe-egg
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
versions = versions
[instance-recipe]
egg = slapos.cookbook
module = slaprunner
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg
mode = 0644
md5sum = e2cbd8fe7b8e4c7e92a19cd775de0aa6
[eggs]
eggs +=
slapos.libnetworkcache
slapos.toolbox
slapos.core
[versions]
slapos.cookbook = 0.12
# Required by slapos.cookbook==0.12
slapos.core = 0.8
collective.recipe.template = 1.8
netaddr = 0.7.5
xml-marshaller = 0.9.7
setuptools = 0.6c12dev-r88795
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
plone.recipe.command = 1.1
slapos.libnetworkcache = 0.2
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-005
...@@ -9,7 +9,6 @@ parts += ...@@ -9,7 +9,6 @@ parts +=
eggs += slapos.core eggs += slapos.core
[instance-recipe] [instance-recipe]
egg = slapos.cookbook
module = vifib module = vifib
[vifib] [vifib]
...@@ -21,8 +20,15 @@ recipe = plone.recipe.command ...@@ -21,8 +20,15 @@ recipe = plone.recipe.command
location = ${buildout:parts-directory}/${:_buildout_section_name_} location = ${buildout:parts-directory}/${:_buildout_section_name_}
stop-on-error = true stop-on-error = true
repository = http://git.erp5.org/repos/slapos.core.git repository = http://git.erp5.org/repos/slapos.core.git
command = ${git:location}/bin/git clone --quiet ${:repository} ${:location} branch = master
update-command = cd ${:location} && ${git:location}/bin/git pull --quiet revision = f95ca3ccda07292895939ef9b48678acb5f524ce
command = ${git:location}/bin/git clone --quiet -b ${:branch} ${:repository} ${:location} && if [ -n ${:revision} ]; then cd ${:location} && ${git:location}/bin/git reset --quiet --hard ${:revision} ; fi
update-command = cd ${:location} && ${git:location}/bin/git pull --quiet && if [ -n ${:revision} ]; then cd ${:location} && ${git:location}/bin/git reset --quiet --hard ${:revision} ; fi
[local-bt5-repository]
# XXX: workaround for zc.buildout bug, as list += ends up with adding new entry
# after newline
list = ${erp5:location}/bt5 ${erp5:location}/product/ERP5/bootstrap ${vifib:location}/master/bt5
[products] [products]
# XXX: Lack of eggification workaround # XXX: Lack of eggification workaround
......
[buildout]
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
parts = instance
[instance]
recipe = ${instance-recipe:egg}:${instance-recipe:module}
source = ${application:location}
template = ${application-template:location}/${application-template:filename}
configuration = ${application-configuration:location}
httpd_binary = ${apache:location}/bin/httpd
mysql_binary = ${mariadb:location}/bin/mysql
mysql_install_binary = ${mariadb:location}/bin/mysql_install_db
mysql_upgrade_binary = ${mariadb:location}/bin/mysql_upgrade
mysqld_binary = ${mariadb:location}/libexec/mysqld
[buildout]
versions = versions
parts =
template
apache-php
mariadb
eggs
instance-recipe-egg
extends =
../../stack/lamp.cfg
../../stack/shacache-client.cfg
[application]
url = http://wordpress.org/latest.tar.gz
#md5sum = Student may put here md5sum of this file, this is good idea
[application-template]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/wp-config.php.in
#md5sum = ${application-configuration:md5sum}
download-only = True
filename = template.in
mode = 0644
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[application-configuration]
location = wp-config.php
[instance-recipe]
egg = slapos.cookbook
module = osoeslaptraining.request
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = efdb8509f40c86b1b73924fc1ce92f13
output = ${buildout:directory}/template.cfg
mode = 0644
[versions]
slapos.cookbook = 0.12
# Required by slapos.cookbook==0.12
slapos.core = 0.8
collective.recipe.template = 1.8
netaddr = 0.7.5
xml-marshaller = 0.9.7
setuptools = 0.6c12dev-r88795
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
plone.recipe.command = 1.1
# Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-005
<?php
/**
* The base configurations of the WordPress.
*
* This file has the following configurations: MySQL settings, Table Prefix,
* Secret Keys, WordPress Language, and ABSPATH. You can find more information
* by visiting {@link http://codex.wordpress.org/Editing_wp-config.php Editing
* wp-config.php} Codex page. You can get the MySQL settings from your web host.
*
* This file is used by the wp-config.php creation script during the
* installation. You don't have to use the web site, you can just copy this file
* to "wp-config.php" and fill in the values.
*
* @package WordPress
*/
// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', '%(mysql_database)s');
/** MySQL database username */
define('DB_USER', '%(mysql_user)s');
/** MySQL database password */
define('DB_PASSWORD', '%(mysql_password)s');
/** MySQL hostname */
define('DB_HOST', '%(mysql_host)s');
/** Database Charset to use in creating database tables. */
define('DB_CHARSET', 'utf8');
/** The Database Collate type. Don't change this if in doubt. */
define('DB_COLLATE', '');
/**#@+
* Authentication Unique Keys and Salts.
*
* Change these to different unique phrases!
* You can generate these using the {@link https://api.wordpress.org/secret-key/1.1/salt/ WordPress.org secret-key service}
* You can change these at any point in time to invalidate all existing cookies. This will force all users to have to log in again.
*
* @since 2.6.0
*/
define('AUTH_KEY', 'put your unique phrase here');
define('SECURE_AUTH_KEY', 'put your unique phrase here');
define('LOGGED_IN_KEY', 'put your unique phrase here');
define('NONCE_KEY', 'put your unique phrase here');
define('AUTH_SALT', 'put your unique phrase here');
define('SECURE_AUTH_SALT', 'put your unique phrase here');
define('LOGGED_IN_SALT', 'put your unique phrase here');
define('NONCE_SALT', 'put your unique phrase here');
/**#@-*/
/**
* WordPress Database Table prefix.
*
* You can have multiple installations in one database if you give each a unique
* prefix. Only numbers, letters, and underscores please!
*/
$table_prefix = 'wp_';
/**
* WordPress Localized Language, defaults to English.
*
* Change this to localize WordPress. A corresponding MO file for the chosen
* language must be installed to wp-content/languages. For example, install
* de_DE.mo to wp-content/languages and set WPLANG to 'de_DE' to enable German
* language support.
*/
define('WPLANG', '');
/**
* For developers: WordPress debugging mode.
*
* Change this to true to enable the display of notices during development.
* It is strongly recommended that plugin and theme developers use WP_DEBUG
* in their development environments.
*/
define('WP_DEBUG', false);
/* That's all, stop editing! Happy blogging. */
/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') )
define('ABSPATH', dirname(__FILE__) . '/');
/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');
...@@ -22,12 +22,13 @@ allow-hosts = ...@@ -22,12 +22,13 @@ allow-hosts =
dist.repoze.org dist.repoze.org
effbot.org effbot.org
github.com github.com
peak.telecommunity.com
psutil.googlecode.com psutil.googlecode.com
www.dabeaz.com www.dabeaz.com
extends = extends =
# Exact version of Zope # Exact version of Zope
http://svn.zope.org/repos/main/Zope/tags/2.12.18/versions.cfg http://svn.zope.org/repos/main/Zope/tags/2.12.19/versions.cfg
../component/logrotate/buildout.cfg ../component/logrotate/buildout.cfg
../component/dcron/buildout.cfg ../component/dcron/buildout.cfg
../component/file/buildout.cfg ../component/file/buildout.cfg
...@@ -51,6 +52,7 @@ extends = ...@@ -51,6 +52,7 @@ extends =
../component/python-2.7/buildout.cfg ../component/python-2.7/buildout.cfg
../component/python-ldap-python/buildout.cfg ../component/python-ldap-python/buildout.cfg
../component/rdiff-backup/buildout.cfg ../component/rdiff-backup/buildout.cfg
../component/sphinx/buildout.cfg
../component/stunnel/buildout.cfg ../component/stunnel/buildout.cfg
../component/subversion/buildout.cfg ../component/subversion/buildout.cfg
../component/tesseract/buildout.cfg ../component/tesseract/buildout.cfg
...@@ -83,6 +85,7 @@ parts = ...@@ -83,6 +85,7 @@ parts =
libpng12 libpng12
ghostscript ghostscript
mariadb mariadb
sphinx
imagemagick imagemagick
kumo kumo
libreoffice-bin libreoffice-bin
...@@ -119,14 +122,36 @@ parts = ...@@ -119,14 +122,36 @@ parts =
# get git repositories # get git repositories
erp5 erp5
genbt5list
[bt5-repository]
# Format:
# <url or path> [...]
#
# Use absolute paths for local repositories, and URLs for non-local otherwise.
#
list = ${local-bt5-repository:list}
[local-bt5-repository]
# Same as bt5-repository, but only local repository.
# Used to generate bt5lists.
list = ${erp5:location}/bt5 ${erp5:location}/product/ERP5/bootstrap
[genbt5list]
recipe = plone.recipe.command
stop-on-error = true
genbt5list = ${erp5:location}/product/ERP5/bin/genbt5list
command =
${buildout:executable} ${:genbt5list} ${local-bt5-repository:list}
update-command = ${:command}
[bootstrap2.6] [bootstrap2.6]
python = python2.6 python = python2.6
[rebootstrap] [rebootstrap]
# Default first version of rebootstrapped python version = 3
version = 2
section = python2.7 section = python2.7
eggs = slapos.libnetworkcache
[itools] [itools]
pkgname = itools-0.50.8 pkgname = itools-0.50.8
...@@ -190,8 +215,9 @@ location = ${buildout:parts-directory}/${:_buildout_section_name_} ...@@ -190,8 +215,9 @@ location = ${buildout:parts-directory}/${:_buildout_section_name_}
stop-on-error = true stop-on-error = true
repository = http://git.erp5.org/repos/erp5.git repository = http://git.erp5.org/repos/erp5.git
branch = master branch = master
command = ${git:location}/bin/git clone --quiet -b ${:branch} ${:repository} ${:location} revision = 336a8d63bdcabd92bfe3d9466685e5cd47fad716
update-command = cd ${:location} && ${git:location}/bin/git pull --quiet command = ${git:location}/bin/git clone --quiet -b ${:branch} ${:repository} ${:location} && if [ -n ${:revision} ]; then cd ${:location} && ${git:location}/bin/git reset --quiet --hard ${:revision} ; fi
update-command = cd ${:location} && ${git:location}/bin/git pull --quiet && if [ -n ${:revision} ]; then cd ${:location} && ${git:location}/bin/git reset --quiet --hard ${:revision} ; fi
[products] [products]
# XXX: ERP5 related products are not defined as python distributions, so it is # XXX: ERP5 related products are not defined as python distributions, so it is
...@@ -217,14 +243,16 @@ initialization = ...@@ -217,14 +243,16 @@ initialization =
import Zope2 import Zope2
os.environ['SOFTWARE_HOME'] = os.path.abspath(os.path.dirname(os.path.dirname(Zope2.__file__))) os.environ['SOFTWARE_HOME'] = os.path.abspath(os.path.dirname(os.path.dirname(Zope2.__file__)))
os.environ['ZOPE_SCRIPTS'] = '' os.environ['ZOPE_SCRIPTS'] = ''
temp_bt5_path_list = ['/'.join(['''${buildout:parts-directory}''', x, 'bt5']) for x in '''${erp5_repository_list:repository_id_list}'''.split(' ')] parts_directory = '''${buildout:parts-directory}'''
repository_id_list = list(reversed('''${erp5_repository_list:repository_id_list}'''.split()))
temp_bt5_path_list = ['/'.join([parts_directory, x, 'bt5']) for x in repository_id_list]
bt5_path_list = [] bt5_path_list = []
[bt5_path_list.extend([bt5_path, '%s/*' % bt5_path]) for bt5_path in temp_bt5_path_list] [bt5_path_list.extend([bt5_path, '%s/*' % bt5_path]) for bt5_path in temp_bt5_path_list]
os.environ['erp5_tests_bt5_path'] = ','.join(bt5_path_list) os.environ['erp5_tests_bt5_path'] = ','.join(bt5_path_list)
sys.path[0:0] = ['/'.join(['''${buildout:parts-directory}''', x, 'tests']) for x in '''${erp5_repository_list:repository_id_list}'''.split(' ')] sys.path[0:0] = ['/'.join([parts_directory, x, 'tests']) for x in repository_id_list]
import glob import glob
product_test_path_list = [] product_test_path_list = []
[product_test_path_list.extend(glob.glob('/'.join(['''${buildout:parts-directory}''', x, 'product/*/tests']))) for x in '''${erp5_repository_list:repository_id_list}'''.split(' ')] [product_test_path_list.extend(glob.glob('/'.join([parts_directory, x, 'product/*/tests']))) for x in repository_id_list]
sys.path[0:0] = product_test_path_list sys.path[0:0] = product_test_path_list
[test_suite_runner] [test_suite_runner]
...@@ -246,7 +274,8 @@ initialization = ...@@ -246,7 +274,8 @@ initialization =
import Zope2 import Zope2
os.environ['SOFTWARE_HOME'] = os.path.abspath(os.path.dirname(os.path.dirname(Zope2.__file__))) os.environ['SOFTWARE_HOME'] = os.path.abspath(os.path.dirname(os.path.dirname(Zope2.__file__)))
os.environ['ZOPE_SCRIPTS'] = '' os.environ['ZOPE_SCRIPTS'] = ''
sys.path[0:0] = ['/'.join(['''${buildout:parts-directory}''', x]) for x in '''${erp5_repository_list:repository_id_list}'''.split(' ')] repository_id_list = list(reversed('''${erp5_repository_list:repository_id_list}'''.split()))
sys.path[0:0] = ['/'.join(['''${buildout:parts-directory}''', x]) for x in repository_id_list]
[instance-recipe-egg] [instance-recipe-egg]
recipe = zc.recipe.egg recipe = zc.recipe.egg
...@@ -364,7 +393,7 @@ scripts = ...@@ -364,7 +393,7 @@ scripts =
[versions] [versions]
# Use SlapOS patched zc.buildout # Use SlapOS patched zc.buildout
zc.buildout = 1.5.3-dev-SlapOS-001 zc.buildout = 1.5.3-dev-SlapOS-005
# pin Acquisition and Products.DCWorkflow to Nexedi flavour of eggs # pin Acquisition and Products.DCWorkflow to Nexedi flavour of eggs
Acquisition = 2.13.7nxd001 Acquisition = 2.13.7nxd001
......
...@@ -4,6 +4,18 @@ extends = ...@@ -4,6 +4,18 @@ extends =
../component/python-2.7/buildout.cfg ../component/python-2.7/buildout.cfg
../component/lxml-python/buildout.cfg ../component/lxml-python/buildout.cfg
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
parts = parts =
eggs eggs
...@@ -11,7 +23,12 @@ parts = ...@@ -11,7 +23,12 @@ parts =
unzip = true unzip = true
[eggs] [eggs]
python = python2.7
recipe = zc.recipe.egg recipe = zc.recipe.egg
eggs = eggs =
${lxml-python:egg} ${lxml-python:egg}
Flask Flask
slapos.cookbook
[lxml-python]
python = python2.7
[buildout]
extends =
shacache-client.cfg
../component/python-2.7/buildout.cfg
../component/lxml-python/buildout.cfg
../component/git/buildout.cfg
../component/zlib/buildout.cfg
../component/readline/buildout.cfg
../component/ncurses/buildout.cfg
../component/libuuid/buildout.cfg
../component/noVNC/buildout.cfg
../component/openssl/buildout.cfg
../component/rdiff-backup/buildout.cfg
../component/dcron/buildout.cfg
parts =
template
gnutls
kvm
eggs
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
versions = versions
[gpg-error]
recipe = hexagonit.recipe.cmmi
url = ftp://ftp.gnupg.org/gcrypt/libgpg-error/libgpg-error-1.10.tar.gz
md5sum = 7c2710ef439f82ac429b88fec88e9a4c
[gcrypt]
recipe = hexagonit.recipe.cmmi
url = ftp://ftp.gnupg.org/gcrypt/libgcrypt/libgcrypt-1.4.6.tar.gz
md5sum = bfd45922eefb8a24d598af77366220d4
configure-options =
--with-gpg-error-prefix=${gpg-error:location}
environment =
CPPFLAGS=-I${gpg-error:location}/include
LDFLAGS=-Wl,-rpath -Wl,${gpg-error:location}/lib -Wl,${gpg-error:location}/lib/libgpg-error.so.0
[gnutls]
recipe = hexagonit.recipe.cmmi
url = ftp://ftp.gnupg.org/gcrypt/gnutls/gnutls-2.8.6.tar.bz2
md5sum = eb0a6d7d3cb9ac684d971c14f9f6d3ba
configure-options =
--with-libgcrypt-prefix=${gcrypt:location}
environment =
CPPFLAGS=-I${zlib:location}/include -I${readline:location}/include -I${ncurses:location}/include -I${ncurses:location}/include/ncursesw -I${gcrypt:location}/include -I${gpg-error:location}/include
LDFLAGS=-L${readline:location}/lib -L${ncurses:location}/lib -L${gcrypt:location}/lib -Wl,-rpath -Wl,${zlib:location}/lib -Wl,-rpath -Wl,${readline:location}/lib -Wl,-rpath -Wl,${ncurses:location}/lib -Wl,-rpath -Wl,${gcrypt:location}/lib -Wl,-rpath -Wl,${gpg-error:location}/lib -Wl,${gcrypt:location}/lib/libgcrypt.so.11
PKG_CONFIG=${zlib:location}/lib/pkgconfig
[kvm]
recipe = hexagonit.recipe.cmmi
path = ${kvmsource:location}/
configure-options =
--disable-sdl
--disable-xen
--enable-vnc-tls
--disable-vnc-sasl
--disable-curses
--disable-curl
--enable-kvm
--disable-docs
--enable-vnc-png
--disable-vnc-jpeg
--extra-cflags="-I${gnutls:location}/include -I${libuuid:location}/include -I${zlib:location}/include"
--extra-ldflags="-Wl,-rpath -Wl,${gnutls:location}/lib -L${libuuid:location}/lib -Wl,-rpath -Wl,${libuuid:location}/lib -L${zlib:location}/lib -Wl,-rpath -Wl,${zlib:location}/lib"
--disable-werror
environment =
PKG_CONFIG_PATH=${gnutls:location}/lib/pkgconfig
[kvmsource]
recipe=plone.recipe.command
location = ${buildout:parts-directory}/${:_buildout_section_name_}
stop-on-error = true
#tag = slapos-v0.1
command =
(${git:location}/bin/git clone --quiet http://git.erp5.org/repos/slapos.kvm.git ${:location} ) || (rm -fr ${:location} ; exit 1)
update-command =
cd ${:location} && ${git:location}/bin/git pull --quiet origin master
[eggs]
python = python2.7
recipe = z3c.recipe.scripts
eggs =
${lxml-python:egg}
slapos.cookbook
[versions]
zc.buildout = 1.5.3-dev-SlapOS-005
[buildout]
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
parts =
template
apache-php
mariadb
eggs
instance-recipe-egg
extends =
../component/mariadb/buildout.cfg
../component/apache/buildout.cfg
../component/apache-php/buildout.cfg
../component/dcron/buildout.cfg
../component/git/buildout.cfg
../component/glib/buildout.cfg
../component/logrotate/buildout.cfg
../component/python-2.7/buildout.cfg
../component/perl/buildout.cfg
../component/sqlite3/buildout.cfg
../component/xtrabackup/buildout.cfg
../component/rdiff-backup/buildout.cfg
../component/lxml-python/buildout.cfg
../component/zlib/buildout.cfg
[application]
recipe = hexagonit.recipe.download
#If provided tarball does not containt top directory this option shall be changed to false
strip-top-level-dir = true
[eggs]
recipe = zc.recipe.egg
eggs =
${lxml-python:egg}
[buildout]
networkcache-section = networkcache
[networkcache]
download-cache-url = http://www.shacache.org/shacache
download-dir-url = http://www.shacache.org/shadir
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment