Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
slapos
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Hardik Juneja
slapos
Commits
f13f9c09
Commit
f13f9c09
authored
Mar 07, 2013
by
Kazuhiko Shiozaki
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
use memory-based kumofs instead of memcached to have no limitation for key length and data size.
parent
7c55ab32
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
141 additions
and
31 deletions
+141
-31
slapos/recipe/generic_kumofs/__init__.py
slapos/recipe/generic_kumofs/__init__.py
+5
-1
stack/erp5/buildout.cfg
stack/erp5/buildout.cfg
+5
-4
stack/erp5/instance-memcached.cfg.in
stack/erp5/instance-memcached.cfg.in
+131
-26
No files found.
slapos/recipe/generic_kumofs/__init__.py
View file @
f13f9c09
...
@@ -35,6 +35,10 @@ class Recipe(GenericBaseRecipe):
...
@@ -35,6 +35,10 @@ class Recipe(GenericBaseRecipe):
kumo_server_listen_port
=
int
(
self
.
options
[
'server-listen-port'
])
kumo_server_listen_port
=
int
(
self
.
options
[
'server-listen-port'
])
kumo_gateway_port
=
int
(
self
.
options
[
'gateway-port'
])
kumo_gateway_port
=
int
(
self
.
options
[
'gateway-port'
])
path_list
=
[]
path_list
=
[]
if
'data-path'
in
self
.
options
:
kumo_server_storage
=
self
.
options
.
get
(
'data-path'
)
else
:
kumo_server_storage
=
os
.
path
.
join
(
self
.
options
[
'data-directory'
],
"kumodb.tch"
)
# XXX: kumo is not storing pid in file, unless it is not running as daemon
# XXX: kumo is not storing pid in file, unless it is not running as daemon
# but running daemons is incompatible with SlapOS, so there is currently
# but running daemons is incompatible with SlapOS, so there is currently
# no way to have Kumo's pid files to rotate logs and send signals to them
# no way to have Kumo's pid files to rotate logs and send signals to them
...
@@ -48,7 +52,7 @@ class Recipe(GenericBaseRecipe):
...
@@ -48,7 +52,7 @@ class Recipe(GenericBaseRecipe):
kumo_server_binary
=
self
.
options
[
'kumo-server-binary'
],
kumo_server_binary
=
self
.
options
[
'kumo-server-binary'
],
kumo_server_ip
=
ip
,
kumo_server_ip
=
ip
,
kumo_server_log
=
self
.
options
[
'kumo-server-log'
],
kumo_server_log
=
self
.
options
[
'kumo-server-log'
],
kumo_server_storage
=
os
.
path
.
join
(
self
.
options
[
'data-directory'
],
"kumodb.tch"
)
,
kumo_server_storage
=
kumo_server_storage
,
kumo_manager_port
=
kumo_manager_port
,
kumo_manager_port
=
kumo_manager_port
,
kumo_server_port
=
kumo_server_port
,
kumo_server_port
=
kumo_server_port
,
kumo_server_listen_port
=
kumo_server_listen_port
,
kumo_server_listen_port
=
kumo_server_listen_port
,
...
...
stack/erp5/buildout.cfg
View file @
f13f9c09
...
@@ -40,7 +40,6 @@ extends =
...
@@ -40,7 +40,6 @@ extends =
../../component/lxml-python/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/percona-toolkit/buildout.cfg
../../component/percona-toolkit/buildout.cfg
../../component/mariadb/buildout.cfg
../../component/mariadb/buildout.cfg
../../component/memcached/buildout.cfg
../../component/mysql-python/buildout.cfg
../../component/mysql-python/buildout.cfg
../../component/pil-python/buildout.cfg
../../component/pil-python/buildout.cfg
../../component/pycrypto-python/buildout.cfg
../../component/pycrypto-python/buildout.cfg
...
@@ -74,7 +73,6 @@ parts =
...
@@ -74,7 +73,6 @@ parts =
apache-antiloris
apache-antiloris
file
file
graphviz
graphviz
memcached
haproxy
haproxy
varnish-3.0
varnish-3.0
w3m
w3m
...
@@ -291,10 +289,13 @@ extra-context =
...
@@ -291,10 +289,13 @@ extra-context =
[template-memcached]
[template-memcached]
< = template-jinja2-base
< = template-jinja2-base
filename = instance-memcached.cfg
filename = instance-memcached.cfg
md5sum =
44c507e9eec1003beb99f9063b5e09be
md5sum =
2de1801236eb78651ecfd50ada46dd3b
extra-context =
extra-context =
key dash_location dash:location
key dash_location dash:location
key memcached_location memcached:location
key dcron_location dcron:location
key gzip_location gzip:location
key kumo_location kumo:location
key logrotate_location logrotate:location
[template-erp5-single]
[template-erp5-single]
recipe = slapos.recipe.build:download
recipe = slapos.recipe.build:download
...
...
stack/erp5/instance-memcached.cfg.in
View file @
f13f9c09
#############################
# memcached-compatible volatile cache using kumofs
#
# that has no limitation for key length and data size
# Instanciate memcached
#
# No slapos parameter needed
#
#############################
[buildout]
[buildout]
parts =
parts =
memcached-instance
publish-kumofs-connection-information
publish-memcached-connection-information
kumofs-instance
promise
logrotate
logrotate-entry-kumofs
cron
cron-entry-logrotate
promise-kumofs-server
promise-kumofs-server-listen
promise-kumofs-gateway
promise-kumofs-manager
eggs-directory = {{ eggs_directory }}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
offline = true
[publish-kumofs-connection-information]
recipe = slapos.cookbook:publishurl
url = memcached://${kumofs-instance:ip}:${kumofs-instance:gateway-port}/
[kumofs-instance]
recipe = slapos.cookbook:generic.kumofs
# Network options
ip = ${slap-network-information:local-ipv4}
manager-port = 13101
server-port = 13201
server-listen-port = 13202
# previous memcached configuration
gateway-port = 11000
# previous memcached configuration
storage-size = 64m
# Paths: Running wrappers
gateway-wrapper = ${basedirectory:services}/volatile_kumofs_gateway
manager-wrapper = ${basedirectory:services}/volatile_kumofs_manager
server-wrapper = ${basedirectory:services}/volatile_kumofs_server
# Paths: Data
data-path = *#capsiz=${:storage-size}
# Paths: Logs
kumo-gateway-log = ${basedirectory:log}/kumo-gateway.log
kumo-manager-log = ${basedirectory:log}/kumo-manager.log
kumo-server-log = ${basedirectory:log}/kumo-server.log
# Binary information
kumo-gateway-binary = {{ kumo_location }}/bin/kumo-gateway
kumo-manager-binary = {{ kumo_location }}/bin/kumo-manager
kumo-server-binary = {{ kumo_location }}/bin/kumo-server
shell-path = {{ dash_location }}/bin/dash
[logrotate-entry-kumofs]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = kumofs
log = ${kumofs-instance:kumo-gateway-log} ${kumofs-instance:kumo-manager-log}
${kumofs-instance:kumo-server-log}
# rest of parts are candidates for some generic stuff
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = {{ logrotate_location }}/usr/sbin/logrotate
gzip-binary = {{ gzip_location }}/bin/gzip
gunzip-binary = {{ gzip_location }}/bin/gunzip
# Directories
wrapper = ${rootdirectory:bin}/logrotate
conf = ${rootdirectory:etc}/logrotate.conf
logrotate-entries = ${directory:logrotate-entries}
backup = ${directory:logrotate-backup}
state-file = ${rootdirectory:srv}/logrotate.status
[basedirectory]
recipe = slapos.cookbook:mkdirectory
log = ${rootdirectory:var}/log
services = ${rootdirectory:etc}/run
promise = ${rootdirectory:etc}/promise
run = ${rootdirectory:var}/run
backup = ${rootdirectory:srv}/backup
[directory]
[directory]
recipe = slapos.cookbook:mkdirectory
recipe = slapos.cookbook:mkdirectory
cron-entries = ${rootdirectory:etc}/cron.d
crontabs = ${rootdirectory:etc}/crontabs
cronstamps = ${rootdirectory:etc}/cronstamps
logrotate-backup = ${basedirectory:backup}/logrotate
logrotate-entries = ${rootdirectory:etc}/logrotate.d
[rootdirectory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
etc = ${buildout:directory}/etc
var = ${buildout:directory}/var
srv = ${buildout:directory}/srv
bin = ${buildout:directory}/bin
bin = ${buildout:directory}/bin
service = ${:etc}/run
promise = ${:etc}/promise
[memcached-instance]
[cron]
recipe = slapos.cookbook:generic.memcached
recipe = slapos.cookbook:cron
wrapper_path = ${directory:service}/memcached
dcrond-binary = {{ dcron_location }}/sbin/crond
binary_path = {{ memcached_location }}/bin/memcached
cron-entries = ${directory:cron-entries}
shell-path = {{ dash_location }}/bin/dash
crontabs = ${directory:crontabs}
ip = ${slap-network-information:local-ipv4}
cronstamps = ${directory:cronstamps}
port = 11000
catcher = ${cron-simplelogger:wrapper}
binary = ${basedirectory:services}/crond
[publish-memcached-connection-information]
[cron-simplelogger]
recipe = slapos.cookbook:publishurl
recipe = slapos.cookbook:simplelogger
url = memcached://${memcached-instance:ip}:${memcached-instance:port}/
wrapper = ${rootdirectory:bin}/cron_simplelogger
log = ${basedirectory:log}/cron.log
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = ${logrotate:wrapper}
# Deploy zope promises scripts
# Deploy zope promises scripts
[promise]
[promise
-template
]
recipe = slapos.cookbook:check_port_listening
recipe = slapos.cookbook:check_port_listening
path = ${directory:promise}/memcache
hostname = ${kumofs-instance:ip}
hostname = ${memcached-instance:ip}
port = ${kumofs-instance:server-listen-port}
port = ${memcached-instance:port}
[promise-kumofs-server]
<= promise-template
path = ${basedirectory:promise}/kumofs-server
port = ${kumofs-instance:server-port}
[promise-kumofs-server-listen]
<= promise-template
path = ${basedirectory:promise}/kumofs-server-listen
port = ${kumofs-instance:server-listen-port}
[promise-kumofs-gateway]
<= promise-template
path = ${basedirectory:promise}/kumofs-gateway
port = ${kumofs-instance:gateway-port}
[promise-kumofs-manager]
<= promise-template
path = ${basedirectory:promise}/kumofs-manager
port = ${kumofs-instance:manager-port}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment