- 05 Nov, 2015 1 commit
-
-
Rafael Monnerat authored
Dump on filesytem the ipv4 used to the node create connections
-
- 04 Nov, 2015 4 commits
-
-
Kirill Smelkov authored
It is well known that UNIX sockets are faster than TCP over loopback. E.g. on my machine according to lmbench[1] they have ~ 2 times lower latency and ~ 2-3 times more throughput compared to TCP over loopback: *Local* Communication latencies in microseconds - smaller is better --------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- teco Linux 4.2.0-1 13.8 29.2 26.8 45.0 47.9 48.5 55.5 45. *Local* Communication bandwidths in MB/s - bigger is better ----------------------------------------------------------------------------- Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem UNIX reread reread (libc) (hand) read write --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- ----- teco Linux 4.2.0-1 1084 4353 1493 2329.1 3720.7 1613.8 1109.2 3402 1404. The same ratio holds for our std shuttle servers. API to work with unix sockets is essentially the same as with TCP/UDP. Because of that it is easy to support both TCP and UNIX socket in one software, and this way a lot of software support unix sockets out of the box, including Redis. Because of lower latencies and higher throughput, for performance reasons, it makes sense to interconnect services on one machine via unix sockets and talk via TCP only to outside world. Here we add support for unix sockets to Redis recipe. [1] http://www.bitmover.com/lmbench/ /reviewed-by @kazuhiko (on !27) /cc @alain.takoudjou, @jerome, @vpelletier
-
Kirill Smelkov authored
Because redis.Redis(...) ctor creates connection pool on initialization and we can rely on it. Another reason: Redis ctor (in form of StrictRedis.__init__()) has logic how to process arguments and does selecting - either it is TCP (`host` and `port` args), or UNIX socket (`unix_socket_path` arg): https://lab.nexedi.com/nexedi/slapos/blob/95dbb5b2/slapos/recipe/redis/MyRedis2410.py#L560 Since we are going to introduce unix socket support to redis recipe in the next patch, and don't want to duplicate StrictRedis.__init__() logic in promise code, let's refactor promise to delegate argument processing logic to Redis. /reviewed-by @kazuhiko (on !27) /cc @alain.takoudjou
-
Kirill Smelkov authored
- update Redis software to latest upstream in 2.8.* series (which now supports IPv6 out of the box); - update Redis instance template to the one from 2.8.23 and re-merge our templating changes to it (file/dir locations, port and binding, master password). The whole diff to pristine 2.8.23 redis conf is now this: diff --git a/.../redis-2.8.23/redis.conf b/slapos/recipe/redis/template/redis.conf.in index 870959f..2895539 100644 --- a/.../redis-2.8.23/redis.conf +++ b/slapos/recipe/redis/template/redis.conf.in @@ -46 +46 @@ daemonize no -pidfile /var/run/redis.pid +pidfile %(pid_file)s @@ -50 +50 @@ pidfile /var/run/redis.pid -port 6379 +port %(port)s @@ -69,0 +70 @@ tcp-backlog 511 +bind %(ipv6)s @@ -108 +109 @@ loglevel notice -logfile "" +logfile %(log_file)s @@ -174 +175 @@ rdbcompression yes -# hit to pay (around 10%) when saving and loading RDB files, so you can disable it +# hit to pay (around 10%%) when saving and loading RDB files, so you can disable it @@ -192 +193 @@ dbfilename dump.rdb -dir ./ +dir %(server_dir)s @@ -217 +218 @@ dir ./ -# masterauth <master-password> +%(master_passwd)s NOTE There are test failures for almost all Redis versions when machine have not small amount of CPUs: https://github.com/antirez/redis/issues/2715#issuecomment-151608948 Because the failure is in replication test, and so far we do not use replication, and there is no feedback from upstream author to handle this (for 7 days for my detailed report, and for ~ 3 months for this issue in general), we can just disable replication test as a temporary solution. ( to handle remote patches with md5 hash easily the building recipe is changed to slapos.recipe.cmmi ) NOTE Redis updated to 2.8 version because GitLab uses this series. If/when we need more recent one we can add [redis30] in addition to [redis28]. /reviewed-by @kazuhiko (on !27 and on !26) /cc @alain.takoudjou, @jerome
-
Kirill Smelkov authored
Ruby 2.1.7 contain security and other bugfixes: https://www.ruby-lang.org/en/news/2015/08/18/ruby-2-1-7-released/ /reviewed-by @kazuhiko (on !28)
-
- 03 Nov, 2015 4 commits
-
-
Kirill Smelkov authored
Compared to our currently-packaged 9.2.8 postgresql 9.2.14 contains a lot of fixes including security ones. On the path of updating, we cleanup postgresql component a bit. Details are in commit logs. /reviewed-by @kazuhiko (on !25)
-
Kirill Smelkov authored
Compared to 9.2.8 postgresql 9.2.14 contains a lot of fixes including security ones: http://www.postgresql.org/docs/current/static/release-9-2-9.html http://www.postgresql.org/docs/current/static/release-9-2-10.html http://www.postgresql.org/docs/current/static/release-9-2-11.html http://www.postgresql.org/docs/current/static/release-9-2-12.html http://www.postgresql.org/docs/current/static/release-9-2-13.html http://www.postgresql.org/docs/current/static/release-9-2-14.html
-
Kirill Smelkov authored
Currently we have recipes to build both postgresql 9.1 and 9.2 (see 9f1f0759 "provide both postgres 9.1 and 9.2") which mostly duplicate each other with minor difference that 9.1 is built with perl and 9.2 without (perl added to 9.1 in dbbd9a96 "Include Perl in Postgres"). To reduce the duplication let's move common compilation bits to common section. NOTE I've tried to add perl to postgresql 9.2 as well and got the following compilation error: ld: .../perl/libs-c/libperl.a(op.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC .../parts/perl/libs-c/libperl.a: could not read symbols: Bad value which happens because libperl.a is not compiled with fPIC. For now we don't need perl in postgresql92 and this is not handled, and we just deduplicate building recipes without any actual change for resulting commands how the software is built.
-
Kirill Smelkov authored
component/postgresql/buildout.cfg defines both postgresql92 and postgresql sections, with postgresql being just [postgresql] <= postgresql92 On the other hand, software/postgresql uses postgresql92 in parts, but bin = ${postgresql:location}/bin in instance, which leads to _both_ postgresql and postgresql92 being built, and postgresql92 not used at all. Let's fix it and use postgresql everywhere. /cc @kazuhiko
-
- 02 Nov, 2015 1 commit
-
-
Julien Muchembled authored
This is an important update for us because we use http-send-name-header. A bug caused HAProxy to segfault.
-
- 29 Oct, 2015 1 commit
-
-
Kirill Smelkov authored
Slaprunner tries to leverage multicore, and spawns multiple jobs when compiling software, based on `cpu-usage-ratio` parameter. But this currently have effect only on Make-based projects (via setting `MAKEFLAGS=-j<n>`) and does not affect software with different build systems. Let's also provide support for parallel building for NumPy-based software and Ruby gems out of the box. /cc @cedric.leninivin, @kazuhiko, @alain.takoudjou /reviewed-by @jerome, @rafael (on !22)
-
- 27 Oct, 2015 8 commits
-
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Kirill Smelkov authored
Most Ruby projects are built via Bundler[1], e.g. the upcoming GitLab, and Bundler supports building/fetching dependency gems in parallel: http://bundler.io/v1.10/bundle_install.html#jobs via using `--jobs <n>` command line option. All bundler options can be also passed in via environment variable, and this way we can use BUNDLE_JOBS=<n> for default `--jobs <n>`. Let's use it, and this way speedup Ruby-related builds (like we already do for Make- and NumPy- based software). [1] http://bundler.io/ /cc @jerome, @cedric.leninivin, @kazuhiko
-
Kirill Smelkov authored
Starting from NumPy 1.10 numpy's distutils support parallel building http://docs.scipy.org/doc/numpy/user/install.html#basic-installation https://github.com/numpy/numpy/commit/23d54617 and this way software which uses numpy's distutils (scipy, scikit-learn, etc) should support it too. Let's use it, like we currently already use MAKEFLAGS for speeding up make-based projects. /cc @jerome, @cedric.leninivin, @kazuhiko
-
Kirill Smelkov authored
Currently we use '%d' and string formatting on max(1, ncpu / cpu-usage-ratio), because `ncpu / cpu-usage-ratio` is float: In [1]: from __future__ import division In [2]: 8 / 4 Out[2]: 2.0 and jinja2 uses future division by default: {{ 8 / 4 }} -> 2.0 We can however make things more explicit, by explicitly using integer division (// operator) and this way avoid the need for '%d' and string formatting. /cc @jerome, @cedric.leninivin
-
Kirill Smelkov authored
The calculation is full line long, and we are going to reuse this number in a couple of new places, so this way it makes sense to first compute jobs number separately, and then reuse the variable. /cc @jerome, @cedric.leninivin
-
- 26 Oct, 2015 7 commits
-
-
Kirill Smelkov authored
/reviewed-by @kazuhiko (on !21)
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
- 23 Oct, 2015 3 commits
-
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
Alain Takoudjou authored
-
- 22 Oct, 2015 6 commits
-
-
Julien Muchembled authored
Commit 6f009a9f broke generation of zope-*.conf
-
Vincent Pelletier authored
Partition root should be used to control overall permissions, which is typically a safe-enough 0750 (depending on slapformat and sysadmin). umask is applied at libc mkdir level, so this typically results in 0755 actual permission. More restrictive permissions should only be used when really needed.
-
Kirill Smelkov authored
/reviewed-by @kazuhiko
-
Julien Muchembled authored
This is a follow-up of commit 6f009a9f
-
Alain Takoudjou authored
-
Kazuhiko Shiozaki authored
Conflicts: stack/slapos.cfg
-
- 21 Oct, 2015 5 commits
-
-
Julien Muchembled authored
NEO is still in heavy development and it must be easy to update code on existing deployment.
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-