We added support for Go in the previous patch. Let's now illustrate how
to use the toolchain and build simple service based on it.

/cc @gabriel, @Camata

Add support for Go language in the form of providing component for Go
toolchain and stdlib.

Current Go stable series is 1.5.* but to bootstrap it one need to have
Go 1.4.* available, which itself needs C compiler to bootstrap.

We need Go support for upcoming GitLab SR, where one server is written
in this language:

    https://gitlab.com/gitlab-org/gitlab-workhorse

/cc @gabriel, @Camata

To illustrate how to build Ruby software, let's add helloweb-ruby
component and integrate it into helloworld SR as a service.

helloweb-ruby installation is a bit tricky, because we do not have it
initially as a released gem, and have to build it with Bundler by hand.

Changes for integration into helloworld services infrastructure are
small, thanks for the "kind" rework we did in previous patch.

/cc @kazuhiko

Prepare to have several helloweb programs each written in its own
language, and installed into bin/ as helloworld-<language> and exposed
as separate service to the web on its own port.

For component/helloweb we just rename installed script and section.

For software/helloworld we split helloweb & helloweb-promise section
into base and generate requested helloweb-<kinds> & friends via jinja2
programming; the same for exposing url for each kind.

So far it is only preparatory changes for new kinds - i.e. instance code
now supports it, but the only kind so far is still python.

/cc @jerome

Jinja2 is currently the preferred way to do instance templating, because
it has control structures.

We'll need control structures in the following patches, but even without
it, it is better to show "how to" do instances the preferred way.

/cc @jerome

This

- better illustrates what components do (provide building recipes for
  programs) and what software do (gather components and provide recipes
  for instances).

- illustrates how to build the software properly - usually we build
  eggs or scripts with zc.recipe.egg:* , not generate python programs via
  jinja2 templating. Because when installing via egg, it is possible to
  import other installed eggs relatively straghtforward, and for jinja2
  way it is hard to do.

The helloweb program itself is moved into newly introduced helloweb.git
repository and, as almost all other software, becomes separate from
slapos.git .

/cc @jerome

Since 7cd2f953 (helloworld: Turn it into real web-service) helloworld SR
implements real web-service which talks to outside world with the help
of shipped-with-it "hello web" program.

Buildout sections which create helloweb service and promise were named
based on helloworld prefix though, which is not very consistent. Thus the
change for such sections to the same name as the "hello web" program uses.

Also for style consistency the "hello web" program is renamed from
hello-web to helloweb, as this naming style is usually used for
helloworld.{c,py.rb,go,etc}

So the final naming is:

    helloweb

for both program and related sections.

Slaprunner tries to leverage multicore, and spawns multiple jobs when compiling software, based on `cpu-usage-ratio` parameter. But this currently have effect only on Make-based projects (via setting `MAKEFLAGS=-j<n>`) and does not affect software with different build systems.

Let's also provide support for parallel building for NumPy-based software and Ruby gems out of the box.

/cc @cedric.leninivin, @kazuhiko, @alain.takoudjou
/reviewed-by @jerome, @rafael  (on !22)

Most Ruby projects are built via Bundler[1], e.g. the upcoming GitLab,
and Bundler supports building/fetching dependency gems in parallel:

    http://bundler.io/v1.10/bundle_install.html#jobs

via using `--jobs <n>` command line option.

All bundler options can be also passed in via environment variable, and
this way we can use BUNDLE_JOBS=<n> for default `--jobs <n>`.

Let's use it, and this way speedup Ruby-related builds (like we already
do for Make- and NumPy- based software).

[1] http://bundler.io/

/cc @jerome, @cedric.leninivin, @kazuhiko

Starting from NumPy 1.10 numpy's distutils support parallel building

    http://docs.scipy.org/doc/numpy/user/install.html#basic-installation
    https://github.com/numpy/numpy/commit/23d54617

and this way software which uses numpy's distutils (scipy,
scikit-learn, etc) should support it too.

Let's use it, like we currently already use MAKEFLAGS for speeding up
make-based projects.

/cc @jerome, @cedric.leninivin, @kazuhiko

Currently we use '%d' and string formatting on max(1, ncpu / cpu-usage-ratio),
because `ncpu / cpu-usage-ratio` is float:

    In [1]: from __future__ import division

    In [2]: 8 / 4
    Out[2]: 2.0

and jinja2 uses future division by default:

    {{ 8 / 4 }}   ->   2.0

We can however make things more explicit, by explicitly using integer
division (// operator) and this way avoid the need for '%d' and string
formatting.

/cc @jerome, @cedric.leninivin

The calculation is full line long, and we are going to reuse this number
in a couple of new places, so this way it makes sense to first compute
jobs number separately, and then reuse the variable.

/cc @jerome, @cedric.leninivin

/reviewed-by @kazuhiko  (on !21)

Commit 6f009a9f broke generation of zope-*.conf

Partition root should be used to control overall permissions, which is
typically a safe-enough 0750 (depending on slapformat and sysadmin).
umask is applied at libc mkdir level, so this typically results in 0755
actual permission.
More restrictive permissions should only be used when really needed.

/reviewed-by @kazuhiko

This is a follow-up of commit 6f009a9f

Conflicts:
	stack/slapos.cfg

NEO is still in heavy development and it must be easy to update code on
existing deployment.

Zope integrated Products.signalstack back in 2010:

    https://github.com/zopefoundation/Zope/commit/16796274
    ( Integrated the Products.signalstack / z3c.deadlockdebugger
      packages. You can now send a SIGUSR1 signal to a Zope process and get a
      stack trace of all threads printed out on the console. This works even
      if all threads are stuck. )

In 404150b7 (wendelin: Teach -dev build to dump all threads traceback on
SIGUSR1) it was my mistake to add Products.signalstack.

/reviewed-by TrustMe
/cc @Tyagov