Commit 1b540151 by Kirill Smelkov

golang: Infrastructure to build Go workspaces / projects

In Go world development workflow is organized around so-called workspace
where multiple packages can be installed / worked on etc. The following
page describes Go workspaces:

A new [gowork] section and infrastructure around it is introduced.
Quoting code:

    # gowork is a top-level section representing workspace
    # users should add `install` field to [gowork] to describe packages they want to
    # be installed (+ automatically their dependencies are installed too). e.g.
    #   [gowork]
    #   install =
    #  \
    #          \

The way it all works inside is:

- gowork organizes to create directory in buildout root
- inside it creates which when sources in shell adjusts all paths
  so that appropriate go compiler is in path, GOPATH is also set
  appropriately, etc - in other words everything needs for using/working
  on this workspace is setup in the environment.
- in actual user software profile the list of Go projects which needs to
  be installed by SlapOS has to be listed. The [gowork] machinery takes
  care about git-cloning these projects and `go install`s them after.
- by SlapOS design builds have to be reproducible and so every component
  state/version has to be fixed.

  A convenient helper (gowork-snapshot) to get snapshot of git go
  packages installed with their exact revisions is added. This should be
  used to automatically (re-)generate list of go git repositories &
  their pinning for a workspace.

For a new Go project that needs to be slaposified the workflow should be:

- first get a project into working state via any convenient way (`go get`, etc.)
- generate list of go packages & their pinning with gowork-snapshot
- extend golang/buildout.cfg and in project software profile add

  	install = your-top-level-packages

  as it was outlined above.


[golang] is removed because

1. it is not used anywhere in slapos.git tree, and
2. the canonical way to work on go projects from now on will be to use
   [gowork]. There latest go version is preconfigured, but users can
   override it to some particular other version in their projects as they

/reviewed-on	 !242
/also-needed-for !243
1 parent 4e24a566
......@@ -2,12 +2,11 @@
extends =
parts = golang
<= golang19
parts = gowork
# ---- Go builds itself ----
recipe = slapos.recipe.cmmi
......@@ -46,3 +45,73 @@ md5sum = 27bce1ffb05f4f6bd90d90081e5d4169
# go1.9 needs go1.4 to bootstrap
environment-extra =
# ---- infrastructure to build Go workspaces / projects ----
# gowork is a top-level section representing workspace
# users should add `install` field to [gowork] to describe packages they want to
# be installed (+ automatically their dependencies are installed too). e.g.
# [gowork]
# install =
# \
# \
directory = ${buildout:directory}/
src = ${:directory}/src
bin = ${:directory}/bin
depends = ${gowork.goinstall:recipe}
# go version used for the workspace (possible to override in applications)
golang = ${golang19:location}
# everything is done by dependent parts
recipe = plone.recipe.command
command = :
# for compiling and running go programs
[gowork] = ${}
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/
output = ${gowork:directory}/
depends = ${gowork.mkdir:recipe}
md5sum = a9a265135931b3da53f4392870748264
# NOTE do not use slapos.cookbook:mkdirectory here - if anything in software (not instance)
# uses slapos.cookbook:* in recipe - slapos.cookbook will get compiled against system
# libxml/libxslt or fail to bootstrap at all if those are not present.
recipe = plone.recipe.command
command = mkdir -p ${gowork:directory}
update-command = ${:command}
stop-on-error = true
# install go packages
# clients should put package list to install to gowork:install ("..." requests installing everything)
recipe = plone.recipe.command
command = bash -c ". ${} && go install -v ${gowork:install}"
update-command = ${:command}
stop-on-error = true
recipe =
git-executable = ${git:location}/bin/git
# a go package should:
# 1) <= go-git-package
# 2) provide go.importpath
# 3) provide repository (which is not the same as importpath in general case)
# the list of go packages for a go workspace state can be automatically
# generated with the help of go-pkg-snapshot tool.
<= git-repository
location = ${gowork:src}/${:go.importpath}
# for a Go workspace
# Usage: [/path/to/]
# ---- 8< ---- (buildout substitution here)
# PATH so that go & friends work out of the box
export PATH=${gowork:golang}/bin:${git:location}/bin:${buildout:bin-directory}:$PATH
# ---- 8< ----
export PATH=$X/bin:$PATH
export PS1="(`basename $X`) $PS1"
# strip trailing : from $GOPATH
# gowork-snapshot - find out installed go packages and produce buildout code to install them pinned
# FIXME currently works only with cwd=gowork/src
echo "# Code generated by gowork-snapshot; DO NOT EDIT."
# list installed go git repositories.
# this gives something like:
# 529a34b1c1
# 1087133bc4
# ...
gogit_list() {
find . -name .git | sort | \
while read repo; do
echo -ne "${importpath}\t"
echo -ne "`git_upstream_url $repo`\t"
git -C $repo describe --long --always --abbrev=10
# git_upstream_url <repo> - show current branch upstream URL
git_upstream_url() {
head="`git -C $repo symbolic-ref --short HEAD`" # current branch - e.g. "t"
remote="`git -C $repo config --get branch.$head.remote`" # upstream name, e.g. "kirr"
url="`git -C $repo config --get remote.$remote.url`" # upstream URL
echo "$url"
# buildout_safe <name> - canonicalize name to be allowed to be used in buildout section name
# e.g. aaa/bbb -> aaa_bbb
# XXX can't use e.g. "go!" because buildout disallows
# "!" or "/" in section name references.
buildout_safe() {
# see _simple regex in slapos.buildout
echo -n "$1" | tr --complement -- "-a-zA-Z0-9 ._" _
echo "# list of go git repositories to fetch"
echo "[gowork.goinstall]"
echo "depends_gitfetch ="
gogit_list | \
while read pkg _ _; do
echo " \${go_`buildout_safe $pkg`:recipe}"
gogit_list | \
while read pkg url rev; do
echo "[go_`buildout_safe $pkg`]"
echo "<= go-git-package"
echo "go.importpath = $pkg"
echo "repository = $url"
echo "revision = $rev"
Styling with Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!