Skip to content

slapos
slapos
Commit 866d9051 authored by Kirill Smelkov's avatar Kirill Smelkov
Browse files
Options

GitLab Software Release - first version 

Hello up there,

Here comes SlapOS port of GitLab. We start from GitLab 8.2.X as that is what we
currently run on KVM on lab.nexedi.com, so that our data can be
straightforwardly migrated.

The SR compiles all needed software and organized all (sub-)services in one
partition and interconnects them with unix sockets for security and speed
reasons (see patch "gitlab: Make a plan to base instance layout on
gitlab-omnibus and to interconnect all internal services").

Services configuration files are originally taken from omnibus-gitlab
"distribution" and incrementally ported to slapos variant. This way we
establish a (imho) good path on how to track upstream changes and minimize our
delta & effort supporting it.

GitLab itself is patched (above patches that were already applied by upstream):

  - to support HTTP(S) only
  - to show site's ICP number
  - to speedup raw blob downloading ~ 17x times
    ( see patch "gitlab: Optimize raw blob downloading" for details and
      https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17 )

Overall it should work and we should finally be able to migrate slapos.git
(because of raw blob downloading is not slow now) to GitLab and all other
Nexedi git repositories.

Thanks,
Kirill

P.S.

Somewhat outdated, but this picture on GitLab architecture might help to
understand how parts are glued together:

https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/architecture.md

P.P.S.

Native resiliency is not implemented yet, but we should be able to use gitlab
inside resilient webrunner already.

/proposed-for-review-on !39
/partly-reviewed-by @kazuhiko, @jerome, @Yanni, @jp
/cc @rafael, @jm
parents 3f4cde22 729be3b8
Expand all Show whitespace changes
Inline Side-by-side
Showing with 2437 additions and 0 deletions
software/gitlab/gitlab-parameters.cfg 0 → 100644
View file @ 866d9051
# Upstream parameters for a GitLab instance
#
# Selected parameters - main ones - names and advanced defaults taken from omnibus-gitlab
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb
#
# TODO better autogenerate from ^^^ (?)
#
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
[gitlab-parameters]
configuration.external_url = http://lab.example.com
# db advanced
configuration.db_pool = 10
# rack-attack
configuration.rate_limit_requests_per_period = 10
configuration.rate_limit_period = 60
configuration.time_zone = UTC
configuration.email_enabled = true
configuration.email_from = lab@example.com
configuration.email_display_name = GitLab
configuration.email_reply_to = noreply@example.com
configuration.smtp_enable = true
configuration.smtp_address = smtp.server
configuration.smtp_port = 465
configuration.smtp_user_name = smtp user
configuration.smtp_password = smtp password
configuration.smtp_domain = lab.example.com
configuration.smtp_authentication = login
configuration.smtp_enable_starttls_auto = true
# none | peer | client_once | fail_if_no_peer_cert -> see gitlab-omnibus links at top
configuration.smtp_openssl_verify_mode = peer
configuration.default_can_create_group = true
configuration.username_changing_enabled = true
configuration.default_theme = 2
configuration.default_projects_features.issues = true
configuration.default_projects_features.merge_requests = true
configuration.default_projects_features.wiki = true
configuration.default_projects_features.snippets = true
# NOTE can be public|private|internal
configuration.default_projects_features.visibility_level= public
#configuration.default_projects_features.builds = false
configuration.webhook_timeout = 10
# 0 means forever (seconds)
configuration.backup_keep_time = 0
# NOTE empty = default gitlab limits
configuration.git_max_size =
configuration.git_timeout =
# sidekiq
configuration.sidekiq_shutdown_timeout = 4
configuration.sidekiq_concurrency = 25
configuration.sidekiq_memory_killer_max_rss = 1000000
# unicorn
configuration.unicorn_worker_timeout = 60
configuration.unicorn_worker_processes = 2
# unicorn advanced
configuration.unicorn_backlog_socket = 1024
configuration.unicorn_worker_memory_limit_min = 200*(1024**2)
configuration.unicorn_worker_memory_limit_max = 250*(1024**2)
# nginx
configuration.nginx_client_max_body_size = 250m
# NOTE: we don't really need old ciphers - usually we talk directly to frontend only
configuration.nginx_ssl_ciphers = ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4
configuration.nginx_ssl_prefer_server_ciphers = on
configuration.nginx_ssl_protocols = TLSv1 TLSv1.1 TLSv1.2
# the following is gitlab-omnibus default but not nginx's default
configuration.nginx_ssl_session_cache = builtin:1000 shared:SSL:10m
configuration.nginx_ssl_session_timeout = 5m
configuration.nginx_proxy_read_timeout = 300
configuration.nginx_proxy_connect_timeout = 300
# nginx advanced
configuration.nginx_worker_processes = 4
configuration.nginx_worker_connections = 10240
configuration.nginx_log_format = $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"
configuration.nginx_sendfile = on
configuration.nginx_tcp_nopush = on
configuration.nginx_tcp_nodelay = on
configuration.nginx_gzip = on
configuration.nginx_gzip_http_version = 1.0
configuration.nginx_gzip_comp_level = 2
configuration.nginx_gzip_proxied = any
configuration.nginx_gzip_types = text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript application/json
configuration.nginx_keepalive_timeout = 65
software/gitlab/gitlab-unicorn-startup.in 0 → 100644
View file @ 866d9051
#!{{ bash_bin }}
# start up gitlab's unicorn with first making sure db is properly setup and all
# migrations are up as pre-condition.
RAKE={{ gitlab_rake }}
die() {
echo "$*" 1>&2
exit 1
}
# 1. what to do when instance is initially setup
# see
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/recipes/database_migrations.rb
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/recipes/postgresql.rb
# initial db setup
pgtables="$({{ psql_bin }} \
-h {{ pgsql['pgdata-directory'] }} \
-U {{ pgsql.superuser }} \
-d {{ pgsql.dbname }} \
-c '\d')" || die "pg query problem"
if echo "$pgtables" | grep -q '^No relations found' ; then
$RAKE db:schema:load db:seed_fu || die "initial db setup failed"
fi
# re-build ssh keys
# (we do not use them - just for cleannes)
force=yes $RAKE gitlab:shell:setup || die "gitlab:shell:setup failed"
# 2. what to do when instance is upgraded
# see
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/deploy/deploy.sh
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/upgrader.rb
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/recipes/gitlab-rails.rb
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-ctl-commands/upgrade.rb
#
# Assets compilation is handled at instance deployment time. We do everything else here.
# make sure all migrations are up
migrate_log="{{ log_dir }}/db-migrate-`date +%s`.log"
$RAKE db:migrate >$migrate_log 2>&1 || die "db:migrate failed"
# if it was a no-op "migration" - we don't need info about that - only keep
# logs of actual migration run.
test -s $migrate_log || rm $migrate_log
# clear cache
$RAKE cache:clear || die "cache:clear failed"
# 3. finally exec to unicorn
exec {{ gitlab_unicorn }} \
-E production \
-c {{ unicorn_rb.rendered }} \
{{ gitlab_work.location }}/config.ru
software/gitlab/instance-gitlab.cfg.in 0 → 100644
View file @ 866d9051
This diff is collapsed.
software/gitlab/instance.cfg.in 0 → 100644
View file @ 866d9051
# GitLab "switch-softwaretype" instance
[buildout]
parts = switch-softwaretype
# std stuff for slapos instance
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${instance-gitlab.cfg:rendered}
# TODO -export, -import, -pull-backup
[instance-gitlab.cfg]
recipe = slapos.recipe.template:jinja2
mode = 0644
template= ${instance-gitlab.cfg.in:target}
rendered= $${buildout:directory}/instance-gitlab.cfg
context =
import os os
import pwd pwd
import multiprocessing multiprocessing
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
raw gitlab_repository_location ${gitlab-repository:location}
raw gitlab_shell_repository_location ${gitlab-shell-repository:location}
# program binaries
raw bash_bin ${bash:location}/bin/bash
raw bundler_4gitlab ${bundler-4gitlab:bundle}
raw curl_bin ${curl:location}/bin/curl
raw dcron_bin ${dcron-output:crond}
raw git ${git:location}/bin/git
raw git_location ${git:location}
raw gitlab_workhorse ${gitlab-workhorse:location}/gitlab-workhorse
raw gunzip_bin ${gzip:location}/bin/gunzip
raw gzip_bin ${gzip:location}/bin/gzip
raw logrotate_bin ${logrotate:location}/usr/sbin/logrotate
raw nginx_bin ${nginx-output:nginx}
raw nginx_mime_types ${nginx-output:mime}
raw openssl_bin ${openssl-output:openssl}
raw postgresql_location ${postgresql92:location}
raw redis_binprefix ${redis28:location}/bin
raw ruby_location ${bundler-4gitlab:ruby-location}
raw watcher_sigkill ${watcher-sigkill:rendered}
# config files
raw config_ru_in ${config.ru.in:target}
raw database_yml_in ${database.yml.in:target}
raw gitconfig_in ${gitconfig.in:target}
raw gitlab_parameters_cfg ${gitlab-parameters.cfg:target}
raw gitlab_shell_config_yml_in ${gitlab-shell-config.yml.in:target}
raw gitlab_unicorn_startup_in ${gitlab-unicorn-startup.in:target}
raw gitlab_yml_in ${gitlab.yml.in:target}
raw macrolib_cfg_in ${macrolib.cfg.in:target}
raw nginx_conf_in ${nginx.conf.in:target}
raw nginx_gitlab_http_conf_in ${nginx-gitlab-http.conf.in:target}
raw rack_attack_rb_in ${rack_attack.rb.in:target}
raw resque_yml_in ${resque.yml.in:target}
raw smtp_settings_rb_in ${smtp_settings.rb.in:target}
raw unicorn_rb_in ${unicorn.rb.in:target}
software/gitlab/macrolib.cfg.in 0 → 100644
View file @ 866d9051
{# common macros for gitlab instance #}
{# cfg(name) -> instance_parameter:configuration.<name> #}
{% macro cfg(name) %}{{ instance_parameter[str("configuration." + name)] }}{% endmacro %}
{# cfg_bool(name) - like cfg(name), but returns 'true'/''
NOTE macros can return only strings - that's why '' is used for false #}
{% macro cfg_bool(name) %}{{ 'true' if (cfg(name).lower() in ('true', 'yes')) else '' }}{% endmacro %}
{# deduce whether to use https from external url
( here - becasue we cannot use jinja2 logic in instance-gitlab.cfg.in to
process instance parameters ) #}
{% set external_url = urlparse.urlparse(cfg('external_url')) %}
{% set cfg_https = (true if external_url.scheme == 'https' else false) %}
{# for convenience #}
{% set fqdn = external_url.hostname %}
software/gitlab/software.cfg 0 → 100644
View file @ 866d9051
# GitLab software-release
[buildout]
extends =
../../stack/slapos.cfg
../../component/ruby/buildout.cfg
../../component/golang/buildout.cfg
../../component/postgresql/buildout.cfg
../../component/redis/buildout.cfg
../../component/cmake/buildout.cfg
../../component/icu/buildout.cfg
../../component/pkgconfig/buildout.cfg
../../component/nodejs/buildout.cfg
../../component/openssl/buildout.cfg
../../component/nginx/buildout.cfg
# for instance
../../component/bash/buildout.cfg
../../component/curl/buildout.cfg
../../component/gzip/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
parts =
ruby2.1
golang15
git
postgresql92
redis28
cmake
icu
pkgconfig
nginx-output
gitlab-shell/vendor
gitlab/vendor/bundle
gitlab-workhorse
# for instance
instance.cfg
slapos-cookbook
eggs
bash
curl
watcher-sigkill
gzip
dcron-output
logrotate
############################
# Software compilation #
############################
# rubygemsrecipe with fixed url and this way pinned rubygems version
[rubygemsrecipe]
recipe = rubygemsrecipe
url = https://rubygems.org/rubygems/rubygems-2.5.1.zip
# bundler, that we'll use to
# - install gems for gitlab
# - run gitlab services / jobs (via `bundle exec ...`)
[bundler-4gitlab]
<= rubygemsrecipe
ruby-location = ${ruby2.1:location}
ruby-executable = ${:ruby-location}/bin/ruby
gems = bundler==1.11.2
# bin installed here
bundle = ${buildout:bin-directory}/bundle
# install together with dependencies of gitlab, which we cannot specify using
# --with-... gem option
# ( reason: rubygemsrecipe hardcodes PATH inside generated bin/* and it is
# impossible to adjust it later )
#
# bundle exec <smth> ; <smth> starts with `#!/usr/bin/env ruby` as rubygems
# Rugged needs: cmake, pkgconfig
# execjs needs: nodejs
# rails needs db client program on path: psql
# gitlab wants to check redis version via running: redis-cli
environment =
PATH = ${:ruby-location}/bin:${cmake:location}/bin:${pkgconfig:location}/bin:${nodejs:location}/bin:${postgresql92:location}/bin:${redis28:location}/bin:%(PATH)s
# gitlab, gitlab-shell & gitlab-workhorse checked out as git repositories
# pinned to exact commit
[git-repository]
recipe = slapos.recipe.build:gitclone
git-executable = ${git:location}/bin/git
[gitlab-repository]
<= git-repository
#repository = https://gitlab.com/gitlab-org/gitlab-ce.git
repository = https://lab.nexedi.com/kirr/gitlab-ce.git
# 8.2.X + NXD patches:
revision = v8.2.3-9-g79c127e6e068a619c53a8c22f1db8c1e28ec87d2
location = ${buildout:parts-directory}/gitlab
[gitlab-shell-repository]
<= git-repository
repository = https://gitlab.com/gitlab-org/gitlab-shell.git
# gitlab 8.2 wants gitlab-shell 2.6.8
# 2.6.8 + NXD patches
revision = v2.6.8-2-g216d7e15fe06917198891a895f762ba84fdcc4d4
location = ${buildout:parts-directory}/gitlab-shell
[gitlab-workhorse-repository]
<= git-repository
#repository = https://gitlab.com/gitlab-org/gitlab-workhorse.git
repository = https://lab.nexedi.com/kirr/gitlab-workhorse.git
# 0.4.X + NXD patches for blob download speedup
# (https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17)
revision = 0.4.1-23-g2beb8c9539433f072e3db540f91f75894ca6b1b0
location = ${buildout:parts-directory}/gitlab-workhorse
# build needed-by-gitlab gems via bundler
[gitlab/vendor/bundle]
recipe = slapos.recipe.cmmi
path = ${gitlab-repository:location}
bundle = ${bundler-4gitlab:bundle}
configure-command = cd ${:path} &&
${:bundle} config --local build.charlock_holmes --with-icu-dir=${icu:location} &&
${:bundle} config --local build.pg --with-pg-config=${postgresql92:location}/bin/pg_config
make-binary =
make-targets= cd ${:path} &&
${:bundle} install --deployment --without development test mysql kerberos
# build needed-by-gitlab-shell gems via bundler
# ( there is not vendor/ dir in gitlab-shell, so to avoid having buildout error
# on mkdir vendor/bundle, this part name is just /vendor )
[gitlab-shell/vendor]
recipe = slapos.recipe.cmmi
path = ${gitlab-shell-repository:location}
bundle = ${bundler-4gitlab:bundle}
configure-command = true
make-binary =
make-targets= cd ${:path} &&
${:bundle} install --deployment --without development test
# build gitlab-workhorse
[gitlab-workhorse]
recipe = slapos.recipe.cmmi
path = ${gitlab-workhorse-repository:location}
configure-command = :
make-targets= ${:_buildout_section_name_}
environment =
PATH=${golang15:location}/bin:%(PATH)s
###############################
# Trampoline for instance #
###############################
# eggs for instance.cfg
[eggs]
recipe = zc.recipe.egg
eggs =
plone.recipe.command
cns.recipe.symlink
[instance.cfg]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg.in
output = ${buildout:directory}/instance.cfg
md5sum = b40cd8824b978da867404d8955b06c18
[watcher-sigkill]
recipe = slapos.recipe.template:jinja2
template= ${:_profile_base_location_}/${:_buildout_section_name_}.in
rendered= ${buildout:bin-directory}/${:_buildout_section_name_}
mode = 0755
md5sum = 2986dcb006dc9e8508ff81f646656131
context =
section bash bash
# macro: download a file named as section name
#
# [filename]
# <= download-file
# md5sum = ...
[download-file]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/${:_buildout_section_name_}
destination = ${buildout:directory}/${:_buildout_section_name_}
# like download-file, but download from template/<filename>
[download-template]
<= download-file
url = ${:_profile_base_location_}/template/${:_buildout_section_name_}
[config.ru.in]
<= download-template
md5sum = bb12852c28079f40a0751f7f3559e2a6
[database.yml.in]
<= download-template
md5sum = ee656cfd96e1c82df167f68bb5773291
[gitconfig.in]
<= download-template
md5sum = f4cb11e8bca379e016b062d0db859b74
[gitlab-parameters.cfg]
<= download-file
md5sum = bc98ec10209bc53f6a49888b1a2b9382
[gitlab-shell-config.yml.in]
<= download-template
md5sum = ea351e16b47f0008f61211eb2d7685e2
[gitlab-unicorn-startup.in]
<= download-file
md5sum = 2716afaa9445c0c429c6b211356ebe8f
[gitlab.yml.in]
<= download-template
md5sum = cc32f5053dd2a2461aa5952a5b925310
[instance-gitlab.cfg.in]
<= download-file
md5sum = dfd2b14f846eda999fe9d12108d513b4
[macrolib.cfg.in]
<= download-file
md5sum = a56a44e96f65f5ed20211bb6a54279f4
[nginx-gitlab-http.conf.in]
<= download-template
md5sum = 590da2b00cd198c7bc261c3d893bc199
[nginx.conf.in]
<= download-template
md5sum = f1a6e2bce3f28a2243fed49d1e1601df
[rack_attack.rb.in]
<= download-template
md5sum = 16503c029159ea6db7d0fb5ab67093a3
[resque.yml.in]
<= download-template
md5sum = 7d9cba658f9315cd058dfc74db943a66
[smtp_settings.rb.in]
<= download-template
md5sum = c7c09c241b5fa8163e4995260be52604
[unicorn.rb.in]
<= download-template
md5sum = 9bdca16362fe19c727bca38383e57068
[versions]
cns.recipe.symlink = 0.2.3
plone.recipe.command = 1.1
rubygemsrecipe = 0.2.2
slapos.recipe.template = 2.9
software/gitlab/template/config.ru.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config.ru
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/gitlab-rails-config.ru.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
# This file is used by Rack-based servers to start the application.
{% from 'macrolib.cfg.in' import cfg with context %}
if defined?(Unicorn)
require 'unicorn'
if ENV['RAILS_ENV'] == 'production' || ENV['RAILS_ENV'] == 'staging'
# Unicorn self-process killer
require 'unicorn/worker_killer'
# Max memory size (RSS) per worker
use Unicorn::WorkerKiller::Oom, ({{ cfg('unicorn_worker_memory_limit_min') }}), ({{ cfg('unicorn_worker_memory_limit_max') }})
end
end
require ::File.expand_path('../config/environment', __FILE__)
map ENV['RAILS_RELATIVE_URL_ROOT'] || "/" do
run Gitlab::Application
end
software/gitlab/template/database.yml.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/database.yml.postgresql
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/database.yml.erb
# (last updated for 8.2.3+ce.0-0-g8eda093)
{% from 'macrolib.cfg.in' import cfg with context %}
production:
adapter: postgresql
encoding: unicode
{# collation is mainly for mysql
collation: <%= @db_collation %>
#}
database: {{ pgsql.dbname }}
pool: {{ cfg('db_pool') }}
{# XXX is it ok to use superuser, even if the whole database is only for gitlab? #}
username: '{{ pgsql.superuser }}'
{# we have no password - access is via unix socket #}
password:
host: '{{ pgsql["pgdata-directory"] }}'
port:
socket:
{# not needed for unix socket
sslmode: <%= single_quote(@db_sslmode) %>
sslrootcert: <%= single_quote(@db_sslrootcert) %>
#}
software/gitlab/template/gitconfig.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# global git configuration for GitLab
# see:
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/gitconfig.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
#
{% from 'macrolib.cfg.in' import cfg with context %}
# don't waste memory when packing (each thread uses own work memory)
# besides it packs better with 1 thread
[pack]
threads = 1
# don't allow corrupt/broken objects to go in
[receive]
fsckObjects = true
[user]
name = {{ cfg('email_display_name') }}
email = {{ cfg('email_from') }}
[core]
autocrlf = input
software/gitlab/template/gitlab-shell-config.yml.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-shell/blob/master/config.yml.example
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/gitlab-shell-config.yml.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
# GitLab user. git by default
user: {{ backend_info.user }}
# Url to gitlab instance. Used for api calls. Should end with a slash.
gitlab_url: "http+unix://{{ urllib.quote_plus(unicorn.socket) }}/"
http_settings:
{# we don't need any
<%= @http_settings.to_json if @http_settings %>
#}
# user: someone
# password: somepass
# ca_file: /etc/ssl/cert.pem
# ca_path: /etc/pki/tls/certs
# self_signed_cert: false
# Repositories path
# Give the canonicalized absolute pathname,
# REPOS_PATH MUST NOT CONTAIN ANY SYMLINK!!!
# Check twice that none of the components is a symlink, including "/home".
repos_path: "{{ gitlab.repositories }}"
# File used as authorized_keys for gitlab user
# NOTE not used in slapos version (all access via https only)
auth_file: "{{ gitlab.var }}/sshkeys-notused"
# File that contains the secret key for verifying access to GitLab.
# Default is .gitlab_shell_secret in the root directory.
secret_file: "{{ gitlab_shell.secret }}"
# Redis settings used for pushing commit notices to gitlab
redis:
bin: {{ redis_binprefix }}/redis-cli
host: {# <%= @redis_host %> #}
port: {# <%= @redis_port %> #}
socket: {{ service_redis.unixsocket }}
{# we don't use password for redis
<% if @redis_password %>
pass: <%= @redis_password %>
<% end %>
#}
database: {# <%= @redis_database %> #}
namespace: resque:gitlab
# Log file.
# Default is gitlab-shell.log in the root directory.
log_file: "{{ gitlab_shell.log }}/gitlab-shell.log"
# Log level. INFO by default
log_level:
# Audit usernames.
# Set to true to see real usernames in the logs instead of key ids, which is easier to follow, but
# incurs an extra API call on every gitlab-shell command.
audit_usernames:
# Enable git-annex support
# git-annex allows managing files with git, without checking the file contents into git
# See https://git-annex.branchable.com/ for documentation
# If enabled, git-annex needs to be installed on the server where gitlab-shell is setup
# For Debian and Ubuntu systems this can be done with: sudo apt-get install git-annex
# For CentOS: sudo yum install epel-release && sudo yum install git-annex
git_annex_enabled:
software/gitlab/template/gitlab.yml.in 0 → 100644
View file @ 866d9051
This diff is collapsed.
software/gitlab/template/nginx-gitlab-http.conf.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
{% from 'macrolib.cfg.in' import cfg, cfg_bool, cfg_https, fqdn with context %}
## GitLab
## Modified from https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab-ssl & https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab
##
## Lines starting with two hashes (##) are comments with information.
## Lines starting with one hash (#) are configuration parameters that can be uncommented.
##
##################################
## CHUNKED TRANSFER ##
##################################
##
## It is a known issue that Git-over-HTTP requires chunked transfer encoding [0]
## which is not supported by Nginx < 1.3.9 [1]. As a result, pushing a large object
## with Git (i.e. a single large file) can lead to a 411 error. In theory you can get
## around this by tweaking this configuration file and either:
## - installing an old version of Nginx with the chunkin module [2] compiled in, or
## - using a newer version of Nginx.
##
## At the time of writing we do not know if either of these theoretical solutions works.
## As a workaround users can use Git over SSH to push large files.
##
## [0] https://git.kernel.org/cgit/git/git.git/tree/Documentation/technical/http-protocol.txt#n99
## [1] https://github.com/agentzh/chunkin-nginx-module#status
## [2] https://github.com/agentzh/chunkin-nginx-module
##
###################################
## configuration ##
###################################
upstream gitlab {
server unix:{{ unicorn.socket }} fail_timeout=0;
}
upstream gitlab-workhorse {
server unix:{{ gitlab_workhorse.socket }};
}
{# not needed for us - the frontend can do the redirection and also
gitlab/nginx speaks HSTS on https port so when we access https port via http
protocol, it gets redirected to https
<% if @https && @redirect_http_to_https %>
## Redirects all HTTP traffic to the HTTPS host
server {
<% @listen_addresses.each do |listen_address| %>
listen <%= listen_address %>:<%= @redirect_http_to_https_port %>;
<% end %>
server_name <%= @fqdn %>;
server_tokens off; ## Don't show the nginx version number, a security best practice
return 301 https://<%= @fqdn %>:<%= @port %>$request_uri;
access_log <%= @log_directory %>/gitlab_access.log gitlab_access;
error_log <%= @log_directory %>/gitlab_error.log;
}
<% end %>
#}
server {
listen [{{ backend_info.host }}]:{{ backend_info.port }}{% if cfg_https %} ssl spdy{% endif %};
{# we don't use: kerbeeros
<% if @kerberos_enabled && @kerberos_use_dedicated_port %>
listen <%= listen_address %>:<%= @kerberos_port %><% if @kerberos_https %> ssl<% end %>;
<% end %>
#}
server_name {{ fqdn }};
server_tokens off; ## Don't show the nginx version number, a security best practice
root {{ gitlab_work.location }}/public;
## Increase this if you want to upload large attachments
## Or if you want to accept large git objects over http
client_max_body_size {{ cfg('nginx_client_max_body_size') }};
{% if cfg_https %}
## Strong SSL Security
## https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/
ssl on;
ssl_certificate {{ nginx.cert_file }};
ssl_certificate_key {{ nginx.key_file }};
{# we don't need - most root CA will be included by default
<% if @ssl_client_certificate %>
ssl_client_certificate <%= @ssl_client_certificate%>;
<% end %>
#}
# GitLab needs backwards compatible ciphers to retain compatibility with Java IDEs
# NOTE(slapos) ^^^ is not relevant for us - we are behind frontend and clients
# directly connects to frontend
ssl_ciphers '{{ cfg("nginx_ssl_ciphers") }}';
ssl_protocols {{ cfg('nginx_ssl_protocols') }};
ssl_prefer_server_ciphers {{ cfg('nginx_ssl_prefer_server_ciphers') }};
ssl_session_cache {{ cfg('nginx_ssl_session_cache') }};
ssl_session_timeout {{ cfg('nginx_ssl_session_timeout') }};
{# we do not use: ssl_dhparam
<% if @ssl_dhparam %>
ssl_dhparam <%= @ssl_dhparam %>;
<% end %>
#}
{% endif %}
## Individual nginx logs for this GitLab vhost
access_log {{ nginx.log }}/gitlab_access.log gitlab_access;
error_log {{ nginx.log }}/gitlab_error.log;
location / {
## Serve static files from defined root folder.
## @gitlab is a named location for the upstream fallback, see below.
try_files $uri /index.html $uri.html @gitlab;
}
location /uploads/ {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
{{ 'gzip off;' if cfg_https else ''}}
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout {{ cfg('nginx_proxy_read_timeout') }};
proxy_connect_timeout {{ cfg('nginx_proxy_connect_timeout') }};
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
{% if cfg_https %}
proxy_set_header X-Forwarded-Ssl on;
{% endif %}
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto {{ "https" if cfg_https else "http" }};
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_pass http://gitlab;
}
## If a file, which is not found in the root folder is requested,
## then the proxy passes the request to the upsteam (gitlab unicorn).
location @gitlab {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
{{ 'gzip off;' if cfg_https else ''}}
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout {{ cfg('nginx_proxy_read_timeout') }};
proxy_connect_timeout {{ cfg('nginx_proxy_connect_timeout') }};
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
{% if cfg_https %}
proxy_set_header X-Forwarded-Ssl on;
{% endif %}
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto {{ "https" if cfg_https else "http" }};
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_pass http://gitlab;
}
location ~ ^/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/api/v3/projects/.*/repository/archive {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
# Build artifacts should be submitted to this location
location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
# Build artifacts should be submitted to this location
location ~ /ci/api/v1/builds/[0-9]+/artifacts {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
# access to raw blobs -> @gitlab-workhorse
location ~ ^/[\w\.-]+/[\w\.-]+/raw/ {
client_max_body_size 0;
error_page 418 = @gitlab-workhorse;
return 418;
}
location @gitlab-workhorse {
client_max_body_size 0;
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
{{ 'gzip off;' if cfg_https else ''}}
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout {{ cfg('nginx_proxy_read_timeout') }};
proxy_connect_timeout {{ cfg('nginx_proxy_connect_timeout') }};
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
{% if cfg_https %}
proxy_set_header X-Forwarded-Ssl on;
{% endif %}
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto {{ "https" if cfg_https else "http" }};
proxy_pass http://gitlab-workhorse;
}
## Enable gzip compression as per rails guide:
## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
## WARNING: If you are using relative urls remove the block below
## See config/application.rb under "Relative url support" for the list of
## other files that need to be changed for relative url support
location ~ ^/(assets)/ {
root {{ gitlab_work.location }}/public;
gzip_static on; # to serve pre-gzipped version
expires max;
add_header Cache-Control public;
}
error_page 502 /502.html;
{# we don't support custom nginx configs
<%= @custom_gitlab_server_config %>
#}
}
software/gitlab/template/nginx.conf.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab-ssl
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/nginx.conf.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
{% from 'macrolib.cfg.in' import cfg with context %}
# user directive makes sense only when running initially as root
# (and nginx will complain if not and directive given)
# user {{ backend_info.user }};
worker_processes {{ cfg('nginx_worker_processes') }};
error_log stderr;
pid {{ directory.run }}/nginx.pid;
daemon off;
events {
worker_connections {{ cfg('nginx_worker_connections') }};
}
http {
log_format gitlab_access '{{ cfg("nginx_log_format") }}';
{# we do not use: ci, mattermost
log_format gitlab_ci_access '<%= @gitlab_ci_access_log_format %>';
log_format gitlab_mattermost_access '<%= @gitlab_mattermost_access_log_format %>';
#}
sendfile {{ cfg('nginx_sendfile') }};
tcp_nopush {{ cfg('nginx_tcp_nopush') }};
tcp_nodelay {{ cfg('nginx_tcp_nodelay') }};
keepalive_timeout {{ cfg('nginx_keepalive_timeout') }};
gzip {{ cfg('nginx_gzip') }};
gzip_http_version {{ cfg('nginx_gzip_http_version') }};
gzip_comp_level {{ cfg('nginx_gzip_comp_level') }};
gzip_proxied {{ cfg('nginx_gzip_proxied') }};
gzip_types {{ cfg('nginx_gzip_types') }};
include {{ nginx_mime_types }};
include {{ nginx_gitlab_http_conf }};
{# we don't need: ci, mattermost
include <%= @gitlab_ci_http_config %>
include <%= @gitlab_mattermost_http_config %>
#}
}
software/gitlab/template/rack_attack.rb.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/initializers/rack_attack.rb.example
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/rack_attack.rb.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
{% from 'macrolib.cfg.in' import cfg with context %}
# 1. Rename this file to rack_attack.rb
# 2. Review the paths_to_be_protected and add any other path you need protecting
#
paths_to_be_protected = [
"#{Rails.application.config.relative_url_root}/users/password",
"#{Rails.application.config.relative_url_root}/users/sign_in",
"#{Rails.application.config.relative_url_root}/api/#{API::API.version}/session.json",
"#{Rails.application.config.relative_url_root}/api/#{API::API.version}/session",
"#{Rails.application.config.relative_url_root}/users",
"#{Rails.application.config.relative_url_root}/users/confirmation",
"#{Rails.application.config.relative_url_root}/unsubscribes/"
]
# Create one big regular expression that matches strings starting with any of
# the paths_to_be_protected.
paths_regex = Regexp.union(paths_to_be_protected.map { |path| /\A#{Regexp.escape(path)}/ })
unless Rails.env.test?
Rack::Attack.throttle('protected paths', limit: {{ cfg('rate_limit_requests_per_period') }}, period: {{ cfg('rate_limit_period') }}.seconds) do |req|
if req.post? && req.path =~ paths_regex
req.ip
end
end
end
software/gitlab/template/resque.yml.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/resque.yml.example
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/resque.yml.erb
# (last udpdated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
production: unix://{{ redis.unixsocket }}
software/gitlab/template/smtp_settings.rb.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/initializers/smtp_settings.rb.sample
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/smtp_settings.rb.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
{% from 'macrolib.cfg.in' import cfg, cfg_bool with context %}
{% if cfg_bool('smtp_enable') %}
if Rails.env.production?
Gitlab::Application.config.action_mailer.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
address: "{{ cfg('smtp_address') }}",
port: {{ cfg('smtp_port') }},
user_name: "{{ cfg('smtp_user_name') }}",
password: "{{ cfg('smtp_password') }}",
domain: "{{ cfg('smtp_domain') }}",
authentication: :{{ cfg('smtp_authentication') }},
enable_starttls_auto: {{ cfg('smtp_enable_starttls_auto') }},
openssl_verify_mode: '{{ cfg("smtp_openssl_verify_mode") }}'
# ca_path:
# ca_file:
}
end
{% else %}
# SMTP disabled in instance configuration (see `smtp_enable` parameter).
# Mail sending, if enabled (see `email_enabled`), will be done via sendmail.
{% endif %}
software/gitlab/template/unicorn.rb.in 0 → 100644
View file @ 866d9051
{{ autogenerated }}
# see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/unicorn.rb.example
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/unicorn.rb.example.development
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/unicorn.rb.erb
# (last updated for omnibus-gitlab 8.2.3+ce.0-0-g8eda093)
{% from 'macrolib.cfg.in' import cfg with context %}
# What ports/sockets to listen on, and what options for them.
# we listen only on unix socket
listen "{{ unicorn.socket }}", :backlog => {{ cfg('unicorn_backlog_socket') }}
#listen "127.0.0.1:8888", :tcp_nopush => true
working_directory '{{ gitlab_work.location }}'
# What the timeout for killing busy workers is, in seconds
timeout {{ cfg('unicorn_worker_timeout') }}
# Whether the app should be pre-loaded
preload_app true
# How many worker processes
worker_processes {{ cfg('unicorn_worker_processes') }}
# about before_fork / after_fork - see:
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/definitions/unicorn_service.rb
# http://bogomips.org/unicorn.git/tree/examples/unicorn.conf.rb?id=3312aca8#n75
# What to do before we fork a worker
before_fork do |server, worker|
# XXX why gitlab does not enable this?
# # the following is highly recomended for Rails + "preload_app true"
# # as there's no need for the master process to hold a connection
# defined?(ActiveRecord::Base) and
# ActiveRecord::Base.connection.disconnect!
# This allows a new master process to incrementally
# phase out the old master process with SIGTTOU to avoid a
# thundering herd (especially in the "preload_app false" case)
# when doing a transparent upgrade. The last worker spawned
# will then kill off the old master process with a SIGQUIT.
old_pid = "#{server.config[:pid]}.oldbin"
if old_pid != server.pid
begin
sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
Process.kill(sig, File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
end
end
end
# What to do after we fork a worker
after_fork do |server, worker|
# per-process listener ports for debugging/admin/migrations
# addr = "127.0.0.1:#{9293 + worker.nr}"
# server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true)
# XXX why gitlab does not enable this?
# # the following is *required* for Rails + "preload_app true",
# defined?(ActiveRecord::Base) and
# ActiveRecord::Base.establish_connection
end
# Where to drop a pidfile
pid '{{ directory.run }}/unicorn.pid'
# Where stderr gets logged
stderr_path '{{ unicorn.log }}/unicorn_stderr.log'
# Where stdout gets logged
stdout_path '{{ unicorn.log }}/unicorn_stdout.log'
software/gitlab/watcher-sigkill.in 0 → 100644
View file @ 866d9051
#!{{ bash.location }}/bin/bash
# run program under SIGKILL watchdog
# watcher-sigkill <prog> [<progargs> ...]
#
# if the program terminates with SIGKILL - it is restarted after grace period.
# if the program terminates otherwise - whole process terminates.
if [ "$#" -lt 1 ]; then
echo "Usage: watcher-sigkill <prog> [<progargs> ...]" 1>&2
exit 1
fi
prog="$@"
progpid=""
killexit="137" # = 128 + 9 (exit code of process terminated by SIGKILL)
# make sure to terminate children, when we exit.
# needed for e.g. when `slapos node stop ...` kills us.
trap 'atexit' EXIT
atexit() {
jobs="$(jobs -p)"
test -n "$jobs" && kill $jobs
}
# run prog under monitoring
while true; do
echo "run $prog"
$prog &
progpid=$!
echo "wait $progpid"
wait $progpid
status=$?
echo "-> $status"
# if program terminated not by SIGKILL - exit
if [ "$status" != "$killexit" ] ; then
echo "exit $status"
exit "$status"
fi
# otherwise sleep a bit and restart
sleep 1
done
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7