1. 07 Aug, 2016 4 commits
    • Kirill Smelkov's avatar
      gitlab: Sync sidekiq service to upstream · af0c5b11
      Kirill Smelkov authored
      Like for 2a835e63
      
          $ git diff 8.6.5+ce.0-0-g342f8be..8.7.9+ce.1-0-gf589ad7 --
                  files/gitlab-cookbooks/gitlab/templates/default/sv-sidekiq-run.erb
      
      is empty.
      af0c5b11
    • Kirill Smelkov's avatar
      gitlab: Sync gitlab-parameters.cfg to upstream · 61177775
      Kirill Smelkov authored
      I've manually reviewed
      
          git diff 8.6.5+ce.0-0-g342f8be..8.7.9+ce.1-0-gf589ad7 --    \
              files/gitlab-config-template/gitlab.rb.template \
              files/gitlab-cookbooks/gitlab/attributes/default.rb
      
      and modulo trusted proxies there are no interesting changes for us.
      61177775
    • Kirill Smelkov's avatar
      gitlab: Slapos'ify gitlab config updates · 35b142e0
      Kirill Smelkov authored
      - config.ru template is gone - pristine gitlab-ce/config.ru can do the
        job because it obtains unicorn OOM killer setting via environment
        variables.
      
        https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/672
      
      - we put TODO there for configuring trusted proxies (gitlab.yml & nginx)
      
      - we restore our slaposified configuration from config.ru to unicorn.rb
      35b142e0
    • Kirill Smelkov's avatar
      gitlab: Update software to gitlab 8.7 · bf688f44
      Kirill Smelkov authored
      Update GitLab software to
      
          - gitlab-ce 8.7.9 + NXD patches
      
          - gitlab-shell to 2.7.2 + 1 patch to remove unneeded hooks.old in *.git
      
          - gitlab-workhorse stays at 0.7.1 + NXD patches because gitlab-ce 8.7.x
            sticks to this version (i.e. no workhorse upgrade for gitlab 8.6 -> 8.7)
      
      This only updates software and begins SR update to 8.7 - for now gitlab
      instance becomes non-working -- we'll pull in configuration files
      updates and fixups in the following patches.
      bf688f44
  2. 05 Aug, 2016 1 commit
  3. 02 Aug, 2016 7 commits
  4. 19 Jul, 2016 1 commit
    • Jérome Perrin's avatar
      gitlab: enable parameters-extra options when creating wrapper · 6082d6e9
      Jérome Perrin authored
      @jerome says at 5f5d5102 (comment 17119):
      
      before f4e51f77, we had:
      `~/srv/runner/instance/slappart0/bin/gitlab-rake` containing:
          ```python
          ...
          if __name__ == '__main__':
              sys.exit(slapos.recipe.librecipe.execute.generic_exec((['/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/bin/bundle', 'exec', 'sh', '-c', 'cd /srv/slapgrid/slappart16/srv/runner/instance/slappart0/gitlab-work && rake "$@"', 'rake'], None, {'BUNDLE_GEMFILE': '/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/parts/gitlab/Gemfile', 'HOME': '/srv/slapgrid/slappart16/srv/runner/instance/slappart0', 'SIDEKIQ_MEMORY_KILLER_MAX_RSS': '1000000', 'RAILS_ENV': 'production'})))
          ```
      
      after, `~/srv/runner/instance/slappart0/bin/gitlab-rake` contains:
          ```shell
          #!/bin/bash
          COMMAND=/srv/slapgrid/slappart16/srv/runner/instance/slappart0/bin/gitlab-rake.py
      
          # If the wrapped command uses a shebang, execute the referenced
          # executable passing the script path as first argument.
          # This is to workaround the limitation of 127 characters in #!
          if [[ -f $COMMAND && x$(head -c2 "$COMMAND") = x"#!" ]]; then
            SHEBANG=$(head -1 "$COMMAND")
            INTERPRETER=( ${SHEBANG#\#!} )
            COMMAND="${INTERPRETER[@]} $COMMAND"
          fi
      
          exec $COMMAND
          ```
      
      which is a wrapper around `gitlab-rake.py` containing:
          ```python
          ...
          if __name__ == '__main__':
              sys.exit(slapos.recipe.librecipe.execute.generic_exec((['/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/bin/bundle', 'exec', 'sh', '-c', 'cd /srv/slapgrid/slappart16/srv/runner/instance/slappart0/gitlab-work && rake "$@"', 'rake'], None, {'BUNDLE_GEMFILE': '/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/parts/gitlab/Gemfile', 'HOME': '/srv/slapgrid/slappart16/srv/runner/instance/slappart0', 'SIDEKIQ_MEMORY_KILLER_MAX_RSS': '1000000', 'RAILS_ENV': 'production'})))
          ```
      
      `gitlab-rake.py` after is same as `gitlab-rake` before.
      
      This [slapos.cookbook:wrapper](https://lab.nexedi.com/nexedi/slapos/blob/cd9faac0/slapos/recipe/wrapper.py#L39) has an argument *parameters-extra* which if set to true, propagate command line arguments to the wrapped script. The default value for this parameter is false.
      
      Before f4e51f77, the generated wrapper was also propagating arguments even when *parameters-extra* was not set, but since this commit, this *parameters-extra* option is now handled as expected.
      
      This is the reason for this regression. In our case, when we see `/srv/slapgrid/slappart16/srv/runner/instance/slappart0/bin/gitlab-rake assets:clean`, it just calls `rake` without arguments.
      
      So a simple patch that fix the problem would be jerome/slapos@d3d05f02 . This way, the generated wrapper becomes:
      
      ```shell
      ...
      exec $COMMAND $@
      ```
      
      and arguments are correctly propagated.
      
      Feel free to cherry-pick that patch for now, but it may be nice to rethink this *parameters-extra* option, after this debugging session, I believe it should be true by default.
      
      /cc @seb for introducing the parameter in 80bb4305 and @vpelletier for touching this code in e7083872
      
      /reviewed-by @kirr
      6082d6e9
  5. 22 Apr, 2016 2 commits
  6. 11 Apr, 2016 1 commit
  7. 14 Mar, 2016 1 commit
  8. 06 Mar, 2016 1 commit
  9. 03 Mar, 2016 1 commit
  10. 02 Mar, 2016 1 commit
  11. 29 Feb, 2016 1 commit
  12. 28 Feb, 2016 7 commits
    • Kirill Smelkov's avatar
      gitlab: Sync sidekiq service to upstream · 0a72505e
      Kirill Smelkov authored
          $ git diff 8.4.4+ce.0-0-g1680742..8.5.1+ce.0-1-ge732b39 --  \
              files/gitlab-cookbooks/gitlab/templates/default/sv-sidekiq-run.erb
      
      shows nothing.
      0a72505e
    • Kirill Smelkov's avatar
      gitlab: Sync gitlab-parameters.cfg to upstream · 043402bb
      Kirill Smelkov authored
      I manually reviewed
      
          $ git diff 8.4.2+ce.0-3-g68d5ee8..8.5.1+ce.0-1-ge732b39 \
              files/gitlab-config-template/gitlab.rb.template \
              files/gitlab-cookbooks/gitlab/attributes/default.rb
      
      in omnibus-gitlab, and module proxy_set_header change, which we already
      addressed in previous patch in Nginx config, there are no more changes
      for us.
      043402bb
    • Kirill Smelkov's avatar
      gitlab: Slapos'ify gitlab config updates · b19d2942
      Kirill Smelkov authored
          - relative URL support: comment out - we do not need it - gitlab is
            always located at /.
      
          - Nginx-http: restore our version for proxy_set_header - upstream
            turned to allowing users to configure this, see e.g.
      
              https://gitlab.com/gitlab-org/omnibus-gitlab/commit/e13d5e42
              https://gitlab.com/gitlab-org/omnibus-gitlab/commit/a450585e
      
            but doing this way creates more complexity for gitlab SR, so I've
            restored our version which essentially does the same as default in
            omnibus-gitlab, and if we'll need to tune it - we can do directly in
            Nginx config.
      
            In other words slapos version does not allow users to tune nginx
            headers as instance parameter.
      b19d2942
    • Kirill Smelkov's avatar
      gitlab: Update software to gitlab 8.5 + friends · 2df034ba
      Kirill Smelkov authored
      Update GitLab software to
      
          - gitlab-ce 8.5.1 + NXD patches
      
            https://lab.nexedi.com/kirr/gitlab-ce/commits/8-5-nxd
      
          - gitlab-shell to 2.6.10 + 1 patch to remove unneeded hooks.old in *.git
      
            https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/40
      
          - gitlab-workhorse 0.6.4 + NXD patches.
      
            https://lab.nexedi.com/kirr/gitlab-workhorse/commits/y/blobraw-4
            https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17
      
            ( download speedup patches got improved, and now also properly
              proxy _gitlab_session cookie to auth backend, so raw files for
              private repositories now open in browser ok )
      
      This only updates software and begins SR update to 8.5 - for now gitlab
      instance becomes non-working -- we'll pull in configuration files
      updates and fixups in the following patches.
      
      P.S. we also pin-up rubygems version, used to build gems, along the way.
      2df034ba
    • Kirill Smelkov's avatar
      gitlab: GitLab wants git to be really on $PATH · caaf6825
      Kirill Smelkov authored
      GitLab uses git executable by full path as defined in gitlab.yml, but
      not all places in code use it, e.g. here git is used just from $PATH
      
          https://gitlab.com/gitlab-org/gitlab_git/blob/2f0d3c1a/lib/gitlab_git/repository.rb#L259
      
      So make sure to include our git into bundler-4gitlab PATH.
      caaf6825
    • Kirill Smelkov's avatar
      gitlab: Sync upstream configs from omnibus-gitlab · 02d0063b
      Kirill Smelkov authored
      Like 8c62b063, d17f1f5f and e8461571 - pristine copy from omnibus-gitlab
      8.5.1+ce.0-1-ge732b39 .
      
      Changes are in
      
          - gitlab.yml.erb, unicorn.rb.erb
      
            * Something related to relative URL root (we do not use)
            * Something related to SAML (we do not use)
            * Misc
      
          - nginx-gitlab-http.conf.erb
      
            * SPDY -> HTTP/2
            * Relative URL root
            * Configurable proxy_set_header passing
      
      The following files stay the same:
      
          - database.yml.erb
          - gitconfig.erb
          - gitlab-rails-config.ru.erb
          - gitlab-shell-config.yml.erb
          - nginx.conf.erb
          - rack_attack.rb.erb
          - resque.yml.erb
          - smtp_settings.rb.erb
      02d0063b
    • Kirill Smelkov's avatar
      gitlab: Establish proper 1 branch for tracking upstream configs · 97dcf455
      Kirill Smelkov authored
      It was my mistake to establish several tracking lines for tracking
      upstream changes - e.g. in
      
          61544d87    (gitlab: Import nginx http configuration from omnibus-gitlab)
      
      we started not from
      
          6fd7b987    (gitlab: Import gitlab-ce & gitlab-shell configs from omnibus-gitlab)
      
      -- the first upstream tracking commit on its own branch -- but from
      
          4c127fdd    (gitlab: Setup sidekiq service)
      
      i.e. from after some changes which already tweaked upstream
      configuration files.
      
      This makes updating gitlab more work than necessary: instead of
      switching to upstream branch only once, importing all files, and
      then switching back to master and merging upstream changes only once, we
      currently have to do that operation 3 times:
      
          - for main gitlab settings,
          - for nginx settings, and
          - for gitconfig settings
      
      which is not convenient and wastes our time.
      
      So establish a proper 1 branch for tracking upstream configs:
      
      Here we cherry-pick the following commits
      
          61544d87    (gitlab: Import nginx http configuration from omnibus-gitlab)
          d17f1f5f    (gitlab: Sync nginx http configuration from omnibus gitlab)
      
          8f945bd2    (gitlab: Import gitconfig from omnibus-gitlab)
          e8461571    (gitlab: Sync gitconfig settings from omnibus-gitlab)
      
      and later we'll be updating upstream files on a branch starting from
      this commit and containing upstream changes only.
      
      /cc @kazuhiko, @jerome
      97dcf455
  13. 17 Feb, 2016 2 commits
  14. 16 Feb, 2016 1 commit
  15. 13 Feb, 2016 1 commit
    • Kirill Smelkov's avatar
      gitlab: Wait a bit for PostgreSQL to be ready in Unicorn startup · 949e55e2
      Kirill Smelkov authored
      As it was outlined in 5a744de7 (gitlab: Compile assets on instantiation
      and make sure DB is properly setup/migrated before unicorn runs) we are
      performing DB initialization and migration in pre-action as part of
      unicorn startup script. But that currently has one drawback:
      
          if all services start at the same time - e.g. both PostgreSQL and
          Unicorn - and that is a common scenario when SR is compiled and
          instantiated / started, PostgreSQL is usually not yet ready to
          process queries from Unicorn startup script, and first-time Unicorn
          startup fails.
      
      Until now this problem was workarounded by manually starting unicorn
      second time - after some time postgresql is started and ready. But why
      do it manually, if we can do the same logic automatically. So fix it:
      
          in Unicorn startup script wait a bit (up to 5 seconds) for
          PostgreSQL to become ready.
      
      /cc @kazuhiko, @jerome
      949e55e2
  16. 11 Feb, 2016 8 commits