Commit 089696b4 authored by Kamil Trzciński's avatar Kamil Trzciński

Merge branch 'workhorse-passthrough' into 'master'

Pass all requests from NGINX to gitlab-workhorse



See merge request !2073
parents a1fd87cd f2db1410
...@@ -348,7 +348,7 @@ GitLab Shell is an SSH access and repository management software developed speci ...@@ -348,7 +348,7 @@ GitLab Shell is an SSH access and repository management software developed speci
cd /home/git cd /home/git
sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git
cd gitlab-workhorse cd gitlab-workhorse
sudo -u git -H git checkout 0.4.2 sudo -u git -H git checkout 0.5.0
sudo -u git -H make sudo -u git -H make
### Initialize Database and Activate Advanced Features ### Initialize Database and Activate Advanced Features
......
...@@ -78,7 +78,7 @@ which should already be on your system from GitLab 8.1. ...@@ -78,7 +78,7 @@ which should already be on your system from GitLab 8.1.
```bash ```bash
cd /home/git/gitlab-workhorse cd /home/git/gitlab-workhorse
sudo -u git -H git fetch --all sudo -u git -H git fetch --all
sudo -u git -H git checkout 0.4.2 sudo -u git -H git checkout 0.5.0
sudo -u git -H make sudo -u git -H make
``` ```
...@@ -115,6 +115,12 @@ git diff origin/8-2-stable:config/gitlab.yml.example origin/8-3-stable:config/gi ...@@ -115,6 +115,12 @@ git diff origin/8-2-stable:config/gitlab.yml.example origin/8-3-stable:config/gi
#### Nginx configuration #### Nginx configuration
GitLab 8.3 introduces major changes in the NGINX configuration.
Because all HTTP requests pass through gitlab-workhorse now a lot of
directives need to be removed from NGINX. During future upgrades there
should be much less changes in the NGINX configuration because of
this.
View changes between the previous recommended Nginx configuration and the View changes between the previous recommended Nginx configuration and the
current one: current one:
...@@ -134,6 +140,18 @@ via [/etc/default/gitlab]. ...@@ -134,6 +140,18 @@ via [/etc/default/gitlab].
[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache [Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-3-stable/lib/support/init.d/gitlab.default.example#L34 [/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-3-stable/lib/support/init.d/gitlab.default.example#L34
#### Init script
We updated the init script for GitLab in order to pass new
configuration options to gitlab-workhorse. We let gitlab-workhorse
connect to the Rails application via a Unix domain socket and we tell
it where the 'public' directory of GitLab is.
```
cd /home/git/gitlab
sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
```
### 8. Use Redis v2.8.0+ ### 8. Use Redis v2.8.0+
Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but
......
...@@ -33,12 +33,13 @@ app_user="git" ...@@ -33,12 +33,13 @@ app_user="git"
app_root="/home/$app_user/gitlab" app_root="/home/$app_user/gitlab"
pid_path="$app_root/tmp/pids" pid_path="$app_root/tmp/pids"
socket_path="$app_root/tmp/sockets" socket_path="$app_root/tmp/sockets"
rails_socket="$socket_path/gitlab.socket"
web_server_pid_path="$pid_path/unicorn.pid" web_server_pid_path="$pid_path/unicorn.pid"
sidekiq_pid_path="$pid_path/sidekiq.pid" sidekiq_pid_path="$pid_path/sidekiq.pid"
mail_room_enabled=false mail_room_enabled=false
mail_room_pid_path="$pid_path/mail_room.pid" mail_room_pid_path="$pid_path/mail_room.pid"
gitlab_workhorse_pid_path="$pid_path/gitlab-workhorse.pid" gitlab_workhorse_pid_path="$pid_path/gitlab-workhorse.pid"
gitlab_workhorse_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080" gitlab_workhorse_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080 -authSocket $rails_socket -documentRoot $app_root/public"
gitlab_workhorse_log="$app_root/log/gitlab-workhorse.log" gitlab_workhorse_log="$app_root/log/gitlab-workhorse.log"
shell_path="/bin/bash" shell_path="/bin/bash"
......
...@@ -36,7 +36,7 @@ gitlab_workhorse_pid_path="$pid_path/gitlab-workhorse.pid" ...@@ -36,7 +36,7 @@ gitlab_workhorse_pid_path="$pid_path/gitlab-workhorse.pid"
# '-listenNetwork tcp -listenAddr localhost:8181'. # '-listenNetwork tcp -listenAddr localhost:8181'.
# The -authBackend setting tells gitlab-workhorse where it can reach # The -authBackend setting tells gitlab-workhorse where it can reach
# Unicorn. # Unicorn.
gitlab_workhorse_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080" gitlab_workhorse_options="-listenUmask 0 -listenNetwork unix -listenAddr $socket_path/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080 -authSocket $socket_path/gitlab.socket -documentRoot $app_root/public"
gitlab_workhorse_log="$app_root/log/gitlab-workhorse.log" gitlab_workhorse_log="$app_root/log/gitlab-workhorse.log"
# mail_room_enabled specifies whether mail_room, which is used to process incoming email, is enabled. # mail_room_enabled specifies whether mail_room, which is used to process incoming email, is enabled.
......
...@@ -10,34 +10,12 @@ ...@@ -10,34 +10,12 @@
## If you change this file in a Merge Request, please also create ## If you change this file in a Merge Request, please also create
## a Merge Request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests ## a Merge Request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
## ##
##################################
## CHUNKED TRANSFER ##
##################################
##
## It is a known issue that Git-over-HTTP requires chunked transfer encoding [0]
## which is not supported by Nginx < 1.3.9 [1]. As a result, pushing a large object
## with Git (i.e. a single large file) can lead to a 411 error. In theory you can get
## around this by tweaking this configuration file and either:
## - installing an old version of Nginx with the chunkin module [2] compiled in, or
## - using a newer version of Nginx.
##
## At the time of writing we do not know if either of these theoretical solutions works.
## As a workaround users can use Git over SSH to push large files.
##
## [0] https://git.kernel.org/cgit/git/git.git/tree/Documentation/technical/http-protocol.txt#n99
## [1] https://github.com/agentzh/chunkin-nginx-module#status
## [2] https://github.com/agentzh/chunkin-nginx-module
##
################################### ###################################
## configuration ## ## configuration ##
################################### ###################################
## ##
## See installation.md#using-https for additional HTTPS configuration details. ## See installation.md#using-https for additional HTTPS configuration details.
upstream gitlab {
server unix:/home/git/gitlab/tmp/sockets/gitlab.socket fail_timeout=0;
}
upstream gitlab-workhorse { upstream gitlab-workhorse {
server unix:/home/git/gitlab/tmp/sockets/gitlab-workhorse.socket fail_timeout=0; server unix:/home/git/gitlab/tmp/sockets/gitlab-workhorse.socket fail_timeout=0;
} }
...@@ -54,10 +32,6 @@ server { ...@@ -54,10 +32,6 @@ server {
server_tokens off; ## Don't show the nginx version number, a security best practice server_tokens off; ## Don't show the nginx version number, a security best practice
root /home/git/gitlab/public; root /home/git/gitlab/public;
## Increase this if you want to upload large attachments
## Or if you want to accept large git objects over http
client_max_body_size 20m;
## See app/controllers/application_controller.rb for headers set ## See app/controllers/application_controller.rb for headers set
## Individual nginx logs for this GitLab vhost ## Individual nginx logs for this GitLab vhost
...@@ -65,103 +39,8 @@ server { ...@@ -65,103 +39,8 @@ server {
error_log /var/log/nginx/gitlab_error.log; error_log /var/log/nginx/gitlab_error.log;
location / { location / {
## Serve static files from defined root folder.
## @gitlab is a named location for the upstream fallback, see below.
try_files $uri /index.html $uri.html @gitlab;
}
## We route uploads through GitLab to prevent XSS and enforce access control.
location /uploads/ {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
# gzip off;
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_pass http://gitlab;
}
## If a file, which is not found in the root folder is requested,
## then the proxy passes the request to the upsteam (gitlab unicorn).
location @gitlab {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
# gzip off;
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_pass http://gitlab;
}
location ~ ^/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/api/v3/projects/.*/repository/archive {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
# Build artifacts should be submitted to this location
location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
client_max_body_size 0; client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block gzip off;
error_page 418 = @gitlab-workhorse;
return 418;
}
# Build artifacts should be submitted to this location
location ~ /ci/api/v1/builds/[0-9]+/artifacts {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location @gitlab-workhorse {
client_max_body_size 0;
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
# gzip off;
## https://github.com/gitlabhq/gitlabhq/issues/694 ## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds. ## Some requests take more than 30 seconds.
...@@ -169,14 +48,7 @@ server { ...@@ -169,14 +48,7 @@ server {
proxy_connect_timeout 300; proxy_connect_timeout 300;
proxy_redirect off; proxy_redirect off;
# Do not buffer Git HTTP responses proxy_http_version 1.1;
proxy_buffering off;
# The following settings only work with NGINX 1.7.11 or newer
#
# # Pass chunked request bodies to gitlab-workhorse as-is
# proxy_request_buffering off;
# proxy_http_version 1.1;
proxy_set_header Host $http_host; proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
...@@ -185,18 +57,4 @@ server { ...@@ -185,18 +57,4 @@ server {
proxy_pass http://gitlab-workhorse; proxy_pass http://gitlab-workhorse;
} }
## Enable gzip compression as per rails guide:
## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
## WARNING: If you are using relative urls remove the block below
## See config/application.rb under "Relative url support" for the list of
## other files that need to be changed for relative url support
location ~ ^/(assets)/ {
root /home/git/gitlab/public;
gzip_static on; # to serve pre-gzipped version
expires max;
add_header Cache-Control public;
}
error_page 502 /502.html;
} }
...@@ -14,34 +14,12 @@ ...@@ -14,34 +14,12 @@
## If you change this file in a Merge Request, please also create ## If you change this file in a Merge Request, please also create
## a Merge Request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests ## a Merge Request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
## ##
##################################
## CHUNKED TRANSFER ##
##################################
##
## It is a known issue that Git-over-HTTP requires chunked transfer encoding [0]
## which is not supported by Nginx < 1.3.9 [1]. As a result, pushing a large object
## with Git (i.e. a single large file) can lead to a 411 error. In theory you can get
## around this by tweaking this configuration file and either:
## - installing an old version of Nginx with the chunkin module [2] compiled in, or
## - using a newer version of Nginx.
##
## At the time of writing we do not know if either of these theoretical solutions works.
## As a workaround users can use Git over SSH to push large files.
##
## [0] https://git.kernel.org/cgit/git/git.git/tree/Documentation/technical/http-protocol.txt#n99
## [1] https://github.com/agentzh/chunkin-nginx-module#status
## [2] https://github.com/agentzh/chunkin-nginx-module
##
################################### ###################################
## configuration ## ## configuration ##
################################### ###################################
## ##
## See installation.md#using-https for additional HTTPS configuration details. ## See installation.md#using-https for additional HTTPS configuration details.
upstream gitlab {
server unix:/home/git/gitlab/tmp/sockets/gitlab.socket fail_timeout=0;
}
upstream gitlab-workhorse { upstream gitlab-workhorse {
server unix:/home/git/gitlab/tmp/sockets/gitlab-workhorse.socket fail_timeout=0; server unix:/home/git/gitlab/tmp/sockets/gitlab-workhorse.socket fail_timeout=0;
} }
...@@ -61,7 +39,6 @@ server { ...@@ -61,7 +39,6 @@ server {
error_log /var/log/nginx/gitlab_error.log; error_log /var/log/nginx/gitlab_error.log;
} }
## HTTPS host ## HTTPS host
server { server {
listen 0.0.0.0:443 ssl; listen 0.0.0.0:443 ssl;
...@@ -70,10 +47,6 @@ server { ...@@ -70,10 +47,6 @@ server {
server_tokens off; ## Don't show the nginx version number, a security best practice server_tokens off; ## Don't show the nginx version number, a security best practice
root /home/git/gitlab/public; root /home/git/gitlab/public;
## Increase this if you want to upload large attachments
## Or if you want to accept large git objects over http
client_max_body_size 20m;
## Strong SSL Security ## Strong SSL Security
## https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/ ## https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/
ssl on; ssl on;
...@@ -110,104 +83,7 @@ server { ...@@ -110,104 +83,7 @@ server {
error_log /var/log/nginx/gitlab_error.log; error_log /var/log/nginx/gitlab_error.log;
location / { location / {
## Serve static files from defined root folder.
## @gitlab is a named location for the upstream fallback, see below.
try_files $uri /index.html $uri.html @gitlab;
}
## We route uploads through GitLab to prevent XSS and enforce access control.
location /uploads/ {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
gzip off;
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_pass http://gitlab;
}
## If a file, which is not found in the root folder is requested,
## then the proxy passes the request to the upsteam (gitlab unicorn).
location @gitlab {
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
gzip off;
## https://github.com/gitlabhq/gitlabhq/issues/694
## Some requests take more than 30 seconds.
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_pass http://gitlab;
}
location ~ ^/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
client_max_body_size 0; client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location ~ ^/api/v3/projects/.*/repository/archive {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
# Build artifacts should be submitted to this location
location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
# Build artifacts should be submitted to this location
location ~ /ci/api/v1/builds/[0-9]+/artifacts {
client_max_body_size 0;
# 'Error' 418 is a hack to re-use the @gitlab-workhorse block
error_page 418 = @gitlab-workhorse;
return 418;
}
location @gitlab-workhorse {
client_max_body_size 0;
## If you use HTTPS make sure you disable gzip compression
## to be safe against BREACH attack.
gzip off; gzip off;
## https://github.com/gitlabhq/gitlabhq/issues/694 ## https://github.com/gitlabhq/gitlabhq/issues/694
...@@ -216,14 +92,7 @@ server { ...@@ -216,14 +92,7 @@ server {
proxy_connect_timeout 300; proxy_connect_timeout 300;
proxy_redirect off; proxy_redirect off;
# Do not buffer Git HTTP responses proxy_http_version 1.1;
proxy_buffering off;
# The following settings only work with NGINX 1.7.11 or newer
#
# # Pass chunked request bodies to gitlab-workhorse as-is
# proxy_request_buffering off;
# proxy_http_version 1.1;
proxy_set_header Host $http_host; proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
...@@ -232,18 +101,4 @@ server { ...@@ -232,18 +101,4 @@ server {
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://gitlab-workhorse; proxy_pass http://gitlab-workhorse;
} }
## Enable gzip compression as per rails guide:
## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
## WARNING: If you are using relative urls remove the block below
## See config/application.rb under "Relative url support" for the list of
## other files that need to be changed for relative url support
location ~ ^/(assets)/ {
root /home/git/gitlab/public;
gzip_static on; # to serve pre-gzipped version
expires max;
add_header Cache-Control public;
}
error_page 502 /502.html;
} }
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment