Commit 9f53a532 authored by Dmitriy Zaporozhets's avatar Dmitriy Zaporozhets

Merge branch 'y/httpunix2' into 'master'

Add support to connect gitlab-shell to Unicorn via UNIX socket  (v2)

Hello up there.

I'm doing SlapOS port of GitLab, and that means several different services could be running on the same machine, including several GitLabs.

So far all internal GitLab subservices could be glued together via UNIX sockets except gitlab-shell -> Unicorn link, which, when done via local TCP, requires firewall/network namespaces to protect services on one machine from each other.

On the other hand access to UNIX domain sockets is managed via regular UNIX permissions on filesystem, and thus is easier to manage. Besides UNIX domain sockets are well known to be faster compared to TCP over loopback - in particular to have ~ 2 times less latency and ~ 2 times more throughput.

From this point of view it makes sense to teach gitlab-shell to talk to Unicorn via UNIX socket and switch to that mode by default eventually.

I've just made a patch for this. Please apply.

Thanks beforehand,  
Kirill

/cc @dzaporozhets, @jacobvosmaer, @rspeicher 

See merge request !30
parents ae498b6c 184385ac
...@@ -10,7 +10,9 @@ user: git ...@@ -10,7 +10,9 @@ user: git
# Default: http://localhost:8080/ # Default: http://localhost:8080/
# You only have to change the default if you have configured Unicorn # You only have to change the default if you have configured Unicorn
# to listen on a custom port, or if you have configured Unicorn to # to listen on a custom port, or if you have configured Unicorn to
# only listen on a Unix domain socket. # only listen on a Unix domain socket. For Unix domain sockets use
# "http+unix://<urlquoted-path-to-socket>/", e.g.
# "http+unix://%2Fpath%2Fto%2Fsocket/"
gitlab_url: "http://localhost:8080/" gitlab_url: "http://localhost:8080/"
# See installation.md#using-https for additional HTTPS configuration details. # See installation.md#using-https for additional HTTPS configuration details.
......
...@@ -5,6 +5,7 @@ require 'json' ...@@ -5,6 +5,7 @@ require 'json'
require_relative 'gitlab_config' require_relative 'gitlab_config'
require_relative 'gitlab_logger' require_relative 'gitlab_logger'
require_relative 'gitlab_access' require_relative 'gitlab_access'
require_relative 'httpunix'
class GitlabNet class GitlabNet
class ApiUnreachableError < StandardError; end class ApiUnreachableError < StandardError; end
...@@ -63,7 +64,11 @@ class GitlabNet ...@@ -63,7 +64,11 @@ class GitlabNet
end end
def http_client_for(uri) def http_client_for(uri)
http = Net::HTTP.new(uri.host, uri.port) if uri.is_a?(URI::HTTPUNIX)
http = Net::HTTPUNIX.new(uri.hostname)
else
http = Net::HTTP.new(uri.host, uri.port)
end
if uri.is_a?(URI::HTTPS) if uri.is_a?(URI::HTTPS)
http.use_ssl = true http.use_ssl = true
......
# support for http+unix://... connection scheme
#
# The URI scheme has the same structure as the similar one for python requests. See:
# http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/
# https://github.com/msabramo/requests-unixsocket
require 'uri'
require 'net/http'
module URI
class HTTPUNIX < HTTP
def hostname
# decode %XX from path to file
v = self.host
URI.decode(v)
end
# port is not allowed in URI
DEFAULT_PORT = nil
def set_port(v)
return v unless v
raise InvalidURIError, "http+unix:// cannot contain port"
end
end
@@schemes['HTTP+UNIX'] = HTTPUNIX
end
# Based on:
# - http://stackoverflow.com/questions/15637226/ruby-1-9-3-simple-get-request-to-unicorn-through-socket
# - Net::HTTP::connect
module Net
class HTTPUNIX < HTTP
def initialize(socketpath, port=nil)
super(socketpath, port)
@port = nil # HTTP will set it to default - override back -> set DEFAULT_PORT
end
# override to prevent ":<port>" being appended to HTTP_HOST
def addr_port
address
end
def connect
D "opening connection to #{address} ..."
s = UNIXSocket.new(address)
D "opened"
@socket = BufferedIO.new(s)
@socket.read_timeout = @read_timeout
@socket.continue_timeout = @continue_timeout
@socket.debug_output = @debug_output
on_connect
end
end
end
require_relative 'spec_helper'
require_relative '../lib/httpunix'
require 'webrick'
describe URI::HTTPUNIX do
describe :parse do
uri = URI::parse('http+unix://%2Fpath%2Fto%2Fsocket/img.jpg')
subject { uri }
it { should be_an_instance_of(URI::HTTPUNIX) }
its(:scheme) { should eq('http+unix') }
its(:hostname) { should eq('/path/to/socket') }
its(:path) { should eq('/img.jpg') }
end
end
# like WEBrick::HTTPServer, but listens on UNIX socket
class HTTPUNIXServer < WEBrick::HTTPServer
def listen(address, port)
socket = Socket.unix_server_socket(address)
socket.autoclose = false
server = UNIXServer.for_fd(socket.fileno)
socket.close
@listeners << server
end
end
def tmp_socket_path
File.join(ROOT_PATH, 'tmp', 'socket')
end
describe Net::HTTPUNIX do
# "hello world" over unix socket server in background thread
FileUtils.mkdir_p(File.dirname(tmp_socket_path))
server = HTTPUNIXServer.new(:BindAddress => tmp_socket_path)
server.mount_proc '/' do |req, resp|
resp.body = "Hello World (at #{req.path})"
end
Thread.start { server.start }
it "talks via HTTP ok" do
VCR.turned_off do
begin
WebMock.allow_net_connect!
http = Net::HTTPUNIX.new(tmp_socket_path)
expect(http.get('/').body).to eq('Hello World (at /)')
expect(http.get('/path').body).to eq('Hello World (at /path)')
ensure
WebMock.disable_net_connect!
end
end
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment