Commit f7b05616 authored by Douwe Maan's avatar Douwe Maan

Merge branch 'ericidema/gitlab-ce-import-with-github-personal-access-tokens' into 'master'

Allow importing from Github using Personal Access Tokens

_Originally opened at !4005 by @ericidema._

------

## What does this MR do?

* Made changes to `Gitlab::GithubImport::Client` so that it can be used with Github Personal Access Tokens without the need for OAuth.
* Added UI to collect Personal Access Token from user.
* Detect if the user has logged in with GitHub and use OAuth to skip the Personal Access Token form.

## Are there points in the code the reviewer needs to double check?

Twin Omnibus MR: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/774

## What are the relevant issue numbers?

Closes #13970.

## Screenshots

### GitHub import is configured

![github_import_configured](/uploads/151e4f0edf3f87bfa03c2d97dda8b3d8/github_import_configured.png)

-----

### GitHub import is not configured

![github_import_not_configured](/uploads/cb129f7e2ffe66cceb28ccd9af480284/github_import_not_configured.png)

-----

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4938
parents 82ccc0af ce663540
......@@ -18,6 +18,7 @@ v 8.10.0 (unreleased)
- Exclude email check from the standard health check
- Fix changing issue state columns in milestone view
- Add notification settings dropdown for groups
- Allow importing from Github using Personal Access Tokens. (Eric K Idema)
- Fix user creation with stronger minimum password requirements !4054 (nathan-pmt)
- PipelinesFinder uses git cache data
- Check for conflicts with existing Project's wiki path when creating a new project.
......
class Import::GithubController < Import::BaseController
before_action :verify_github_import_enabled
before_action :github_auth, except: :callback
before_action :github_auth, only: [:status, :jobs, :create]
rescue_from Octokit::Unauthorized, with: :github_unauthorized
helper_method :logged_in_with_github?
def new
if logged_in_with_github?
go_to_github_for_permissions
elsif session[:github_access_token]
redirect_to status_import_github_url
end
end
def callback
session[:github_access_token] = client.get_token(params[:code])
redirect_to status_import_github_url
end
def personal_access_token
session[:github_access_token] = params[:personal_access_token]
redirect_to status_import_github_url
end
def status
@repos = client.repos
@already_added_projects = current_user.created_projects.where(import_type: "github")
......@@ -57,10 +72,14 @@ class Import::GithubController < Import::BaseController
end
def github_unauthorized
go_to_github_for_permissions
session[:github_access_token] = nil
redirect_to new_import_github_url,
alert: 'Access denied to your GitHub account.'
end
private
def logged_in_with_github?
current_user.identities.exists?(provider: 'github')
end
def access_params
{ github_access_token: session[:github_access_token] }
......
- page_title "GitHub Import"
- header_title "Projects", root_path
%h3.page-title
= icon 'github', text: 'Import Projects from GitHub'
- if github_import_configured?
%p
To import a GitHub project, you first need to authorize GitLab to access
the list of your GitHub repositories:
= link_to 'List Your GitHub Repositories', status_import_github_path, class: 'btn btn-success'
%hr
%p
- if github_import_configured?
Alternatively,
- else
To import a GitHub project,
you can use a
= succeed '.' do
= link_to 'Personal Access Token', 'https://github.com/settings/tokens'
When you create your Personal Access Token,
you will need to select the <code>repo</code> scope, so we can display a
list of your public and private repositories which are available for import.
= form_tag personal_access_token_import_github_path, method: :post, class: 'form-inline' do
.form-group
= text_field_tag :personal_access_token, '', class: 'form-control', placeholder: "Personal Access Token", size: 40
= submit_tag 'List Your GitHub Repositories', class: 'btn btn-success'
- unless github_import_configured?
%hr
%p
Note:
- if current_user.admin?
As an administrator you may like to configure
- else
Consider asking your GitLab administrator to configure
= link_to 'GitHub integration', help_page_path("integration", "github")
which will allow login via GitHub and allow importing projects without
generating a Personal Access Token.
%div#github_import_modal.modal
.modal-dialog
.modal-content
.modal-header
%a.close{href: "#", "data-dismiss" => "modal"} ×
%h3 Import projects from GitHub
.modal-body
To enable importing projects from GitHub,
- if current_user.admin?
as administrator you need to configure
- else
ask your Gitlab administrator to configure
== #{link_to 'OAuth integration', help_page_path("integration", "github")}.
......@@ -23,6 +23,7 @@
.input-group-addon
= root_url
= f.select :namespace_id, namespaces_options(params[:namespace_id] || :current_user, display_path: true), {}, {class: 'select2 js-select-namespace', tabindex: 1}
- else
.input-group-addon.static-namespace
#{root_url}#{current_user.username}/
......@@ -44,15 +45,8 @@
.col-sm-12.import-buttons
%div
- if github_import_enabled?
- if github_import_configured?
= link_to status_import_github_path, class: 'btn import_github' do
%i.fa.fa-github
GitHub
- else
= link_to '#', class: 'how_to_import_link btn import_github' do
%i.fa.fa-github
GitHub
= render 'github_import_modal'
= link_to new_import_github_path, class: 'btn import_github' do
= icon 'github', text: 'GitHub'
%div
- if bitbucket_import_enabled?
- if bitbucket_import_configured?
......
......@@ -10,7 +10,8 @@ paths_to_be_protected = [
"#{Rails.application.config.relative_url_root}/api/#{API::API.version}/session",
"#{Rails.application.config.relative_url_root}/users",
"#{Rails.application.config.relative_url_root}/users/confirmation",
"#{Rails.application.config.relative_url_root}/unsubscribes/"
"#{Rails.application.config.relative_url_root}/unsubscribes/",
"#{Rails.application.config.relative_url_root}/import/github/personal_access_token"
]
......
......@@ -139,6 +139,7 @@ Rails.application.routes.draw do
#
namespace :import do
resource :github, only: [:create, :new], controller: :github do
post :personal_access_token
get :status
get :callback
get :jobs
......
# Import your project from GitHub to GitLab
>**Note:**
In order to enable the GitHub import setting, you should first
enable the [GitHub integration][gh-import] in your GitLab instance.
In order to enable the GitHub import setting, you may also want to
enable the [GitHub integration][gh-import] in your GitLab instance. This
configuration is optional, you will be able import your GitHub repositories
with a Personal Access Token.
At its current state, GitHub importer can import:
......@@ -20,9 +22,15 @@ It is not yet possible to import your cross-repository pull requests (those from
forks). We are working on improving this in the near future.
The importer page is visible when you [create a new project][new-project].
Click on the **GitHub** link and you will be redirected to GitHub for
permission to access your projects. After accepting, you'll be automatically
redirected to the importer.
Click on the **GitHub** link and, if you are logged in via the GitHub
integration, you will be redirected to GitHub for permission to access your
projects. After accepting, you'll be automatically redirected to the importer.
If you are not using the GitHub integration, you can still perform a one-off
authorization with GitHub to access your projects.
Alternatively, you can also enter a GitHub Personal Access Token. Once you enter
your token, you'll be taken to the importer.
![New project page on GitLab](img/import_projects_from_github_new_project_page.png)
......
......@@ -21,7 +21,7 @@ Background:
Scenario: I should see instructions on how to import from GitHub
Given I see "New Project" page
When I click on "Import project from GitHub"
Then I see instructions on how to import from GitHub
Then I am redirected to the GitHub import page
@javascript
Scenario: I should see Google Code import page
......
......@@ -28,14 +28,8 @@ class Spinach::Features::NewProject < Spinach::FeatureSteps
first('.import_github').click
end
step 'I see instructions on how to import from GitHub' do
github_modal = first('.modal-body')
expect(github_modal).to be_visible
expect(github_modal).to have_content "To enable importing projects from GitHub"
page.all('.modal-body').each do |element|
expect(element).not_to be_visible unless element == github_modal
end
step 'I am redirected to the GitHub import page' do
expect(current_path).to eq new_import_github_path
end
step 'I click on "Repo by URL"' do
......
......@@ -4,26 +4,39 @@ module Gitlab
GITHUB_SAFE_REMAINING_REQUESTS = 100
GITHUB_SAFE_SLEEP_TIME = 500
attr_reader :client, :api
attr_reader :access_token
def initialize(access_token)
@client = ::OAuth2::Client.new(
config.app_id,
config.app_secret,
github_options.merge(ssl: { verify: config['verify_ssl'] })
)
@access_token = access_token
if access_token
::Octokit.auto_paginate = false
end
end
def api
@api ||= ::Octokit::Client.new(
access_token: access_token,
api_endpoint: github_options[:site],
# If there is no config, we're connecting to github.com and we
# should verify ssl.
connection_options: {
ssl: { verify: config ? config['verify_ssl'] : true }
}
)
end
@api = ::Octokit::Client.new(
access_token: access_token,
api_endpoint: github_options[:site],
connection_options: {
ssl: { verify: config['verify_ssl'] }
}
)
def client
unless config
raise Projects::ImportService::Error,
'OAuth configuration for GitHub missing.'
end
@client ||= ::OAuth2::Client.new(
config.app_id,
config.app_secret,
github_options.merge(ssl: { verify: config['verify_ssl'] })
)
end
def authorize_url(redirect_uri)
......@@ -56,7 +69,11 @@ module Gitlab
end
def github_options
config["args"]["client_options"].deep_symbolize_keys
if config
config["args"]["client_options"].deep_symbolize_keys
else
OmniAuth::Strategies::GitHub.default_options[:client_options].symbolize_keys
end
end
def rate_limit
......
......@@ -16,6 +16,24 @@ describe Import::GithubController do
allow(controller).to receive(:github_import_enabled?).and_return(true)
end
describe "GET new" do
it "redirects to GitHub for an access token if logged in with GitHub" do
allow(controller).to receive(:logged_in_with_github?).and_return(true)
expect(controller).to receive(:go_to_github_for_permissions)
get :new
end
it "redirects to status if we already have a token" do
assign_session_token
allow(controller).to receive(:logged_in_with_github?).and_return(false)
get :new
expect(controller).to redirect_to(status_import_github_url)
end
end
describe "GET callback" do
it "updates access token" do
token = "asdasd12345"
......@@ -32,6 +50,20 @@ describe Import::GithubController do
end
end
describe "POST personal_access_token" do
it "updates access token" do
token = "asdfasdf9876"
allow_any_instance_of(Gitlab::GithubImport::Client).
to receive(:user).and_return(true)
post :personal_access_token, personal_access_token: token
expect(session[:github_access_token]).to eq(token)
expect(controller).to redirect_to(status_import_github_url)
end
end
describe "GET status" do
before do
@repo = OpenStruct.new(login: 'vim', full_name: 'asd/vim')
......@@ -59,6 +91,17 @@ describe Import::GithubController do
expect(assigns(:already_added_projects)).to eq([@project])
expect(assigns(:repos)).to eq([])
end
it "handles an invalid access token" do
allow_any_instance_of(Gitlab::GithubImport::Client).
to receive(:repos).and_raise(Octokit::Unauthorized)
get :status
expect(session[:github_access_token]).to eq(nil)
expect(controller).to redirect_to(new_import_github_url)
expect(flash[:alert]).to eq('Access denied to your GitHub account.')
end
end
describe "POST create" do
......
......@@ -20,6 +20,20 @@ describe Gitlab::GithubImport::Client, lib: true do
expect { client.api }.not_to raise_error
end
context 'when config is missing' do
before do
allow(Gitlab.config.omniauth).to receive(:providers).and_return([])
end
it 'is still possible to get an Octokit client' do
expect { client.api }.not_to raise_error
end
it 'is not be possible to get an OAuth2 client' do
expect { client.client }.to raise_error(Projects::ImportService::Error)
end
end
context 'allow SSL verification to be configurable on API' do
before do
github_provider['verify_ssl'] = false
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment