Commit 7586ed28 authored by Harish Ramachandran's avatar Harish Ramachandran Committed by Mike Jang

LDAP troubleshooting docs for admins

Flesh out some more

More about user logins and errors

More fleshing out

Reorganize outline on the page

Updated a few sections

Clarify ldapsearch and groupsync docs

Clarify phrasing

Update UserSync logs description

Clarify some headings and modify other stuff

Add some links; flesh out section on failed logins

Finish the User logins section

Finish the Connection section

New links; updated workflows

Link to the SSOT in the workflows

Simplify the membership debugging steps

Some final checks

Un-indent a few bulletted lists

Correct a few broken links

Resolve pending comments

Remove the Workflows section

Move the Tools section to the bottom

Rename the log output to be console outputs

Move user_filter debugging to the logins section

Move no-users-found section to user logins

Move most of the console docs inline

Update codeblock formatting; use the new Ldap module
parent 04852a58
......@@ -405,198 +405,4 @@ network and LDAP server response time will affect these metrics.
## Troubleshooting
### Referral error
If you see `LDAP search error: Referral` in the logs, or when troubleshooting
LDAP Group Sync, this error may indicate a configuration problem. The LDAP
configuration `/etc/gitlab/gitlab.rb` (Omnibus) or `config/gitlab.yml` (source)
is in YAML format and is sensitive to indentation. Check that `group_base` and
`admin_group` configuration keys are indented 2 spaces past the server
identifier. The default identifier is `main` and an example snippet looks like
the following:
```yaml
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'LDAP'
host: 'ldap.example.com'
...
group_base: 'cn=my_group,ou=groups,dc=example,dc=com'
admin_group: 'my_admin_group'
```
[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
[restart]: ../restart_gitlab.md#installations-from-source
[^1]: In Active Directory, a user is marked as disabled/blocked if the user
account control attribute (`userAccountControl:1.2.840.113556.1.4.803`)
has bit 2 set. See <https://ctovswild.com/2009/09/03/bitmask-searches-in-ldap/>
for more information.
### User DN has changed
When an LDAP user is created in GitLab, their LDAP DN is stored for later reference.
If GitLab cannot find a user by their DN, it will attempt to fallback
to finding the user by their email. If the lookup is successful, GitLab will
update the stored DN to the new value.
### User is not being added to a group
Sometimes you may think a particular user should be added to a GitLab group via
LDAP group sync, but for some reason it's not happening. There are several
things to check to debug the situation.
- Ensure LDAP configuration has a `group_base` specified. This configuration is
required for group sync to work properly.
- Ensure the correct LDAP group link is added to the GitLab group. Check group
links by visiting the GitLab group, then **Settings dropdown > LDAP groups**.
- Check that the user has an LDAP identity:
1. Sign in to GitLab as an administrator user.
1. Navigate to **Admin Area > Users**.
1. Search for the user
1. Open the user, by clicking on their name. Do not click 'Edit'.
1. Navigate to the **Identities** tab. There should be an LDAP identity with
an LDAP DN as the 'Identifier'.
If all of the above looks good, jump in to a little more advanced debugging.
Often, the best way to learn more about why group sync is behaving a certain
way is to enable debug logging. There is verbose output that details every
step of the sync.
1. Start a Rails console:
```shell
# For Omnibus installations
sudo gitlab-rails console
# For installations from source
sudo -u git -H bundle exec rails console -e production
```
1. Set the log level to debug (only for this session):
```ruby
Rails.logger.level = Logger::DEBUG
```
1. Choose a GitLab group to test with. This group should have an LDAP group link
already configured. If the output is `nil`, the group could not be found.
If a bunch of group attributes are output, your group was found successfully.
```ruby
group = Group.find_by(name: 'my_group')
# Output
=> #<Group:0x007fe825196558 id: 1234, name: "my_group"...>
```
1. Run a group sync for this particular group.
```ruby
EE::Gitlab::Auth::Ldap::Sync::Group.execute_all_providers(group)
```
1. Look through the output of the sync. See [example log output](#example-log-output)
below for more information about the output.
1. If you still aren't able to see why the user isn't being added, query the
LDAP group directly to see what members are listed. Still in the Rails console,
run the following query:
```ruby
adapter = Gitlab::Auth::Ldap::Adapter.new('ldapmain') # If `main` is the LDAP provider
ldap_group = EE::Gitlab::Auth::Ldap::Group.find_by_cn('group_cn_here', adapter)
# Output
=> #<EE::Gitlab::Auth::Ldap::Group:0x007fcbdd0bb6d8
```
1. Query the LDAP group's member DNs and see if the user's DN is in the list.
One of the DNs here should match the 'Identifier' from the LDAP identity
checked earlier. If it doesn't, the user does not appear to be in the LDAP
group.
```ruby
ldap_group.member_dns
# Output
=> ["uid=john,ou=people,dc=example,dc=com", "uid=mary,ou=people,dc=example,dc=com"]
```
1. Some LDAP servers don't store members by DN. Rather, they use UIDs instead.
If you didn't see results from the last query, try querying by UIDs instead.
```ruby
ldap_group.member_uids
# Output
=> ['john','mary']
```
#### Example log output
The output of the last command will be very verbose, but contains lots of
helpful information. For the most part you can ignore log entries that are SQL
statements.
Indicates the point where syncing actually begins:
```shell
Started syncing all providers for 'my_group' group
```
The follow entry shows an array of all user DNs GitLab sees in the LDAP server.
Note that these are the users for a single LDAP group, not a GitLab group. If
you have multiple LDAP groups linked to this GitLab group, you will see multiple
log entries like this - one for each LDAP group. If you don't see an LDAP user
DN in this log entry, LDAP is not returning the user when we do the lookup.
Verify the user is actually in the LDAP group.
```shell
Members in 'ldap_group_1' LDAP group: ["uid=john0,ou=people,dc=example,dc=com",
"uid=mary0,ou=people,dc=example,dc=com", "uid=john1,ou=people,dc=example,dc=com",
"uid=mary1,ou=people,dc=example,dc=com", "uid=john2,ou=people,dc=example,dc=com",
"uid=mary2,ou=people,dc=example,dc=com", "uid=john3,ou=people,dc=example,dc=com",
"uid=mary3,ou=people,dc=example,dc=com", "uid=john4,ou=people,dc=example,dc=com",
"uid=mary4,ou=people,dc=example,dc=com"]
```
Shortly after each of the above entries, you will see a hash of resolved member
access levels. This hash represents all user DNs GitLab thinks should have
access to this group, and at which access level (role). This hash is additive,
and more DNs may be added, or existing entries modified, based on additional
LDAP group lookups. The very last occurrence of this entry should indicate
exactly which users GitLab believes should be added to the group.
NOTE: **Note:**
10 is 'Guest', 20 is 'Reporter', 30 is 'Developer', 40 is 'Maintainer'
and 50 is 'Owner'.
```shell
Resolved 'my_group' group member access: {"uid=john0,ou=people,dc=example,dc=com"=>30,
"uid=mary0,ou=people,dc=example,dc=com"=>30, "uid=john1,ou=people,dc=example,dc=com"=>30,
"uid=mary1,ou=people,dc=example,dc=com"=>30, "uid=john2,ou=people,dc=example,dc=com"=>30,
"uid=mary2,ou=people,dc=example,dc=com"=>30, "uid=john3,ou=people,dc=example,dc=com"=>30,
"uid=mary3,ou=people,dc=example,dc=com"=>30, "uid=john4,ou=people,dc=example,dc=com"=>30,
"uid=mary4,ou=people,dc=example,dc=com"=>30}
```
It's not uncommon to see warnings like the following. These indicate that GitLab
would have added the user to a group, but the user could not be found in GitLab.
Usually this is not a cause for concern.
If you think a particular user should already exist in GitLab, but you're seeing
this entry, it could be due to a mismatched DN stored in GitLab. See
[User DN has changed](#User-DN-has-changed) to update the user's LDAP identity.
```shell
User with DN `uid=john0,ou=people,dc=example,dc=com` should have access
to 'my_group' group but there is no user in GitLab with that
identity. Membership will be updated once the user signs in for
the first time.
```
Finally, the following entry says syncing has finished for this group:
```shell
Finished syncing all providers for 'my_group' group
```
Please see our [administrator guide to troubleshooting LDAP](ldap-troubleshooting.md).
This diff is collapsed.
......@@ -552,74 +552,4 @@ be mandatory and clients cannot be authenticated with the TLS protocol.
## Troubleshooting
If a user account is blocked or unblocked due to the LDAP configuration, a
message will be logged to `application.log`.
If there is an unexpected error during an LDAP lookup (configuration error,
timeout), the login is rejected and a message will be logged to
`production.log`.
### Debug LDAP user filter with ldapsearch
This example uses `ldapsearch` and assumes you are using ActiveDirectory. The
following query returns the login names of the users that will be allowed to
log in to GitLab if you configure your own user_filter.
```shell
ldapsearch -H ldaps://$host:$port -D "$bind_dn" -y bind_dn_password.txt -b "$base" "$user_filter" sAMAccountName
```
- Variables beginning with a `$` refer to a variable from the LDAP section of
your configuration file.
- Replace `ldaps://` with `ldap://` if you are using the plain authentication method.
Port `389` is the default `ldap://` port and `636` is the default `ldaps://`
port.
- We are assuming the password for the bind_dn user is in bind_dn_password.txt.
### Invalid credentials when logging in
- Make sure the user you are binding with has enough permissions to read the user's
tree and traverse it.
- Check that the `user_filter` is not blocking otherwise valid users.
- Run the following check command to make sure that the LDAP settings are
correct and GitLab can see your users:
```shell
# For Omnibus installations
sudo gitlab-rake gitlab:ldap:check
# For installations from source
sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production
```
### Connection refused
If you are getting 'Connection Refused' errors when trying to connect to the
LDAP server please double-check the LDAP `port` and `encryption` settings used by
GitLab. Common combinations are `encryption: 'plain'` and `port: 389`, OR
`encryption: 'simple_tls'` and `port: 636`.
### Connection times out
If GitLab cannot reach your LDAP endpoint, you will see a message like this:
```plaintext
Could not authenticate you from Ldapmain because "Connection timed out - user specified timeout".
```
If your configured LDAP provider and/or endpoint is offline or otherwise unreachable by GitLab, no LDAP user will be able to authenticate and log in. GitLab does not cache or store credentials for LDAP users to provide authentication during an LDAP outage.
Contact your LDAP provider or administrator if you are seeing this error.
### No file specified as Settingslogic source
If `sudo gitlab-ctl reconfigure` fails with the following error, or you are seeing it in
the logs, you may have malformed YAML in `/etc/gitlab/gitlab.rb`:
```plaintext
Errno::ENOENT: No such file or directory - No file specified as Settingslogic source
```
This issue is frequently due to the spacing in your YAML file. To fix the problem,
verify the syntax with **spacing** against the
[documentation for the configuration of LDAP](#configuration).
Please see our [administrator guide to troubleshooting LDAP](ldap-troubleshooting.md).
......@@ -28,7 +28,7 @@ rake gitlab:ldap:check[50]
## Run a Group Sync
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/14735) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.3.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/14735) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.2.
The following task will run a [group sync](../auth/ldap-ee.md#group-sync) immediately. This is valuable
when you'd like to update all configured group memberships against LDAP without
......
......@@ -598,98 +598,6 @@ group = Group.find_by_path_or_name('group-name')
group.project_creation_level=0
```
## LDAP
### LDAP commands in the rails console
TIP: **TIP:**
Use the rails runner to avoid entering the rails console in the first place.
This is great when only a single command (such as a UserSync or GroupSync)
is needed.
```ruby
# Get debug output
Rails.logger.level = Logger::DEBUG
# Run a UserSync (normally performed once a day)
LdapSyncWorker.new.perform
# Run a GroupSync for all groups (9.3-)
LdapGroupSyncWorker.new.perform
# Run a GroupSync for all groups (9.3+)
LdapAllGroupsSyncWorker.new.perform
# Run a GroupSync for a single group (10.6-)
group = Group.find_by(name: 'my_gitlab_group')
EE::Gitlab::LDAP::Sync::Group.execute_all_providers(group)
# Run a GroupSync for a single group (10.6+)
group = Group.find_by(name: 'my_gitlab_group')
EE::Gitlab::Auth::Ldap::Sync::Group.execute_all_providers(group)
# Query an LDAP group directly (10.6-)
adapter = Gitlab::LDAP::Adapter.new('ldapmain') # If `main` is the LDAP provider
ldap_group = EE::Gitlab::LDAP::Group.find_by_cn('group_cn_here', adapter)
ldap_group.member_dns
ldap_group.member_uids
# Query an LDAP group directly (10.6+)
adapter = Gitlab::Auth::Ldap::Adapter.new('ldapmain') # If `main` is the LDAP provider
ldap_group = EE::Gitlab::Auth::Ldap::Group.find_by_cn('group_cn_here', adapter)
ldap_group.member_dns
ldap_group.member_uids
# Lookup a particular user (10.6+)
# This could expose potential errors connecting to and/or querying LDAP that may seem to
# fail silently in the GitLab UI
adapter = Gitlab::Auth::Ldap::Adapter.new('ldapmain') # If `main` is the LDAP provider
user = Gitlab::Auth::Ldap::Person.find_by_uid('<username>',adapter)
# Query the LDAP server directly (10.6+)
## For an example, see https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/ee/gitlab/auth/ldap/adapter.rb
adapter = Gitlab::Auth::Ldap::Adapter.new('ldapmain')
options = {
# the :base is required
# use adapter.config.base for the base or .group_base for the group_base
base: adapter.config.group_base,
# :filter is optional
# 'cn' looks for all "cn"s under :base
# '*' is the search string - here, it's a wildcard
filter: Net::LDAP::Filter.eq('cn', '*'),
# :attributes is optional
# the attributes we want to get returned
attributes: %w(dn cn memberuid member submember uniquemember memberof)
}
adapter.ldap_search(options)
```
### Update user accounts when the `dn` and email change
The following will require that any accounts with the new email address are removed.
Emails have to be unique in GitLab. This is expected to work but unverified as of yet:
```ruby
# Here's an example with a couple users.
# Each entry will have to include the old username and the new email
emails = {
'ORIGINAL_USERNAME' => 'NEW_EMAIL_ADDRESS',
...
}
emails.each do |username, email|
user = User.find_by_username(username)
user.email = email
user.skip_reconfirmation!
user.save!
end
# Run the UserSync to update the above users' data
LdapSyncWorker.new.perform
```
## Routes
### Remove redirecting routes
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment