Merge branch 'mk/add-rate-limit-docs' into 'master'

Add rate limit docs

Closes #41308

See merge request gitlab-org/gitlab-ce!30829

doc/api/README.md

@@ -695,6 +695,13 @@ The correct encoding for the query parameter would be:
 There are many unofficial GitLab API Clients for most of the popular
 programming languages. Visit the [GitLab website] for a complete list.
 
+## Rate limits
+
+For administrator documentation on rate limit settings, check out
+[Rate limits](../security/rate_limits.md). To find the settings that are
+specifically used by GitLab.com, see
+[GitLab.com-specific rate limits](../user/gitlab_com/index.md).
+
 [GitLab website]: https://about.gitlab.com/applications/#api-clients "Clients using the GitLab API"
 [lib-api-url]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/api/api.rb
 [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749

doc/security/README.md

@@ -7,7 +7,7 @@ type: index
 
 - [Password length limits](password_length_limits.md)
 - [Restrict SSH key technologies and minimum length](ssh_keys_restrictions.md)
-- [Rack attack](rack_attack.md)
+- [Rate limits](rate_limits.md)
 - [Webhooks and insecure internal web services](webhooks.md)
 - [Information exclusivity](information_exclusivity.md)
 - [Reset your root password](reset_root_password.md)

doc/security/rack_attack.md

@@ -2,7 +2,9 @@
 type: reference, howto
 ---
 
-# Rack Attack
+# Rack Attack initializer
+
+## Overview
 
 [Rack Attack](https://github.com/kickstarter/rack-attack), also known as Rack::Attack, is a Ruby gem
 that is meant to protect GitLab with the ability to customize throttling and
@@ -14,19 +16,72 @@ If you find throttling is not enough to protect you against abusive clients,
 Rack Attack offers IP whitelisting, blacklisting, Fail2ban style filtering, and
 tracking.
 
-**Note:** Starting with 11.2, Rack Attack is disabled by default. To continue
-using Rack Attack, please enable it by [configuring `gitlab.rb` as described in Settings](#settings).
+For more information on how to use these options see the [Rack Attack README](https://github.com/kickstarter/rack-attack/blob/master/README.md).
+
+NOTE: **Note:** See
+[User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md)
+for simpler throttles that are configured in UI.
+
+NOTE: **Note:** Starting with 11.2, Rack Attack is disabled by default. If your
+instance is not exposed to the public internet, it is recommended that you leave
+Rack Attack disabled.
+
+## Behavior
+
+If set up as described in the [Settings](#settings) section below, two behaviors
+will be enabled:
+
+- Protected paths will be throttled
+- Failed authentications for Git and container registry requests will trigger a temporary IP ban
+
+### Protected paths throttle
+
+GitLab responds with HTTP status code 429 to POST requests at protected paths
+over 10 requests per minute per IP address.
+
+By default, protected paths are:
+
+```ruby
+default['gitlab']['gitlab-rails']['rack_attack_protected_paths'] = [
+  '/users/password',
+  '/users/sign_in',
+  '/api/#{API::API.version}/session.json',
+  '/api/#{API::API.version}/session',
+  '/users',
+  '/users/confirmation',
+  '/unsubscribes/',
+  '/import/github/personal_access_token'
+]
+```
+
+This header is included in responses to blocked requests:
+
+```
+Retry-After: 60
+```
+
+For example, the following are limited to a maximum 10 requests per minute:
+
+- user sign-in
+- user sign-up (if enabled)
+- user password reset
+
+After trying for 10 times, the client will
+have to wait a minute before to be able to try again.
+
+### Git and container registry failed authentication ban
+
+GitLab responds with HTTP status code 403 for 1 hour, if 30 failed
+authentication requests were received in a 3-minute period from a single IP address.
 
-By default, user sign-in, user sign-up (if enabled), and user password reset is
-limited to 6 requests per minute. After trying for 6 times, the client will
-have to wait for the next minute to be able to try again.
+This applies only to Git requests and container registry (`/jwt/auth`) requests
+(combined).
 
-If you installed or upgraded GitLab by following the [official guides](../install/README.md),
-Rack Attack should be disabled by default. If your instance is not exposed to any incoming
-connections, it is recommended that you leave Rack Attack disabled.
+This limit is reset by requests that authenticate successfully. For example, 29
+failed authentication requests followed by 1 successful request, followed by 29
+more failed authentication requests would not trigger a ban.
 
-For more information on how to use these options check out
-[rack-attack README](https://github.com/kickstarter/rack-attack/blob/master/README.md).
+No response headers are provided.
 
 ## Settings
 

doc/security/rate_limits.md

+---
+type: reference, howto
+---
+
+# Rate limits
+
+NOTE: **Note:**
+For GitLab.com, please see
+[GitLab.com-specific rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits).
+
+Rate limiting is a common technique used to improve the security and durability
+of a web application.
+
+For example, a simple script can make thousands of web requests per second.
+Whether malicious, apathetic, or just a bug, your application and infrastructure
+may not be able to cope with the load. For more details, see
+[Denial-of-service attack](https://en.wikipedia.org/wiki/Denial-of-service_attack).
+Most cases can be mitigated by limiting the rate of requests from a single IP address.
+
+Most [brute-force attacks](https://en.wikipedia.org/wiki/Brute-force_attack) are
+similarly mitigated by a rate limit.
+
+## Admin Area settings
+
+See
+[User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md).
+
+## Rack Attack initializer
+
+This method of rate limiting is cumbersome, but has some advantages. It allows
+throttling of specific paths, and is also integrated into Git and container
+registry requests. See [Rack Attack initializer](rack_attack.md).

doc/user/admin_area/settings/index.md

@@ -18,6 +18,7 @@ include:
 - [Third party offers](third_party_offers.md)
 - [Usage statistics](usage_statistics.md)
 - [Visibility and access controls](visibility_and_access_controls.md)
+- [User and IP rate limits](user_and_ip_rate_limits.md)
 - [Custom templates repository](instance_template_repository.md) **(PREMIUM)**
 
 NOTE: **Note:**

doc/user/admin_area/settings/user_and_ip_rate_limits.md

+---
+type: reference
+---
+
+# User and IP rate limits
+
+Rate limiting is a common technique used to improve the security and durability
+of a web application. For more details, see
+[Rate limits](../../../security/rate_limits.md).
+
+The following limits can be enforced in **Admin Area > Network > User and
+IP rate limits**:
+
+- Unauthenticated requests
+- Authenticated API requests
+- Authenticated web requests
+
+These limits are disabled by default.
+
+![user-and-ip-rate-limits](img/user_and_ip_rate_limits.png)
+
+<!-- ## Troubleshooting
+
+Include any troubleshooting steps that you can foresee. If you know beforehand what issues
+one might have when setting this up, or when something is changed, or on upgrading, it's
+important to describe those, too. Think of things that may go wrong and include them here.
+This is important to minimize requests for support, and to avoid doc comments with
+questions that you know someone might ask.
+
+Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+If you have none to add when creating a doc, leave this section in place
+but commented out to help encourage others to add to it in the future. -->

doc/user/gitlab_com/index.md

@@ -298,6 +298,79 @@ Web front-ends:
 - `memory_limit_min` = 1024MiB
 - `memory_limit_max` = 1280MiB
 
+## GitLab.com-specific rate limits
+
+NOTE: **Note:**
+See [Rate limits](../../security/rate_limits.md) for administrator
+documentation.
+
+IP blocks usually happen when GitLab.com receives unusual traffic from a single
+IP address that the system views as potentially malicious based on rate limit
+settings. After the unusual traffic ceases, the IP address will be automatically
+released depending on the type of block, as described below.
+
+If you receive a `403 Forbidden` error for all requests to GitLab.com, please
+check for any automated processes that may be triggering a block. For
+assistance, contact [GitLab Support](https://support.gitlab.com)
+with details, such as the affected IP address.
+
+### HAProxy API throttle
+
+GitLab.com responds with HTTP status code 429 to API requests over 10 requests
+per second per IP address.
+
+The following example headers are included for all API requests:
+
+```
+RateLimit-Limit: 600
+RateLimit-Observed: 6
+RateLimit-Remaining: 594
+RateLimit-Reset: 1563325137
+RateLimit-ResetTime: Wed, 17 Jul 2019 00:58:57 GMT
+```
+
+Source:
+
+- Search for `rate_limit_http_rate_per_minute` and `rate_limit_sessions_per_second` in [GitLab.com's current HAProxy settings](https://gitlab.com/gitlab-cookbooks/gitlab-haproxy/blob/master/attributes/default.rb).
+
+### Rack Attack initializer
+
+#### Protected paths throttle
+
+GitLab.com responds with HTTP status code 429 to POST requests at protected
+paths over 10 requests per **minute** per IP address.
+
+See the source below for which paths are protected. This includes user creation,
+user confirmation, user sign in, and password reset.
+
+This header is included in responses to blocked requests:
+
+```
+Retry-After: 60
+```
+
+Source:
+
+- Search for `rate_limit_requests_per_period`, `rate_limit_period`, and `rack_attack_protected_paths` in [GitLab.com's current Rails app settings](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb).
+
+#### Git and container registry failed authentication ban
+
+GitLab.com responds with HTTP status code 403 for 1 hour, if 30 failed
+authentication requests were received in a 3-minute period from a single IP address.
+
+This applies only to Git requests and container registry (`/jwt/auth`) requests
+(combined).
+
+This limit is reset by requests that authenticate successfully. For example, 29
+failed authentication requests followed by 1 successful request, followed by 29
+more failed authentication requests would not trigger a ban.
+
+No response headers are provided.
+
+### Admin Area settings
+
+GitLab.com does not currently use these settings.
+
 ## GitLab.com at scale
 
 In addition to the GitLab Enterprise Edition Omnibus install, GitLab.com uses