info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Multiple Kubernetes clusters for Auto DevOps **(FREE)**
When using Auto DevOps, you can deploy different environments to
different Kubernetes clusters, due to the 1:1 connection
[existing between them](../../user/project/clusters/multiple_kubernetes_clusters.md).
The [Deploy Job template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml)
used by Auto DevOps defines 3 environment names:
-`review/` (every environment starting with `review/`)
-`staging`
-`production`
Those environments are tied to jobs using [Auto Deploy](stages.md#auto-deploy), so
except for the environment scope, they must have a different deployment domain.
You must define a separate `KUBE_INGRESS_BASE_DOMAIN` variable for each of the above
[based on the environment](../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable).
The following table is an example of how to configure the three different clusters:
| Cluster name | Cluster environment scope | `KUBE_INGRESS_BASE_DOMAIN` variable value | Variable environment scope | Notes |
| review | `review/*` | `review.example.com` | `review/*` | The review cluster which runs all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, used by every environment name starting with `review/`. |
| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which runs the deployments of the staging environments. You must [enable it first](customize.md#deploy-policy-for-staging-and-production-environments). |
| production | `production` | `example.com` | `production` | The production cluster which runs the production environment deployments. You can use [incremental rollouts](customize.md#incremental-rollout-to-production). |
To add a different cluster for each environment:
1. Navigate to your project's **Infrastructure > Kubernetes clusters**.
1. Create the Kubernetes clusters with their respective environment scope, as
described from the table above.
1. After creating the clusters, navigate to each cluster and [install
Ingress](quick_start_guide.md#install-ingress). Wait for the Ingress IP address to be assigned.
1. Make sure you've [configured your DNS](requirements.md#auto-devops-base-domain) with the
specified Auto DevOps domains.
1. Navigate to each cluster's page, through **Infrastructure > Kubernetes clusters**,
and add the domain based on its Ingress IP address.
After completing configuration, test your setup by creating a merge request.
Verify whether your application deployed as a Review App in the Kubernetes
cluster with the `review/*` environment scope. Similarly, check the
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Prepare Auto DevOps for deployment **(FREE)**
If you enable Auto DevOps without setting the base domain and deployment
strategy, GitLab can't deploy your application directly. Therefore, we
recommend that you prepare them before enabling Auto DevOps.
## Deployment strategy
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38542) in GitLab 11.0.
When using Auto DevOps to deploy your applications, choose the
| **Continuous deployment to production** | Enables [Auto Deploy](stages.md#auto-deploy) with the default branch continuously deployed to production. | Continuous deployment to production.|
| **Continuous deployment to production using timed incremental rollout** | Sets the [`INCREMENTAL_ROLLOUT_MODE`](customize.md#timed-incremental-rollout-to-production) variable to `timed`. | Continuously deploy to production with a 5 minutes delay between rollouts. |
| **Automatic deployment to staging, manual deployment to production** | Sets [`STAGING_ENABLED`](customize.md#deploy-policy-for-staging-and-production-environments) to `1` and [`INCREMENTAL_ROLLOUT_MODE`](customize.md#incremental-rollout-to-production) to `manual`. | The default branch is continuously deployed to staging and continuously delivered to production. |
You can choose the deployment method when enabling Auto DevOps or later:
1. In GitLab, go to your project's **Settings > CI/CD > Auto DevOps**.
1. Choose the deployment strategy.
1. Select **Save changes**.
NOTE:
Use the [blue-green deployment](../../ci/environments/incremental_rollouts.md#blue-green-deployment) technique
to minimize downtime and risk.
## Auto DevOps base domain
The Auto DevOps base domain is required to use
[Auto Review Apps](stages.md#auto-review-apps), [Auto Deploy](stages.md#auto-deploy), and
[Auto Monitoring](stages.md#auto-monitoring).
To define the base domain, either:
- In the project, group, or instance level: go to your cluster settings and add it there.
- In the project or group level: add it as an environment variable: `KUBE_INGRESS_BASE_DOMAIN`.
- In the instance level: go to **Menu >****{admin}****Admin > Settings > CI/CD> Continuous Integration and Delivery** and add it there.
The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence
as other environment [variables](../../ci/variables/index.md#cicd-variable-precedence).
If you don't specify the base domain in your projects and groups, Auto DevOps uses the instance-wide **Auto DevOps domain**.
Auto DevOps requires a wildcard DNS A record matching the base domain(s). For
a base domain of `example.com`, you'd need a DNS entry like:
```plaintext
*.example.com 3600 A 1.2.3.4
```
In this case, the deployed applications are served from `example.com`, and `1.2.3.4`
is the IP address of your load balancer, generally NGINX ([see requirements](requirements.md)).
Setting up the DNS record is beyond the scope of this document; check with your
DNS provider for information.
Alternatively, you can use free public services like [nip.io](https://nip.io)
which provide automatic wildcard DNS without any configuration. For [nip.io](https://nip.io),
set the Auto DevOps base domain to `1.2.3.4.nip.io`.
After completing setup, all requests hit the load balancer, which routes requests
| **Continuous deployment to production** | Enables [Auto Deploy](stages.md#auto-deploy) with the default branch continuously deployed to production. | Continuous deployment to production.|
| **Continuous deployment to production using timed incremental rollout** | Sets the [`INCREMENTAL_ROLLOUT_MODE`](customize.md#timed-incremental-rollout-to-production) variable to `timed`. | Continuously deploy to production with a 5 minutes delay between rollouts. |
| **Automatic deployment to staging, manual deployment to production** | Sets [`STAGING_ENABLED`](customize.md#deploy-policy-for-staging-and-production-environments) to `1` and [`INCREMENTAL_ROLLOUT_MODE`](customize.md#incremental-rollout-to-production) to `manual`. | The default branch is continuously deployed to staging and continuously delivered to production. |
You can choose the deployment method when enabling Auto DevOps or later:
1. In GitLab, go to your project's **Settings > CI/CD > Auto DevOps**.
1. Choose the deployment strategy.
1. Select **Save changes**.
NOTE:
Use the [blue-green deployment](../../ci/environments/incremental_rollouts.md#blue-green-deployment) technique
to minimize downtime and risk.
## Auto DevOps base domain
The Auto DevOps base domain is required to use
[Auto Review Apps](stages.md#auto-review-apps), [Auto Deploy](stages.md#auto-deploy), and
[Auto Monitoring](stages.md#auto-monitoring).
To define the base domain, either:
- In the project, group, or instance level: go to your cluster settings and add it there.
- In the project or group level: add it as an environment variable: `KUBE_INGRESS_BASE_DOMAIN`.
- In the instance level: go to **Menu >****{admin}****Admin > Settings > CI/CD> Continuous Integration and Delivery** and add it there.
The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence
as other environment [variables](../../ci/variables/index.md#cicd-variable-precedence).
If you don't specify the base domain in your projects and groups, Auto DevOps uses the instance-wide **Auto DevOps domain**.
Auto DevOps requires a wildcard DNS A record matching the base domain(s). For
a base domain of `example.com`, you'd need a DNS entry like:
```plaintext
*.example.com 3600 A 1.2.3.4
```
In this case, the deployed applications are served from `example.com`, and `1.2.3.4`
is the IP address of your load balancer, generally NGINX ([see requirements](requirements.md)).
Setting up the DNS record is beyond the scope of this document; check with your
DNS provider for information.
Alternatively, you can use free public services like [nip.io](https://nip.io)
which provide automatic wildcard DNS without any configuration. For [nip.io](https://nip.io),
set the Auto DevOps base domain to `1.2.3.4.nip.io`.
After completing setup, all requests hit the load balancer, which routes requests
to the Kubernetes pods running your application.
## Auto DevOps requirements for Kubernetes
...
...
@@ -49,7 +123,7 @@ To make full use of Auto DevOps with Kubernetes, you need:
