Make usage of SSH keys in CI/CD more clear

doc/ci/ssh_keys/README.md

-# Using SSH keys
+---
+last_updated: 2017-12-13
+---
+
+# Using SSH keys with GitLab CI/CD
 
 GitLab currently doesn't have built-in support for managing SSH keys in a build
-environment.
+environment (where the GitLab Runner runs).
 
 The SSH keys can be useful when:
 
 1. You want to checkout internal submodules
-2. You want to download private packages using your package manager (eg. bundler)
-3. You want to deploy your application to eg. Heroku or your own server
-4. You want to execute SSH commands from the build server to the remote server
-5. You want to rsync files from your build server to the remote server
+1. You want to download private packages using your package manager (e.g., Bundler)
+1. You want to deploy your application to your own server, or, for example, Heroku
+1. You want to execute SSH commands from the build environment to a remote server
+1. You want to rsync files from the build environment to a remote server
 
 If anything of the above rings a bell, then you most likely need an SSH key.
 
-## Inject keys in your build server
-
 The most widely supported method is to inject an SSH key into your build
-environment by extending your `.gitlab-ci.yml`.
-
-This is the universal solution which works with any type of executor
-(docker, shell, etc.).
-
-### How it works
-
-1. Create a new SSH key pair with [ssh-keygen][]
-2. Add the private key as a **Secret Variable** to the project
-3. Run the [ssh-agent][] during job to load the private key.
+environment by extending your `.gitlab-ci.yml`, and it's a solution which works
+with any type of [executor](https://docs.gitlab.com/runner/executors/)
+(Docker, shell, etc.).
+
+## How it works
+
+1. Create a new SSH key pair locally with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen)
+1. Add the private key as a [secret variable](../variables/README.md) to
+   your project
+1. Run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during job to load
+   the private key.
+1. Copy the public key to the servers you want to have access to (usually in
+   `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys)
+   if you are accessing a private GitLab repository.
+
+NOTE: **Note:**
+The private key will not be displayed in the job trace, unless you enable
+[debug tracing](../variables/README.md#debug-tracing). You might also want to
+check the [visibility of your pipelines](../../user/project/pipelines/settings.md#visibility-of-pipelines).
 
 ## SSH keys when using the Docker executor
 
-You will first need to create an SSH key pair. For more information, follow the
-instructions to [generate an SSH key](../../ssh/README.md). Do not add a
-passphrase to the SSH key, or the `before_script` will prompt for it.
-
-Then, create a new **Secret Variable** in your project settings on GitLab
-following **Settings > CI/CD** and look for the "Secret Variables" section.
-As **Key** add the name `SSH_PRIVATE_KEY` and in the **Value** field paste the 
-content of your _private_ key that you created earlier.
-
-It is also good practice to check the server's own public key to make sure you
-are not being targeted by a man-in-the-middle attack. To do this, add another
-variable named `SSH_SERVER_HOSTKEYS`. To find out the hostkeys of your server, run
-the `ssh-keyscan YOUR_SERVER` command from a trusted network (ideally, from the
-server itself), and paste its output into the `SSH_SERVER_HOSTKEYS` variable. If
-you need to connect to multiple servers, concatenate all the server public keys
-that you collected into the **Value** of the variable. There must be one key per
-line.
-
-Next you need to modify your `.gitlab-ci.yml` with a `before_script` action.
-Add it to the top:
-
-```
-before_script:
-  # Install ssh-agent if not already installed, it is required by Docker.
-  # (change apt-get to yum if you use a CentOS-based image)
-  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
-
-  # Run ssh-agent (inside the build environment)
-  - eval $(ssh-agent -s)
-
-  # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
-  - ssh-add <(echo "$SSH_PRIVATE_KEY")
-
-  # For Docker builds disable host key checking. Be aware that by adding that
-  # you are suspectible to man-in-the-middle attacks.
-  # WARNING: Use this only with the Docker executor, if you use it with shell
-  # you will overwrite your user's SSH config.
-  - mkdir -p ~/.ssh
-  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
-  # In order to properly check the server's host key, assuming you created the
-  # SSH_SERVER_HOSTKEYS variable previously, uncomment the following two lines
-  # instead.
-  # - mkdir -p ~/.ssh
-  # - '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'
-```
-
-As a final step, add the _public_ key from the one you created earlier to the
-services that you want to have an access to from within the build environment.
-If you are accessing a private GitLab repository you need to add it as a
-[deploy key](../../ssh/README.md#deploy-keys).
+When your CI/CD jobs run inside Docker containers (meaning the environment is
+contained) and you want to deploy your code in a private server, you need a way
+to access it. This is where an SSH key pair comes in handy.
+
+1. You will first need to create an SSH key pair. For more information, follow
+   the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).
+   **Do not** add a passphrase to the SSH key, or the `before_script` will\
+   prompt for it.
+
+1. Create a new [secret variable](../variables/README.md#secret-variables).
+   As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste
+   the content of your _private_ key that you created earlier.
+
+1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following
+   example, a Debian based image is assumed. Edit to your needs:
+
+    ```yaml
+    before_script:
+      ##
+      ## Install ssh-agent if not already installed, it is required by Docker.
+      ## (change apt-get to yum if you use an RPM-based image)
+      ##
+      - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
+
+      ##
+      ## Run ssh-agent (inside the build environment)
+      ##
+      - eval $(ssh-agent -s)
+
+      ##
+      ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
+      ## We're using tr to fix line endings which makes ed25519 keys work
+      ## without extra base64 encoding.
+      ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
+      ##
+      - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
+
+      ##
+      ## Create the SSH directory and give it the right permissions
+      ##
+      - mkdir -p ~/.ssh
+      - chmod 700 ~/.ssh
+
+      ##
+      ## Optionally, if you will be using any Git commands, set the user name and
+      ## and email.
+      ##
+      #- git config --global user.email "user@example.com"
+      #- git config --global user.name "User name"
+    ```
+
+    NOTE: **Note:**
+    The [`before_script`](../yaml/README.md#before-script) can be set globally
+    or per-job.
+
+1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys).
+
+1. As a final step, add the _public_ key from the one you created in the first
+   step to the services that you want to have an access to from within the build
+   environment. If you are accessing a private GitLab repository you need to add
+   it as a [deploy key](../../ssh/README.md#deploy-keys).
 
 That's it! You can now have access to private servers or repositories in your
 build environment.
@@ -91,24 +113,93 @@ SSH key.
 You can generate the SSH key from the machine that GitLab Runner is installed
 on, and use that key for all projects that are run on this machine.
 
-First, you need to login to the server that runs your jobs.
+1. First, you need to login to the server that runs your jobs.
+
+1. Then from the terminal login as the `gitlab-runner` user:
 
-Then from the terminal login as the `gitlab-runner` user and generate the SSH
-key pair as described in the [SSH keys documentation](../../ssh/README.md).
+    ```
+    sudo su - gitlab-runner
+    ```
 
-As a final step, add the _public_ key from the one you created earlier to the
-services that you want to have an access to from within the build environment.
-If you are accessing a private GitLab repository you need to add it as a
-[deploy key](../../ssh/README.md#deploy-keys).
+1. Generate the SSH key pair as described in the instructions to
+   [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair).
+   **Do not** add a passphrase to the SSH key, or the `before_script` will
+   prompt for it.
+
+1. As a final step, add the _public_ key from the one you created earlier to the
+   services that you want to have an access to from within the build environment.
+   If you are accessing a private GitLab repository you need to add it as a
+   [deploy key](../../ssh/README.md#deploy-keys).
 
 Once done, try to login to the remote server in order to accept the fingerprint:
 
 ```bash
-ssh <address-of-my-server>
+ssh example.com
+```
+
+For accessing repositories on GitLab.com, you would use `git@gitlab.com`.
+
+## Verifying the SSH host keys
+
+It is a good practice to check the private server's own public key to make sure
+you are not being targeted by a man-in-the-middle attack. In case anything
+suspicious happens, you will notice it since the job would fail (the SSH
+connection would fail if the public keys would not match).
+
+To find out the host keys of your server, run the `ssh-keyscan` command from a
+trusted network (ideally, from the private server itself):
+
+```sh
+## Use the domain name
+ssh-keyscan example.com
+
+## Or use an IP
+ssh-keyscan 1.2.3.4
 ```
 
-For accessing repositories on GitLab.com, the `<address-of-my-server>` would be
-`git@gitlab.com`.
+Create a new [secret variable](../variables/README.md#secret-variables) with
+`SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`.
+
+NOTE: **Note:**
+If you need to connect to multiple servers, all the server host keys
+need to be collected in the **Value** of the variable, one key per line.
+
+TIP: **Tip:**
+By using a secret variable instead of `ssh-keyscan` directly inside
+`.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml`
+if the host domain name changes for some reason. Also, the values are predefined
+by you, meaning that if the host keys suddenly change, the CI/CD job will fail,
+and you'll know there's something wrong with the server or the network.
+
+Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the
+[content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor)
+above, here's what more you need to add:
+
+ ```yaml
+before_script:
+  ##
+  ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the
+  ## following two lines.
+  ##
+  - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts'
+  - chmod 644 ~/.ssh/known_hosts
+
+  ##
+  ## Alternatively, use ssh-keyscan to scan the keys of your private server.
+  ## Replace example.com with your private server's domain name. Repeat that
+  ## command if you have more than one server to connect to.
+  ##
+  #- ssh-keyscan example.com >> ~/.ssh/known_hosts
+  #- chmod 644 ~/.ssh/known_hosts
+
+  ##
+  ## You can optionally disable host key checking. Be aware that by adding that
+  ## you are suspectible to man-in-the-middle attacks.
+  ## WARNING: Use this only with the Docker executor, if you use it with shell
+  ## you will overwrite your user's SSH config.
+  ##
+  #- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
+```
 
 ## Example project
 
@@ -119,6 +210,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available
 Want to hack on it? Simply fork it, commit and push your changes. Within a few
 moments the changes will be picked by a public runner and the job will begin.
 
-[ssh-keygen]: http://linux.die.net/man/1/ssh-keygen
-[ssh-agent]: http://linux.die.net/man/1/ssh-agent
 [ssh-example-repo]: https://gitlab.com/gitlab-examples/ssh-private-key/

doc/ci/variables/README.md

@@ -213,14 +213,15 @@ An example project service that defines deployment variables is
 ## Debug tracing
 
 > Introduced in GitLab Runner 1.7.
->
-> **WARNING:** Enabling debug tracing can have severe security implications. The
-  output **will** contain the content of all your secret variables and any other
-  secrets! The output **will** be uploaded to the GitLab server and made visible
-  in job traces!
+
+CAUTION: **Warning:**
+Enabling debug tracing can have severe security implications. The
+output **will** contain the content of all your secret variables and any other
+secrets! The output **will** be uploaded to the GitLab server and made visible
+in job traces!
 
 By default, GitLab Runner hides most of the details of what it is doing when
-processing a job. This behaviour keeps job traces short, and prevents secrets
+processing a job. This behavior keeps job traces short, and prevents secrets
 from being leaked into the trace unless your script writes them to the screen.
 
 If a job isn't working as expected, this can make the problem difficult to

doc/user/project/pipelines/settings.md

@@ -68,7 +68,7 @@ in the pipelines settings page.
 
 Access to pipelines and job details (including output of logs and artifacts)
 is checked against your current user access level and the **Public pipelines**
-project setting.
+project setting under your project's **Settings > CI/CD > General pipelines settings**.
 
 If **Public pipelines** is enabled (default):