Document MailRoom gem update deprecation

Gemfile

@@ -443,7 +443,8 @@ end
 
 gem 'octokit', '~> 4.15'
 
-# https://gitlab.com/gitlab-org/gitlab/issues/207207
+# Updating this gem version here is deprecated. See:
+# https://docs.gitlab.com/ee/development/emails.html#mailroom-gem-updates
 gem 'gitlab-mail_room', '~> 0.0.9', require: 'mail_room'
 
 gem 'email_reply_trimmer', '~> 0.1'

doc/administration/incoming_email.md

@@ -170,6 +170,15 @@ Reply by email should now be working.
    cd /home/git/gitlab
    ```
 
+1. Install the `gitlab-mail_room` gem manually:
+
+   ```shell
+   gem install gitlab-mail_room
+   ```
+
+   NOTE: This step is necessary to avoid thread deadlocks and to support the latest MailRoom features. See
+   [this explanation](../development/emails.md#mailroom-gem-updates) for more details.
+
 1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature
   and fill in the details for your specific IMAP server and email account (see [examples](#configuration-examples) below).
 

doc/development/emails.md

@@ -139,6 +139,44 @@ These are the only valid legacy formats for an email handler:
 
 In GitLab, the handler for the Service Desk feature is `path/to/project`.
 
+### MailRoom Gem updates
+
+We use [`gitlab-mail_room`](https://gitlab.com/gitlab-org/gitlab-mail_room), a
+fork of [`MailRoom`](https://github.com/tpitale/mail_room/), to ensure
+that we can make updates quickly to the gem if necessary. We try to upstream
+changes as soon as possible and keep the two projects in sync.
+
+Updating the `gitlab-mail_room` dependency in `Gemfile` is deprecated at
+the moment in favor of updating the version in
+[Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/software/mail_room.rb)
+(see [example merge request](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5816))
+and Helm Chart configuration (see [example merge request](https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/854)).
+
+#### Rationale
+
+This was done because to avoid [thread deadlocks](https://github.com/ruby/net-imap/issues/14), `MailRoom` needs
+an updated version of the `net-imap` gem. However, this [version of the net-imap cannot be installed by an unprivileged
+user](https://github.com/ruby/net-imap/issues/14) due to [an error installing the digest
+gem](https://github.com/ruby/digest/issues/14). [This bug in the Ruby interpreter](https://bugs.ruby-lang.org/issues/17761) was fixed in Ruby
+3.0.2.
+
+Updating the gem directly in the GitLab Rails `Gemfile` caused a [production incident](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/4053)
+since Kubernetes pods do not run as privileged users.
+
+Note that Omnibus GitLab runs `/opt/gitlab/embedded/bin/mail_room`
+instead of `bundle exec rake` to avoid loading the older version. With a
+Kubernetes install, the MailRoom version has always been explicitly set
+in the Helm Chart configuration rather than the `Gemfile`.
+
+#### Preserving backwards compatibility
+
+Removing the `Gemfile` would break incoming e-mail processing for source
+installs. For now, source installs are advised to upgrade manually to
+the version specified in Omnibus and run `bin/mail_room` directly as
+done with Omnibus.
+
+We can reconsider this deprecation once we upgrade to Ruby 3.0.
+
 ---
 
 [Return to Development documentation](index.md)

doc/update/plan_your_upgrade.md

@@ -60,6 +60,16 @@ to ensure the major components of GitLab are working:
 
 1. If using Elasticsearch, verify that searches are successful.
 
+1. If you are using [Reply by Email](../administration/reply_by_email.md) or [Service Desk](../user/project/service_desk.md),
+   manually install the latest version of `gitlab-mail_room`:
+
+   ```shell
+   gem install gitlab-mail_room
+   ```
+
+   NOTE: This step is necessary to avoid thread deadlocks and to support the latest MailRoom features. See
+   [this explanation](../development/emails.md#mailroom-gem-updates) for more details.
+
 If in any case something goes wrong, see [how to troubleshoot](#troubleshooting).
 
 ## Rollback plan