Index trial groups in Elasticsearch

As part of https://gitlab.com/gitlab-org/gitlab/-/issues/220386 we want
to index trial groups in Elasticsearch so they can make use of Advanced
search.

This change also refactored the existing logic around indexing paid
groups a bit:

1. Removing the feature flag that has been on for a while now along
with the documentation explaining this feature flag. We already have
separate docs explaining that Advanced Global Search is enabled for
bronze and above groups so this is not necessary.
2. This will now also exclude `expired?` groups which we shouldn't have
been indexing to begin with. The assumption is that the `expired?` check
will also ensure we skip expired trials. These expired trials also have
a `trial_ends_on` date but based on looking at some data I think we can
expect this will be the same or less than `end_date`. The expired trials
that are in the index will be cleaned up in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38551
3. Don't check `saved_changes.key?('hosted_plan_id')` anymore since we
don't really care why the plan is changing. As long as we use
`safe_find_or_create_by!` we'll be fine to run this multiple times.

doc/development/integrations/elasticsearch_for_paid_tiers_on_gitlabcom.md

-# Elasticsearch for paid tiers on GitLab.com
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220246) in GitLab 13.2
-> - It's deployed behind a feature flag, disabled by default.
-> - It's disabled on GitLab.com.
-> - It's not recommended for use in GitLab self-managed instances.
-
-This document describes how to enable Elasticsearch with GitLab for all paid tiers on GitLab.com. Once enabled,
-all paid tiers will have access to the [Advanced Global Search feature](../../integration/elasticsearch.md) on GitLab.com.
-
-## Enable or disable Elasticsearch for all paid tiers on GitLab.com
-
-Since we're still in the process of rolling this out and want to control the timing this is behind a feature flag
-which defaults to off.
-
-To enable it:
-
-```ruby
-# Instance-wide
-Feature.enable(:elasticsearch_index_only_paid_groups)
-```
-
-To disable it:
-
-```ruby
-# Instance-wide
-Feature.disable(:elasticsearch_index_only_paid_groups)
-```

ee/app/models/gitlab_subscription.rb

@@ -36,8 +36,8 @@ class GitlabSubscription < ApplicationRecord
     [0, max_seats_used - seats].max
   end
 
-  def has_a_paid_hosted_plan?
-    !trial? &&
+  def has_a_paid_hosted_plan?(include_trials: false)
+    (include_trials || !trial?) &&
       hosted? &&
       seats > 0 &&
       Plan::PAID_HOSTED_PLANS.include?(plan_name)
@@ -88,13 +88,18 @@ class GitlabSubscription < ApplicationRecord
     namespace_id.present?
   end
 
+  def automatically_index_in_elasticsearch?
+    return false unless ::Gitlab.dev_env_or_com?
+    return false if expired?
+
+    has_a_paid_hosted_plan?(include_trials: true)
+  end
+
   # Kick off Elasticsearch indexing for paid groups with new or upgraded paid, hosted subscriptions
   # Uses safe_find_or_create_by to avoid ActiveRecord::RecordNotUnique exception when upgrading from
   # one paid plan to another paid plan
   def index_namespace
-    return unless ::Feature.enabled?(:elasticsearch_index_only_paid_groups) &&
-        has_a_paid_hosted_plan? &&
-        saved_changes.key?('hosted_plan_id')
+    return unless automatically_index_in_elasticsearch?
 
     ElasticsearchIndexedNamespace.safe_find_or_create_by!(namespace_id: namespace_id)
   end

ee/changelogs/unreleased/220386-index-trial-groups.yml

+---
+title: Index trial groups in Elasticsearch
+merge_request: 38541
+author:
+type: added

ee/spec/models/gitlab_subscription_spec.rb

@@ -254,40 +254,72 @@ RSpec.describe GitlabSubscription do
 
   describe 'callbacks' do
     context 'after_commit :index_namespace' do
-      where(:elasticsearch_index_only_paid_groups_setting, :operation, :initial_plan, :should_update_plan?, :should_index_namespace?) do
-        false   | :create   | :bronze         | false   | false
-        true    | :create   | :bronze         | false   | true
-        true    | :create   | :free           | false   | false
-        true    | :create   | { trial: true } | false   | false
-        false   | :update   | :bronze         | false   | false
-        true    | :update   | :bronze         | false   | false
-        true    | :update   | :bronze         | true    | true
-        true    | :update   | :free           | true    | true
-        true    | :update   | { trial: true } | true    | true
+      let(:gitlab_subscription) { build(:gitlab_subscription, plan) }
+      let(:dev_env_or_com) { true }
+      let(:expiration_date) { Date.today + 10 }
+      let(:plan) { :bronze }
+
+      before do
+        allow(::Gitlab).to receive(:dev_env_or_com?).and_return(dev_env_or_com)
+        gitlab_subscription.end_date = expiration_date
       end
 
-      with_them do
-        let!(:gitlab_subscription) { operation == :create ? build(:gitlab_subscription, initial_plan) : create(:gitlab_subscription, initial_plan) }
+      it 'indexes the namespace' do
+        expect(ElasticsearchIndexedNamespace).to receive(:safe_find_or_create_by!).with(namespace_id: gitlab_subscription.namespace_id)
 
-        before do
-          stub_feature_flags(elasticsearch_index_only_paid_groups: elasticsearch_index_only_paid_groups_setting)
+        gitlab_subscription.save!
       end
 
-        it 'indexes namespace' do
-          if should_index_namespace?
+      context 'when it is a trial' do
+        let(:gitlab_subscription) { build(:gitlab_subscription, :active_trial) }
+
+        it 'indexes the namespace' do
           expect(ElasticsearchIndexedNamespace).to receive(:safe_find_or_create_by!).with(namespace_id: gitlab_subscription.namespace_id)
-          else
+
+          gitlab_subscription.save!
+        end
+      end
+
+      context 'when not ::Gitlab.dev_env_or_com?' do
+        let(:dev_env_or_com) { false }
+
+        it 'does not index the namespace' do
+          expect(ElasticsearchIndexedNamespace).not_to receive(:safe_find_or_create_by!)
+
+          gitlab_subscription.save!
+        end
+      end
+
+      context 'when the plan has expired' do
+        let(:expiration_date) { Date.today - 8.days }
+
+        it 'does not index the namespace' do
           expect(ElasticsearchIndexedNamespace).not_to receive(:safe_find_or_create_by!)
+
+          gitlab_subscription.save!
+        end
+      end
+
+      context 'when it is not a hosted plan' do
+        before do
+          gitlab_subscription.namespace_id = nil
         end
 
-          case operation
-          when :create
+        it 'does not index anything' do
+          expect(ElasticsearchIndexedNamespace).not_to receive(:safe_find_or_create_by!)
+
           gitlab_subscription.save!
-          when :update
-            update_attributes = should_update_plan? ? { hosted_plan: create(:silver_plan), trial: false } : { max_seats_used: 32 }
-            gitlab_subscription.update!(update_attributes)
         end
       end
+
+      context 'when it is a free plan' do
+        let(:plan) { :free }
+
+        it 'does not index the namespace' do
+          expect(ElasticsearchIndexedNamespace).not_to receive(:safe_find_or_create_by!)
+
+          gitlab_subscription.save!
+        end
       end
     end