Remove feature flag elastic_bulk_incremental_updates

We also refactor now since the only time we don't use the bulk indexer
is in the project model hooks it makes things cleaner to just overload
the methods in project model hooks rather than keeping all the logic in
the parent model.

doc/development/elasticsearch.md

@@ -36,11 +36,7 @@ Additionally, if you need large repos or multiple forks for testing, please cons
 
 The Elasticsearch integration depends on an external indexer. We ship an [indexer written in Go](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer). The user must trigger the initial indexing via a rake task but, after this is done, GitLab itself will trigger reindexing when required via `after_` callbacks on create, update, and destroy that are inherited from [/ee/app/models/concerns/elastic/application_versioned_search.rb](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/app/models/concerns/elastic/application_versioned_search.rb).
 
-After initial indexing is complete, updates proceed in one of two ways, depending on the `:elastic_bulk_incremental_updates` feature flag.
-
-If disabled, every create, update, or delete operation on an Elasticsearch-tracked model enqueues a new `ElasticIndexerWorker` Sidekiq job which takes care of updating just that document. This is quite inefficient.
-
-If the feature flag is enabled, create, update, and delete operations for all models except projects (see [#207494](https://gitlab.com/gitlab-org/gitlab/issues/207494)) are tracked in a Redis [`ZSET`](https://redis.io/topics/data-types#sorted-sets) instead. A regular `sidekiq-cron` `ElasticIndexBulkCronWorker` processes this queue, updating many Elasticsearch documents at a time with the [Bulk Request API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html).
+After initial indexing is complete, create, update, and delete operations for all models except projects (see [#207494](https://gitlab.com/gitlab-org/gitlab/issues/207494)) are tracked in a Redis [`ZSET`](https://redis.io/topics/data-types#sorted-sets). A regular `sidekiq-cron` `ElasticIndexBulkCronWorker` processes this queue, updating many Elasticsearch documents at a time with the [Bulk Request API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html).
 
 Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them!
 

ee/app/models/concerns/elastic/application_versioned_search.rb

@@ -45,36 +45,15 @@ module Elastic
     end
 
     def maintain_elasticsearch_create
-      return if maintain_elasticsearch_incremental_bulk
-
-      ElasticIndexerWorker.perform_async(:index, self.class.to_s, self.id, self.es_id)
+      ::Elastic::ProcessBookkeepingService.track!(self)
     end
 
     def maintain_elasticsearch_update
-      return if maintain_elasticsearch_incremental_bulk
-
-      ElasticIndexerWorker.perform_async(
-        :update,
-        self.class.to_s,
-        self.id,
-        self.es_id
-      )
+      ::Elastic::ProcessBookkeepingService.track!(self)
     end
 
     def maintain_elasticsearch_destroy
-      return if maintain_elasticsearch_incremental_bulk
-
-      ElasticIndexerWorker.perform_async(
-        :delete, self.class.to_s, self.id, self.es_id, es_parent: self.es_parent
-      )
-    end
-
-    def maintain_elasticsearch_incremental_bulk
-      return false unless Feature.enabled?(:elastic_bulk_incremental_updates, self.project)
-
       ::Elastic::ProcessBookkeepingService.track!(self)
-
-      true
     end
 
     class_methods do

ee/app/models/concerns/elastic/projects_search.rb

@@ -19,12 +19,20 @@ module Elastic
         ::Gitlab::CurrentSettings.elasticsearch_indexes_project?(self)
       end
 
-      def maintain_elasticsearch_incremental_bulk
-        # TODO: ElasticIndexerWorker does extra work for project hooks, so we
-        # can't use the incremental-bulk indexer for projects yet.
-        #
-        # https://gitlab.com/gitlab-org/gitlab/issues/207494
-        false
+      # TODO: ElasticIndexerWorker does extra work for project hooks, so we
+      # can't use the incremental-bulk indexer for projects yet.
+      #
+      # https://gitlab.com/gitlab-org/gitlab/issues/207494
+      def maintain_elasticsearch_create
+        ElasticIndexerWorker.perform_async(:index, self.class.to_s, self.id, self.es_id)
+      end
+
+      def maintain_elasticsearch_update
+        ElasticIndexerWorker.perform_async(:update, self.class.to_s, self.id, self.es_id)
+      end
+
+      def maintain_elasticsearch_destroy
+        ElasticIndexerWorker.perform_async(:delete, self.class.to_s, self.id, self.es_id, es_parent: self.es_parent)
       end
 
       def each_indexed_association

ee/changelogs/unreleased/208717-remove-ff-for-bulk-indexer.yml

+---
+title: Remove feature flag elastic_bulk_incremental_updates
+merge_request: 27293
+author:
+type: performance

ee/spec/models/concerns/elastic/note_spec.rb

@@ -106,21 +106,7 @@ describe Note, :elastic do
     expect(note.__elasticsearch__.as_indexed_json).to eq(expected_hash)
   end
 
-  it "does not create ElasticIndexerWorker job for system messages" do
-    stub_feature_flags(elastic_bulk_incremental_updates: false)
-
-    project = create :project, :repository
-    # We have to set one minute delay because of https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/15682
-    issue = create :issue, project: project, updated_at: 1.minute.ago
-
-    # Only issue should be updated
-    expect(ElasticIndexerWorker).to receive(:perform_async).with(:update, 'Issue', anything, anything)
-    create :note, :system, project: project, noteable: issue
-  end
-
-  it 'does not track system note updates via the bulk updater' do
-    stub_feature_flags(elastic_bulk_incremental_updates: true)
-
+  it 'does not track system note updates' do
     note = create(:note, :system)
 
     expect(Elastic::ProcessBookkeepingService).not_to receive(:track!)