Merge branch '328-other' into 'master'

Elasticsearch versioned schema for other ActiveRecord models

See merge request gitlab-org/gitlab-ee!14522

config/initializers/zz_metrics.rb

@@ -100,11 +100,7 @@ def instrument_classes(instrumentation)
     instrumentation.instrument_instance_methods(Gitlab::Elastic::SnippetSearchResults)
     instrumentation.instrument_methods(Gitlab::Elastic::Helper)
 
-    instrumentation.instrument_instance_methods(Elastic::ApplicationSearch)
-    instrumentation.instrument_instance_methods(Elastic::IssuesSearch)
-    instrumentation.instrument_instance_methods(Elastic::MergeRequestsSearch)
-    instrumentation.instrument_instance_methods(Elastic::MilestonesSearch)
-    instrumentation.instrument_instance_methods(Elastic::NotesSearch)
+    instrumentation.instrument_instance_methods(Elastic::ApplicationVersionedSearch)
     instrumentation.instrument_instance_methods(Elastic::ProjectsSearch)
     instrumentation.instrument_instance_methods(Elastic::RepositoriesSearch)
     instrumentation.instrument_instance_methods(Elastic::SnippetsSearch)

doc/development/elasticsearch.md

@@ -148,26 +148,49 @@ Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/
 - Searches can have their own analyzers. Remember to check when editing analyzers
 - `Character` filters (as opposed to token filters) always replace the original character, so they're not a good choice as they can hinder exact searches
 
-## Architecture
+## Zero downtime reindexing with multiple indices
 
-GitLab uses `elasticsearch-rails` for handling communication with Elasticsearch server. However, in order to achieve zero-downtime deployment during schema changes, an extra abstraction layer is built to allow:
+Currently GitLab can only handle a single version of setting. Any setting/schema changes would require reindexing everything from scratch. Since reindexing can take a long time, this can cause search functionality downtime.
 
-* Indexing (writes) to multiple indexes, with different mappings
-* Switching to different index for searches (reads) on the fly
+To avoid downtime, GitLab is working to support multiple indices that
+can function at the same time. Whenever the schema changes, the admin
+will be able to create a new index and reindex to it, while searches
+continue to go to the older, stable index. Any data updates will be
+forwarded to both indices. Once the new index is ready, an admin can
+mark it active, which will direct all searches to it, and remove the old
+index.
 
-Currently we are on the process of migrating models to this new design (e.g. `Snippet`), and it is hardwired to work with a single version for now.
+This is also helpful for migrating to new servers, e.g. moving to/from AWS.
 
-Traditionally, `elasticsearch-rails` provides class and instance level `__elasticsearch__` proxy methods. If you call `Issue.__elasticsearch__`, you will get an instance of `Elasticsearch::Model::Proxy::ClassMethodsProxy`, and if you call `Issue.first.__elasticsearch__`, you will get an instance of `Elasticsearch::Model::Proxy::InstanceMethodsProxy`. These proxy objects would talk to Elasticsearch server directly.
+Currently we are on the process of migrating to this new design. Everything is hardwired to work with one single version for now.
 
-In the new design, `__elasticsearch__` instead represents one extra layer of proxy. It would keep multiple versions of the actual proxy objects, and it would forward read and write calls to the proxy of the intended version.
+### Architecture
 
-The `elasticsearch-rails`'s way of specifying each model's mappings and other settings is to create a module for the model to include. However in the new design, each model would have its own corresponding subclassed proxy object, where the settings reside in. For example, snippet related setting in the past reside in `SnippetsSearch` module, but in the new design would reside in `SnippetClassProxy` (which is a subclass of `Elasticsearch::Model::Proxy::ClassMethodsProxy`). This reduces namespace pollution in model classes.
+The traditional setup, provided by `elasticsearch-rails`, is to communicate through its internal proxy classes. Developers would write model-specific logic in a module for the model to include in (e.g. `SnippetsSearch`). The `__elasticsearch__` methods would return a proxy object, e.g.:
+
+- `Issue.__elasticsearch__` returns an instance of `Elasticsearch::Model::Proxy::ClassMethodsProxy`
+- `Issue.first.__elasticsearch__` returns an instance of `Elasticsearch::Model::Proxy::InstanceMethodsProxy`.
+
+These proxy objects would talk to Elasticsearch server directly (see top half of the diagram).
+
+![Elasticsearch Architecture](img/elasticsearch_architecture.svg)
+
+In the planned new design, each model would have a pair of corresponding subclassed proxy objects, in which model-specific logic is located. For example, `Snippet` would have `SnippetClassProxy` and `SnippetInstanceProxy` (being subclass of `Elasticsearch::Model::Proxy::ClassMethodsProxy` and `Elasticsearch::Model::Proxy::InstanceMethodsProxy`, respectively).
+
+`__elasticsearch__` would represent another layer of proxy object, keeping track of multiple actual proxy objects. It would forward method calls to the appropriate index. For example:
+
+- `model.__elasticsearch__.search` would be forwarded to the one stable index, since it is a read operation.
+- `model.__elasticsearch__.update_document` would be forwarded to all indices, to keep all indices up-to-date.
 
 The global configurations per version are now in the `Elastic::(Version)::Config` class. You can change mappings there.
 
 ### Creating new version of schema
 
-Currently GitLab would still work with a single version of setting. Once it is implemented, multiple versions of setting can exists in different folders (e.g. `ee/lib/elastic/v12p1` and `ee/lib/elastic/v12p3`). To keep a continuous git history, the latest version lives under the `/latest` folder, but is aliased as the latest version.
+NOTE: **Note:** this is not applicable yet as multiple indices functionality is not fully implemented.
+
+Folders like `ee/lib/elastic/v12p1` contain snapshots of search logic from different versions. To keep a continuous git history, the latest version lives under `ee/lib/elastic/latest`, but its classes are aliased under an actual version (e.g. `ee/lib/elastic/v12p3`). When referencing these classes, never use the `Latest` namespace directly, but use the actual version (e.g. `V12p3`).
+
+The version name basically follows GitLab's release version. If setting is changed in 12.3, we will create a new namespace called `V12p3` (p stands for "point"). Raise an issue if there is a need to name a version differently.
 
 If the current version is `v12p1`, and we need to create a new version for `v12p3`, the steps are as follows:
 
@@ -176,7 +199,7 @@ If the current version is `v12p1`, and we need to create a new version for `v12p
 1. Delete `v12p1` folder
 1. Copy the entire folder of `latest` as `v12p1`
 1. Change the namespace for files under `v12p1` folder from `Latest` to `V12p1`
-1. Make changes to `Latest` as needed
+1. Make changes to files under the `latest` folder as needed
 
 ## Troubleshooting
 

ee/app/models/concerns/elastic/issues_search.rb

-# frozen_string_literal: true
-
-module Elastic
-  module IssuesSearch
-    extend ActiveSupport::Concern
-
-    included do
-      include ApplicationSearch
-
-      def as_indexed_json(options = {})
-        data = {}
-
-        # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
-        # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
-        [:id, :iid, :title, :description, :created_at, :updated_at, :state, :project_id, :author_id, :confidential].each do |attr|
-          data[attr.to_s] = safely_read_attribute_for_elasticsearch(attr)
-        end
-
-        data['assignee_id'] = safely_read_attribute_for_elasticsearch(:assignee_ids)
-
-        data.merge(generic_attributes)
-      end
-
-      def self.nested?
-        true
-      end
-
-      def self.elastic_search(query, options: {})
-        query_hash =
-          if query =~ /#(\d+)\z/
-            iid_query_hash(Regexp.last_match(1))
-          else
-            basic_query_hash(%w(title^2 description), query)
-          end
-
-        options[:features] = 'issues'
-        query_hash = project_ids_filter(query_hash, options)
-        query_hash = confidentiality_filter(query_hash, options[:current_user])
-
-        self.__elasticsearch__.search(query_hash)
-      end
-
-      def self.confidentiality_filter(query_hash, current_user)
-        return query_hash if current_user && current_user.full_private_access?
-
-        filter = if current_user
-                   {
-                     bool: {
-                       should: [
-                         { term: { confidential: false } },
-                         {
-                           bool: {
-                             must: [
-                               { term: { confidential: true } },
-                               {
-                                 bool: {
-                                   should: [
-                                     { term: { author_id: current_user.id } },
-                                     { term: { assignee_id: current_user.id } },
-                                     { terms: { project_id: current_user.authorized_projects(Gitlab::Access::REPORTER).pluck(:id) } }
-                                   ]
-                                 }
-                               }
-                             ]
-                           }
-                         }
-                       ]
-                     }
-                   }
-                 else
-                   { term: { confidential: false } }
-                 end
-
-        query_hash[:query][:bool][:filter] << filter
-        query_hash
-      end
-    end
-  end
-end

ee/app/models/concerns/elastic/projects_search.rb

@@ -4,13 +4,7 @@ module Elastic
   module ProjectsSearch
     extend ActiveSupport::Concern
 
-    TRACKED_FEATURE_SETTINGS = %w(
-      issues_access_level
-      merge_requests_access_level
-      snippets_access_level
-      wiki_access_level
-      repository_access_level
-    ).freeze
+    include ApplicationVersionedSearch
 
     INDEXED_ASSOCIATIONS = [
       :issues,
@@ -21,97 +15,10 @@ module Elastic
     ].freeze
 
     included do
-      include ApplicationSearch
-
       def use_elasticsearch?
         ::Gitlab::CurrentSettings.elasticsearch_indexes_project?(self)
       end
 
-      def as_indexed_json(options = {})
-        # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
-        # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
-        data = {}
-
-        [
-          :id,
-          :name,
-          :path,
-          :description,
-          :namespace_id,
-          :created_at,
-          :updated_at,
-          :archived,
-          :visibility_level,
-          :last_activity_at,
-          :name_with_namespace,
-          :path_with_namespace
-        ].each do |attr|
-          data[attr.to_s] = safely_read_attribute_for_elasticsearch(attr)
-        end
-
-        # Set it as a parent in our `project => child` JOIN field
-        data['join_field'] = es_type
-
-        # ES6 is now single-type per index, so we implement our own typing
-        data['type'] = 'project'
-
-        TRACKED_FEATURE_SETTINGS.each do |feature|
-          data[feature] = project_feature.public_send(feature) # rubocop:disable GitlabSecurity/PublicSend
-        end
-
-        data
-      end
-
-      def self.elastic_search(query, options: {})
-        options[:in] = %w(name^10 name_with_namespace^2 path_with_namespace path^9 description)
-
-        query_hash = basic_query_hash(options[:in], query)
-
-        filters = []
-
-        if options[:namespace_id]
-          filters << {
-            terms: {
-              namespace_id: [options[:namespace_id]].flatten
-            }
-          }
-        end
-
-        if options[:non_archived]
-          filters << {
-            terms: {
-              archived: [!options[:non_archived]].flatten
-            }
-          }
-        end
-
-        if options[:visibility_levels]
-          filters << {
-            terms: {
-              visibility_level: [options[:visibility_levels]].flatten
-            }
-          }
-        end
-
-        if options[:project_ids]
-          filters << {
-            bool: project_ids_query(options[:current_user], options[:project_ids], options[:public_and_internal_projects])
-          }
-        end
-
-        query_hash[:query][:bool][:filter] = filters
-
-        query_hash[:sort] = [:_score]
-
-        self.__elasticsearch__.search(query_hash)
-      end
-
-      def self.indexed_association_classes
-        INDEXED_ASSOCIATIONS.map do |association_name|
-          reflect_on_association(association_name).klass
-        end
-      end
-
       def each_indexed_association
         INDEXED_ASSOCIATIONS.each do |association_name|
           association = self.association(association_name)

ee/app/models/ee/issue.rb

@@ -11,7 +11,7 @@ module EE
       WEIGHT_ANY = 'Any'.freeze
       WEIGHT_NONE = 'None'.freeze
 
-      include Elastic::IssuesSearch
+      include Elastic::ApplicationVersionedSearch
 
       scope :order_weight_desc, -> { reorder ::Gitlab::Database.nulls_last_order('weight', 'DESC') }
       scope :order_weight_asc, -> { reorder ::Gitlab::Database.nulls_last_order('weight') }

ee/app/models/ee/legacy_diff_note.rb

@@ -5,7 +5,7 @@ module EE
     extend ActiveSupport::Concern
 
     prepended do
-      include Elastic::NotesSearch
+      include Elastic::ApplicationVersionedSearch
     end
   end
 end

ee/app/models/ee/merge_request.rb

@@ -11,7 +11,7 @@ module EE
     include FromUnion
 
     prepended do
-      include Elastic::MergeRequestsSearch
+      include Elastic::ApplicationVersionedSearch
 
       has_many :reviews, inverse_of: :merge_request
       has_many :approvals, dependent: :delete_all # rubocop:disable Cop/ActiveRecordDependent

ee/app/models/ee/milestone.rb

@@ -5,7 +5,7 @@ module EE
     extend ActiveSupport::Concern
 
     prepended do
-      include Elastic::MilestonesSearch
+      include Elastic::ApplicationVersionedSearch
 
       has_many :boards
     end

ee/app/models/ee/note.rb

@@ -7,7 +7,7 @@ module EE
 
     prepended do
       include ::ObjectStorage::BackgroundMove
-      include Elastic::NotesSearch
+      include Elastic::ApplicationVersionedSearch
 
       belongs_to :review, inverse_of: :notes
 

ee/lib/elastic/latest/issue_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class IssueClassProxy < ApplicationClassProxy
+      def nested?
+        true
+      end
+
+      def elastic_search(query, options: {})
+        query_hash =
+          if query =~ /#(\d+)\z/
+            iid_query_hash(Regexp.last_match(1))
+          else
+            basic_query_hash(%w(title^2 description), query)
+          end
+
+        options[:features] = 'issues'
+        query_hash = project_ids_filter(query_hash, options)
+        query_hash = confidentiality_filter(query_hash, options[:current_user])
+
+        search(query_hash)
+      end
+
+      private
+
+      def confidentiality_filter(query_hash, current_user)
+        return query_hash if current_user && current_user.full_private_access?
+
+        filter =
+          if current_user
+            {
+              bool: {
+                should: [
+                  { term: { confidential: false } },
+                  {
+                    bool: {
+                      must: [
+                        { term: { confidential: true } },
+                        {
+                          bool: {
+                            should: [
+                              { term: { author_id: current_user.id } },
+                              { term: { assignee_id: current_user.id } },
+                              { terms: { project_id: current_user.authorized_projects(Gitlab::Access::REPORTER).pluck_primary_key } }
+                            ]
+                          }
+                        }
+                      ]
+                    }
+                  }
+                ]
+              }
+            }
+          else
+            { term: { confidential: false } }
+          end
+
+        query_hash[:query][:bool][:filter] << filter
+        query_hash
+      end
+    end
+  end
+end

ee/lib/elastic/latest/issue_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class IssueInstanceProxy < ApplicationInstanceProxy
+      def as_indexed_json(options = {})
+        data = {}
+
+        # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
+        # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
+        [:id, :iid, :title, :description, :created_at, :updated_at, :state, :project_id, :author_id, :confidential].each do |attr|
+          data[attr.to_s] = safely_read_attribute_for_elasticsearch(attr)
+        end
+
+        data['assignee_id'] = safely_read_attribute_for_elasticsearch(:assignee_ids)
+
+        data.merge(generic_attributes)
+      end
+    end
+  end
+end

ee/lib/elastic/latest/merge_request_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class MergeRequestClassProxy < ApplicationClassProxy
+      def nested?
+        true
+      end
+
+      def elastic_search(query, options: {})
+        query_hash =
+          if query =~ /\!(\d+)\z/
+            iid_query_hash(Regexp.last_match(1))
+          else
+            basic_query_hash(%w(title^2 description), query)
+          end
+
+        options[:features] = 'merge_requests'
+        query_hash = project_ids_filter(query_hash, options)
+
+        search(query_hash)
+      end
+    end
+  end
+end

ee/app/models/concerns/elastic/merge_requests_search.rb → ee/lib/elastic/latest/merge_request_instance_proxy.rb

 # frozen_string_literal: true
 
 module Elastic
-  module MergeRequestsSearch
-    extend ActiveSupport::Concern
-
-    included do
-      include ApplicationSearch
-
+  module Latest
+    class MergeRequestInstanceProxy < ApplicationInstanceProxy
       def as_indexed_json(options = {})
         # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
         # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
@@ -32,28 +28,6 @@ module Elastic
 
         data.merge(generic_attributes)
       end
-
-      def es_parent
-        "project_#{target_project_id}"
-      end
-
-      def self.nested?
-        true
-      end
-
-      def self.elastic_search(query, options: {})
-        query_hash =
-          if query =~ /\!(\d+)\z/
-            iid_query_hash(Regexp.last_match(1))
-          else
-            basic_query_hash(%w(title^2 description), query)
-          end
-
-        options[:features] = 'merge_requests'
-        query_hash = project_ids_filter(query_hash, options)
-
-        self.__elasticsearch__.search(query_hash)
-      end
     end
   end
 end

ee/lib/elastic/latest/milestone_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class MilestoneClassProxy < ApplicationClassProxy
+      def nested?
+        true
+      end
+
+      def elastic_search(query, options: {})
+        options[:in] = %w(title^2 description)
+
+        query_hash = basic_query_hash(options[:in], query)
+
+        query_hash = project_ids_filter(query_hash, options)
+
+        search(query_hash)
+      end
+    end
+  end
+end

ee/app/models/concerns/elastic/milestones_search.rb → ee/lib/elastic/latest/milestone_instance_proxy.rb

 # frozen_string_literal: true
 
 module Elastic
-  module MilestonesSearch
-    extend ActiveSupport::Concern
-
-    included do
-      include ApplicationSearch
-
+  module Latest
+    class MilestoneInstanceProxy < ApplicationInstanceProxy
       def as_indexed_json(options = {})
         # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
         # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
@@ -18,20 +14,6 @@ module Elastic
 
         data.merge(generic_attributes)
       end
-
-      def self.nested?
-        true
-      end
-
-      def self.elastic_search(query, options: {})
-        options[:in] = %w(title^2 description)
-
-        query_hash = basic_query_hash(options[:in], query)
-
-        query_hash = project_ids_filter(query_hash, options)
-
-        self.__elasticsearch__.search(query_hash)
-      end
     end
   end
 end

ee/app/models/concerns/elastic/notes_search.rb → ee/lib/elastic/latest/note_class_proxy.rb

 # frozen_string_literal: true
 
 module Elastic
-  module NotesSearch
-    extend ActiveSupport::Concern
-
-    included do
-      include ApplicationSearch
-
+  module Latest
+    class NoteClassProxy < ApplicationClassProxy
       def es_type
         'note'
       end
 
-      def as_indexed_json(options = {})
-        data = {}
-
-        # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
-        # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
-        [:id, :note, :project_id, :noteable_type, :noteable_id, :created_at, :updated_at].each do |attr|
-          data[attr.to_s] = safely_read_attribute_for_elasticsearch(attr)
-        end
-
-        if noteable.is_a?(Issue)
-          data['issue'] = {
-            assignee_id: noteable.assignee_ids,
-            author_id: noteable.author_id,
-            confidential: noteable.confidential
-          }
-        end
-
-        data.merge(generic_attributes)
-      end
-
-      def self.nested?
+      def nested?
         true
       end
 
-      def self.elastic_search(query, options: {})
+      def elastic_search(query, options: {})
         options[:in] = ['note']
 
         query_hash = basic_query_hash(%w[note], query)
@@ -49,11 +25,13 @@ module Elastic
 
         query_hash[:highlight] = highlight_options(options[:in])
 
-        self.__elasticsearch__.search(query_hash)
+        search(query_hash)
       end
 
-      def self.confidentiality_filter(query_hash, current_user)
-        return query_hash if current_user && current_user.full_private_access?
+      private
+
+      def confidentiality_filter(query_hash, current_user)
+        return query_hash if current_user&.full_private_access?
 
         filter = {
           bool: {
@@ -74,7 +52,7 @@ module Elastic
                     should: [
                       { term: { "issue.author_id" => current_user.id } },
                       { term: { "issue.assignee_id" => current_user.id } },
-                      { terms: { "project_id" => current_user.authorized_projects(Gitlab::Access::REPORTER).pluck(:id) } }
+                      { terms: { "project_id" => current_user.authorized_projects(Gitlab::Access::REPORTER).pluck_primary_key } }
                     ]
                   }
                 }

ee/lib/elastic/latest/note_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class NoteInstanceProxy < ApplicationInstanceProxy
+      delegate :noteable, to: :target
+
+      def as_indexed_json(options = {})
+        data = {}
+
+        # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
+        # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
+        [:id, :note, :project_id, :noteable_type, :noteable_id, :created_at, :updated_at].each do |attr|
+          data[attr.to_s] = safely_read_attribute_for_elasticsearch(attr)
+        end
+
+        if noteable.is_a?(Issue)
+          data['issue'] = {
+            assignee_id: noteable.assignee_ids,
+            author_id: noteable.author_id,
+            confidential: noteable.confidential
+          }
+        end
+
+        data.merge(generic_attributes)
+      end
+    end
+  end
+end

ee/lib/elastic/latest/project_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class ProjectClassProxy < ApplicationClassProxy
+      def elastic_search(query, options: {})
+        options[:in] = %w(name^10 name_with_namespace^2 path_with_namespace path^9 description)
+
+        query_hash = basic_query_hash(options[:in], query)
+
+        filters = []
+
+        if options[:namespace_id]
+          filters << {
+            terms: {
+              namespace_id: [options[:namespace_id]].flatten
+            }
+          }
+        end
+
+        if options[:non_archived]
+          filters << {
+            terms: {
+              archived: [!options[:non_archived]].flatten
+            }
+          }
+        end
+
+        if options[:visibility_levels]
+          filters << {
+            terms: {
+              visibility_level: [options[:visibility_levels]].flatten
+            }
+          }
+        end
+
+        if options[:project_ids]
+          filters << {
+            bool: project_ids_query(options[:current_user], options[:project_ids], options[:public_and_internal_projects])
+          }
+        end
+
+        query_hash[:query][:bool][:filter] = filters
+
+        query_hash[:sort] = [:_score]
+
+        search(query_hash)
+      end
+    end
+  end
+end

ee/lib/elastic/latest/project_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module Latest
+    class ProjectInstanceProxy < ApplicationInstanceProxy
+      TRACKED_FEATURE_SETTINGS = %w(
+        issues_access_level
+        merge_requests_access_level
+        snippets_access_level
+        wiki_access_level
+        repository_access_level
+      ).freeze
+
+      def as_indexed_json(options = {})
+        # We don't use as_json(only: ...) because it calls all virtual and serialized attributtes
+        # https://gitlab.com/gitlab-org/gitlab-ee/issues/349
+        data = {}
+
+        [
+          :id,
+          :name,
+          :path,
+          :description,
+          :namespace_id,
+          :created_at,
+          :updated_at,
+          :archived,
+          :visibility_level,
+          :last_activity_at,
+          :name_with_namespace,
+          :path_with_namespace
+        ].each do |attr|
+          data[attr.to_s] = safely_read_attribute_for_elasticsearch(attr)
+        end
+
+        # Set it as a parent in our `project => child` JOIN field
+        data['join_field'] = es_type
+
+        # ES6 is now single-type per index, so we implement our own typing
+        data['type'] = 'project'
+
+        TRACKED_FEATURE_SETTINGS.each do |feature|
+          data[feature] = target.project_feature.public_send(feature) # rubocop:disable GitlabSecurity/PublicSend
+        end
+
+        data
+      end
+    end
+  end
+end

ee/lib/elastic/multi_version_util.rb

@@ -36,6 +36,19 @@ module Elastic
       klass < ActiveRecord::Base ? klass.base_class : klass
     end
 
+    # Handles which method calls should be forwarded to all targets,
+    # and which calls should be forwarded to just one target.
+    #
+    # This first sets up forwarding for methods which should go to all targets.
+    # This is specified by implementing `methods_for_all_write_targets`.
+    # Examples include document indexing/updating operations.
+    #
+    # Then other methods are forwarded to just the single read target.
+    # Examples include user searches.
+    #
+    # Special write operations specified in `methods_for_one_write_target` are left out.
+    # The caller must always specify the version the call should be triggered.
+    # Examples include deleting the whole index.
     def generate_forwarding
       methods_for_all_write_targets = elastic_writing_targets.first.real_class.methods_for_all_write_targets
       methods_for_one_write_target = elastic_writing_targets.first.real_class.methods_for_one_write_target

ee/lib/elastic/v12p1/issue_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    IssueClassProxy = Elastic::Latest::IssueClassProxy
+  end
+end

ee/lib/elastic/v12p1/issue_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    IssueInstanceProxy = Elastic::Latest::IssueInstanceProxy
+  end
+end

ee/lib/elastic/v12p1/merge_request_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    MergeRequestClassProxy = Elastic::Latest::MergeRequestClassProxy
+  end
+end

ee/lib/elastic/v12p1/merge_request_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    MergeRequestInstanceProxy = Elastic::Latest::MergeRequestInstanceProxy
+  end
+end

ee/lib/elastic/v12p1/milestone_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    MilestoneClassProxy = Elastic::Latest::MilestoneClassProxy
+  end
+end

ee/lib/elastic/v12p1/milestone_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    MilestoneInstanceProxy = Elastic::Latest::MilestoneInstanceProxy
+  end
+end

ee/lib/elastic/v12p1/note_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    NoteClassProxy = Elastic::Latest::NoteClassProxy
+  end
+end

ee/lib/elastic/v12p1/note_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    NoteInstanceProxy = Elastic::Latest::NoteInstanceProxy
+  end
+end

ee/lib/elastic/v12p1/project_class_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    ProjectClassProxy = Elastic::Latest::ProjectClassProxy
+  end
+end

ee/lib/elastic/v12p1/project_instance_proxy.rb

+# frozen_string_literal: true
+
+module Elastic
+  module V12p1
+    ProjectInstanceProxy = Elastic::Latest::ProjectInstanceProxy
+  end
+end

ee/lib/gitlab/elastic/helper.rb

@@ -4,8 +4,7 @@ module Gitlab
   module Elastic
     class Helper
       # rubocop: disable CodeReuse/ActiveRecord
-      def self.create_empty_index
-        index_name = Project.index_name
+      def self.create_empty_index(version = ::Elastic::MultiVersionUtil::TARGET_VERSION)
         settings = {}
         mappings = {}
 
@@ -23,7 +22,9 @@ module Gitlab
           mappings.deep_merge!(klass.__elasticsearch__.mappings.to_hash)
         end
 
-        client = Project.__elasticsearch__.client
+        proxy = Project.__elasticsearch__.version(version)
+        client = proxy.client
+        index_name = proxy.index_name
 
         # ES5.6 needs a setting enabled to support JOIN datatypes that ES6 does not support...
         if Gitlab::VersionInfo.parse(client.info['version']['number']) < Gitlab::VersionInfo.new(6)
@@ -42,16 +43,16 @@ module Gitlab
       end
       # rubocop: enable CodeReuse/ActiveRecord
 
-      def self.delete_index
-        Project.__elasticsearch__.delete_index!
+      def self.delete_index(version = ::Elastic::MultiVersionUtil::TARGET_VERSION)
+        Project.__elasticsearch__.version(version).delete_index!
       end
 
       def self.refresh_index
         Project.__elasticsearch__.refresh_index!
       end
 
-      def self.index_size
-        Project.__elasticsearch__.client.indices.stats['indices'][Project.__elasticsearch__.index_name]['total']
+      def self.index_size(version = ::Elastic::MultiVersionUtil::TARGET_VERSION)
+        Project.__elasticsearch__.version(version).client.indices.stats['indices'][Project.__elasticsearch__.index_name]['total']
       end
     end
   end

ee/lib/gitlab/elastic/search_results.rb

@@ -199,7 +199,7 @@ module Gitlab
       def milestones
         strong_memoize(:milestones) do
           # Must pass 'issues' and 'merge_requests' to check
-          # if any of the features is available for projects in Elastic::ApplicationSearch#project_ids_query
+          # if any of the features is available for projects in ApplicationClassProxy#project_ids_query
           # Otherwise it will ignore project_ids and return milestones
           # from projects with milestones disabled.
           options = base_options

ee/lib/tasks/gitlab/elastic/test.rake

@@ -4,7 +4,7 @@ namespace :gitlab do
       desc 'GitLab | Elasticsearch | Test | Measure space taken by ES indices'
       task index_size: :environment do
         puts "===== Size stats for index: #{Project.__elasticsearch__.index_name} ====="
-        pp Gitlab::Elastic::Helper.index_size.slice(*%w(docs store))
+        pp Gitlab::Elastic::Helper.index_size(::Elastic::MultiVersionUtil::TARGET_VERSION).slice(*%w(docs store))
       end
 
       desc 'GitLab | Elasticsearch | Test | Measure space taken by ES indices, reindex, and measure space taken again'

ee/spec/models/concerns/elastic/issue_spec.rb

@@ -89,7 +89,7 @@ describe Issue, :elastic do
 
     expected_hash['assignee_id'] = [assignee.id]
 
-    expect(issue.as_indexed_json).to eq(expected_hash)
+    expect(issue.__elasticsearch__.as_indexed_json).to eq(expected_hash)
   end
 
   it_behaves_like 'no results when the user cannot read cross project' do

ee/spec/models/concerns/elastic/merge_request_spec.rb

@@ -61,7 +61,7 @@ describe MergeRequest, :elastic do
               'type' => merge_request.es_type
             })
 
-    expect(merge_request.as_indexed_json).to eq(expected_hash)
+    expect(merge_request.__elasticsearch__.as_indexed_json).to eq(expected_hash)
   end
 
   it_behaves_like 'no results when the user cannot read cross project' do

ee/spec/models/concerns/elastic/milestone_spec.rb

@@ -53,7 +53,7 @@ describe Milestone, :elastic do
       'type' => milestone.es_type
     })
 
-    expect(milestone.as_indexed_json).to eq(expected_hash)
+    expect(milestone.__elasticsearch__.as_indexed_json).to eq(expected_hash)
   end
 
   it_behaves_like 'no results when the user cannot read cross project' do

ee/spec/models/concerns/elastic/note_spec.rb

@@ -88,7 +88,7 @@ describe Note, :elastic do
       type
     )
 
-    expect(note.as_indexed_json.keys).to eq(expected_hash_keys)
+    expect(note.__elasticsearch__.as_indexed_json.keys).to eq(expected_hash_keys)
   end
 
   it "does not create ElasticIndexerWorker job for system messages" do
@@ -105,7 +105,7 @@ describe Note, :elastic do
     Note.subclasses.each do |note_class|
       expect(note_class.index_name).to eq(Note.index_name)
       expect(note_class.document_type).to eq(Note.document_type)
-      expect(note_class.mappings.to_hash).to eq(Note.mappings.to_hash)
+      expect(note_class.__elasticsearch__.mappings.to_hash).to eq(Note.__elasticsearch__.mappings.to_hash)
     end
   end
 

ee/spec/models/concerns/elastic/project_spec.rb

@@ -159,6 +159,6 @@ describe Project, :elastic do
     expected_hash['name_with_namespace'] = project.full_name
     expected_hash['path_with_namespace'] = project.full_path
 
-    expect(project.as_indexed_json).to eq(expected_hash)
+    expect(project.__elasticsearch__.as_indexed_json).to eq(expected_hash)
   end
 end