Merge branch 'fix-extra-new-line-crlf' into 'master'

Trim extra displayed carriage returns in diffs and files with CRLFs

## What does this MR do?

This MR strips out extra carriage returns at the end of each tokenized string.

## Are there points in the code the reviewer needs to double check?

Does this remove more than we need?

## Why was this MR needed?

Windows users would see extra lines in their files. Newline characters are already added with every line in the HTML formatter.

## What are the relevant issue numbers?

#20440, gitlab-com/support-forum#892, #20468 

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5588

CHANGELOG

@@ -46,6 +46,7 @@ v 8.10.3 (unreleased)
   - Properly abort a merge when merge conflicts occur
   - Ignore invalid IPs in X-Forwarded-For when trusted proxies are configured.
   - Fix Import/Export issue importing milestones and labels not associated properly
+  - Trim extra displayed carriage returns in diffs and files with CRLFs
 
 v 8.10.2
   - User can now search branches by name. !5144

lib/rouge/formatters/html_gitlab.rb

@@ -18,7 +18,7 @@ module Rouge
           is_first = false
 
           yield %(<span id="LC#{@line_number}" class="line">)
-          line.each { |token, value| yield span(token, value) }
+          line.each { |token, value| yield span(token, value.chomp) }
           yield %(</span>)
 
           @line_number += 1

spec/finders/branches_finder_spec.rb

@@ -20,7 +20,7 @@ describe BranchesFinder do
 
         result = branches_finder.execute
 
-        expect(result.first.name).to eq('video')
+        expect(result.first.name).to eq('crlf-diff')
       end
 
       it 'sorts by last_updated' do

spec/lib/gitlab/highlight_spec.rb

@@ -17,6 +17,18 @@ describe Gitlab::Highlight, lib: true do
       expect(lines[21]).to eq(%Q{<span id="LC22" class="line">    <span class="k">unless</span> <span class="no">File</span><span class="p">.</span><span class="nf">directory?</span><span class="p">(</span><span class="n">path</span><span class="p">)</span></span>\n})
       expect(lines[26]).to eq(%Q{<span id="LC27" class="line">    <span class="vi">@cmd_status</span> <span class="o">=</span> <span class="mi">0</span></span>\n})
     end
+
+    describe 'with CRLF' do
+      let(:branch) { 'crlf-diff' }
+      let(:blob) { repository.blob_at_branch(branch, path) }
+      let(:lines) do
+        Gitlab::Highlight.highlight_lines(project.repository, 'crlf-diff', 'files/whitespace')
+      end
+
+      it 'strips extra LFs' do
+        expect(lines[0]).to eq("<span id=\"LC1\" class=\"line\">test  </span>")
+      end
+    end
   end
 
   describe 'custom highlighting from .gitattributes' do

spec/support/test_env.rb

@@ -21,7 +21,8 @@ module TestEnv
     'expand-collapse-diffs' => '4842455',
     'expand-collapse-files' => '025db92',
     'expand-collapse-lines' => '238e82d',
-    'video'                 => '8879059'
+    'video'                 => '8879059',
+    'crlf-diff'             => '5938907'
   }
 
   # gitlab-test-fork is a fork of gitlab-fork, but we don't necessarily