Commit 80ddd2c0 authored by Dmitriy Zaporozhets's avatar Dmitriy Zaporozhets

Better encoding handling. Updated grit

parent 3706a974
...@@ -14,7 +14,7 @@ GIT ...@@ -14,7 +14,7 @@ GIT
GIT GIT
remote: https://github.com/gitlabhq/grit.git remote: https://github.com/gitlabhq/grit.git
revision: ff015074ef35bd94cba943f9c0f98e161ab5851c revision: 3fc864f3c637e06e2fa7a81f6b48a5df58a9bc5b
specs: specs:
grit (2.4.1) grit (2.4.1)
diff-lcs (~> 1.1) diff-lcs (~> 1.1)
...@@ -148,7 +148,7 @@ GEM ...@@ -148,7 +148,7 @@ GEM
mime-types (~> 1.16) mime-types (~> 1.16)
treetop (~> 1.4.8) treetop (~> 1.4.8)
method_source (0.7.0) method_source (0.7.0)
mime-types (1.17.2) mime-types (1.18)
modularity (0.6.1) modularity (0.6.1)
multi_json (1.0.4) multi_json (1.0.4)
multi_xml (0.4.1) multi_xml (0.4.1)
......
require 'benchmark'
require "base64" require "base64"
class CommitsController < ApplicationController class CommitsController < ApplicationController
......
...@@ -17,13 +17,35 @@ Grit::GitRuby::Internal::RawObject.class_eval do ...@@ -17,13 +17,35 @@ Grit::GitRuby::Internal::RawObject.class_eval do
end end
private private
def transcoding(content) def transcoding(content)
content ||= "" content ||= ""
detection = CharlockHolmes::EncodingDetector.detect(content) hash = CharlockHolmes::EncodingDetector.detect(content)
if hash = detection
content = CharlockHolmes::Converter.convert(content, hash[:encoding], 'UTF-8') if hash[:encoding] if hash
end return content if hash[:type] == :binary
if hash[:encoding] == "UTF-8"
content = if hash[:confidence] < 100
content content
else
content.force_encoding("UTF-8")
end
return content
end
CharlockHolmes::Converter.convert(content, hash[:encoding], 'UTF-8') if hash[:encoding]
else
content.force_encoding("UTF-8")
end
end
def z_binary?(string)
string.each_byte do |x|
x.nonzero? or return true
end
false
end end
end end
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment