Commit b3a7e0d5 authored by Stan Hu's avatar Stan Hu

Fix auto-renew of LetsEncrypt domains for Pages

As described in
https://community.letsencrypt.org/t/production-chain-changes/150739, the
LetsEncrypt DST Root CA X3 expired on September 30, 2021. Domains that
needed to be renewed via PagesDomainSslWorker would fail with the error,
"Certificate misses intermediates".

Newly-issued certificates would come with this chain of trust:

End-entity certificate (A) ← R3 (B) ← ISRG Root X1 (C) ← DST Root CA X3

Previously, this is what was happening:

1. LetsEncrypt returned a bundle containing A, B, and C.
2. `PagesDomain#has_intermediates?` took B and C and added them to the
OpenSSL certificate store.
3. `OpenSSL::X509::Store#verify` returned `false` because C was a
trusted certificate, but DST Root CA X3 had expired.

The crux of the problem is that we aren't using `verify` properly: we
should be passing in an untrusted chain and allow OpenSSL to verify that
C is indeed trusted from the system store. This emulates the behavior of
the `-untrusted` parameter in the `openssl` command-line:

```
All certificates (typically of intermediate CAs) are considered
untrusted and may be used to construct a certificate chain from the
target certificate to a trust anchor.
```

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/342326

Changelog: fixed
parent 018e51a2
...@@ -129,18 +129,15 @@ class PagesDomain < ApplicationRecord ...@@ -129,18 +129,15 @@ class PagesDomain < ApplicationRecord
store = OpenSSL::X509::Store.new store = OpenSSL::X509::Store.new
store.set_default_paths store.set_default_paths
# This forces to load all intermediate certificates stored in `certificate` store.verify(x509, untrusted_ca_certs_bundle)
Tempfile.open('certificate_chain') do |f|
f.write(certificate)
f.flush
store.add_file(f.path)
end
store.verify(x509)
rescue OpenSSL::X509::StoreError rescue OpenSSL::X509::StoreError
false false
end end
def untrusted_ca_certs_bundle
::Gitlab::Email::Smime::Certificate.load_ca_certs_bundle(certificate)
end
def expired? def expired?
return false unless x509 return false unless x509
......
...@@ -258,6 +258,18 @@ ZDXgrA== ...@@ -258,6 +258,18 @@ ZDXgrA==
certificate_source { :gitlab_provided } certificate_source { :gitlab_provided }
end end
# This contains:
# webdioxide.com
# Let's Encrypt R3
# ISRG Root X1 (issued by DST Root CA X3)
#
# DST Root CA X3 expired on 2021-09-30, but ISRG Root X1 should be trusted on most systems.
trait :letsencrypt_expired_x3_root do
certificate do
File.read(Rails.root.join('spec/fixtures/ssl', 'letsencrypt_expired_x3.pem'))
end
end
trait :explicit_ecdsa do trait :explicit_ecdsa do
certificate do certificate do
'-----BEGIN CERTIFICATE----- '-----BEGIN CERTIFICATE-----
......
...@@ -29,6 +29,7 @@ RSpec.describe 'factories' do ...@@ -29,6 +29,7 @@ RSpec.describe 'factories' do
[:pages_domain, :with_trusted_chain], [:pages_domain, :with_trusted_chain],
[:pages_domain, :with_trusted_expired_chain], [:pages_domain, :with_trusted_expired_chain],
[:pages_domain, :explicit_ecdsa], [:pages_domain, :explicit_ecdsa],
[:pages_domain, :letsencrypt_expired_x3_root],
[:project_member, :blocked], [:project_member, :blocked],
[:remote_mirror, :ssh], [:remote_mirror, :ssh],
[:user_preference, :only_comments], [:user_preference, :only_comments],
......
-----BEGIN CERTIFICATE-----
MIIGJDCCBQygAwIBAgISBOSAE/WwQGsTbDJI1vDL9+eKMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTEwMDEyMjIxMTlaFw0yMTEyMzAyMjIxMThaMBkxFzAVBgNVBAMT
DndlYmRpb3hpZGUuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA
wf/TpE5AjzoLXMFQ+WHle7Dn5rlEe0bPee2JU386cZmMYnGFS5DR251FerSX28U4
pqk2yS8oefHGi2PS6h8/MWxr+Zy/6hk3WkgwdIK3uPiUcfCdPV/btXDd4YqikEDm
BoOE4fQlqKQwtLOnhEZu9y8FQoxxoQ+7DndHrDixDoMbpUloxpqUZwziQnH4QHXE
32rQhq25+NUK/lVFGKOFnmZ2s/yUildKafqulHrLHOhumKMOEivzlFDZbtqP+RKt
nsrJ3i9O+nSQz6j5dv3Du6eaResrtK7tT1MFDNhcg2cgjNW64VLXQdFXYXE1OYsw
yAuXUnHNzWFhinyf80qeh2046YR21dlG8voIDQH4fGG5GmWLyu7glsWYVwQQ36VA
TTxPmAoaqUTl8A7cnlJpAo+BJ00mS/9DwJ7pkgGC7dYOhJzWlI7lPqzEfmJ+o8pj
CJlLIuqsn0vcCZQlmqCFMxK4asn+puLLnMjRLHIYEJKDNyPGHQEr2e5t4GUYZKaN
MEpXMwJd97tUamUKWeBPNIND/kOuqexe+okbOTRp34VAsK5oCpawEJckoNkK+sv0
OrSWFOdfLBHv66p9qsrz8LQXxmN5JUBUe51SBSUo1Ul4/vGYdhuKd/8KcLw9/Al+
HJN2hAeo3v+2fVey4hgGna7XNe8e3+E+OEQb4zpQDLkCAwEAAaOCAkswggJHMA4G
A1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYD
VR0TAQH/BAIwADAdBgNVHQ4EFgQU4PbvqCKatjx6GZMXy7v9GwykZq4wHwYDVR0j
BBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsG
AQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6
Ly9yMy5pLmxlbmNyLm9yZy8wGQYDVR0RBBIwEIIOd2ViZGlveGlkZS5jb20wTAYD
VR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAmBggrBgEFBQcCARYa
aHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEGBgorBgEEAdZ5AgQCBIH3BIH0
APIAdwBc3EOS/uarRUSxXprUVuYQN/vV+kfcoXOUsl7m9scOygAAAXw+KYGHAAAE
AwBIMEYCIQCqD6jMtHrGlE02Qh1FzFd4+qYzJTrChHmHBFIncPGQKAIhALeYk0Vf
/Lw2tX2beVlKN4/h1o8srNJv+06xkr1N6XmiAHcAfT7y+I//iFVoJMLAyp5SiXkr
xQ54CX8uapdomX4i8NcAAAF8PimBogAABAMASDBGAiEA0h883FFj1dSYKGym9+Wa
XgJRj526X7YlkhkZ5J1TjioCIQDyjMPrbo5liVi/e5b8gfDw5Fd9WNiTu1W1LKKu
UpE/qTANBgkqhkiG9w0BAQsFAAOCAQEAcx10nqp1kh2awwoqwf7Jo8Gycqx2bA2O
E2rveQ/BK9UhwvrNeEpE9SG6liMsYJKxGar0vbbBHvxzuMU00bhGjXFtUT5XuQ8q
FcU0OdycyZj8fjZmUNsJr82l8HvfJ50jfxFORTgj8Ln5MWVUFlbl0nD+06l28sDc
V+r/B4394fkoMsKXtiTA4/ZeOD1tHNsdxQ7sNQtEfqCG0wFCYHK3rs7XTZ1K0F3c
M051JShko1UKP/k5blrendOwVRwLtq+9pavGnJBeqNIVgugTER/IHlp4427WyhdY
KYjKoytW+XQyWqxU/Mh/O4rxkD8cZaE+FdZpP67VZ185AuZMbn+LcQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1ow
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEB
AQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XC
ov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpL
wYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+D
LtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK
4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5
bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5y
sR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZ
Xmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4
FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBc
SLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2ql
PRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TND
TwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYw
SwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1
c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx
+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEB
ATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQu
b3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9E
U1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26Ztu
MA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC
5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW
9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuG
WCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9O
he8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFC
Dfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5
-----END CERTIFICATE-----
...@@ -287,6 +287,19 @@ RSpec.describe PagesDomain do ...@@ -287,6 +287,19 @@ RSpec.describe PagesDomain do
it { is_expected.to be_truthy } it { is_expected.to be_truthy }
end end
# The LetsEncrypt DST Root CA X3 expired on 2021-09-30, but the
# cross-sign in ISRG Root X1 enables it to function provided a chain
# of trust can be established with the system store. See:
#
# 1. https://community.letsencrypt.org/t/production-chain-changes/150739
# 2. https://letsencrypt.org/2020/12/21/extending-android-compatibility.html
# 3. https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/
context 'with a LetsEncrypt bundle with an expired DST Root CA X3' do
let(:domain) { build(:pages_domain, :letsencrypt_expired_x3_root) }
it { is_expected.to be_truthy }
end
end end
describe '#expired?' do describe '#expired?' do
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment