I realized where the N+1 queries were actually coming from
project.protected_branches, but how come we cannot preload this,
or cache this at all?

Then I found that this is somehow a Rails limitation. What we're
doing before, eventually come to:

    project.protected_branches.matching

But why it's not cached? (project.protected_branches.loaded? is always
false) It's because matching is a class method, which is called on
the proxy. In this case, Rails cannot cache the result. I don't know
if this is possible to implement or not, because clearly this would
require some tricks to implement class methods on associations.

So instead, we could just pass project.protected_branches to
ProtectedRef.matching, then it would work regularly.

With this change, there's no more N+1 queries.