Found a workaround for that weird SQL error:

Without that, Rails would complain:

> bind message supplies 5 parameters, but prepared statement "a4" requires 4

However without this workaround it would still work in Rails console,
so I still think this is a Rails bug somewhere.

app/models/ci/runner.rb

@@ -27,7 +27,9 @@ module Ci
     end
 
     scope :specific_for, ->(project) do
-      where(locked: false).where.not(id: project.runners).specific
+      # TODO: That `to_sql` is needed to workaround a weird Rails bug.
+      #       Without that, placeholders would miss one and couldn't match.
+      where(locked: false).where.not(id: project.runners.to_sql).specific
     end
 
     validate :tag_constraints