Only force reconfig for old app versions

See merge request !1998
6 jobs for master
in 0 seconds, using 0 compute credits, and was queued for 0 seconds