Only skip group when it's actually a group in the "Share with group" select

Fixes #23961

See merge request !7262