Optimize group labels page

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/23684

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/1148

See merge request !7123
Signed-off-by: Rémy Coutable <remy@rymai.me>