We need to do this earlier to prevent aux channel timeouts in resume
paths on certain systems.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>