Since the atomic modeset conversion, this should not be an issue
anymore.
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>