The issue was found when we enabled CONFIG_PM and tested edge events using
libgpiod.

> # gpiomon -r gpiochip0 55
> gpiomon: error waiting for events: Permission denied

`gpiomon` will call irq_chip_pm_get() and then call pm_runtime_resume_and_get()
if (IS_ENABLED(CONFIG_PM) && sfp->gc.irq.domain->pm_dev).
pm_runtime_resume_and_get() will fail if the runtime pm of pinctrl device
is disabled.

As we expect the pinctrl driver can be always working and never suspend
during runtime, unset sfp->gc.irq.domain->pm_dev to make sure
pm_runtime_resume_and_get() won't be called when setting irq.

Fixes: 447976ab ("pinctrl: starfive: Add StarFive JH7110 sys controller driver")
Signed-off-by: Hal Feng <hal.feng@starfivetech.com>
Link: https://lore.kernel.org/r/20230905122105.117000-2-hal.feng@starfivetech.comSigned-off-by: Linus Walleij <linus.walleij@linaro.org>