When dynamically unloading overlays, it is important that freed pins are
restored to being inputs to prevent functions from being enabled in
multiple places at once.

Cc: Stefan Wahren <stefan.wahren@i2se.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Acked-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>