This allows to remove the .remove() callback, and all functions and data
it needed for its own bookkeeping.
Suggested-by: Laxman Dewangan <ldewangan@nvidia.com>
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Linus Walleij <linus.walleij@linaro.org>