In case some drivers are unloading, they can remove lookup tables which
they had registered during their load time to avoid redundant entries if
loaded again.

CC: Samuel Ortiz <sameo@linux.intel.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Alexandre Courbot <gnurou@gmail.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Signed-off-by: Shobhit Kumar <shobhit.kumar@intel.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com>