This takes a reference count when fbdev adds the connector,
and drops it when it removes the connector.

It also drops the now unneeded code to find connectors
and remove the from the modeset as they are reference counted.

v2: drop references when removing all connectors at end.
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>