Remove the connector as well as all the links and only connect the
PTN5150 with xHCI controller. This is sufficient to implement the
role switching.

Furthermore, this makes resume work without hanging. Without this
patch, the platform would hang on resume of 'connector'.
Signed-off-by: Marek Vasut <marex@denx.de>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>