Presently the port defs are lamely copied around between competing data
structures (one platform facing, the other driver internal). As we pretty
much require all of the data from the platform facing structure within
the driver too, simply nest the pointer directly and kill off the
duplication.
Signed-off-by: Paul Mundt <lethal@linux-sh.org>