tty: serial - fix various misuses/mishandlings of port->tty

Make it robust against hang up events. In most cases we can do this simply
by passing the right things in the first place.
Signed-off-by: Alan Cox <alan@linux.intel.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>