The old code used an extra function call and switch to inspect the
current token and determine the new state of curio.nlsemi. However,
the lexer knows the token w/o the need of an extra test and thus
can set curio.nlsemi directly:

- removed need for extra function call in next
- renamed _yylex to next
- set nlsemi at the point a token is identified
- moved nlsemi from curio to lexer - it's really part of the lexer state

This change makes the lexer call sequence less convoluted and should
also speed up the lexing a bit.

Change-Id: Iaf2683081f04231cb62c94e1400d455f98f6f82a
Reviewed-on: https://go-review.googlesource.com/19765Reviewed-by: Matthew Dempsky <mdempsky@google.com>