feat: remove semicolon token from lexer and treat it as invalid syntax

Remove the `;` case from the `nextToken` function in `wren_compiler.c` so that semicolons are no longer lexed as `TOKEN_LINE`. Update the documentation in `doc/site/syntax.markdown` to describe newlines as the sole statement separator, and adjust all test files (`test/limit/many_constants.wren`, `test/limit/many_globals.wren`, `test/limit/too_many_constants.wren`, `test/semicolon.wren`) to reflect that semicolons now produce an error instead of being treated as line breaks.
This commit is contained in:
Bob Nystrom
2015-01-03 18:02:45 +00:00
parent 3b63f25690
commit 1951059c8f
6 changed files with 196557 additions and 24585 deletions
-4
View File
@@ -705,10 +705,6 @@ static void nextToken(Parser* parser)
case ']': makeToken(parser, TOKEN_RIGHT_BRACKET); return;
case '{': makeToken(parser, TOKEN_LEFT_BRACE); return;
case '}': makeToken(parser, TOKEN_RIGHT_BRACE); return;
// TODO: Should treat ";" differently from "\n". An elided newline
// makes sense, but an elided explicit ";" is weird. This should not be
// valid: a + ; b
case ';': makeToken(parser, TOKEN_LINE); return;
case ':': makeToken(parser, TOKEN_COLON); return;
case '.':
if (peekChar(parser) == '.')