Commit Graph
2 Commits
Author SHA1 Message Date
Yury Korchemkin c769f93833 fix: treat carriage return as whitespace instead of line token in lexer 2015-01-06 02:33:49 +00:00
Yury Korchemkin a137ea8ccd feat: add support for carriage return as line terminator in lexer
- Open source files in binary mode to prevent OS-level newline translation
- Treat '\r' character as equivalent to '\n' in the tokenizer
- Handle '\r' followed by '\n' as a single line break via twoCharToken
2015-01-04 22:25:17 +00:00