Bob Nystrom
184126372c
When the lexer encounters an invalid character, it now explicitly sets the current token's type to TOKEN_ERROR and its length to 0. Previously, the token retained the previous token's type, which could cause the compiler to get stuck in an infinite loop in certain code paths. Also improves the error message for non-ASCII invalid bytes by showing the raw hex value instead of attempting to display the character, since the lexer operates on raw bytes without UTF-8 decoding. Adds regression test for issue #428 that was crashing the compiler with an out-of-bounds memory access due to this loop.
This contains the Wren source code. It is organized like so:
-
optional: the Wren and C source code for the optional modules. These are built in to the VM and can be used even when you embed the VM in your own application. But they are also optional and can be compiled out by setting defines. -
cli: the source code for the command line interface. This is a custom executable that embeds the VM in itself. Code here handles reading command-line, running the REPL, loading modules from disc, etc. -
include: the public header directory for the VM. If you are embedding the VM in your own application, you will add this to your include path. -
module: the source code for the built-in modules that come with the CLI. These modules are written in a mixture of C and Wren and generally use libuv to implement their underlying functionality. -
vm: the source code for the Wren VM itself. Unlike code incliandmodule, this has no dependencies on libuv. If you are embedding the VM in your own application from source, you will compile the files here into your app.