Commit Graph

122 Commits

Author SHA1 Message Date
Bob Nystrom
ee239f2bb6 feat: add undefined sentinel value and implicit global declaration for forward references
Add a new singleton value `VAL_UNDEFINED` to represent globals that have been implicitly declared via forward reference but not yet explicitly defined. Introduce `wrenDeclareGlobal()` to create such entries in the global table, and modify `wrenDefineGlobal()` to check for and resolve undefined slots. Update the compiler's `findUpvalue()` to stop closure capture at method boundaries, preventing methods from closing over local variables. Refactor `wrenSymbolTableAdd()` to always add symbols without duplicate checking, moving that logic to callers. Adjust error messages in the compiler to handle EOF tokens and improve newline formatting. Fix the delta_blue benchmark by renaming the global `planner` variable to `ThePlanner` to avoid shadowing the forward-declared function.
2015-01-15 07:08:34 +00:00
Bob Nystrom
72291f8b74 feat: implicitly declare nonlocal names as globals for forward references
Add support for forward references to top-level variables by implicitly
declaring unresolved capitalized names as globals during compilation.
If a real definition is not found later, a compile-time error is raised.
This enables mutual recursion at the top level and fixes issues #101 and #106.

Introduce an `UNDEFINED` singleton value to mark implicitly declared globals
that have not yet been explicitly defined. Update `wrenDeclareGlobal` to add
such entries, and modify `wrenDefineGlobal` to check for and replace undefined
placeholders. Adjust the symbol table API to allow duplicate additions for
implicit declarations. Add test cases for mutual recursion, forward references
in functions and methods, and undefined variable errors.
2015-01-15 07:08:25 +00:00
Bob Nystrom
82ef82cdb9 fix: prevent redundant keyword checks after first match in readName
Replace sequential `if` statements with `else if` chains in `readName` so that once a keyword matches (e.g. "break"), the remaining keyword checks are skipped, fixing issue #110 where later keywords could incorrectly override an already-matched token type.
2015-01-14 14:46:02 +00:00
Bob Nystrom
54ef9bbc73 feat: stop closure upvalue search at method boundaries and treat capitalized names as globals
Modify `findUpvalue` in the compiler to return -1 when the enclosing function is a method, preventing closures from capturing outer local variables. Add a new `nonlocal` variable resolution path that treats capitalized identifiers as global variables rather than method receivers. Update existing test files to use capitalized global names and add new test cases for nonlocal assignment, duplicate nonlocal declarations, block scoping, and method-local variable shadowing.
2015-01-14 05:36:42 +00:00
Bob Nystrom
a2c7499e0a fix: add tab character to whitespace skipping in nextToken parser
The nextToken function in wren_compiler.c previously only skipped spaces and carriage returns when advancing past whitespace, causing tab characters to be treated as non-whitespace tokens. This change adds '\t' to the whitespace check in both the initial case statement and the peekChar loop, ensuring tabs are properly skipped like other whitespace characters. A new test file test/whitespace.wren is added to verify correct handling of tabs in various indentation patterns and inline code positions.
2015-01-12 04:01:30 +00:00
Bob Nystrom
cd6d6c1f91 style: reformat while-loop body in nextToken to use braces 2015-01-12 03:52:59 +00:00
Bob Nystrom
34d3a8edb7 fix: open source files in binary mode and skip CR in whitespace
Change `fopen` mode from `"r"` to `"rb"` in `readFile()` to prevent CRLF-to-LF conversion on Windows, and add `'\r'` as a skipped whitespace character in `nextToken()` to handle carriage returns embedded in source files. This fixes parsing errors when Wren source files use Windows-style line endings.
2015-01-12 03:52:08 +00:00
Bob Nystrom
4e461f0b0c chore: clarify comment for super outside method error handling path 2015-01-12 03:44:35 +00:00
Kyle Marek-Spartz
1175d93efa fix: allow compiler to continue after super used outside method body
Previously, when `super` was used at the top level or outside any method, the compiler would error but still attempt to read `enclosingClass->methodLength` and `enclosingClass->methodName` from a NULL pointer, leading to undefined behavior. This restructures the error handling in `super_()` to always call `loadThis()` and guard the unnamed super call path with a NULL check on `enclosingClass`, setting an empty method signature when no enclosing class exists. Two additional test cases for `super.foo("bar")` and `super("foo")` are added to verify the compiler continues past the error without crashing.
2015-01-11 23:43:51 +00:00
Kyle Marek-Spartz
dd0760a54c fix: guard super() compilation against null enclosingClass to prevent segfault
When `super` is called at top-level with no arguments, `enclosingClass` is null, causing a segfault when trying to access its method name. This fix wraps the unnamed super call logic inside an else branch that only executes when `enclosingClass` is non-null, and adds a test case for the unnamed `super` call at top-level.
2015-01-09 21:14:06 +00:00
Evan Shaw
0526cee655 fix: stop lexing negative numbers as single tokens to fix parsing of subtraction
Previously, the lexer greedily consumed a `-` followed by a digit as a single negative number token, causing "1 -1" to be lexed as two number tokens instead of a minus operator and a positive literal. This broke parsing of subtraction expressions without spaces.

Now `-` always emits a TOKEN_MINUS, and the parser handles unary negation. This is a breaking change: `-16.sqrt` is now parsed as `-(16.sqrt)` instead of `(-16).sqrt`, requiring parentheses for negative method receivers. All number method tests (abs, ceil, floor, sqrt, toString) are updated to use explicit parentheses for negative receivers.
2015-01-09 07:06:50 +00:00
Yury Korchemkin
c769f93833 fix: treat carriage return as whitespace instead of line token in lexer 2015-01-06 02:33:49 +00:00
Yury Korchemkin
a137ea8ccd feat: add support for carriage return as line terminator in lexer
- Open source files in binary mode to prevent OS-level newline translation
- Treat '\r' character as equivalent to '\n' in the tokenizer
- Handle '\r' followed by '\n' as a single line break via twoCharToken
2015-01-04 22:25:17 +00:00
Bob Nystrom
b61e55d4c9 fix: skip duplicate lexer error reporting in compiler error handler
The `error()` function in the compiler now returns early when the offending token
is of type `TOKEN_ERROR`, preventing the lexer's already-reported error from
being emitted a second time to stderr. This eliminates redundant error messages
for malformed tokens.
2015-01-04 21:10:46 +00:00
Bob Nystrom
96d893a637 feat: ignore shebang lines starting with #! on first line of source
Add special handling in the lexer's nextToken function to detect a '#' character followed by '!' on the first line (currentLine == 1) and skip the entire line as a comment. If the shebang appears on any other line, or if a '#' is not followed by '!', emit a lex error for invalid character. Include test cases for valid shebang execution, shebang at EOF, shebang on a non-first line (expected error), and an invalid shebang pattern (expected error).
2015-01-04 21:02:42 +00:00
Bob Nystrom
ed31544ead fix: handle empty and comment-only files by removing early skip in repl and compiler
Remove the empty-line check in runRepl that skipped interpretation for blank input, and refactor the main compilation loop in wrenCompile to use a while loop with TOKEN_EOF matching instead of a for loop with redundant EOF break. This allows the compiler to process files containing only comments or whitespace without prematurely terminating, fixing issue #57. Add test fixtures for comment-only source files.
2015-01-04 21:00:53 +00:00
mauvm
6a69382275 refactor: inline unmatchedChar logic into nextToken and add shebang error test 2015-01-04 18:56:16 +00:00
mauvm
685f791fb0 fix: remove duplicate comment in unmatchedChar function in wren_compiler.c 2015-01-04 07:52:40 +00:00
mauvm
9828fafee7 feat: ignore shebang lines starting with #! on first line in compiler lexer 2015-01-03 19:53:12 +00:00
Bob Nystrom
1951059c8f feat: remove semicolon token from lexer and treat it as invalid syntax
Remove the `;` case from the `nextToken` function in `wren_compiler.c` so that semicolons are no longer lexed as `TOKEN_LINE`. Update the documentation in `doc/site/syntax.markdown` to describe newlines as the sole statement separator, and adjust all test files (`test/limit/many_constants.wren`, `test/limit/many_globals.wren`, `test/limit/too_many_constants.wren`, `test/semicolon.wren`) to reflect that semicolons now produce an error instead of being treated as line breaks.
2015-01-03 18:02:45 +00:00
Aleksander
d67036b95d fix: extend whitespace skipping in nextToken to handle tabs and add test coverage 2015-01-03 13:57:34 +00:00
Aleksander
4236d1e2de fix: treat tab character as valid whitespace in tokenizer
Add explicit case for '\t' alongside space character in the nextToken function's whitespace-skipping logic. Previously, only spaces were consumed during whitespace skipping, causing tab characters to be incorrectly treated as unexpected tokens. The fix ensures tabs are properly skipped by first matching the tab case and then falling through to the existing space-handling loop.
2015-01-02 22:22:09 +00:00
Bob Nystrom
520e841044 feat: add specialized LOAD_LOCAL_0..8 instructions to skip arg byte decode
Introduce nine new bytecode instructions (CODE_LOAD_LOCAL_0 through CODE_LOAD_LOCAL_8) that directly encode the local slot number in the opcode itself, eliminating the need to read and decode a separate argument byte for the most frequently accessed local variables. This optimization targets the hottest path in the interpreter, as LOAD_LOCAL is the most executed instruction across benchmarks (e.g., 3.75M times in delta_blue). The compiler's `loadLocal` helper emits the specialized opcode when the slot index is ≤8, falling back to the generic `CODE_LOAD_LOCAL` with an argument byte otherwise. The VM dispatch table and debug printer are updated accordingly. Benchmarks show consistent speedups: fib +7.3%, method_call +5.9%, binary_trees +1.9%, for +2.8%, delta_blue +1.0%.
2014-12-07 03:59:11 +00:00
Bob Nystrom
9b498b4638 fix: ensure all switch and unreachable paths return a value in release builds
In `getNumArguments`, add a `return 0` after the `UNREACHABLE()` default case to prevent
undefined behavior when assertions are disabled. In `wrenGetClass`, reorder the
`UNREACHABLE()` macro before the `return NULL` so the unreachable annotation is
always emitted. In `runInterpreter`, replace the `ASSERT(0, ...)` with an explicit
`UNREACHABLE()` and `return false` to guarantee a return value on all control paths
when the interpreter exits unexpectedly.
2014-12-02 05:36:06 +00:00
Bob Nystrom
3266491cf5 fix: correct break statement instruction size from 2 to 3 in endLoop to prevent code walker from misinterpreting arguments 2014-11-27 02:33:54 +00:00
Bob Nystrom
d9a42f21ba fix: add missing -lm link flag and fix GCC warnings for getline and switch coverage
Add -lm flag to both debug and release linker commands in Makefile to resolve
undefined math library references. Define _GNU_SOURCE in main.c to make getline()
available under GCC. Add default cases to switch statements in runFile and
getNumArguments to suppress -Wswitch warnings. Fix u_int8_t to uint8_t type in
wrenNewFunction declaration and definition for POSIX compliance.
2014-11-26 22:03:05 +00:00
Bob Nystrom
8abbc4197c feat: add MAX_VARIABLE_NAME limit and improve field overflow error message
Add a 64-character maximum identifier length with compile-time error reporting, and enhance the runtime field overflow error to include the class name and remove the TODO comment. Update test expectations for both limit scenarios.
2014-04-22 14:06:26 +00:00
Bob Nystrom
e665d4a9d3 feat: add closure test for shadowing closed-over variable with local
Add a new test case `shadow_closure_with_local.wren` that verifies a closed-over
variable can be shadowed by a local variable in an inner scope within the same
closure. The test ensures that the outer closure variable retains its value
outside the inner scope after the shadowing local goes out of scope.

Also remove an incomplete TODO comment from `reuse_closure_slot.wren` and add
a detailed TODO comment in `wren_compiler.c` discussing the design trade-off
between treating capitalized names as globals versus implicit `this` calls
when walking the scope chain.
2014-04-20 00:48:06 +00:00
Bob Nystrom
29f4b8167c feat: add zero-arg IO.print overload and remove newline skipping in parameter parsing
Add a static `print` method with no arguments that writes only a newline, enabling blank-line output. Remove the `ignoreNewlines` calls before closing parentheses in parameter lists and method calls, tightening syntax rules. Add a `strlen` TODO comment in symbol table lookup for future optimization. Move the `Foo.toString` test from `test/io/write.wren` to `test/io/print.wren` and delete the old write test file.
2014-04-19 18:44:51 +00:00
Bob Nystrom
fca506fdf1 feat: require parentheses around operator and setter method parameters in Wren
Enforce syntactic requirement for parentheses in operator method signatures (e.g., `+(other)` instead of `+ other`) and setter definitions (e.g., `bar=(value)` instead of `bar = value`). Update the compiler's `infixSignature`, `mixedSignature`, and `namedSignature` functions to consume explicit `(` and `)` tokens around parameter names. Migrate all built-in core library operator definitions and test fixtures to the new parenthesized syntax.
2014-04-09 00:54:37 +00:00
Bob Nystrom
2aaf6e5489 chore: rename debug and feature flags to consistent WREN_DEBUG_ and WREN_USE_ prefixes
Rename boolean defines from `true`/`false` to `1`/`0` for consistency, prefix debug flags with `WREN_DEBUG_`, add `WREN_USE_LIB_IO` flag to guard IO library loading, and wrap IO module includes and function declarations in conditional compilation blocks.
2014-04-07 14:45:09 +00:00
Bob Nystrom
e1cd942656 refactor: rename SymbolTable fields and flatten struct to StringBuffer alias
Replace the nested `SymbolTable` struct (containing a `StringBuffer names` field) with a direct `typedef StringBuffer SymbolTable`, updating all internal accesses from `symbols->names.data` to `symbols->data` and `symbols->names.count` to `symbols->count`. Rename `vm->methods` to `vm->methodNames` and `vm->globalSymbols` to `vm->globalNames` across the codebase, including debug printing, compiler symbol resolution, core initialization, and VM interpreter error messages. Remove the stale TODO comment about error codes in `wren.h` and adjust the `instantiate` method name to include a leading space to prevent direct user invocation.
2014-04-07 01:59:53 +00:00
Bob Nystrom
e7d2684d30 feat: replace fixed-size global array with growable ValueBuffer to support more than 256 globals
Replace the static `Value globals[MAX_GLOBALS]` array in WrenVM with a dynamic `ValueBuffer` that grows on demand, removing the 256-global limit. Introduce `wrenDefineGlobal()` helper that adds a named global to the symbol table and stores its value in the buffer, returning -2 when the 65536 limit is reached. Update all global access sites (compiler emit, runtime interpreter, GC marking, debug disassembly, core class definition) to use the buffer's data pointer and short (16-bit) opcode arguments instead of byte-sized ones. Add a regression test `many_globals.wren` that defines 8200 globals to verify the new capacity.
2014-04-06 23:16:15 +00:00
Bob Nystrom
4c4f32b5b2 feat: validate argument type in Fn.new and add runtime error test
Add type checking to the `fn_new` native function to ensure the argument
passed to `Fn.new` is either a function or closure. Previously, passing
an invalid type like an integer would silently succeed. Now it returns
a clear runtime error message: "Argument must be a function."

Also remove two stale TODO comments about newline handling in the
compiler that were no longer relevant to current parsing logic.
2014-04-05 01:13:13 +00:00
Bob Nystrom
093da7befb refactor: move newline handling from lexer into parser grammar rules
This change shifts newline tokenization from the lexer's readRawToken into the parser's nextToken, allowing the grammar to selectively ignore newlines in specific contexts (e.g., after operators like "|", "is", "for", "if", "while") while treating them as errors in others (e.g., after "var", "new", "class", "static", before commas in lists). The lexer now emits TOKEN_LINE for semicolons and newlines uniformly, and the parser decides which to elide. Test files are updated to reflect the new error expectations and to remove tests for newlines that are now grammatically allowed.
2014-04-04 14:51:18 +00:00
Bob Nystrom
a1365c4183 feat: unify block, function, and method body parsing to support single-expression implicit returns
Refactor `finishBlock` in `wren_compiler.c` to detect single-expression bodies (no newline after `{`) and leave a value on the stack, enabling implicit returns for methods and functions. Add `finishBody` wrapper to handle constructor flag. Update all benchmark and test `.wren` files to remove explicit `return` keywords from getters and single-expression methods, relying on the new implicit return behavior. Remove old `return_null_if_brace.wren` test, add `newline_body.wren`, `newline_in_expression_block.wren`, and `no_newline_before_close.wren` tests for edge cases.
2014-04-03 14:48:19 +00:00
Bob Nystrom
de2650b0ba fix: remove deprecated TOKEN_FN and old function syntax from wren compiler
Remove the TOKEN_FN enum value, its keyword mapping in readName, and the
associated function syntax handling from the tokenizer. Also clean up
unnecessary operator tokens from the newline-skipping switch in nextToken
to simplify the token processing logic.
2014-04-03 03:22:36 +00:00
Bob Nystrom
e20da2f070 feat: migrate all function literals from fn syntax to new Fn block syntax
Replace all occurrences of the old `fn` function literal syntax with the new `new Fn {|params| body}` block syntax across benchmark, test, and source files. Update the Wren compiler to handle the pipe operator as a block argument delimiter by removing `TOKEN_PIPE` from the newline-swallowing token list and adding explicit newline matching in `infixOp`. Adjust test expectations in `conditional/precedence.wren` to remove `fn`-based expressions and update `test/assignment/local.wren` to use the new syntax.
2014-04-03 02:41:53 +00:00
Bob Nystrom
81f20caf33 feat: replace CODE_NEW instruction with instantiate method call for class and Fn
Refactor the 'new' operator compilation to emit a method call to 'instantiate' on the class instead of using a dedicated bytecode instruction. This allows built-in types like Fn to override instantiation behavior, enabling `new Fn { ... }` to return the block argument directly. Remove the CODE_NEW opcode from the VM, compiler, and debug printer, and add native methods `class_instantiate`, `fn_instantiate`, `fn_new`, and `object_instantiate` to handle the new dispatch. Update test files to reflect the renamed `Function` class to `Fn`.
2014-04-03 00:42:30 +00:00
Bob Nystrom
c606e93516 feat: convert fiber tests to use block arguments instead of anonymous function literals
Update all fiber test files to replace `Fiber.create(fn { ... })` with `Fiber.create { ... }` block syntax, and add compiler support for parsing block bodies without requiring a newline after the opening brace. The compiler changes include removing `TOKEN_LEFT_BRACE` from the newline-discarding token list and adding `match(compiler, TOKEN_LINE)` calls in `finishBlock`, `methodCall`, `function`, and `block` to handle empty block bodies and inline expressions.
2014-04-03 00:31:58 +00:00
Bob Nystrom
5a017d17de feat: add block argument parsing with pipe-delimited parameters in method calls 2014-04-03 00:26:39 +00:00
Kyle Marek-Spartz
4f4251882d feat: enable custom bitwise operators | and & in wren compiler and add test
Add INFIX_OPERATOR entries for TOKEN_PIPE and TOKEN_AMP with PREC_BITWISE precedence in the compiler's grammar rules, replacing the previous UNUSED entries. This allows user-defined classes to implement custom `|` and `&` operator methods.

Include a new test file `test/custom_operators/bitwise.wren` that defines Amp and Pipe classes with their respective operator methods to validate the feature works correctly.
2014-02-16 18:01:40 +00:00
Bob Nystrom
73b92ca7f3 feat: add conditional operator and fn toString to wren compiler and core
Add TOKEN_QUESTION token type and implement conditional() parsing function in wren_compiler.c to support the ternary conditional operator (condition ? then : else). Register fn_toString native in wren_core.c to return "<fn>" string for function objects. Include comprehensive test suite covering precedence, short-circuit evaluation, and error cases for missing tokens.
2014-02-15 19:36:01 +00:00
Kyle Marek-Spartz
632cbf31b7 fix: correct typo in comment and remove redundant struct tag in Token definition 2014-02-15 06:15:51 +00:00
Bob Nystrom
5f505c2a02 fix: allow class body to end without newline after last method definition
In classDefinition in wren_compiler.c, add a check for TOKEN_RIGHT_BRACE before
requiring a newline after the last method definition, so that a closing brace
immediately following a method is accepted. Update test/class/syntax.wren with
a test case for this syntax.
2014-02-15 04:15:49 +00:00
Bob Nystrom
e3b3fef0fb feat: make this the implicit receiver for method calls inside class bodies
In the compiler, when a bare name is encountered inside a class definition and it does not resolve to a local variable or global, emit an implicit `this.` load before the call, enabling getter, setter, and method calls without explicit receiver. Update all benchmark, builtin, and test files to remove redundant `this.` qualifiers, and add comprehensive test suites for implicit receiver behavior across instance, inherited, static, nested, and shadowing scenarios.
2014-02-13 01:33:35 +00:00
Bob Nystrom
86015f4e48 feat: add numParams tracking and arity-checked fn.call variants for extra/missing args
Add `numParams` field to `ObjFn` and `Compiler` structs to track expected parameter count. Refactor `parameterList` to return count and store it in compiler. Modify `endCompiler` to pass `numParams` to `wrenNewFunction`. Replace single `fn_call` native with 17 arity-specific `fn_call0` through `fn_call16` natives that validate argument count against `fn->numParams` via `callFunction` helper, returning error on missing args. Remove stale TODO comment in VM interpreter. Add test files `call_extra_arguments.wren` and `call_missing_arguments.wren`.
2014-02-04 17:34:05 +00:00
Bob Nystrom
19e362f977 fix: add lexError calls for invalid string literals and unterminated strings in wren_compiler.c
Replace TODO comments with proper error reporting for unterminated strings, incomplete Unicode escapes, and invalid escape sequences. Add five new test files covering each error case.
2014-02-02 18:31:46 +00:00
Bob Nystrom
ea2ec33ce0 refactor: move class name from CODE_CLASS argument to stack in compiler and VM
In the compiler's classDefinition function, the class name constant is now emitted as a separate CODE_CONSTANT instruction before CODE_CLASS, instead of being passed as a short argument to CODE_CLASS. The VM's CLASS case handler is updated to pop the class name string from the stack using PEEK2() instead of reading it from the constant table via READ_SHORT(). The comment in wren_vm.h is revised to document that the class name is now on the stack below the superclass.
2014-01-29 15:47:09 +00:00
Bob Nystrom
e83296ea53 feat: add class name storage and expose via name method on class objects
Store the class name string in `ObjClass` during compilation and class creation, and expose it through a new `class_name` native method. Also update `object_toString` to return the class name for class instances and "instance of <name>" for regular instances.
2014-01-28 23:31:11 +00:00