2265 lines
63 KiB
C
Raw Normal View History

feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
#include <stdarg.h>
#include <stdbool.h>
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
#include <stdio.h>
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
#include <stdlib.h>
#include <string.h>
chore: rename source files to wren_ prefix and add module-level documentation headers
2013-11-28 17:11:50 +01:00
#include "wren_common.h"
#include "wren_compiler.h"
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
chore: rename source files to wren_ prefix and add module-level documentation headers
2013-11-28 17:11:50 +01:00
// This is written in bottom-up order, so the tokenization comes first, then
// parsing/code generation. This minimizes the number of explicit forward
// declarations needed.
feat: raise max method parameters from 10 to 16 with proper error handling Add MAX_PARAMETERS constant set to 16 and extend VM bytecode instructions CODE_CALL_11 through CODE_CALL_16 to support the new limit. Introduce compile-time error messages when parameter or argument count exceeds 16 in function definitions, method declarations, subscript calls, and regular method calls. Add new native fiber call stubs fn_call9 through fn_call16 and register them in the core initialization. Include comprehensive test coverage for 9-16 parameter functions and methods, plus error tests for exceeding the limit in all contexts.
2013-11-30 00:08:27 +01:00
// The maximum number of arguments that can be passed to a method. Note that
// this limtation is hardcoded in other places in the VM, in particular, the
// `CODE_CALL_XX` instructions assume a certain maximum number.
#define MAX_PARAMETERS (16)
feat: enforce MAX_LOCALS limit of 256 in declareVariable and add test cases for boundary conditions Add a compile-time check in declareVariable() that emits an error when the number of simultaneous local variables reaches MAX_LOCALS (256), replacing the previous placeholder TODO. The limit is derived from the bytecode encoding constraint where CODE_LOAD_LOCAL and CODE_STORE_LOCAL use a single argument byte. Also move the MAX_LOCALS definition from a TODO comment to a proper #define, increasing the value from 255 to 256. Include four new test files: many_locals.wren (exactly 256 simultaneous locals), many_nonsimultaneous_locals.wren (more than 256 locals across nested scopes), too_many_locals.wren (257 simultaneous locals triggering the error), and too_many_locals_nested.wren (exceeding the limit across nested blocks).
2013-12-01 03:51:27 +01:00
// The maximum number of local (i.e. non-global) variables that can be declared
// in a single function, method, or chunk of top level code. This is the
// maximum number of variables in scope at one time, and spans block scopes.
//
// Note that this limitation is also explicit in the bytecode. Since
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// `CODE_LOAD_LOCAL` and `CODE_STORE_LOCAL` use a single argument byte to
feat: enforce MAX_LOCALS limit of 256 in declareVariable and add test cases for boundary conditions Add a compile-time check in declareVariable() that emits an error when the number of simultaneous local variables reaches MAX_LOCALS (256), replacing the previous placeholder TODO. The limit is derived from the bytecode encoding constraint where CODE_LOAD_LOCAL and CODE_STORE_LOCAL use a single argument byte. Also move the MAX_LOCALS definition from a TODO comment to a proper #define, increasing the value from 255 to 256. Include four new test files: many_locals.wren (exactly 256 simultaneous locals), many_nonsimultaneous_locals.wren (more than 256 locals across nested scopes), too_many_locals.wren (257 simultaneous locals triggering the error), and too_many_locals_nested.wren (exceeding the limit across nested blocks).
2013-12-01 03:51:27 +01:00
// identify the local, only 256 can be in scope at one time.
#define MAX_LOCALS (256)
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// The maximum number of upvalues (i.e. variables from enclosing functions)
// that a function can close over.
#define MAX_UPVALUES (256)
feat: add MAX_CONSTANTS limit and overflow check with deduplication in wren_compiler.c
2013-12-24 00:08:47 +01:00
// The maximum number of distinct constants that a function can contain. This
// value is explicit in the bytecode since `CODE_CONSTANT` only takes a single
// argument.
#define MAX_CONSTANTS (256)
feat: enforce maximum method name length of 64 characters in compiler Add MAX_METHOD_NAME constant and validation in copyName() to prevent method names exceeding 64 characters. When a method name is too long, the compiler now reports an error and truncates the name to the maximum allowed length. Also update MAX_METHOD_SIGNATURE to be derived from MAX_METHOD_NAME and MAX_PARAMETERS. Add test cases for valid 64-character method names, names that exceed the limit, and a call to an overlong method name.
2013-12-07 05:09:43 +01:00
// The maximum name of a method, not including the signature. This is an
// arbitrary but enforced maximum just so we know how long the method name
// strings need to be in the parser.
#define MAX_METHOD_NAME (64)
// The maximum length of a method signature. This includes the name, and the
// extra spaces added to handle arity, and another byte to terminate the string.
#define MAX_METHOD_SIGNATURE (MAX_METHOD_NAME + MAX_PARAMETERS + 1)
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Get rid of this and use a growable buffer.
feat: enforce maximum method name length of 64 characters in compiler Add MAX_METHOD_NAME constant and validation in copyName() to prevent method names exceeding 64 characters. When a method name is too long, the compiler now reports an error and truncates the name to the maximum allowed length. Also update MAX_METHOD_SIGNATURE to be derived from MAX_METHOD_NAME and MAX_PARAMETERS. Add test cases for valid 64-character method names, names that exceed the limit, and a call to an overlong method name.
2013-12-07 05:09:43 +01:00
#define MAX_STRING (1024)
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
typedef enum
{
TOKEN_LEFT_PAREN,
TOKEN_RIGHT_PAREN,
TOKEN_LEFT_BRACKET,
TOKEN_RIGHT_BRACKET,
TOKEN_LEFT_BRACE,
TOKEN_RIGHT_BRACE,
TOKEN_COLON,
TOKEN_DOT,
TOKEN_COMMA,
TOKEN_STAR,
TOKEN_SLASH,
TOKEN_PERCENT,
TOKEN_PLUS,
TOKEN_MINUS,
TOKEN_PIPE,
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
TOKEN_PIPEPIPE,
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
TOKEN_AMP,
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
TOKEN_AMPAMP,
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
TOKEN_BANG,
feat: add bitwise NOT operator (~) with token, parser, and native implementation Add TOKEN_TILDE token type and lexer support for the '~' character in the compiler. Implement the bitwise NOT operation as a native method on the Num class, operating on 32-bit unsigned integers. Include a new OPERATOR macro for prefix/infix operator grammar rules and add comprehensive test cases covering positive, negative, floating-point, and boundary values.
2013-12-05 07:09:31 +01:00
TOKEN_TILDE,
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
TOKEN_EQ,
TOKEN_LT,
TOKEN_GT,
TOKEN_LTEQ,
TOKEN_GTEQ,
TOKEN_EQEQ,
TOKEN_BANGEQ,
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
TOKEN_BREAK,
feat: add class parsing and object type system with ObjBlock, ObjClass, and ObjNum structs Introduce token types TOKEN_CLASS and TOKEN_META for class declaration parsing in the compiler. Refactor the internal object model from a single Obj union to distinct ObjNum, ObjBlock, and ObjClass structs, replacing the old Block type with ObjBlock throughout the compiler, VM, and main entry point. Implement makeClass() and makeBlock() factory functions, update makeNum() to return ObjNum*, and add CODE_CLASS and CODE_DUP bytecodes to support class definition at compile time. Modify the symbol table and method dispatch to use a Method union (primitive or block) stored in ObjClass, and update the VM's callBlock signature and newVM initialization accordingly.
2013-10-24 07:50:04 +02:00
TOKEN_CLASS,
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
TOKEN_ELSE,
feat: add boolean literals and number comparison operators to compiler and VM Add TOKEN_TRUE/TOKEN_FALSE parsing with boolean() prefix rule emitting CODE_TRUE/CODE_FALSE bytecodes. Wire comparison operators <, >, <=, >= as infix rules with PREC_COMPARISON precedence. Implement num_lt, num_gt, num_lte, num_gte primitives returning bool values. Register boolClass with toString primitive and move class initialization into registerPrimitives(). Add OBJ_TRUE/OBJ_FALSE object types with makeBool() helper and AS_BOOL macro. Include test files for bool_toString and number_comparison.
2013-11-04 06:38:58 +01:00
TOKEN_FALSE,
feat: replace block syntax with fn keyword for function literals across compiler and tests Rename `block` parsing rule to `fn` by adding TOKEN_FN token and function prefix rule, removing the old block prefix from TOKEN_LEFT_BRACE. Update compiler.c to replace `block()` with `function()` and add fn equality primitives (fn_eqeq, fn_bangeq) in primitives.c. Migrate test files: rename block_call.wren to fn_call.wren, block_syntax.wren to fn_syntax.wren, and replace `{ ... }` with `fn { ... }` or `fn expr` syntax. Remove obsolete bool_equality.wren test.
2013-11-06 16:47:47 +01:00
TOKEN_FN,
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
TOKEN_FOR,
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
TOKEN_IF,
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
TOKEN_IN,
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
TOKEN_IS,
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
TOKEN_NEW,
feat: add null literal token and compiler support with keyword parsing and test cases Introduce TOKEN_NULL to the token enum and register it as a prefix parse rule that emits CODE_NULL. Implement the null() compiler function to emit the null constant. Extend readName() to recognize the "null" keyword and map it to TOKEN_NULL. Add test files verifying null output, and that null, true, and false cannot be used as variable names.
2013-11-06 03:22:22 +01:00
TOKEN_NULL,
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
TOKEN_RETURN,
feat: add static method support to class definitions with metaclass dispatch Add TOKEN_STATIC keyword to the lexer, replacing the unused TOKEN_META, and extend the method() compiler function with an isStatic parameter. When isStatic is true, emit a new CODE_METACLASS instruction before CODE_METHOD to push the class's metaclass onto the stack, enabling static method dispatch. Also fix a buffer overflow bug in string_plus by allocating one extra byte for the null terminator, and refactor object initialization across vm.c by introducing a shared initObj helper to eliminate repeated type/flag assignments.
2013-11-10 20:46:13 +01:00
TOKEN_STATIC,
feat: add TOKEN_SUPER enum and keyword recognition to wren compiler Add the TOKEN_SUPER token type to the token enum and register "super" as a reserved keyword in the readName function. Include TOKEN_SUPER in the nextToken skip-newlines switch and mark it as UNUSED in the grammar rules table, preparing the compiler for future super keyword support.
2013-12-11 17:02:17 +01:00
TOKEN_SUPER,
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
TOKEN_THIS,
feat: add boolean literals and number comparison operators to compiler and VM Add TOKEN_TRUE/TOKEN_FALSE parsing with boolean() prefix rule emitting CODE_TRUE/CODE_FALSE bytecodes. Wire comparison operators <, >, <=, >= as infix rules with PREC_COMPARISON precedence. Implement num_lt, num_gt, num_lte, num_gte primitives returning bool values. Register boolClass with toString primitive and move class initialization into registerPrimitives(). Add OBJ_TRUE/OBJ_FALSE object types with makeBool() helper and AS_BOOL macro. Include test files for bool_toString and number_comparison.
2013-11-04 06:38:58 +01:00
TOKEN_TRUE,
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
TOKEN_VAR,
feat: add while loop token, parser support, CODE_LOOP opcode, and patchJump helper Add TOKEN_WHILE to the token enum and keyword lookup in the compiler, enabling the parser to recognize 'while' statements. Introduce the CODE_LOOP opcode in the VM for backward jumps, and implement the patchJump helper to resolve forward jump offsets during compilation. Include comprehensive test cases for while loops covering single-expression bodies, block bodies, newline handling, and null result semantics.
2013-11-18 07:38:59 +01:00
TOKEN_WHILE,
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
TOKEN_FIELD,
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
TOKEN_NAME,
TOKEN_NUMBER,
TOKEN_STRING,
TOKEN_LINE,
TOKEN_ERROR,
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
TOKEN_EOF
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
} TokenType;
typedef struct Token_s
{
TokenType type;
feat: add compile error detection and line tracking to test runner and compiler Add support for testing compile errors by introducing `// expect error` annotations in test files. The test runner now validates that expected errors occur on specific lines and that the compiler exits with code 1 when errors are present. The compiler's error output is enhanced to include line numbers in the format `[Line N] Error`, and the `Token` struct gains a `line` field to track token positions. The main entry point returns exit code 1 when compilation fails, enabling the test runner to verify error exit codes.
2013-10-28 15:12:39 +01:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
// The beginning of the token, pointing directly into the source.
const char* start;
feat: add compile error detection and line tracking to test runner and compiler Add support for testing compile errors by introducing `// expect error` annotations in test files. The test runner now validates that expected errors occur on specific lines and that the compiler exits with code 1 when errors are present. The compiler's error output is enhanced to include line numbers in the format `[Line N] Error`, and the `Token` struct gains a `line` field to track token positions. The main entry point returns exit code 1 when compilation fails, enabling the test runner to verify error exit codes.
2013-10-28 15:12:39 +01:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
// The length of the token in characters.
int length;
feat: add compile error detection and line tracking to test runner and compiler Add support for testing compile errors by introducing `// expect error` annotations in test files. The test runner now validates that expected errors occur on specific lines and that the compiler exits with code 1 when errors are present. The compiler's error output is enhanced to include line numbers in the format `[Line N] Error`, and the `Token` struct gains a `line` field to track token positions. The main entry point returns exit code 1 when compilation fails, enabling the test runner to verify error exit codes.
2013-10-28 15:12:39 +01:00
// The 1-based line where the token appears.
int line;
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
} Token;
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
typedef struct
{
feat: add public C API header wren.h and refactor internal types to WrenVM prefix
2013-11-25 16:47:02 +01:00
WrenVM* vm;
feat: replace recursive descent parser with grammar-driven approach and add method call infrastructure Refactor compiler.c to remove recursive descent parsing infrastructure (InfixCompiler, CompileFn, precedence enums) and replace with a simpler grammar-driven approach using statement(), call(), primary(), and number() functions. Add VM struct to compiler.h and vm.h with symbol table support, including getSymbol() for method name resolution. Introduce CODE_CALL opcode in VM interpreter with basic call dispatch. Update main.c to use new VM lifecycle (newVM/freeVM) and pass VM through compile and interpret calls.
2013-10-23 22:51:41 +02:00
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
const char* source;
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
// The beginning of the currently-being-lexed token in [source].
const char* tokenStart;
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
// The current character being lexed in [source].
const char* currentChar;
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: add compile error detection and line tracking to test runner and compiler Add support for testing compile errors by introducing `// expect error` annotations in test files. The test runner now validates that expected errors occur on specific lines and that the compiler exits with code 1 when errors are present. The compiler's error output is enhanced to include line numbers in the format `[Line N] Error`, and the `Token` struct gains a `line` field to track token positions. The main entry point returns exit code 1 when compilation fails, enabling the test runner to verify error exit codes.
2013-10-28 15:12:39 +01:00
// The 1-based line number of [currentChar].
int currentLine;
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
// The most recently lexed token.
Token current;
// The most recently consumed/advanced token.
Token previous;
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// If subsequent newline tokens should be discarded.
bool skipNewlines;
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// If a syntax or compile error has occurred.
bool hasError;
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Dynamically allocate this.
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
// A buffer for the unescaped text of the current token if it's a string
// literal. Unlike the raw token, this will have escape sequences translated
// to their literal equivalent.
char currentString[MAX_STRING];
int currentStringLength;
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
} Parser;
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
typedef struct
feat: add block scope support with push/pop macros and scope tracking in compiler Introduce Scope struct and PUSH_SCOPE/POP_SCOPE macros to manage nested variable scopes during compilation. Update declareVariable to distinguish global declarations from local ones based on scope presence rather than parent compiler. Add truncateSymbolTable helper to clean up local symbols when exiting a scope. Include new test files verifying scoped variable isolation in if, while, and block constructs.
2013-11-18 18:19:03 +01:00
{
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// The name of the local variable. This points directly into the original
// source code string.
const char* name;
// The length of the local variable's name.
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
int length;
feat: add block scope support with push/pop macros and scope tracking in compiler Introduce Scope struct and PUSH_SCOPE/POP_SCOPE macros to manage nested variable scopes during compilation. Update declareVariable to distinguish global declarations from local ones based on scope presence rather than parent compiler. Add truncateSymbolTable helper to clean up local symbols when exiting a scope. Include new test files verifying scoped variable isolation in if, while, and block constructs.
2013-11-18 18:19:03 +01:00
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// The depth in the scope chain that this variable was declared at. Zero is
// the outermost scope--parameters for a method, or the first local block in
// top level code. One is the scope within that, etc.
int depth;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// If this local variable is being used as an upvalue.
bool isUpvalue;
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
} Local;
feat: add block scope support with push/pop macros and scope tracking in compiler Introduce Scope struct and PUSH_SCOPE/POP_SCOPE macros to manage nested variable scopes during compilation. Update declareVariable to distinguish global declarations from local ones based on scope presence rather than parent compiler. Add truncateSymbolTable helper to clean up local symbols when exiting a scope. Include new test files verifying scoped variable isolation in if, while, and block constructs.
2013-11-18 18:19:03 +01:00
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
typedef struct
{
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// True if this upvalue is capturing a local variable from the enclosing
// function. False if it's capturing an upvalue.
bool isLocal;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// The index of the local or upvalue being captured in the enclosing function.
int index;
} CompilerUpvalue;
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
typedef struct sCompiler
{
Parser* parser;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// The compiler for the function enclosing this one, or NULL if it's the
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
// top level.
struct sCompiler* parent;
refactor: rename ObjBlock to ObjFn and replace block_call with fn_call across compiler, vm, and primitives
2013-11-06 15:52:28 +01:00
// The function being compiled.
ObjFn* fn;
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
int numCodes;
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
// Symbol table for the fields of the nearest enclosing class, or NULL if not
// currently inside a class.
SymbolTable* fields;
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// The number of local variables currently in scope.
int numLocals;
// The current level of block scope nesting, where zero is no nesting. A -1
// here means top-level code is being compiled and there is no block scope
// in effect at all. Any variables declared will be global.
int scopeDepth;
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
// Index of the first instruction of the body of the innermost loop currently
// being compiled. Will be -1 if not currently inside a loop.
int loopBody;
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
// The name of the method this compiler is compiling, or NULL if this
// compiler is not for a method. Note that this is just the bare method name,
// and not its full signature.
const char* methodName;
// The length of the method name being compiled.
int methodLength;
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// The currently in scope local variables.
Local locals[MAX_LOCALS];
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// The upvalues that this function has captured from outer scopes. The count
// of them is stored in `fn->numUpvalues`.
CompilerUpvalue upvalues[MAX_UPVALUES];
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
} Compiler;
feat: add MAX_CONSTANTS limit and overflow check with deduplication in wren_compiler.c
2013-12-24 00:08:47 +01:00
// Outputs a compile or syntax error. This also marks the compilation as having
// an error, which ensures that the resulting code will be discarded and never
// run. This means that after calling lexError(), it's fine to generate whatever
// invalid bytecode you want since it won't be used.
static void lexError(Parser* parser, const char* format, ...)
{
parser->hasError = true;
fprintf(stderr, "[Line %d] Error: ", parser->currentLine);
va_list args;
va_start(args, format);
vfprintf(stderr, format, args);
va_end(args);
fprintf(stderr, "\n");
}
// Outputs a compile or syntax error. This also marks the compilation as having
// an error, which ensures that the resulting code will be discarded and never
// run. This means that after calling error(), it's fine to generate whatever
// invalid bytecode you want since it won't be used.
//
// You'll note that most places that call error() continue to parse and compile
// after that. That's so that we can try to find as many compilation errors in
// one pass as possible instead of just bailing at the first one.
static void error(Compiler* compiler, const char* format, ...)
{
compiler->parser->hasError = true;
Token* token = &compiler->parser->previous;
fprintf(stderr, "[Line %d] Error on ", token->line);
if (token->type == TOKEN_LINE)
{
// Don't print the newline itself since that looks wonky.
fprintf(stderr, "newline: ");
}
else
{
fprintf(stderr, "'%.*s': ", token->length, token->start);
}
va_list args;
va_start(args, format);
vfprintf(stderr, format, args);
va_end(args);
fprintf(stderr, "\n");
}
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
// Adds [constant] to the constant pool and returns its index.
static int addConstant(Compiler* compiler, Value constant)
{
feat: add MAX_CONSTANTS limit and overflow check with deduplication in wren_compiler.c
2013-12-24 00:08:47 +01:00
// See if an equivalent constant has already been added.
for (int i = 0; i < compiler->fn->numConstants; i++)
{
if (wrenValuesEqual(compiler->fn->constants[i], constant)) return i;
}
if (compiler->fn->numConstants < MAX_CONSTANTS)
{
compiler->fn->constants[compiler->fn->numConstants++] = constant;
}
else
{
error(compiler, "A function may only contain %d unique constants.",
MAX_CONSTANTS);
}
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
return compiler->fn->numConstants - 1;
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Initializes [compiler].
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
static int initCompiler(Compiler* compiler, Parser* parser, Compiler* parent,
const char* methodName, int methodLength)
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
compiler->parser = parser;
compiler->parent = parent;
compiler->numCodes = 0;
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
compiler->loopBody = -1;
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
compiler->methodName = methodName;
compiler->methodLength = methodLength;
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
if (parent == NULL)
{
// Compiling top-level code, so the initial scope is global.
compiler->scopeDepth = -1;
compiler->numLocals = 0;
}
else
{
feat: implement closure over "this" in methods and fix upvalue closing order Add support for closing over "this" in function closures defined inside methods by naming the receiver local variable "this" in method compilers. Fix upvalue closing to copy the actual value instead of the stack top, and reorder return handling to close upvalues before storing the result. Add test cases for closure over "this", nested closures, and nested classes.
2013-12-22 00:55:08 +01:00
// Declare a fake local variable for the receiver so that it's slot in the
// stack is taken. For methods, we call this "this", so that we can resolve
// references to that like a normal variable. For functions, they have no
// explicit "this". So we pick a bogus name. That way references to "this"
// inside a function will try to walk up the parent chain to find a method
// enclosing the function whose "this" we can close over.
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
compiler->numLocals = 1;
feat: implement closure over "this" in methods and fix upvalue closing order Add support for closing over "this" in function closures defined inside methods by naming the receiver local variable "this" in method compilers. Fix upvalue closing to copy the actual value instead of the stack top, and reorder return handling to close upvalues before storing the result. Add test cases for closure over "this", nested closures, and nested classes.
2013-12-22 00:55:08 +01:00
if (methodName != NULL)
{
compiler->locals[0].name = "this";
compiler->locals[0].length = 4;
}
else
{
compiler->locals[0].name = NULL;
compiler->locals[0].length = 0;
}
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
compiler->locals[0].depth = -1;
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
compiler->locals[0].isUpvalue = false;
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// The initial scope for function or method is a local scope.
compiler->scopeDepth = 0;
}
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
// Propagate the enclosing class downwards.
compiler->fields = parent != NULL ? parent->fields : NULL;
refactor: rename internal value creation functions to wren-prefixed public API names Move `allocate`, `initObj`, `newClass`, and related static helpers from `vm.c` into `value.c` with consistent `wren` prefix, and update all call sites in `compiler.c`, `primitives.c`, and `vm.c` to use the new names (`wrenNewFunction`, `wrenNewString`, `wrenNewClass`, `wrenNewInstance`, `wrenNewList`). Remove the old declarations from `vm.h` and add comprehensive documentation comments to `value.h` explaining the NaN-tagging representation and type system.
2013-11-28 05:00:03 +01:00
compiler->fn = wrenNewFunction(parser->vm);
feat: add block scope support with push/pop macros and scope tracking in compiler Introduce Scope struct and PUSH_SCOPE/POP_SCOPE macros to manage nested variable scopes during compilation. Update declareVariable to distinguish global declarations from local ones based on scope presence rather than parent compiler. Add truncateSymbolTable helper to clean up local symbols when exiting a scope. Include new test files verifying scoped variable isolation in if, while, and block constructs.
2013-11-18 18:19:03 +01:00
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
if (parent == NULL) return -1;
// Add the block to the constant table. Do this eagerly so it's reachable by
// the GC.
return addConstant(parent, OBJ_VAL(compiler->fn));
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Lexing ----------------------------------------------------------------------
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// Returns true if [c] is a valid (non-initial) identifier character.
static bool isName(char c)
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
{
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '_';
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// Returns true if [c] is a digit.
static bool isDigit(char c)
feat: implement class method definition and instance creation in compiler and VM Add method parsing and compilation in `statement()` for class bodies, including symbol interning and block constant emission. Introduce `addConstant()` helper in compiler.c. Extend VM with `CODE_METHOD` opcode, `ObjInstance` type, `makeInstance()` factory, metaclass support in `makeClass()`, and `getSymbolName()` for symbol table lookup. Update vm.h with new type enum, struct definitions, and function declarations.
2013-10-25 06:32:17 +02:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return c >= '0' && c <= '9';
feat: implement class method definition and instance creation in compiler and VM Add method parsing and compilation in `statement()` for class bodies, including symbol interning and block constant emission. Introduce `addConstant()` helper in compiler.c. Extend VM with `CODE_METHOD` opcode, `ObjInstance` type, `makeInstance()` factory, metaclass support in `makeClass()`, and `getSymbolName()` for symbol table lookup. Update vm.h with new type enum, struct definitions, and function declarations.
2013-10-25 06:32:17 +02:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Returns the current character the parser is sitting on.
static char peekChar(Parser* parser)
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
return *parser->currentChar;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
feat: implement floating-point number literal parsing and arithmetic operations Add support for floating-point number literals in the parser by detecting decimal points with trailing digits in `readNumber()`, and switch the compiler's `number()` function from `strtol` to `strtod` for proper double conversion. Update the `num_abs` primitive to use `fabs()` instead of manual negation. Extend test coverage across arithmetic, comparison, equality, modulus, and string conversion tests with floating-point cases, and add a new test for decimal point at end-of-file error detection.
2013-11-10 06:32:57 +01:00
// Returns the character after the current character.
static char peekNextChar(Parser* parser)
{
// If we're at the end of the source, don't read past it.
if (peekChar(parser) == '\0') return '\0';
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
return *(parser->currentChar + 1);
feat: implement floating-point number literal parsing and arithmetic operations Add support for floating-point number literals in the parser by detecting decimal points with trailing digits in `readNumber()`, and switch the compiler's `number()` function from `strtol` to `strtod` for proper double conversion. Update the `num_abs` primitive to use `fabs()` instead of manual negation. Extend test coverage across arithmetic, comparison, equality, modulus, and string conversion tests with floating-point cases, and add a new test for decimal point at end-of-file error detection.
2013-11-10 06:32:57 +01:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Advances the parser forward one character.
static char nextChar(Parser* parser)
{
char c = peekChar(parser);
parser->currentChar++;
fix: add lexError function and fix line counting for unterminated block comments Add a dedicated lexError function that marks compilation as failed and formats error messages consistently with a trailing newline. Move line counting from readRawToken into nextChar so that newlines are tracked during all lexer phases, including block comment scanning. Replace the TODO placeholder in skipBlockComment with a proper lexError call for unterminated block comments, and add test cases for unterminated block and nested block comments.
2013-12-23 23:51:06 +01:00
if (c == '\n') parser->currentLine++;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return c;
}
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Sets the parser's current token to the given [type] and current character
// range.
static void makeToken(Parser* parser, TokenType type)
{
parser->current.type = type;
parser->current.start = parser->tokenStart;
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
parser->current.length = (int)(parser->currentChar - parser->tokenStart);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
parser->current.line = parser->currentLine;
}
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
// If the current character is [c], then consumes it and makes a token of type
// [two]. Otherwise makes a token of type [one].
static void twoCharToken(Parser* parser, char c, TokenType two, TokenType one)
{
if (peekChar(parser) == c)
{
nextChar(parser);
makeToken(parser, two);
return;
}
makeToken(parser, one);
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Skips the rest of the current line.
static void skipLineComment(Parser* parser)
{
while (peekChar(parser) != '\n' && peekChar(parser) != '\0')
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
nextChar(parser);
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
}
}
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
// Skips the rest of a block comment.
static void skipBlockComment(Parser* parser)
{
nextChar(parser); // The opening "*".
int nesting = 1;
while (nesting > 0)
{
fix: add lexError function and fix line counting for unterminated block comments Add a dedicated lexError function that marks compilation as failed and formats error messages consistently with a trailing newline. Move line counting from readRawToken into nextChar so that newlines are tracked during all lexer phases, including block comment scanning. Replace the TODO placeholder in skipBlockComment with a proper lexError call for unterminated block comments, and add test cases for unterminated block and nested block comments.
2013-12-23 23:51:06 +01:00
if (peekChar(parser) == '\0')
{
lexError(parser, "Unterminated block comment.");
return;
}
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
if (peekChar(parser) == '/' && peekNextChar(parser) == '*')
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
{
nextChar(parser);
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
nextChar(parser);
nesting++;
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
continue;
}
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
if (peekChar(parser) == '*' && peekNextChar(parser) == '/')
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
{
nextChar(parser);
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
nextChar(parser);
nesting--;
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
continue;
}
// Regular comment character.
nextChar(parser);
}
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// Returns true if the current token's text matches [keyword].
static bool isKeyword(Parser* parser, const char* keyword)
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
size_t length = parser->currentChar - parser->tokenStart;
size_t keywordLength = strlen(keyword);
return length == keywordLength &&
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
strncmp(parser->tokenStart, keyword, length) == 0;
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Finishes lexing a number literal.
static void readNumber(Parser* parser)
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Hex, scientific, etc.
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
while (isDigit(peekChar(parser))) nextChar(parser);
feat: implement floating-point number literal parsing and arithmetic operations Add support for floating-point number literals in the parser by detecting decimal points with trailing digits in `readNumber()`, and switch the compiler's `number()` function from `strtol` to `strtod` for proper double conversion. Update the `num_abs` primitive to use `fabs()` instead of manual negation. Extend test coverage across arithmetic, comparison, equality, modulus, and string conversion tests with floating-point cases, and add a new test for decimal point at end-of-file error detection.
2013-11-10 06:32:57 +01:00
// See if it has a floating point. Make sure there is a digit after the "."
// so we don't get confused by method calls on number literals.
if (peekChar(parser) == '.' && isDigit(peekNextChar(parser)))
{
nextChar(parser);
while (isDigit(peekChar(parser))) nextChar(parser);
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
makeToken(parser, TOKEN_NUMBER);
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Finishes lexing an identifier. Handles reserved words.
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
static void readName(Parser* parser, TokenType type)
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
while (isName(peekChar(parser)) || isDigit(peekChar(parser)))
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
nextChar(parser);
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
}
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
if (isKeyword(parser, "break")) type = TOKEN_BREAK;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
if (isKeyword(parser, "class")) type = TOKEN_CLASS;
if (isKeyword(parser, "else")) type = TOKEN_ELSE;
if (isKeyword(parser, "false")) type = TOKEN_FALSE;
if (isKeyword(parser, "fn")) type = TOKEN_FN;
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
if (isKeyword(parser, "for")) type = TOKEN_FOR;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
if (isKeyword(parser, "if")) type = TOKEN_IF;
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
if (isKeyword(parser, "in")) type = TOKEN_IN;
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
if (isKeyword(parser, "is")) type = TOKEN_IS;
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
if (isKeyword(parser, "new")) type = TOKEN_NEW;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
if (isKeyword(parser, "null")) type = TOKEN_NULL;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (isKeyword(parser, "return")) type = TOKEN_RETURN;
feat: add static method support to class definitions with metaclass dispatch Add TOKEN_STATIC keyword to the lexer, replacing the unused TOKEN_META, and extend the method() compiler function with an isStatic parameter. When isStatic is true, emit a new CODE_METACLASS instruction before CODE_METHOD to push the class's metaclass onto the stack, enabling static method dispatch. Also fix a buffer overflow bug in string_plus by allocating one extra byte for the null terminator, and refactor object initialization across vm.c by introducing a shared initObj helper to eliminate repeated type/flag assignments.
2013-11-10 20:46:13 +01:00
if (isKeyword(parser, "static")) type = TOKEN_STATIC;
feat: add TOKEN_SUPER enum and keyword recognition to wren compiler Add the TOKEN_SUPER token type to the token enum and register "super" as a reserved keyword in the readName function. Include TOKEN_SUPER in the nextToken skip-newlines switch and mark it as UNUSED in the grammar rules table, preparing the compiler for future super keyword support.
2013-12-11 17:02:17 +01:00
if (isKeyword(parser, "super")) type = TOKEN_SUPER;
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
if (isKeyword(parser, "this")) type = TOKEN_THIS;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
if (isKeyword(parser, "true")) type = TOKEN_TRUE;
if (isKeyword(parser, "var")) type = TOKEN_VAR;
feat: add while loop token, parser support, CODE_LOOP opcode, and patchJump helper Add TOKEN_WHILE to the token enum and keyword lookup in the compiler, enabling the parser to recognize 'while' statements. Introduce the CODE_LOOP opcode in the VM for backward jumps, and implement the patchJump helper to resolve forward jump offsets during compilation. Include comprehensive test cases for while loops covering single-expression bodies, block bodies, newline handling, and null result semantics.
2013-11-18 07:38:59 +01:00
if (isKeyword(parser, "while")) type = TOKEN_WHILE;
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
makeToken(parser, type);
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
}
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
// Adds [c] to the current string literal being tokenized.
static void addStringChar(Parser* parser, char c)
{
parser->currentString[parser->currentStringLength++] = c;
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Finishes lexing a string literal.
static void readString(Parser* parser)
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
{
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
parser->currentStringLength = 0;
for (;;)
{
char c = nextChar(parser);
if (c == '"') break;
if (c == '\\')
{
switch (nextChar(parser))
{
case '"': addStringChar(parser, '"'); break;
case '\\': addStringChar(parser, '\\'); break;
case 'n': addStringChar(parser, '\n'); break;
feat: add binary_trees benchmark suite and fix string escape for tab character Add Lua, Python, Ruby, and Wren implementations of the binary_trees benchmark from the Computer Language Benchmarks Game. Update the benchmark runner to support multiple benchmarks with output validation, and fix the Wren compiler to handle the '\t' escape sequence in string literals.
2013-11-30 01:19:13 +01:00
case 't': addStringChar(parser, '\t'); break;
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
default:
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Emit error token.
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
break;
}
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Other escapes (\r, etc.), Unicode escape sequences.
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
}
else
{
addStringChar(parser, c);
}
}
feat: add class parsing and object type system with ObjBlock, ObjClass, and ObjNum structs Introduce token types TOKEN_CLASS and TOKEN_META for class declaration parsing in the compiler. Refactor the internal object model from a single Obj union to distinct ObjNum, ObjBlock, and ObjClass structs, replacing the old Block type with ObjBlock throughout the compiler, VM, and main entry point. Implement makeClass() and makeBlock() factory functions, update makeNum() to return ObjNum*, and add CODE_CLASS and CODE_DUP bytecodes to support class definition at compile time. Modify the symbol table and method dispatch to use a Method union (primitive or block) stored in ObjClass, and update the VM's callBlock signature and newVM initialization accordingly.
2013-10-24 07:50:04 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
makeToken(parser, TOKEN_STRING);
}
feat: add class parsing and object type system with ObjBlock, ObjClass, and ObjNum structs Introduce token types TOKEN_CLASS and TOKEN_META for class declaration parsing in the compiler. Refactor the internal object model from a single Obj union to distinct ObjNum, ObjBlock, and ObjClass structs, replacing the old Block type with ObjBlock throughout the compiler, VM, and main entry point. Implement makeClass() and makeBlock() factory functions, update makeNum() to return ObjNum*, and add CODE_CLASS and CODE_DUP bytecodes to support class definition at compile time. Modify the symbol table and method dispatch to use a Method union (primitive or block) stored in ObjClass, and update the VM's callBlock signature and newVM initialization accordingly.
2013-10-24 07:50:04 +02:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
// Lex the next token and store it in [parser.current]. Does not do any newline
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// filtering.
static void readRawToken(Parser* parser)
{
while (peekChar(parser) != '\0')
{
parser->tokenStart = parser->currentChar;
feat: add class parsing and object type system with ObjBlock, ObjClass, and ObjNum structs Introduce token types TOKEN_CLASS and TOKEN_META for class declaration parsing in the compiler. Refactor the internal object model from a single Obj union to distinct ObjNum, ObjBlock, and ObjClass structs, replacing the old Block type with ObjBlock throughout the compiler, VM, and main entry point. Implement makeClass() and makeBlock() factory functions, update makeNum() to return ObjNum*, and add CODE_CLASS and CODE_DUP bytecodes to support class definition at compile time. Modify the symbol table and method dispatch to use a Method union (primitive or block) stored in ObjClass, and update the VM's callBlock signature and newVM initialization accordingly.
2013-10-24 07:50:04 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
char c = nextChar(parser);
switch (c)
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '(': makeToken(parser, TOKEN_LEFT_PAREN); return;
case ')': makeToken(parser, TOKEN_RIGHT_PAREN); return;
case '[': makeToken(parser, TOKEN_LEFT_BRACKET); return;
case ']': makeToken(parser, TOKEN_RIGHT_BRACKET); return;
case '{': makeToken(parser, TOKEN_LEFT_BRACE); return;
case '}': makeToken(parser, TOKEN_RIGHT_BRACE); return;
feat: treat semicolons as line tokens in wren compiler lexer Add a new case in the `readRawToken` function within `src/wren_compiler.c` to map the semicolon character to the `TOKEN_LINE` token type, enabling semicolons to function as line separators in the Wren language parser. Include a test file `test/semicolon.wren` that verifies the new behavior by using a semicolon to separate two statements on a single line, with the expected output "ok".
2013-12-05 04:12:21 +01:00
case ';': makeToken(parser, TOKEN_LINE); return;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case ':': makeToken(parser, TOKEN_COLON); return;
case '.': makeToken(parser, TOKEN_DOT); return;
case ',': makeToken(parser, TOKEN_COMMA); return;
case '*': makeToken(parser, TOKEN_STAR); return;
feat: add bitwise NOT operator (~) with token, parser, and native implementation Add TOKEN_TILDE token type and lexer support for the '~' character in the compiler. Implement the bitwise NOT operation as a native method on the Num class, operating on 32-bit unsigned integers. Include a new OPERATOR macro for prefix/infix operator grammar rules and add comprehensive test cases covering positive, negative, floating-point, and boundary values.
2013-12-05 07:09:31 +01:00
case '%': makeToken(parser, TOKEN_PERCENT); return;
case '+': makeToken(parser, TOKEN_PLUS); return;
case '~': makeToken(parser, TOKEN_TILDE); return;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '/':
if (peekChar(parser) == '/')
{
skipLineComment(parser);
break;
}
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
if (peekChar(parser) == '*')
feat: add skipBlockComment parser with nesting support and three test cases Implement a new skipBlockComment function in the compiler that handles nested /* */ block comments by tracking a nesting counter. The parser advances through comment characters, incrementing nesting on opening /* and decrementing on closing */. Also add three test files: comment_block.wren verifying inline and nested block comments, comment_block_at_eof.wren for a block comment at end of file without newline, and comment_line_at_eof.wren for a line comment at end of file without newline.
2013-11-14 02:09:47 +01:00
{
skipBlockComment(parser);
break;
}
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
makeToken(parser, TOKEN_SLASH);
return;
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '-':
if (isDigit(peekChar(parser)))
{
readNumber(parser);
}
else
{
makeToken(parser, TOKEN_MINUS);
}
return;
feat: implement class method definition and instance creation in compiler and VM Add method parsing and compilation in `statement()` for class bodies, including symbol interning and block constant emission. Introduce `addConstant()` helper in compiler.c. Extend VM with `CODE_METHOD` opcode, `ObjInstance` type, `makeInstance()` factory, metaclass support in `makeClass()`, and `getSymbolName()` for symbol table lookup. Update vm.h with new type enum, struct definitions, and function declarations.
2013-10-25 06:32:17 +02:00
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
case '|':
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
twoCharToken(parser, '|', TOKEN_PIPEPIPE, TOKEN_PIPE);
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
return;
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
case '&':
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
twoCharToken(parser, '&', TOKEN_AMPAMP, TOKEN_AMP);
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
return;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '=':
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
twoCharToken(parser, '=', TOKEN_EQEQ, TOKEN_EQ);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
feat: implement class method definition and instance creation in compiler and VM Add method parsing and compilation in `statement()` for class bodies, including symbol interning and block constant emission. Introduce `addConstant()` helper in compiler.c. Extend VM with `CODE_METHOD` opcode, `ObjInstance` type, `makeInstance()` factory, metaclass support in `makeClass()`, and `getSymbolName()` for symbol table lookup. Update vm.h with new type enum, struct definitions, and function declarations.
2013-10-25 06:32:17 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '<':
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
twoCharToken(parser, '=', TOKEN_LTEQ, TOKEN_LT);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
feat: add class parsing and object type system with ObjBlock, ObjClass, and ObjNum structs Introduce token types TOKEN_CLASS and TOKEN_META for class declaration parsing in the compiler. Refactor the internal object model from a single Obj union to distinct ObjNum, ObjBlock, and ObjClass structs, replacing the old Block type with ObjBlock throughout the compiler, VM, and main entry point. Implement makeClass() and makeBlock() factory functions, update makeNum() to return ObjNum*, and add CODE_CLASS and CODE_DUP bytecodes to support class definition at compile time. Modify the symbol table and method dispatch to use a Method union (primitive or block) stored in ObjClass, and update the VM's callBlock signature and newVM initialization accordingly.
2013-10-24 07:50:04 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '>':
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
twoCharToken(parser, '=', TOKEN_GTEQ, TOKEN_GT);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
feat: add class parsing and object type system with ObjBlock, ObjClass, and ObjNum structs Introduce token types TOKEN_CLASS and TOKEN_META for class declaration parsing in the compiler. Refactor the internal object model from a single Obj union to distinct ObjNum, ObjBlock, and ObjClass structs, replacing the old Block type with ObjBlock throughout the compiler, VM, and main entry point. Implement makeClass() and makeBlock() factory functions, update makeNum() to return ObjNum*, and add CODE_CLASS and CODE_DUP bytecodes to support class definition at compile time. Modify the symbol table and method dispatch to use a Method union (primitive or block) stored in ObjClass, and update the VM's callBlock signature and newVM initialization accordingly.
2013-10-24 07:50:04 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '!':
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
twoCharToken(parser, '=', TOKEN_BANGEQ, TOKEN_BANG);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '\n':
makeToken(parser, TOKEN_LINE);
return;
feat: implement local variable declarations and symbol table in compiler and VM Add support for local variable declarations with `var` keyword in the Wren compiler, including a symbol table for tracking locals within blocks. Introduce new bytecode instructions (CODE_POP, CODE_LOAD_LOCAL, CODE_STORE_LOCAL) and a call frame stack in the VM to manage local variable scopes during execution. Refactor the symbol table from a flat array to a proper SymbolTable struct with init/clear/add/ensure/find functions, and add a primitive method table for the num class with an `abs` method.
2013-10-24 00:32:59 +02:00
refactor: extract twoCharToken helper and remove skipWhitespace from lexer Extract repeated two-character token matching logic into a dedicated twoCharToken function, reducing code duplication in readRawToken. Remove the unused skipWhitespace helper that only handled single spaces and was replaced by more robust whitespace handling elsewhere. Fix indentation in isKeyword and remove stale TODO comment in readName.
2013-11-20 05:19:02 +01:00
case ' ':
// Skip forward until we run out of whitespace.
while (peekChar(parser) == ' ') nextChar(parser);
break;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case '"': readString(parser); return;
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
case '_': readName(parser, TOKEN_FIELD); return;
feat: implement local variable declarations and symbol table in compiler and VM Add support for local variable declarations with `var` keyword in the Wren compiler, including a symbol table for tracking locals within blocks. Introduce new bytecode instructions (CODE_POP, CODE_LOAD_LOCAL, CODE_STORE_LOCAL) and a call frame stack in the VM to manage local variable scopes during execution. Refactor the symbol table from a flat array to a proper SymbolTable struct with init/clear/add/ensure/find functions, and add a primitive method table for the num class with an `abs` method.
2013-10-24 00:32:59 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
default:
if (isName(c))
{
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
readName(parser, TOKEN_NAME);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
else if (isDigit(c))
{
readNumber(parser);
}
else
{
fix: add lexError function and fix line counting for unterminated block comments Add a dedicated lexError function that marks compilation as failed and formats error messages consistently with a trailing newline. Move line counting from readRawToken into nextChar so that newlines are tracked during all lexer phases, including block comment scanning. Replace the TODO placeholder in skipBlockComment with a proper lexError call for unterminated block comments, and add test cases for unterminated block and nested block comments.
2013-12-23 23:51:06 +01:00
lexError(parser, "Invalid character '%c'.", c);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
return;
}
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// If we get here, we're out of source, so just make EOF tokens.
parser->tokenStart = parser->currentChar;
makeToken(parser, TOKEN_EOF);
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Lex the next token in the source file and store it in parser.current. Omits
// newlines that aren't meaningful.
static void nextToken(Parser* parser)
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
parser->previous = parser->current;
feat: disallow field references outside class definitions with explicit error Add a check in the `field()` function to detect when a field is referenced outside of a class context. When `compiler->fields` is NULL (indicating no enclosing class), emit a clear error message "Cannot reference a field outside of a class definition." and assign a dummy field index of 255 to allow continued parsing with minimal cascading errors. Also includes minor cleanup: remove two stale TODO comments about EOF checking and better error messages in `nextToken()` and `consume()`, and add a new test file `test/field/outside_class.wren` that verifies the error is produced for top-level field assignment.
2013-11-29 19:53:56 +01:00
// If we are out of tokens, don't try to tokenize any more. We *do* still
// copy the TOKEN_EOF to previous so that code that expects it to be consumed
// will still work.
if (parser->current.type == TOKEN_EOF) return;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
for (;;)
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
readRawToken(parser);
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
switch (parser->current.type)
{
case TOKEN_LINE:
if (!parser->skipNewlines)
{
// Collapse multiple newlines into one.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
parser->skipNewlines = true;
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Emit this newline.
return;
}
break;
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Discard newlines after tokens that cannot end an expression.
case TOKEN_LEFT_PAREN:
case TOKEN_LEFT_BRACKET:
case TOKEN_LEFT_BRACE:
case TOKEN_DOT:
case TOKEN_COMMA:
case TOKEN_STAR:
case TOKEN_SLASH:
case TOKEN_PERCENT:
case TOKEN_PLUS:
case TOKEN_MINUS:
case TOKEN_PIPE:
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
case TOKEN_PIPEPIPE:
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case TOKEN_AMP:
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
case TOKEN_AMPAMP:
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case TOKEN_BANG:
feat: add bitwise NOT operator (~) with token, parser, and native implementation Add TOKEN_TILDE token type and lexer support for the '~' character in the compiler. Implement the bitwise NOT operation as a native method on the Num class, operating on 32-bit unsigned integers. Include a new OPERATOR macro for prefix/infix operator grammar rules and add comprehensive test cases covering positive, negative, floating-point, and boundary values.
2013-12-05 07:09:31 +01:00
case TOKEN_TILDE:
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case TOKEN_EQ:
case TOKEN_LT:
case TOKEN_GT:
case TOKEN_LTEQ:
case TOKEN_GTEQ:
case TOKEN_EQEQ:
case TOKEN_BANGEQ:
case TOKEN_CLASS:
case TOKEN_ELSE:
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
case TOKEN_FOR:
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case TOKEN_IF:
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
case TOKEN_IN:
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
case TOKEN_IS:
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
case TOKEN_NEW:
feat: add static method support to class definitions with metaclass dispatch Add TOKEN_STATIC keyword to the lexer, replacing the unused TOKEN_META, and extend the method() compiler function with an isStatic parameter. When isStatic is true, emit a new CODE_METACLASS instruction before CODE_METHOD to push the class's metaclass onto the stack, enabling static method dispatch. Also fix a buffer overflow bug in string_plus by allocating one extra byte for the null terminator, and refactor object initialization across vm.c by introducing a shared initObj helper to eliminate repeated type/flag assignments.
2013-11-10 20:46:13 +01:00
case TOKEN_STATIC:
feat: add TOKEN_SUPER enum and keyword recognition to wren compiler Add the TOKEN_SUPER token type to the token enum and register "super" as a reserved keyword in the readName function. Include TOKEN_SUPER in the nextToken skip-newlines switch and mark it as UNUSED in the grammar rules table, preparing the compiler for future super keyword support.
2013-12-11 17:02:17 +01:00
case TOKEN_SUPER:
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
case TOKEN_VAR:
feat: add while loop token, parser support, CODE_LOOP opcode, and patchJump helper Add TOKEN_WHILE to the token enum and keyword lookup in the compiler, enabling the parser to recognize 'while' statements. Introduce the CODE_LOOP opcode in the VM for backward jumps, and implement the patchJump helper to resolve forward jump offsets during compilation. Include comprehensive test cases for while loops covering single-expression bodies, block bodies, newline handling, and null result semantics.
2013-11-18 07:38:59 +01:00
case TOKEN_WHILE:
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
parser->skipNewlines = true;
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Emit this token.
return;
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Newlines are meaningful after other tokens.
default:
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
parser->skipNewlines = false;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
}
}
}
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Parsing ---------------------------------------------------------------------
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Returns the type of the current token.
static TokenType peek(Compiler* compiler)
{
return compiler->parser->current.type;
}
feat: add if/else expression parsing with jump bytecodes and null object support Implement initial support for `if` expressions in the compiler, including new `TOKEN_IF` and `TOKEN_ELSE` tokens, `CODE_JUMP` and `CODE_JUMP_IF` bytecodes for conditional branching, and a `makeNull()` function in the VM to represent the value returned when an if-condition is false without an else clause. The `emit()` function now returns the index of the emitted bytecode to enable patching jump offsets. Also add a `nullClass` to the VM and a `OBJ_NULL` object type, along with a `dumpCode()` debugging helper and new test files for block syntax and if-expression behavior.
2013-11-06 00:40:21 +01:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
// Consumes the current token if its type is [expected]. Returns true if a
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// token was consumed.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static bool match(Compiler* compiler, TokenType expected)
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
{
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
if (peek(compiler) != expected) return false;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
nextToken(compiler->parser);
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
return true;
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Consumes the current token. Emits an error if its type is not [expected].
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
// Returns the consumed token.
static Token* consume(Compiler* compiler, TokenType expected,
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
const char* errorMessage)
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
{
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
nextToken(compiler->parser);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
if (compiler->parser->previous.type != expected)
{
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
error(compiler, errorMessage);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
return &compiler->parser->previous;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
// Variables and scopes --------------------------------------------------------
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
// Emits one bytecode instruction or argument. Returns its index.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
static int emit(Compiler* compiler, Code code)
{
compiler->fn->bytecode[compiler->numCodes++] = code;
return compiler->numCodes - 1;
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
// Emits one bytecode instruction followed by an argument. Returns the index of
// the argument in the bytecode.
static int emit1(Compiler* compiler, Code instruction, uint8_t arg)
{
emit(compiler, instruction);
return emit(compiler, arg);
}
// Create a new local variable with [name]. Assumes the current scope is local
// and the name is unique.
static int defineLocal(Compiler* compiler, const char* name, int length)
{
Local* local = &compiler->locals[compiler->numLocals];
local->name = name;
local->length = length;
local->depth = compiler->scopeDepth;
local->isUpvalue = false;
return compiler->numLocals++;
}
refactor: rename `symbols` to `methods` and `defineName` to `declareVariable` in compiler and VM Rename the `SymbolTable symbols` field in `VM` struct to `methods` to clarify its purpose for method dispatch, and update all references across `vm.c`, `vm.h`, `primitives.c`, and `compiler.c`. Rename `defineName` to `declareVariable` to better reflect its actual behavior of declaring a variable without defining it. Remove the unused `internSymbol` helper function. Add `clearSymbolTable(&vm->globalSymbols)` to `freeVM` to fix a memory leak. Update `dumpCode` signature to take `VM*` and `ObjFn*` instead of `ObjBlock*`, and display method names via `getSymbolName`. Add `stackStart` field to `CallFrame` replacing `locals` for clearer semantics. Introduce `test/method_arity.wren` to validate method dispatch with up to 10 parameters.
2013-11-09 20:18:27 +01:00
// Parses a name token and declares a variable in the current scope with that
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// name. Returns its symbol.
refactor: rename `symbols` to `methods` and `defineName` to `declareVariable` in compiler and VM Rename the `SymbolTable symbols` field in `VM` struct to `methods` to clarify its purpose for method dispatch, and update all references across `vm.c`, `vm.h`, `primitives.c`, and `compiler.c`. Rename `defineName` to `declareVariable` to better reflect its actual behavior of declaring a variable without defining it. Remove the unused `internSymbol` helper function. Add `clearSymbolTable(&vm->globalSymbols)` to `freeVM` to fix a memory leak. Update `dumpCode` signature to take `VM*` and `ObjFn*` instead of `ObjBlock*`, and display method names via `getSymbolName`. Add `stackStart` field to `CallFrame` replacing `locals` for clearer semantics. Introduce `test/method_arity.wren` to validate method dispatch with up to 10 parameters.
2013-11-09 20:18:27 +01:00
static int declareVariable(Compiler* compiler)
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
{
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
Token* token = consume(compiler, TOKEN_NAME, "Expected variable name.");
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// Top-level global scope.
if (compiler->scopeDepth == -1)
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
{
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
SymbolTable* symbols = &compiler->parser->vm->globalSymbols;
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
int symbol = addSymbol(symbols, token->start, token->length);
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
if (symbol == -1)
{
error(compiler, "Global variable is already defined.");
}
return symbol;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// See if there is already a variable with this name declared in the current
// scope. (Outer scopes are OK: those get shadowed.)
for (int i = compiler->numLocals - 1; i >= 0; i--)
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
{
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
Local* local = &compiler->locals[i];
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// Once we escape this scope and hit an outer one, we can stop.
if (local->depth < compiler->scopeDepth) break;
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
if (local->length == token->length &&
strncmp(local->name, token->start, token->length) == 0)
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
{
error(compiler, "Variable is already declared in this scope.");
return i;
}
feat: implement infix arithmetic operators for numbers and string concatenation Add parser support for +, -, *, / operators with correct precedence (term/factor) and emit method calls. Implement primitive methods for num_plus, num_minus, num_multiply, num_divide, and string_plus. Update VM to pass VM pointer to primitives and add unsupported value for type errors. Include test files for each operator.
2013-10-31 15:04:44 +01:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
feat: enforce MAX_LOCALS limit of 256 in declareVariable and add test cases for boundary conditions Add a compile-time check in declareVariable() that emits an error when the number of simultaneous local variables reaches MAX_LOCALS (256), replacing the previous placeholder TODO. The limit is derived from the bytecode encoding constraint where CODE_LOAD_LOCAL and CODE_STORE_LOCAL use a single argument byte. Also move the MAX_LOCALS definition from a TODO comment to a proper #define, increasing the value from 255 to 256. Include four new test files: many_locals.wren (exactly 256 simultaneous locals), many_nonsimultaneous_locals.wren (more than 256 locals across nested scopes), too_many_locals.wren (257 simultaneous locals triggering the error), and too_many_locals_nested.wren (exceeding the limit across nested blocks).
2013-12-01 03:51:27 +01:00
if (compiler->numLocals == MAX_LOCALS)
{
error(compiler, "Cannot declare more than %d variables in one scope.",
MAX_LOCALS);
return -1;
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
return defineLocal(compiler, token->start, token->length);
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Stores a variable with the previously defined symbol in the current scope.
fix: correct local variable storage and remove premature local slot allocation in vm The compiler's storeVariable function was incorrectly emitting CODE_STORE_LOCAL for all variables, including locals that already have their value in the correct stack slot. This change renames storeVariable to defineVariable and adds logic to only emit CODE_STORE_GLOBAL for globals, while for locals it emits CODE_DUP to preserve the value through the subsequent POP. Additionally, the vm's callFunction no longer pre-allocates empty slots for locals, as they are now placed on the stack during compilation. New test files verify duplicate local/parameter detection and correct local variable ordering in block scopes.
2013-11-09 20:38:01 +01:00
static void defineVariable(Compiler* compiler, int symbol)
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
{
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Store the variable. If it's a local, the result of the initializer is
// in the correct slot on the stack already so we're done.
if (compiler->scopeDepth >= 0) return;
// It's a global variable, so store the value in the global slot and then
// discard the temporary for the initializer.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_STORE_GLOBAL, symbol);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
emit(compiler, CODE_POP);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// Starts a new local block scope.
static void pushScope(Compiler* compiler)
{
compiler->scopeDepth++;
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Closes the last pushed block scope. This should only be called in a statement
// context where no temporaries are still on the stack.
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
static void popScope(Compiler* compiler)
{
ASSERT(compiler->scopeDepth > -1, "Cannot pop top-level scope.");
// Pop locals off the stack.
while (compiler->numLocals > 0 &&
compiler->locals[compiler->numLocals - 1].depth ==
compiler->scopeDepth)
{
compiler->numLocals--;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// If the local was closed over, make sure the upvalue gets closed when it
// goes out of scope on the stack.
if (compiler->locals[compiler->numLocals].isUpvalue)
{
emit(compiler, CODE_CLOSE_UPVALUE);
}
else
{
emit(compiler, CODE_POP);
}
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
}
compiler->scopeDepth--;
}
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
// Attempts to look up the name in the local variables of [compiler]. If found,
// returns its index, otherwise returns -1.
static int resolveLocal(Compiler* compiler, const char* name, int length)
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
{
// Look it up in the local scopes. Look in reverse order so that the most
// nested variable is found first and shadows outer ones.
for (int i = compiler->numLocals - 1; i >= 0; i--)
{
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
if (compiler->locals[i].length == length &&
strncmp(name, compiler->locals[i].name, length) == 0)
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
{
return i;
}
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
return -1;
}
// Adds an upvalue to [compiler]'s function with the given properties. Does not
// add one if an upvalue for that variable is already in the list. Returns the
// index of the uvpalue.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static int addUpvalue(Compiler* compiler, bool isLocal, int index)
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
{
// Look for an existing one.
for (int i = 0; i < compiler->fn->numUpvalues; i++)
{
CompilerUpvalue* upvalue = &compiler->upvalues[i];
if (upvalue->index == index && upvalue->isLocal == isLocal) return i;
}
// If we got here, it's a new upvalue.
compiler->upvalues[compiler->fn->numUpvalues].isLocal = isLocal;
compiler->upvalues[compiler->fn->numUpvalues].index = index;
return compiler->fn->numUpvalues++;
}
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
// Attempts to look up [name] in the functions enclosing the one being compiled
// by [compiler]. If found, it adds an upvalue for it to this compiler's list
// of upvalues (unless it's already in there) and returns its index. If not
// found, returns -1.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
//
// If the name is found outside of the immediately enclosing function, this
// will flatten the closure and add upvalues to all of the intermediate
// functions so that it gets walked down to this one.
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
static int findUpvalue(Compiler* compiler, const char* name, int length)
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
{
// If we are out of enclosing functions, it can't be an upvalue.
if (compiler->parent == NULL)
{
return -1;
}
// See if it's a local variable in the immediately enclosing function.
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
int local = resolveLocal(compiler->parent, name, length);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (local != -1)
{
// Mark the local as an upvalue so we know to close it when it goes out of
// scope.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
compiler->parent->locals[local].isUpvalue = true;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
return addUpvalue(compiler, true, local);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
}
// See if it's an upvalue in the immediately enclosing function. In other
// words, if its a local variable in a non-immediately enclosing function.
// This will "flatten" closures automatically: it will add upvalues to all
// of the intermediate functions to get from the function where a local is
// declared all the way into the possibly deeply nested function that is
// closing over it.
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
int upvalue = findUpvalue(compiler->parent, name, length);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (upvalue != -1)
{
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
return addUpvalue(compiler, false, upvalue);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
}
// If we got here, we walked all the way up the parent chain and couldn't
// find it.
return -1;
}
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
// Look up [name] in the current scope to see what name it is bound to. Returns
// the index of the name either in global scope, local scope, or the enclosing
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
// function's upvalue list. Returns -1 if not found.
//
// Sets [loadInstruction] to the instruction needed to load the variable. Will
// be one of [CODE_LOAD_LOCAL], [CODE_LOAD_UPVALUE], or [CODE_LOAD_GLOBAL].
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
static int resolveName(Compiler* compiler, const char* name, int length,
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
Code* loadInstruction)
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
{
// Look it up in the local scopes. Look in reverse order so that the most
// nested variable is found first and shadows outer ones.
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
*loadInstruction = CODE_LOAD_LOCAL;
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
int local = resolveLocal(compiler, name, length);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (local != -1) return local;
// If we got here, it's not a local, so lets see if we are closing over an
// outer local.
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
*loadInstruction = CODE_LOAD_UPVALUE;
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
int upvalue = findUpvalue(compiler, name, length);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (upvalue != -1) return upvalue;
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// If we got here, it wasn't in a local scope, so try the global scope.
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
*loadInstruction = CODE_LOAD_GLOBAL;
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
return findSymbol(&compiler->parser->vm->globalSymbols, name, length);
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
}
refactor: rename MAX_METHOD_NAME to MAX_METHOD_SIGNATURE and extract copyName helper Extract identifier copying logic from subscript and call functions into a dedicated copyName helper function, reducing code duplication. Rename the method signature length constant to MAX_METHOD_SIGNATURE with updated documentation to clarify it includes arity spaces. Update subscript function to use the new constant name and remove stale TODO comment about handling >10 args.
2013-12-01 09:04:46 +01:00
// Copies the identifier from the previously consumed `TOKEN_NAME` into [name],
// which should point to a buffer large enough to contain it. Returns the
// length of the name.
static int copyName(Compiler* compiler, char* name)
{
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
Token* token = &compiler->parser->previous;
feat: enforce maximum method name length of 64 characters in compiler Add MAX_METHOD_NAME constant and validation in copyName() to prevent method names exceeding 64 characters. When a method name is too long, the compiler now reports an error and truncates the name to the maximum allowed length. Also update MAX_METHOD_SIGNATURE to be derived from MAX_METHOD_NAME and MAX_PARAMETERS. Add test cases for valid 64-character method names, names that exceed the limit, and a call to an overlong method name.
2013-12-07 05:09:43 +01:00
int length = token->length;
if (length > MAX_METHOD_NAME)
{
error(compiler, "Method names cannot be longer than %d characters.",
MAX_METHOD_NAME);
length = MAX_METHOD_NAME;
}
strncpy(name, token->start, length);
return length;
refactor: rename MAX_METHOD_NAME to MAX_METHOD_SIGNATURE and extract copyName helper Extract identifier copying logic from subscript and call functions into a dedicated copyName helper function, reducing code duplication. Rename the method signature length constant to MAX_METHOD_SIGNATURE with updated documentation to clarify it includes arity spaces. Update subscript function to use the new constant name and remove stale TODO comment about handling >10 args.
2013-12-01 09:04:46 +01:00
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Finishes [compiler], which is compiling a function, method, or chunk of top
// level code. If there is a parent compiler, then this emits code in the
// parent compiler to load the resulting function.
static void endCompiler(Compiler* compiler, int constant)
{
// Mark the end of the bytecode. Since it may contain multiple early returns,
// we can't rely on CODE_RETURN to tell us we're at the end.
emit(compiler, CODE_END);
// In the function that contains this one, load the resulting function object.
if (compiler->parent != NULL)
{
// If the function has no upvalues, we don't need to create a closure.
// We can just load and run the function directly.
if (compiler->fn->numUpvalues == 0)
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler->parent, CODE_CONSTANT, constant);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
}
else
{
// Capture the upvalues in the new closure object.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler->parent, CODE_CLOSURE, constant);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Emit arguments for each upvalue to know whether to capture a local or
// an upvalue.
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Do something more efficient here?
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
for (int i = 0; i < compiler->fn->numUpvalues; i++)
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler->parent, compiler->upvalues[i].isLocal ? 1 : 0,
compiler->upvalues[i].index);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
}
}
}
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Grammar ---------------------------------------------------------------------
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
typedef enum
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
{
PREC_NONE,
PREC_LOWEST,
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
PREC_ASSIGNMENT, // =
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
PREC_LOGIC, // && ||
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
PREC_IS, // is
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
PREC_EQUALITY, // == !=
PREC_COMPARISON, // < > <= >=
PREC_BITWISE, // | &
PREC_TERM, // + -
PREC_FACTOR, // * / %
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
PREC_UNARY, // unary - ! ~
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
PREC_CALL // . () []
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
} Precedence;
// Forward declarations since the grammar is recursive.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
static void expression(Compiler* compiler);
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
static void statement(Compiler* compiler);
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
static void definition(Compiler* compiler);
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void parsePrecedence(Compiler* compiler, bool allowAssignment,
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
Precedence precedence);
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
typedef void (*GrammarFn)(Compiler*, bool allowAssignment);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
typedef void (*SignatureFn)(Compiler* compiler, char* name, int* length);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
typedef struct
{
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
GrammarFn prefix;
GrammarFn infix;
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
SignatureFn method;
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
Precedence precedence;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
const char* name;
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
} GrammarRule;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
GrammarRule rules[];
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
// Replaces the placeholder argument for a previous CODE_JUMP or CODE_JUMP_IF
// instruction with an offset that jumps to the current end of bytecode.
static void patchJump(Compiler* compiler, int offset)
{
compiler->fn->bytecode[offset] = compiler->numCodes - offset - 1;
}
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
// Parses a block body, after the initial "{" has been consumed.
static void finishBlock(Compiler* compiler)
{
for (;;)
{
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
definition(compiler);
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
// If there is no newline, it must be the end of the block on the same line.
if (!match(compiler, TOKEN_LINE))
{
consume(compiler, TOKEN_RIGHT_BRACE, "Expect '}' after block body.");
break;
}
if (match(compiler, TOKEN_RIGHT_BRACE)) break;
}
}
feat: add list subscript setter and refactor parameter validation Implement list subscript setter (`[ ]=`) native method in wren_core.c, enabling assignment to list elements via bracket syntax. Extract duplicate parameter count validation logic into a shared `validateNumParameters` function in wren_compiler.c, replacing inline checks in both `parameterList` and `methodCall`. Add comprehensive test suite covering basic assignment, return value semantics, negative indices, and error cases for out-of-range and excessive arguments.
2013-12-21 18:37:59 +01:00
// The VM can only handle a certain number of parameters, so check that we
// haven't exceeded that and give a usable error.
static void validateNumParameters(Compiler* compiler, int numArgs)
{
if (numArgs == MAX_PARAMETERS + 1)
{
// Only show an error at exactly max + 1 so that we can keep parsing the
// parameters and minimize cascaded errors.
error(compiler, "Methods cannot have more than %d parameters.",
MAX_PARAMETERS);
}
}
feat: enforce maximum method name length of 64 characters in compiler Add MAX_METHOD_NAME constant and validation in copyName() to prevent method names exceeding 64 characters. When a method name is too long, the compiler now reports an error and truncates the name to the maximum allowed length. Also update MAX_METHOD_SIGNATURE to be derived from MAX_METHOD_NAME and MAX_PARAMETERS. Add test cases for valid 64-character method names, names that exceed the limit, and a call to an overlong method name.
2013-12-07 05:09:43 +01:00
// Parses an optional parenthesis-delimited parameter list. If the parameter
// list is for a method, [name] will be the name of the method and this will
// modify it to handle arity in the signature. For functions, [name] will be
// `null`.
feat: add parameter list parsing and up to 8 function call primitives Extract shared parameter list parsing from namedSignature into a new parameterList function called by function, and replace single fn_call primitive with fn_call0 through fn_call8 primitives to support up to 8 arguments. Add comprehensive fn_parameters.wren test covering 0-8 parameter calls.
2013-11-15 03:25:28 +01:00
static void parameterList(Compiler* compiler, char* name, int* length)
{
// Parse the parameter list, if any.
if (match(compiler, TOKEN_LEFT_PAREN))
{
feat: raise max method parameters from 10 to 16 with proper error handling Add MAX_PARAMETERS constant set to 16 and extend VM bytecode instructions CODE_CALL_11 through CODE_CALL_16 to support the new limit. Introduce compile-time error messages when parameter or argument count exceeds 16 in function definitions, method declarations, subscript calls, and regular method calls. Add new native fiber call stubs fn_call9 through fn_call16 and register them in the core initialization. Include comprehensive test coverage for 9-16 parameter functions and methods, plus error tests for exceeding the limit in all contexts.
2013-11-30 00:08:27 +01:00
int numParams = 0;
feat: add parameter list parsing and up to 8 function call primitives Extract shared parameter list parsing from namedSignature into a new parameterList function called by function, and replace single fn_call primitive with fn_call0 through fn_call8 primitives to support up to 8 arguments. Add comprehensive fn_parameters.wren test covering 0-8 parameter calls.
2013-11-15 03:25:28 +01:00
do
{
feat: add list subscript setter and refactor parameter validation Implement list subscript setter (`[ ]=`) native method in wren_core.c, enabling assignment to list elements via bracket syntax. Extract duplicate parameter count validation logic into a shared `validateNumParameters` function in wren_compiler.c, replacing inline checks in both `parameterList` and `methodCall`. Add comprehensive test suite covering basic assignment, return value semantics, negative indices, and error cases for out-of-range and excessive arguments.
2013-12-21 18:37:59 +01:00
validateNumParameters(compiler, ++numParams);
feat: raise max method parameters from 10 to 16 with proper error handling Add MAX_PARAMETERS constant set to 16 and extend VM bytecode instructions CODE_CALL_11 through CODE_CALL_16 to support the new limit. Introduce compile-time error messages when parameter or argument count exceeds 16 in function definitions, method declarations, subscript calls, and regular method calls. Add new native fiber call stubs fn_call9 through fn_call16 and register them in the core initialization. Include comprehensive test coverage for 9-16 parameter functions and methods, plus error tests for exceeding the limit in all contexts.
2013-11-30 00:08:27 +01:00
feat: add parameter list parsing and up to 8 function call primitives Extract shared parameter list parsing from namedSignature into a new parameterList function called by function, and replace single fn_call primitive with fn_call0 through fn_call8 primitives to support up to 8 arguments. Add comprehensive fn_parameters.wren test covering 0-8 parameter calls.
2013-11-15 03:25:28 +01:00
// Define a local variable in the method for the parameter.
declareVariable(compiler);
// Add a space in the name for the parameter.
if (name != NULL) name[(*length)++] = ' ';
}
while (match(compiler, TOKEN_COMMA));
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after parameters.");
}
}
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
// Compiles an (optional) argument list and then calls it.
static void methodCall(Compiler* compiler, Code instruction,
char name[MAX_METHOD_SIGNATURE], int length)
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
{
// Parse the argument list, if any.
int numArgs = 0;
if (match(compiler, TOKEN_LEFT_PAREN))
{
do
{
feat: add list subscript setter and refactor parameter validation Implement list subscript setter (`[ ]=`) native method in wren_core.c, enabling assignment to list elements via bracket syntax. Extract duplicate parameter count validation logic into a shared `validateNumParameters` function in wren_compiler.c, replacing inline checks in both `parameterList` and `methodCall`. Add comprehensive test suite covering basic assignment, return value semantics, negative indices, and error cases for out-of-range and excessive arguments.
2013-12-21 18:37:59 +01:00
validateNumParameters(compiler, ++numArgs);
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
expression(compiler);
// Add a space in the name for each argument. Lets us overload by
// arity.
name[length++] = ' ';
}
while (match(compiler, TOKEN_COMMA));
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after arguments.");
}
int symbol = ensureSymbol(&compiler->parser->vm->methods, name, length);
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, instruction + numArgs, symbol);
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
}
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
// Compiles an expression that starts with ".name". That includes getters,
// method calls with arguments, and setter calls.
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
static void namedCall(Compiler* compiler, bool allowAssignment,
Code instruction)
{
// Build the method name.
consume(compiler, TOKEN_NAME, "Expect method name after '.'.");
char name[MAX_METHOD_SIGNATURE];
int length = copyName(compiler, name);
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
if (match(compiler, TOKEN_EQ))
{
if (!allowAssignment) error(compiler, "Invalid assignment.");
name[length++] = '=';
name[length++] = ' ';
// Compile the assigned value.
expression(compiler);
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
int symbol = ensureSymbol(&compiler->parser->vm->methods, name, length);
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, instruction + 1, symbol);
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
}
else
{
methodCall(compiler, instruction, name, length);
}
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
}
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
// Loads the receiver of the currently enclosing method. Correctly handles
// functions defined inside methods.
static void loadThis(Compiler* compiler)
{
Code loadInstruction;
int index = resolveName(compiler, "this", 4, &loadInstruction);
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, loadInstruction, index);
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void grouping(Compiler* compiler, bool allowAssignment)
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
{
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
expression(compiler);
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after expression.");
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
}
feat: support multi-part method calls with name mangling for arity Extend the compiler's call expression handler to support multi-part method names (e.g., `foo.bar(arg) nextPart { arg } lastBit`) by building a mangled name that concatenates all name parts and appends a space for each argument. Update the PRIMITIVE macro in the VM to accept an explicit name string, enabling correct symbol registration for methods with spaces in their mangled names.
2013-10-27 19:20:19 +01:00
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void list(Compiler* compiler, bool allowAssignment)
feat: implement list literal syntax and List class with count property Add support for list literals in the compiler with bracket-delimited syntax and comma-separated elements. Introduce the ObjList runtime type with a contiguous elements array, garbage collection marking, and proper memory deallocation. Expose a `List` class in the core library with a `count` method returning the number of elements. Update the grammar table to register `[` as a prefix token for list compilation instead of an infix subscript operator, and reorder opcode enum entries to place CODE_LIST before the load/store group.
2013-11-24 22:38:31 +01:00
{
// Compile the list elements.
int numElements = 0;
if (peek(compiler) != TOKEN_RIGHT_BRACKET)
{
do
{
numElements++;
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
expression(compiler);
fix: add list newline handling and resolve multiple test TODOs across compiler and test suite - Add TOKEN_LINE matching after list elements in compiler to allow newlines before commas or closing brackets - Add tests for list newline handling, EOF after comma/element, and class name inside body - Remove resolved TODOs for default constructors, inherited fields, metaclass is checks, zero comparison, and bitwise not behavior
2013-12-20 21:42:11 +01:00
// Ignore a newline after the element but before the ',' or ']'.
match(compiler, TOKEN_LINE);
feat: implement list literal syntax and List class with count property Add support for list literals in the compiler with bracket-delimited syntax and comma-separated elements. Introduce the ObjList runtime type with a contiguous elements array, garbage collection marking, and proper memory deallocation. Expose a `List` class in the core library with a `count` method returning the number of elements. Update the grammar table to register `[` as a prefix token for list compilation instead of an infix subscript operator, and reorder opcode enum entries to place CODE_LIST before the load/store group.
2013-11-24 22:38:31 +01:00
} while (match(compiler, TOKEN_COMMA));
}
consume(compiler, TOKEN_RIGHT_BRACKET, "Expect ']' after list elements.");
// Create the list.
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Handle lists >255 elements.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_LIST, numElements);
feat: implement list literal syntax and List class with count property Add support for list literals in the compiler with bracket-delimited syntax and comma-separated elements. Introduce the ObjList runtime type with a contiguous elements array, garbage collection marking, and proper memory deallocation. Expose a `List` class in the core library with a `count` method returning the number of elements. Update the grammar table to register `[` as a prefix token for list compilation instead of an infix subscript operator, and reorder opcode enum entries to place CODE_LIST before the load/store group.
2013-11-24 22:38:31 +01:00
}
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
// Unary operators like `-foo`.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void unaryOp(Compiler* compiler, bool allowAssignment)
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
{
GrammarRule* rule = &rules[compiler->parser->previous.type];
// Compile the argument.
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
parsePrecedence(compiler, false, PREC_UNARY + 1);
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
// Call the operator method on the left-hand side.
int symbol = ensureSymbol(&compiler->parser->vm->methods, rule->name, 1);
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_CALL_0, symbol);
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void boolean(Compiler* compiler, bool allowAssignment)
feat: add boolean literals and number comparison operators to compiler and VM Add TOKEN_TRUE/TOKEN_FALSE parsing with boolean() prefix rule emitting CODE_TRUE/CODE_FALSE bytecodes. Wire comparison operators <, >, <=, >= as infix rules with PREC_COMPARISON precedence. Implement num_lt, num_gt, num_lte, num_gte primitives returning bool values. Register boolClass with toString primitive and move class initialization into registerPrimitives(). Add OBJ_TRUE/OBJ_FALSE object types with makeBool() helper and AS_BOOL macro. Include test files for bool_toString and number_comparison.
2013-11-04 06:38:58 +01:00
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit(compiler,
compiler->parser->previous.type == TOKEN_FALSE ? CODE_FALSE : CODE_TRUE);
feat: add boolean literals and number comparison operators to compiler and VM Add TOKEN_TRUE/TOKEN_FALSE parsing with boolean() prefix rule emitting CODE_TRUE/CODE_FALSE bytecodes. Wire comparison operators <, >, <=, >= as infix rules with PREC_COMPARISON precedence. Implement num_lt, num_gt, num_lte, num_gte primitives returning bool values. Register boolClass with toString primitive and move class initialization into registerPrimitives(). Add OBJ_TRUE/OBJ_FALSE object types with makeBool() helper and AS_BOOL macro. Include test files for bool_toString and number_comparison.
2013-11-04 06:38:58 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void function(Compiler* compiler, bool allowAssignment)
feat: replace block syntax with fn keyword for function literals across compiler and tests Rename `block` parsing rule to `fn` by adding TOKEN_FN token and function prefix rule, removing the old block prefix from TOKEN_LEFT_BRACE. Update compiler.c to replace `block()` with `function()` and add fn equality primitives (fn_eqeq, fn_bangeq) in primitives.c. Migrate test files: rename block_call.wren to fn_call.wren, block_syntax.wren to fn_syntax.wren, and replace `{ ... }` with `fn { ... }` or `fn expr` syntax. Remove obsolete bool_equality.wren test.
2013-11-06 16:47:47 +01:00
{
Compiler fnCompiler;
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
int constant = initCompiler(&fnCompiler, compiler->parser, compiler,
NULL, 0);
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
feat: add parameter list parsing and up to 8 function call primitives Extract shared parameter list parsing from namedSignature into a new parameterList function called by function, and replace single fn_call primitive with fn_call0 through fn_call8 primitives to support up to 8 arguments. Add comprehensive fn_parameters.wren test covering 0-8 parameter calls.
2013-11-15 03:25:28 +01:00
parameterList(&fnCompiler, NULL, NULL);
feat: replace block syntax with fn keyword for function literals across compiler and tests Rename `block` parsing rule to `fn` by adding TOKEN_FN token and function prefix rule, removing the old block prefix from TOKEN_LEFT_BRACE. Update compiler.c to replace `block()` with `function()` and add fn equality primitives (fn_eqeq, fn_bangeq) in primitives.c. Migrate test files: rename block_call.wren to fn_call.wren, block_syntax.wren to fn_syntax.wren, and replace `{ ... }` with `fn { ... }` or `fn expr` syntax. Remove obsolete bool_equality.wren test.
2013-11-06 16:47:47 +01:00
if (match(&fnCompiler, TOKEN_LEFT_BRACE))
{
// Block body.
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
finishBlock(&fnCompiler);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Implicitly return null.
emit(&fnCompiler, CODE_NULL);
emit(&fnCompiler, CODE_RETURN);
feat: replace block syntax with fn keyword for function literals across compiler and tests Rename `block` parsing rule to `fn` by adding TOKEN_FN token and function prefix rule, removing the old block prefix from TOKEN_LEFT_BRACE. Update compiler.c to replace `block()` with `function()` and add fn equality primitives (fn_eqeq, fn_bangeq) in primitives.c. Migrate test files: rename block_call.wren to fn_call.wren, block_syntax.wren to fn_syntax.wren, and replace `{ ... }` with `fn { ... }` or `fn expr` syntax. Remove obsolete bool_equality.wren test.
2013-11-06 16:47:47 +01:00
}
else
{
// Single expression body.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
expression(&fnCompiler);
emit(&fnCompiler, CODE_RETURN);
feat: replace block syntax with fn keyword for function literals across compiler and tests Rename `block` parsing rule to `fn` by adding TOKEN_FN token and function prefix rule, removing the old block prefix from TOKEN_LEFT_BRACE. Update compiler.c to replace `block()` with `function()` and add fn equality primitives (fn_eqeq, fn_bangeq) in primitives.c. Migrate test files: rename block_call.wren to fn_call.wren, block_syntax.wren to fn_syntax.wren, and replace `{ ... }` with `fn { ... }` or `fn expr` syntax. Remove obsolete bool_equality.wren test.
2013-11-06 16:47:47 +01:00
}
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
endCompiler(&fnCompiler, constant);
feat: replace block syntax with fn keyword for function literals across compiler and tests Rename `block` parsing rule to `fn` by adding TOKEN_FN token and function prefix rule, removing the old block prefix from TOKEN_LEFT_BRACE. Update compiler.c to replace `block()` with `function()` and add fn equality primitives (fn_eqeq, fn_bangeq) in primitives.c. Migrate test files: rename block_call.wren to fn_call.wren, block_syntax.wren to fn_syntax.wren, and replace `{ ... }` with `fn { ... }` or `fn expr` syntax. Remove obsolete bool_equality.wren test.
2013-11-06 16:47:47 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void field(Compiler* compiler, bool allowAssignment)
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
{
feat: disallow field references outside class definitions with explicit error Add a check in the `field()` function to detect when a field is referenced outside of a class context. When `compiler->fields` is NULL (indicating no enclosing class), emit a clear error message "Cannot reference a field outside of a class definition." and assign a dummy field index of 255 to allow continued parsing with minimal cascading errors. Also includes minor cleanup: remove two stale TODO comments about EOF checking and better error messages in `nextToken()` and `consume()`, and add a new test file `test/field/outside_class.wren` that verifies the error is produced for top-level field assignment.
2013-11-29 19:53:56 +01:00
int field;
if (compiler->fields != NULL)
{
// Look up the field, or implicitly define it.
field = ensureSymbol(compiler->fields,
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
compiler->parser->previous.start,
compiler->parser->previous.length);
feat: disallow field references outside class definitions with explicit error Add a check in the `field()` function to detect when a field is referenced outside of a class context. When `compiler->fields` is NULL (indicating no enclosing class), emit a clear error message "Cannot reference a field outside of a class definition." and assign a dummy field index of 255 to allow continued parsing with minimal cascading errors. Also includes minor cleanup: remove two stale TODO comments about EOF checking and better error messages in `nextToken()` and `consume()`, and add a new test file `test/field/outside_class.wren` that verifies the error is produced for top-level field assignment.
2013-11-29 19:53:56 +01:00
}
else
{
error(compiler, "Cannot reference a field outside of a class definition.");
// Initialize it with a fake value so we can keep parsing and minimize the
// number of cascaded errors.
field = 255;
}
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
// If there's an "=" after a field name, it's an assignment.
if (match(compiler, TOKEN_EQ))
{
if (!allowAssignment) error(compiler, "Invalid assignment.");
// Compile the right-hand side.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
expression(compiler);
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
// If we're directly inside a method, use a more optimal instruction.
if (compiler->methodName != NULL)
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_STORE_FIELD_THIS, field);
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
}
else
{
loadThis(compiler);
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_STORE_FIELD, field);
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
}
}
else
{
// If we're directly inside a method, use a more optimal instruction.
if (compiler->methodName != NULL)
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_LOAD_FIELD_THIS, field);
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
}
else
{
loadThis(compiler);
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_LOAD_FIELD, field);
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
}
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
}
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void name(Compiler* compiler, bool allowAssignment)
feat: replace recursive descent parser with grammar-driven approach and add method call infrastructure Refactor compiler.c to remove recursive descent parsing infrastructure (InfixCompiler, CompileFn, precedence enums) and replace with a simpler grammar-driven approach using statement(), call(), primary(), and number() functions. Add VM struct to compiler.h and vm.h with symbol table support, including getSymbol() for method name resolution. Introduce CODE_CALL opcode in VM interpreter with basic call dispatch. Update main.c to use new VM lifecycle (newVM/freeVM) and pass VM through compile and interpret calls.
2013-10-23 22:51:41 +02:00
{
feat: replace Scope stack with Local array to fix variable shadowing in nested blocks Replace the linked-list Scope structure with a flat Local array indexed by depth, enabling proper shadowing semantics where inner declarations correctly hide outer ones. Remove the PUSH_SCOPE/POP_SCOPE macros and the SymbolTable-based locals tracking, replacing them with a MAX_LOCALS (255) fixed-size array that stores variable name, length, and depth. Add four new test cases covering local/parameter collision, nested block scoping, global shadowing, and local shadowing to validate the new behavior.
2013-12-01 03:28:01 +01:00
// Look up the name in the scope chain.
feat: add superclass call support inside closures in wren compiler Add a new test case `test/super/closure.wren` verifying that `super.toString` correctly resolves to the base class method when called from within a closure defined in a derived class. Refactor `resolveLocal` and `resolveUpvalue` to accept explicit name and length parameters instead of relying on the previously consumed token, enabling proper name resolution in nested closure scopes during superclass method dispatch.
2013-12-23 05:46:12 +01:00
Token* token = &compiler->parser->previous;
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
Code loadInstruction;
int index = resolveName(compiler, token->start, token->length,
&loadInstruction);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (index == -1) error(compiler, "Undefined variable.");
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
// If there's an "=" after a bare name, it's a variable assignment.
if (match(compiler, TOKEN_EQ))
feat: add string literal parsing and runtime string object support Add a new string() compiler function to handle string literal tokens by stripping surrounding quotes, allocating a heap copy, and emitting a constant load instruction. Introduce the OBJ_STRING object type with ObjString struct and makeString() factory, wire it into the VM's class dispatch for string receivers, and implement printValue output for string values. Extend the VM state with a stringClass field and update the hello.wren example to include a string literal.
2013-10-26 05:40:24 +02:00
{
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
if (!allowAssignment) error(compiler, "Invalid assignment.");
// Compile the right-hand side.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
expression(compiler);
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
// Emit the store instruction.
switch (loadInstruction)
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
case CODE_LOAD_LOCAL: emit1(compiler, CODE_STORE_LOCAL, index); break;
case CODE_LOAD_UPVALUE: emit1(compiler, CODE_STORE_UPVALUE, index); break;
case CODE_LOAD_GLOBAL: emit1(compiler, CODE_STORE_GLOBAL, index); break;
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
default:
UNREACHABLE();
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
}
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
}
else
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, loadInstruction, index);
feat: add string literal parsing and runtime string object support Add a new string() compiler function to handle string literal tokens by stripping surrounding quotes, allocating a heap copy, and emitting a constant load instruction. Introduce the OBJ_STRING object type with ObjString struct and makeString() factory, wire it into the VM's class dispatch for string receivers, and implement printValue output for string values. Extend the VM state with a stringClass field and update the hello.wren example to include a string literal.
2013-10-26 05:40:24 +02:00
}
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void null(Compiler* compiler, bool allowAssignment)
feat: add null literal token and compiler support with keyword parsing and test cases Introduce TOKEN_NULL to the token enum and register it as a prefix parse rule that emits CODE_NULL. Implement the null() compiler function to emit the null constant. Extend readName() to recognize the "null" keyword and map it to TOKEN_NULL. Add test files verifying null output, and that null, true, and false cannot be used as variable names.
2013-11-06 03:22:22 +01:00
{
emit(compiler, CODE_NULL);
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void number(Compiler* compiler, bool allowAssignment)
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
{
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
Token* token = &compiler->parser->previous;
fix: parse number literal using strtol and handle invalid input in compiler Replace the hardcoded constant value 234 with actual parsing of the token's numeric string via strtol, adding error handling for invalid number literals and a TODO for truncation when casting long to int. Also add a newline after printing the interpreted value in main.c.
2013-10-22 22:37:53 +02:00
char* end;
feat: implement floating-point number literal parsing and arithmetic operations Add support for floating-point number literals in the parser by detecting decimal points with trailing digits in `readNumber()`, and switch the compiler's `number()` function from `strtol` to `strtod` for proper double conversion. Update the `num_abs` primitive to use `fabs()` instead of manual negation. Extend test coverage across arithmetic, comparison, equality, modulus, and string conversion tests with floating-point cases, and add a new test for decimal point at end-of-file error detection.
2013-11-10 06:32:57 +01:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
double value = strtod(token->start, &end);
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Check errno == ERANGE here.
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
if (end == token->start)
fix: parse number literal using strtol and handle invalid input in compiler Replace the hardcoded constant value 234 with actual parsing of the token's numeric string via strtol, adding error handling for invalid number literals and a TODO for truncation when casting long to int. Also add a newline after printing the interpreted value in main.c.
2013-10-22 22:37:53 +02:00
{
error(compiler, "Invalid number literal.");
value = 0;
}
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
// Define a constant for the literal.
refactor: unbox numeric values by storing double directly in Value struct Remove the ObjNum heap-allocated wrapper for numbers, storing the double value inline in the Value union. This eliminates OBJ_NUM enum case, the newNum() constructor, and all ObjNum-related code in the GC, printing, and type dispatch. Replace all (Value)newNum(vm, ...) calls with the NUM_VAL() macro and update AS_NUM() to read from the inline field. Adjust IS_FN/IS_STRING macros to use IS_OBJ() guard and update all Value literal macros to include the new num field initializer.
2013-11-16 20:41:29 +01:00
int constant = addConstant(compiler, NUM_VAL(value));
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
// Compile the code to load the constant.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_CONSTANT, constant);
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void string(Compiler* compiler, bool allowAssignment)
feat: add string literal parsing and runtime string object support Add a new string() compiler function to handle string literal tokens by stripping surrounding quotes, allocating a heap copy, and emitting a constant load instruction. Introduce the OBJ_STRING object type with ObjString struct and makeString() factory, wire it into the VM's class dispatch for string receivers, and implement printValue output for string values. Extend the VM state with a stringClass field and update the hello.wren example to include a string literal.
2013-10-26 05:40:24 +02:00
{
// Define a constant for the literal.
refactor: rename internal value creation functions to wren-prefixed public API names Move `allocate`, `initObj`, `newClass`, and related static helpers from `vm.c` into `value.c` with consistent `wren` prefix, and update all call sites in `compiler.c`, `primitives.c`, and `vm.c` to use the new names (`wrenNewFunction`, `wrenNewString`, `wrenNewClass`, `wrenNewInstance`, `wrenNewList`). Remove the old declarations from `vm.h` and add comprehensive documentation comments to `value.h` explaining the NaN-tagging representation and type system.
2013-11-28 05:00:03 +01:00
int constant = addConstant(compiler, wrenNewString(compiler->parser->vm,
feat: implement string escape sequence parsing in lexer and add test coverage Add escape sequence handling to the string literal lexer in compiler.c, translating common escape sequences (\", \\, \n) into their literal character equivalents during tokenization. Introduce a fixed-size buffer (MAX_STRING 1024) in the Parser struct to accumulate the unescaped string content, along with a helper function addStringChar() to append characters. The readString() function now loops over input characters, breaking on an unescaped double quote, and processing backslash-prefixed escapes via a switch statement. Remove the previous TODO placeholder comment and the elapsed time computation simplification in benchmark/fib.lua. Add a new test file test/string_escapes.wren that verifies the three supported escape sequences, and remove the now-obsolete TODO about escapes from test/string_literals.wren.
2013-11-24 06:00:47 +01:00
compiler->parser->currentString, compiler->parser->currentStringLength));
feat: implement mark-sweep garbage collector with object linked list and pinned roots Replace manual malloc/free for Obj allocations with GC-managed heap. Add Obj->next linked list, markObj traversal for all object types, and sweep phase in VM. Introduce MAX_PINNED stack for temporary GC roots during compilation. Refactor all object constructors (newBool, newFunction, newClass, newNum, newString) to accept VM pointer and register allocations. Move function constant registration earlier in compiler to ensure GC reachability. Add DEBUG_GC_STRESS and TRACE_MEMORY macros in new common.h header.
2013-11-12 17:32:35 +01:00
feat: add string literal parsing and runtime string object support Add a new string() compiler function to handle string literal tokens by stripping surrounding quotes, allocating a heap copy, and emitting a constant load instruction. Introduce the OBJ_STRING object type with ObjString struct and makeString() factory, wire it into the VM's class dispatch for string receivers, and implement printValue output for string values. Extend the VM state with a stringClass field and update the hello.wren example to include a string literal.
2013-10-26 05:40:24 +02:00
// Compile the code to load the constant.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_CONSTANT, constant);
feat: add string literal parsing and runtime string object support Add a new string() compiler function to handle string literal tokens by stripping surrounding quotes, allocating a heap copy, and emitting a constant load instruction. Introduce the OBJ_STRING object type with ObjString struct and makeString() factory, wire it into the VM's class dispatch for string receivers, and implement printValue output for string values. Extend the VM state with a stringClass field and update the hello.wren example to include a string literal.
2013-10-26 05:40:24 +02:00
}
feat: add compile-time error for 'super' used outside method context Add isInsideMethod helper that walks the compiler's parent chain to detect if the current compilation scope is within a method. Use it in super_() to emit a compile error when 'super' is used at top level or inside a top-level function, with corresponding test cases for both scenarios.
2013-12-16 07:08:40 +01:00
// Returns true if [compiler] is compiling a chunk of code that is either
// directly or indirectly contained in a method for a class.
static bool isInsideMethod(Compiler* compiler)
{
// Walk up the parent chain to see if there is an enclosing method.
while (compiler != NULL)
{
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
if (compiler->methodName != NULL) return true;
feat: add compile-time error for 'super' used outside method context Add isInsideMethod helper that walks the compiler's parent chain to detect if the current compilation scope is within a method. Use it in super_() to emit a compile error when 'super' is used at top level or inside a top-level function, with corresponding test cases for both scenarios.
2013-12-16 07:08:40 +01:00
compiler = compiler->parent;
}
return false;
}
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
static void new_(Compiler* compiler, bool allowAssignment)
{
// TODO: Instead of an expression, explicitly only allow a dotted name.
// Compile the class.
parsePrecedence(compiler, false, PREC_CALL);
// Create the instance of the class.
emit(compiler, CODE_NEW);
// Invoke the constructor on the new instance.
char name[MAX_METHOD_SIGNATURE];
strcpy(name, "new");
methodCall(compiler, CODE_CALL_0, name, 3);
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void super_(Compiler* compiler, bool allowAssignment)
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
{
feat: add compile-time error for 'super' used outside method context Add isInsideMethod helper that walks the compiler's parent chain to detect if the current compilation scope is within a method. Use it in super_() to emit a compile error when 'super' is used at top level or inside a top-level function, with corresponding test cases for both scenarios.
2013-12-16 07:08:40 +01:00
if (!isInsideMethod(compiler))
{
error(compiler, "Cannot use 'super' outside of a method.");
}
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
loadThis(compiler);
refactor: replace ResolvedName enum with direct Code instruction in resolveName Remove the ResolvedName enum and its typedef, replacing the resolved parameter with a Code* loadInstruction pointer that directly returns the appropriate load instruction (CODE_LOAD_LOCAL, CODE_LOAD_UPVALUE, or CODE_LOAD_GLOBAL). This eliminates the intermediate enum and the TODO comment about ditching it, simplifying the variable resolution logic in the compiler.
2013-12-23 05:53:35 +01:00
feat: implement superclass constructor calls and metaclass inheritance chain Add support for explicit superclass constructor invocation in subclass constructors using `super.constructorName(args)` syntax before the opening brace. The metaclass inheritance hierarchy now mirrors the regular class chain, with `Class` as the root metaclass superclass instead of `Object`. Includes test cases for multi-level constructor delegation and static method inheritance.
2013-12-17 18:39:05 +01:00
// TODO: Super operator calls.
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
// See if it's a named super call, or an unnamed one.
if (match(compiler, TOKEN_DOT))
{
// Compile the superclass call.
namedCall(compiler, allowAssignment, CODE_SUPER_0);
}
else
{
// No explicit name, so use the name of the enclosing method.
char name[MAX_METHOD_SIGNATURE];
int length = 0;
Compiler* thisCompiler = compiler;
while (thisCompiler != NULL)
{
if (thisCompiler->methodName != NULL)
{
length = thisCompiler->methodLength;
strncpy(name, thisCompiler->methodName, length);
break;
}
thisCompiler = thisCompiler->parent;
}
// Call the superclass method with the same name.
methodCall(compiler, CODE_SUPER_0, name, length);
}
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void this_(Compiler* compiler, bool allowAssignment)
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
{
feat: add compile-time error for 'super' used outside method context Add isInsideMethod helper that walks the compiler's parent chain to detect if the current compilation scope is within a method. Use it in super_() to emit a compile error when 'super' is used at top level or inside a top-level function, with corresponding test cases for both scenarios.
2013-12-16 07:08:40 +01:00
if (!isInsideMethod(compiler))
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
{
error(compiler, "Cannot use 'this' outside of a method.");
feat: implement closure over "this" in methods and fix upvalue closing order Add support for closing over "this" in function closures defined inside methods by naming the receiver local variable "this" in method compilers. Fix upvalue closing to copy the actual value instead of the stack top, and reorder return handling to close upvalues before storing the result. Add test cases for closure over "this", nested closures, and nested classes.
2013-12-22 00:55:08 +01:00
return;
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
}
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
loadThis(compiler);
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
}
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
// Subscript or "array indexing" operator like `foo[bar]`.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
static void subscript(Compiler* compiler, bool allowAssignment)
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
{
refactor: rename MAX_METHOD_NAME to MAX_METHOD_SIGNATURE and extract copyName helper Extract identifier copying logic from subscript and call functions into a dedicated copyName helper function, reducing code duplication. Rename the method signature length constant to MAX_METHOD_SIGNATURE with updated documentation to clarify it includes arity spaces. Update subscript function to use the new constant name and remove stale TODO comment about handling >10 args.
2013-12-01 09:04:46 +01:00
char name[MAX_METHOD_SIGNATURE];
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
int length = 1;
int numArgs = 0;
// Build the method name. To allow overloading by arity, we add a space to
// the name for each argument.
name[0] = '[';
// Parse the argument list.
do
{
feat: add list subscript setter and refactor parameter validation Implement list subscript setter (`[ ]=`) native method in wren_core.c, enabling assignment to list elements via bracket syntax. Extract duplicate parameter count validation logic into a shared `validateNumParameters` function in wren_compiler.c, replacing inline checks in both `parameterList` and `methodCall`. Add comprehensive test suite covering basic assignment, return value semantics, negative indices, and error cases for out-of-range and excessive arguments.
2013-12-21 18:37:59 +01:00
validateNumParameters(compiler, ++numArgs);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
expression(compiler);
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
// Add a space in the name for each argument. Lets us overload by
// arity.
name[length++] = ' ';
}
while (match(compiler, TOKEN_COMMA));
feat: raise max method parameters from 10 to 16 with proper error handling Add MAX_PARAMETERS constant set to 16 and extend VM bytecode instructions CODE_CALL_11 through CODE_CALL_16 to support the new limit. Introduce compile-time error messages when parameter or argument count exceeds 16 in function definitions, method declarations, subscript calls, and regular method calls. Add new native fiber call stubs fn_call9 through fn_call16 and register them in the core initialization. Include comprehensive test coverage for 9-16 parameter functions and methods, plus error tests for exceeding the limit in all contexts.
2013-11-30 00:08:27 +01:00
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
consume(compiler, TOKEN_RIGHT_BRACKET, "Expect ']' after arguments.");
name[length++] = ']';
feat: add list subscript setter and refactor parameter validation Implement list subscript setter (`[ ]=`) native method in wren_core.c, enabling assignment to list elements via bracket syntax. Extract duplicate parameter count validation logic into a shared `validateNumParameters` function in wren_compiler.c, replacing inline checks in both `parameterList` and `methodCall`. Add comprehensive test suite covering basic assignment, return value semantics, negative indices, and error cases for out-of-range and excessive arguments.
2013-12-21 18:37:59 +01:00
if (match(compiler, TOKEN_EQ))
{
if (!allowAssignment) error(compiler, "Invalid assignment.");
name[length++] = '=';
// Compile the assigned value.
validateNumParameters(compiler, ++numArgs);
expression(compiler);
}
int symbol = ensureSymbol(&compiler->parser->vm->methods, name, length);
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
// Compile the method call.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_CALL_0 + numArgs, symbol);
feat: add subscript operator for array indexing and string character access Implement the `[]` subscript operator as an infix parser at `PREC_CALL` precedence, compiling to a method call on the receiver. Add a `string_subscript` primitive that handles integer indices (including negative for reverse indexing), returns `null` for out-of-bounds or non-integer arguments, and returns a one-character string. Register the `[ ]` method on the string class and include tests for basic indexing, negative indices, out-of-bounds, wrong types, and non-integer indices.
2013-11-24 19:45:07 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void call(Compiler* compiler, bool allowAssignment)
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
{
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
namedCall(compiler, allowAssignment, CODE_CALL_0);
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void is(Compiler* compiler, bool allowAssignment)
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
{
// Compile the right-hand side.
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
parsePrecedence(compiler, false, PREC_CALL);
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
emit(compiler, CODE_IS);
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void and(Compiler* compiler, bool allowAssignment)
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
{
// Skip the right argument if the left is false.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
int jump = emit1(compiler, CODE_AND, 255);
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
parsePrecedence(compiler, false, PREC_LOGIC);
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
patchJump(compiler, jump);
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void or(Compiler* compiler, bool allowAssignment)
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
{
// Skip the right argument if the left is true.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
int jump = emit1(compiler, CODE_OR, 255);
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
parsePrecedence(compiler, false, PREC_LOGIC);
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
patchJump(compiler, jump);
}
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void infixOp(Compiler* compiler, bool allowAssignment)
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
{
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
GrammarRule* rule = &rules[compiler->parser->previous.type];
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
// Compile the right-hand side.
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
parsePrecedence(compiler, false, rule->precedence + 1);
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
// Call the operator method on the left-hand side.
refactor: rename `symbols` to `methods` and `defineName` to `declareVariable` in compiler and VM Rename the `SymbolTable symbols` field in `VM` struct to `methods` to clarify its purpose for method dispatch, and update all references across `vm.c`, `vm.h`, `primitives.c`, and `compiler.c`. Rename `defineName` to `declareVariable` to better reflect its actual behavior of declaring a variable without defining it. Remove the unused `internSymbol` helper function. Add `clearSymbolTable(&vm->globalSymbols)` to `freeVM` to fix a memory leak. Update `dumpCode` signature to take `VM*` and `ObjFn*` instead of `ObjBlock*`, and display method names via `getSymbolName`. Add `stackStart` field to `CallFrame` replacing `locals` for clearer semantics. Introduce `test/method_arity.wren` to validate method dispatch with up to 10 parameters.
2013-11-09 20:18:27 +01:00
int symbol = ensureSymbol(&compiler->parser->vm->methods,
feat: add boolean literals and number comparison operators to compiler and VM Add TOKEN_TRUE/TOKEN_FALSE parsing with boolean() prefix rule emitting CODE_TRUE/CODE_FALSE bytecodes. Wire comparison operators <, >, <=, >= as infix rules with PREC_COMPARISON precedence. Implement num_lt, num_gt, num_lte, num_gte primitives returning bool values. Register boolClass with toString primitive and move class initialization into registerPrimitives(). Add OBJ_TRUE/OBJ_FALSE object types with makeBool() helper and AS_BOOL macro. Include test files for bool_toString and number_comparison.
2013-11-04 06:38:58 +01:00
rule->name, strlen(rule->name));
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_CALL_1, symbol);
feat: replace recursive descent parsing with Pratt parser for expression handling Replace the hand-written recursive descent parsing functions (term, factor, primary) with a table-driven Pratt parser using ParseRule structs. The new implementation introduces a parsePrecedence function that dispatches to prefix and infix parse functions based on token type and precedence levels, enabling cleaner extensibility for future operators. Adds precedence enum (PREC_NONE through PREC_CALL) and ParseRule table mapping token types to their corresponding parse functions and binding power.
2013-11-03 19:16:14 +01:00
}
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
// Compiles a method signature for an infix operator.
void infixSignature(Compiler* compiler, char* name, int* length)
{
// Add a space for the RHS parameter.
name[(*length)++] = ' ';
// Parse the parameter name.
declareVariable(compiler);
}
// Compiles a method signature for an unary operator (i.e. "!").
void unarySignature(Compiler* compiler, char* name, int* length)
{
// Do nothing. The name is already complete.
}
// Compiles a method signature for an operator that can either be unary or
// infix (i.e. "-").
void mixedSignature(Compiler* compiler, char* name, int* length)
{
// If there is a parameter name, it's an infix operator, otherwise it's unary.
if (compiler->parser->current.type == TOKEN_NAME)
{
// Add a space for the RHS parameter.
name[(*length)++] = ' ';
// Parse the parameter name.
declareVariable(compiler);
}
}
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
// Compiles a method signature for a named method or setter.
void namedSignature(Compiler* compiler, char* name, int* length)
{
if (match(compiler, TOKEN_EQ))
{
// It's a setter.
// TODO: Allow setters with parameters? Like: foo.bar(1, 2) = "blah"
name[(*length)++] = '=';
name[(*length)++] = ' ';
// Parse the value parameter.
declareVariable(compiler);
}
else
{
// Regular named method with an optional parameter list.
parameterList(compiler, name, length);
}
}
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
// Compiles a method signature for a constructor.
void constructorSignature(Compiler* compiler, char* name, int* length)
{
// Add the parameters, if there are any.
parameterList(compiler, name, length);
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// This table defines all of the parsing rules for the prefix and infix
// expressions in the grammar. Expressions are parsed using a Pratt parser.
//
// See: http://journal.stuffwithstuff.com/2011/03/19/pratt-parsers-expression-parsing-made-easy/
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
#define UNUSED { NULL, NULL, NULL, PREC_NONE, NULL }
#define PREFIX(fn) { fn, NULL, NULL, PREC_NONE, NULL }
#define INFIX(prec, fn) { NULL, fn, NULL, prec, NULL }
#define INFIX_OPERATOR(prec, name) { NULL, infixOp, infixSignature, prec, name }
#define PREFIX_OPERATOR(name) { unaryOp, NULL, unarySignature, PREC_NONE, name }
feat: add bitwise NOT operator (~) with token, parser, and native implementation Add TOKEN_TILDE token type and lexer support for the '~' character in the compiler. Implement the bitwise NOT operation as a native method on the Num class, operating on 32-bit unsigned integers. Include a new OPERATOR macro for prefix/infix operator grammar rules and add comprehensive test cases covering positive, negative, floating-point, and boundary values.
2013-12-05 07:09:31 +01:00
#define OPERATOR(name) { unaryOp, infixOp, mixedSignature, PREC_TERM, name }
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
GrammarRule rules[] =
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_LEFT_PAREN */ PREFIX(grouping),
/* TOKEN_RIGHT_PAREN */ UNUSED,
feat: implement list literal syntax and List class with count property Add support for list literals in the compiler with bracket-delimited syntax and comma-separated elements. Introduce the ObjList runtime type with a contiguous elements array, garbage collection marking, and proper memory deallocation. Expose a `List` class in the core library with a `count` method returning the number of elements. Update the grammar table to register `[` as a prefix token for list compilation instead of an infix subscript operator, and reorder opcode enum entries to place CODE_LIST before the load/store group.
2013-11-24 22:38:31 +01:00
/* TOKEN_LEFT_BRACKET */ { list, subscript, NULL, PREC_CALL, NULL },
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_RIGHT_BRACKET */ UNUSED,
/* TOKEN_LEFT_BRACE */ UNUSED,
/* TOKEN_RIGHT_BRACE */ UNUSED,
/* TOKEN_COLON */ UNUSED,
/* TOKEN_DOT */ INFIX(PREC_CALL, call),
/* TOKEN_COMMA */ UNUSED,
/* TOKEN_STAR */ INFIX_OPERATOR(PREC_FACTOR, "* "),
/* TOKEN_SLASH */ INFIX_OPERATOR(PREC_FACTOR, "/ "),
/* TOKEN_PERCENT */ INFIX_OPERATOR(PREC_TERM, "% "),
/* TOKEN_PLUS */ INFIX_OPERATOR(PREC_TERM, "+ "),
feat: add bitwise NOT operator (~) with token, parser, and native implementation Add TOKEN_TILDE token type and lexer support for the '~' character in the compiler. Implement the bitwise NOT operation as a native method on the Num class, operating on 32-bit unsigned integers. Include a new OPERATOR macro for prefix/infix operator grammar rules and add comprehensive test cases covering positive, negative, floating-point, and boundary values.
2013-12-05 07:09:31 +01:00
/* TOKEN_MINUS */ OPERATOR("- "),
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_PIPE */ UNUSED,
feat: add logical OR operator with short-circuit evaluation and token support Add TOKEN_PIPEPIPE token for '||' syntax, implement or() parsing function with CODE_OR bytecode, and add VM interpreter logic that short-circuits on truthy left operand. Update .gitignore to exclude *.xccheckout files, extend test coverage for falsy behavior in and.wren and if.wren, and add comprehensive or.wren test suite verifying short-circuit semantics and truthiness rules.
2013-11-20 03:24:58 +01:00
/* TOKEN_PIPEPIPE */ INFIX(PREC_LOGIC, or),
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_AMP */ UNUSED,
feat: implement logical AND operator with short-circuit evaluation and new CODE_AND instruction Add TOKEN_AMPAMP token to the lexer for parsing '&&' operator, introduce PREC_LOGIC precedence level for proper expression parsing, implement the `and()` compiler function that emits CODE_AND with a patchable jump offset, and add runtime support in the VM with IS_FALSE macro for truthiness checking. Include comprehensive test cases in test/and.wren verifying short-circuit behavior and return values.
2013-11-19 16:35:25 +01:00
/* TOKEN_AMPAMP */ INFIX(PREC_LOGIC, and),
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
/* TOKEN_BANG */ PREFIX_OPERATOR("!"),
feat: add bitwise NOT operator (~) with token, parser, and native implementation Add TOKEN_TILDE token type and lexer support for the '~' character in the compiler. Implement the bitwise NOT operation as a native method on the Num class, operating on 32-bit unsigned integers. Include a new OPERATOR macro for prefix/infix operator grammar rules and add comprehensive test cases covering positive, negative, floating-point, and boundary values.
2013-12-05 07:09:31 +01:00
/* TOKEN_TILDE */ PREFIX_OPERATOR("~"),
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_EQ */ UNUSED,
/* TOKEN_LT */ INFIX_OPERATOR(PREC_COMPARISON, "< "),
/* TOKEN_GT */ INFIX_OPERATOR(PREC_COMPARISON, "> "),
/* TOKEN_LTEQ */ INFIX_OPERATOR(PREC_COMPARISON, "<= "),
/* TOKEN_GTEQ */ INFIX_OPERATOR(PREC_COMPARISON, ">= "),
/* TOKEN_EQEQ */ INFIX_OPERATOR(PREC_EQUALITY, "== "),
/* TOKEN_BANGEQ */ INFIX_OPERATOR(PREC_EQUALITY, "!= "),
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
/* TOKEN_BREAK */ UNUSED,
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_CLASS */ UNUSED,
/* TOKEN_ELSE */ UNUSED,
/* TOKEN_FALSE */ PREFIX(boolean),
/* TOKEN_FN */ PREFIX(function),
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
/* TOKEN_FOR */ UNUSED,
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_IF */ UNUSED,
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
/* TOKEN_IN */ UNUSED,
feat: add `is` operator for type checking and refactor primitives registration Add TOKEN_IS token and CODE_IS bytecode to support runtime type checking via the `is` keyword. Refactor primitive registration by replacing `registerPrimitives` with `loadCore`, which compiles a core library string to define built-in classes (Bool, Class, Function, Num, Null, String, IO) and then attaches primitive methods to them. Update Primitive typedef to remove unused numArgs parameter, and add findGlobal helper to retrieve globals by name. Include test files for class syntax, class instantiation, and the new `is` operator.
2013-11-08 02:07:32 +01:00
/* TOKEN_IS */ INFIX(PREC_IS, is),
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
/* TOKEN_NEW */ { new_, NULL, constructorSignature, PREC_NONE, NULL },
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_NULL */ PREFIX(null),
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
/* TOKEN_RETURN */ UNUSED,
feat: add static method support to class definitions with metaclass dispatch Add TOKEN_STATIC keyword to the lexer, replacing the unused TOKEN_META, and extend the method() compiler function with an isStatic parameter. When isStatic is true, emit a new CODE_METACLASS instruction before CODE_METHOD to push the class's metaclass onto the stack, enabling static method dispatch. Also fix a buffer overflow bug in string_plus by allocating one extra byte for the null terminator, and refactor object initialization across vm.c by introducing a shared initObj helper to eliminate repeated type/flag assignments.
2013-11-10 20:46:13 +01:00
/* TOKEN_STATIC */ UNUSED,
feat: implement superclass method dispatch with new SUPER bytecodes and named call compiler Add CODE_SUPER_0 through CODE_SUPER_16 bytecodes to the VM for invoking inherited methods, along with a `namedCall` compiler helper that parses method names and argument lists after a dot. The interpreter dispatches super calls by walking up the class hierarchy from the receiver's immediate class, skipping methods defined on that class. Includes three test cases covering calling a different superclass method, calling the same method name, and indirectly inherited methods. Also fixes error formatting to handle newline tokens gracefully and adds a TODO note about not deduplicating placeholder constants for super calls.
2013-12-13 17:37:49 +01:00
/* TOKEN_SUPER */ PREFIX(super_),
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
/* TOKEN_THIS */ PREFIX(this_),
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_TRUE */ PREFIX(boolean),
/* TOKEN_VAR */ UNUSED,
feat: add while loop token, parser support, CODE_LOOP opcode, and patchJump helper Add TOKEN_WHILE to the token enum and keyword lookup in the compiler, enabling the parser to recognize 'while' statements. Introduce the CODE_LOOP opcode in the VM for backward jumps, and implement the patchJump helper to resolve forward jump offsets during compilation. Include comprehensive test cases for while loops covering single-expression bodies, block bodies, newline handling, and null result semantics.
2013-11-18 07:38:59 +01:00
/* TOKEN_WHILE */ UNUSED,
feat: add instance field support with lexer, compiler, and GC tracing Add TOKEN_FIELD token type and '_' prefix handling in lexer to distinguish field identifiers from regular names. Extend Compiler struct with fields symbol table pointer for tracking enclosing class fields. Implement field() parse function and CODE_LOAD_FIELD/CODE_STORE_FIELD bytecodes in VM. Modify ObjClass to store numFields count and ObjInstance to use flexible array member for field storage. Update GC markInstance to trace all instance fields. Add newClass signature to accept numFields parameter. Include test cases for field default null, multiple fields, object references with GC, and use-before-set ordering.
2013-11-23 23:55:05 +01:00
/* TOKEN_FIELD */ PREFIX(field),
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
/* TOKEN_NAME */ { name, NULL, namedSignature, PREC_NONE, NULL },
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
/* TOKEN_NUMBER */ PREFIX(number),
/* TOKEN_STRING */ PREFIX(string),
/* TOKEN_LINE */ UNUSED,
/* TOKEN_ERROR */ UNUSED,
/* TOKEN_EOF */ UNUSED
};
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// The main entrypoint for the top-down operator precedence parser.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void parsePrecedence(Compiler* compiler, bool allowAssignment,
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
Precedence precedence)
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
{
refactor: reorganize compiler source by grouping static function declarations and adding doc comments for parsing and lexing phases
2013-10-28 05:59:14 +01:00
nextToken(compiler->parser);
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
GrammarFn prefix = rules[compiler->parser->previous.type].prefix;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
if (prefix == NULL)
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
{
feat: disallow field references outside class definitions with explicit error Add a check in the `field()` function to detect when a field is referenced outside of a class context. When `compiler->fields` is NULL (indicating no enclosing class), emit a clear error message "Cannot reference a field outside of a class definition." and assign a dummy field index of 255 to allow continued parsing with minimal cascading errors. Also includes minor cleanup: remove two stale TODO comments about EOF checking and better error messages in `nextToken()` and `consume()`, and add a new test file `test/field/outside_class.wren` that verifies the error is produced for top-level field assignment.
2013-11-29 19:53:56 +01:00
error(compiler, "Unexpected token for expression.");
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
}
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
prefix(compiler, allowAssignment);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
while (precedence <= rules[compiler->parser->current.type].precedence)
{
nextToken(compiler->parser);
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
GrammarFn infix = rules[compiler->parser->previous.type].infix;
infix(compiler, allowAssignment);
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Parses an expression. Unlike statements, expressions leave a resulting value
// on the stack.
void expression(Compiler* compiler)
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
{
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
parsePrecedence(compiler, true, PREC_LOWEST);
feat: add unary operators and variable assignment with precedence handling Implement unary negation for numbers and logical not for booleans, introduce PREC_ASSIGNMENT and PREC_UNARY precedence levels, refactor parse functions to accept allowAssignment parameter, add assignment tests for global/local/chained/error cases, and update if-syntax to support statement branches and assignment in conditions.
2013-11-13 16:10:52 +01:00
}
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
// Compiles a method definition inside a class body.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
void method(Compiler* compiler, Code instruction, bool isConstructor,
feat: replace constructor method special-casing with dedicated CODE_NEW instruction Remove the METHOD_CTOR method type and CODE_METHOD_CTOR opcode, replacing them with a new CODE_NEW instruction that creates class instances directly in the bytecode. This simplifies the compiler by emitting CODE_NEW at the start of constructor bodies instead of relying on a separate method dispatch path, and eliminates the redundant METHOD_CTOR enum value from the method lookup logic.
2013-12-13 20:32:47 +01:00
SignatureFn signature)
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
{
// Build the method name.
refactor: rename MAX_METHOD_NAME to MAX_METHOD_SIGNATURE and extract copyName helper Extract identifier copying logic from subscript and call functions into a dedicated copyName helper function, reducing code duplication. Rename the method signature length constant to MAX_METHOD_SIGNATURE with updated documentation to clarify it includes arity spaces. Update subscript function to use the new constant name and remove stale TODO comment about handling >10 args.
2013-12-01 09:04:46 +01:00
char name[MAX_METHOD_SIGNATURE];
int length = copyName(compiler, name);
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
Compiler methodCompiler;
int constant = initCompiler(&methodCompiler, compiler->parser, compiler,
name, length);
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
// Compile the method signature.
signature(&methodCompiler, name, &length);
int symbol = ensureSymbol(&compiler->parser->vm->methods, name, length);
feat: implement superclass constructor calls and metaclass inheritance chain Add support for explicit superclass constructor invocation in subclass constructors using `super.constructorName(args)` syntax before the opening brace. The metaclass inheritance hierarchy now mirrors the regular class chain, with `Class` as the root metaclass superclass instead of `Object`. Includes test cases for multi-level constructor delegation and static method inheritance.
2013-12-17 18:39:05 +01:00
consume(compiler, TOKEN_LEFT_BRACE, "Expect '{' to begin method body.");
feat: replace constructor method special-casing with dedicated CODE_NEW instruction Remove the METHOD_CTOR method type and CODE_METHOD_CTOR opcode, replacing them with a new CODE_NEW instruction that creates class instances directly in the bytecode. This simplifies the compiler by emitting CODE_NEW at the start of constructor bodies instead of relying on a separate method dispatch path, and eliminates the redundant METHOD_CTOR enum value from the method lookup logic.
2013-12-13 20:32:47 +01:00
chore: disable DEBUG_GC_STRESS and refactor initCompiler with addConstant helper
2013-11-20 05:54:47 +01:00
finishBlock(&methodCompiler);
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Single-expression methods that implicitly return the result.
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// If it's a constructor, return "this".
feat: replace constructor method special-casing with dedicated CODE_NEW instruction Remove the METHOD_CTOR method type and CODE_METHOD_CTOR opcode, replacing them with a new CODE_NEW instruction that creates class instances directly in the bytecode. This simplifies the compiler by emitting CODE_NEW at the start of constructor bodies instead of relying on a separate method dispatch path, and eliminates the redundant METHOD_CTOR enum value from the method lookup logic.
2013-12-13 20:32:47 +01:00
if (isConstructor)
feat: add named user-defined constructors with this keyword prefix in class bodies
2013-11-20 16:20:16 +01:00
{
// The receiver is always stored in the first local slot.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(&methodCompiler, CODE_LOAD_LOCAL, 0);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
}
else
{
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
// Implicitly return null in case there is no explicit return.
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
emit(&methodCompiler, CODE_NULL);
feat: add named user-defined constructors with this keyword prefix in class bodies
2013-11-20 16:20:16 +01:00
}
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
emit(&methodCompiler, CODE_RETURN);
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
endCompiler(&methodCompiler, constant);
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
// Compile the code to define the method.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, instruction, symbol);
feat: add user-defined operator methods with signature compilation Add support for user-defined operator methods in classes by introducing a `SignatureFn` field to `GrammarRule` and implementing four signature compilation functions: `namedSignature`, `infixSignature`, `unarySignature`, and `mixedSignature`. These functions handle parameter parsing and name mangling for different operator types (infix, prefix, and mixed unary/infix). The change includes a new test file `test/method_operators.wren` that validates all supported operator overloads including arithmetic, comparison, and prefix operators.
2013-11-14 19:27:33 +01:00
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Parses a curly block or an expression statement. Used in places like the
// arms of an if statement where either a single expression or a curly body is
// allowed.
void block(Compiler* compiler)
{
// Curly block.
if (match(compiler, TOKEN_LEFT_BRACE))
{
pushScope(compiler);
finishBlock(compiler);
popScope(compiler);
return;
}
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
// Single statement body.
statement(compiler);
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
// Returns the number of arguments to the instruction at [ip] in [fn]'s
// bytecode.
static int getNumArguments(ObjFn* fn, int ip)
{
Code instruction = fn->bytecode[ip];
switch (instruction)
{
case CODE_NULL:
case CODE_FALSE:
case CODE_TRUE:
case CODE_POP:
case CODE_IS:
case CODE_CLOSE_UPVALUE:
case CODE_RETURN:
case CODE_NEW:
return 0;
// Instructions with two arguments:
case CODE_METHOD_INSTANCE:
case CODE_METHOD_STATIC:
return 2;
case CODE_CLOSURE:
{
int constant = fn->bytecode[ip + 1];
ObjFn* loadedFn = AS_FN(fn->constants[constant]);
// There is an argument for the constant, then one for each upvalue.
return 1 + loadedFn->numUpvalues;
}
default:
// Most instructions have one argument.
return 1;
}
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
static int startLoopBody(Compiler* compiler)
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
{
int outerLoopBody = compiler->loopBody;
compiler->loopBody = compiler->numCodes;
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
return outerLoopBody;
}
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
static void endLoopBody(Compiler* compiler, int outerLoopBody)
{
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
// Find any break placeholder instructions (which will be CODE_END in the
// bytecode) and replace them with real jumps.
int i = compiler->loopBody;
while (i < compiler->numCodes)
{
if (compiler->fn->bytecode[i] == CODE_END)
{
compiler->fn->bytecode[i] = CODE_JUMP;
patchJump(compiler, i + 1);
i += 2;
}
else
{
// Skip this instruction and its arguments.
i += 1 + getNumArguments(compiler->fn, i);
}
}
compiler->loopBody = outerLoopBody;
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
static void forStatement(Compiler* compiler)
{
// A for statement like:
//
// for (i in sequence.expression) {
// IO.write(i)
// }
//
// Is compiled to bytecode almost as if the source looked like this:
//
// {
// var seq_ = sequence.expression
// var iter_
// while (true) {
// iter_ = seq_.iterate(iter_)
// if (!iter_) break
// var i = set_.iteratorValue(iter_)
// IO.write(i)
// }
// }
//
// It's not exactly this, because the synthetic variables `seq_` and `iter_`
// actually get names that aren't valid Wren identfiers. Also, the `while`
// and `break` are just the bytecode for explicit loops and jumps. But that's
// the basic idea.
//
// The important parts are:
// - The sequence expression is only evaluated once.
// - The .iterate() method is used to advance the iterator and determine if
// it should exit the loop.
// - The .iteratorValue() method is used to get the value at the current
// iterator position.
// Create a scope for the hidden local variables used for the iterator.
pushScope(compiler);
consume(compiler, TOKEN_LEFT_PAREN, "Expect '(' after 'for'.");
consume(compiler, TOKEN_NAME, "Expect for loop variable name.");
// Remember the name of the loop variable.
const char* name = compiler->parser->previous.start;
int length = compiler->parser->previous.length;
consume(compiler, TOKEN_IN, "Expect 'in' after loop variable.");
// Evaluate the sequence expression and store it in a hidden local variable.
// The space in the variable name ensures it won't collide with a user-defined
// variable.
expression(compiler);
int seqSlot = defineLocal(compiler, "seq ", 4);
// Create another hidden local for the iterator object.
null(compiler, false);
int iterSlot = defineLocal(compiler, "iter ", 5);
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after loop expression.");
// Remember what instruction to loop back to.
int loopStart = compiler->numCodes - 1;
// Advance the iterator by calling the ".iterate" method on the sequence.
emit1(compiler, CODE_LOAD_LOCAL, seqSlot);
emit1(compiler, CODE_LOAD_LOCAL, iterSlot);
int iterateSymbol = ensureSymbol(&compiler->parser->vm->methods,
"iterate ", 8);
emit1(compiler, CODE_CALL_1, iterateSymbol);
// Store the iterator back in its local for the next iteration.
emit1(compiler, CODE_STORE_LOCAL, iterSlot);
// TODO: We can probably get this working with a bit less stack juggling.
// If it returned something falsy, jump out of the loop.
int exitJump = emit1(compiler, CODE_JUMP_IF, 255);
// Create a scope for the loop body.
pushScope(compiler);
// Get the current value in the sequence by calling ".iteratorValue".
emit1(compiler, CODE_LOAD_LOCAL, seqSlot);
emit1(compiler, CODE_LOAD_LOCAL, iterSlot);
int iteratorValueSymbol = ensureSymbol(&compiler->parser->vm->methods,
"iteratorValue ", 14);
emit1(compiler, CODE_CALL_1, iteratorValueSymbol);
// Bind it to the loop variable.
defineLocal(compiler, name, length);
// Compile the body.
int outerLoopBody = startLoopBody(compiler);
block(compiler);
popScope(compiler);
// Loop back to the top.
emit(compiler, CODE_LOOP);
int loopOffset = compiler->numCodes - loopStart;
emit(compiler, loopOffset);
patchJump(compiler, exitJump);
endLoopBody(compiler, outerLoopBody);
popScope(compiler);
}
static void whileStatement(Compiler* compiler)
{
// Remember what instruction to loop back to.
int loopStart = compiler->numCodes - 1;
// Compile the condition.
consume(compiler, TOKEN_LEFT_PAREN, "Expect '(' after 'while'.");
expression(compiler);
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after while condition.");
int exitJump = emit1(compiler, CODE_JUMP_IF, 255);
// Compile the body.
int outerLoopBody = startLoopBody(compiler);
block(compiler);
// Loop back to the top.
emit(compiler, CODE_LOOP);
int loopOffset = compiler->numCodes - loopStart;
emit(compiler, loopOffset);
patchJump(compiler, exitJump);
endLoopBody(compiler, outerLoopBody);
}
feat: implement setter syntax and assignment in named method calls Add support for setter methods in the compiler by detecting '=' after a method name in namedCall, compiling the assigned value, and emitting the setter instruction. Includes new test cases for instance setters, static setters, assignment associativity, grouping, operator precedence errors, setter result values, and same-name getter/setter methods. Also updates the super call test with a TODO for super setter calls.
2013-12-20 16:05:31 +01:00
// Compiles a statement. These can only appear at the top-level or within
// curly blocks. Unlike expressions, these do not leave a value on the stack.
void statement(Compiler* compiler)
{
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
if (match(compiler, TOKEN_BREAK))
{
if (compiler->loopBody == -1)
{
error(compiler, "Cannot use 'break' outside of a loop.");
}
// Emit a placeholder instruction for the jump to the end of the body. When
// we're done compiling the loop body and know where the end is, we'll
// replace these with `CODE_JUMP` instructions with appropriate offsets.
// We use `CODE_END` here because that can't occur in the middle of
// bytecode.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
emit1(compiler, CODE_END, 0);
feat: add break statement support with loop tracking and error handling Add TOKEN_BREAK keyword recognition and a loopBody field to the compiler to track the innermost loop's first instruction index. The break statement emits a jump to the end of the current loop, and compile-time errors are reported when break appears outside a loop body, inside a function defined within a loop, or inside a method defined within a loop. New test files verify correct behavior for break in while loops, nested loops, and error cases for invalid break usage.
2013-12-24 19:15:50 +01:00
return;
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
if (match(compiler, TOKEN_FOR)) return forStatement(compiler);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
if (match(compiler, TOKEN_IF))
{
// Compile the condition.
consume(compiler, TOKEN_LEFT_PAREN, "Expect '(' after 'if'.");
expression(compiler);
consume(compiler, TOKEN_RIGHT_PAREN, "Expect ')' after if condition.");
// Jump to the else branch if the condition is false.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
int ifJump = emit1(compiler, CODE_JUMP_IF, 255);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// Compile the then branch.
block(compiler);
// Compile the else branch if there is one.
if (match(compiler, TOKEN_ELSE))
{
feat: refactor method_call benchmark to use inheritance and fix constructor binding Refactor the method_call benchmark across Lua, Python, Ruby, and Wren to use proper inheritance with super calls instead of direct state manipulation. This ensures all languages benchmark super method invocations consistently. Fix a bug in the Wren compiler where constructors weren't being bound correctly by extracting method binding logic into a dedicated `bindMethod` function that properly handles both instance and static methods against the class object. Optimize if-statement compilation by restructuring jump instruction emission to eliminate unnecessary CODE_JUMP instructions when no else branch exists, reducing bytecode size for conditional blocks. Update the superclass constructor test to verify inherited field access through super constructor chains, confirming correct field initialization across multiple inheritance levels.
2013-12-18 16:37:41 +01:00
// Jump over the else branch when the if branch is taken.
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
int elseJump = emit1(compiler, CODE_JUMP, 255);
feat: refactor method_call benchmark to use inheritance and fix constructor binding Refactor the method_call benchmark across Lua, Python, Ruby, and Wren to use proper inheritance with super calls instead of direct state manipulation. This ensures all languages benchmark super method invocations consistently. Fix a bug in the Wren compiler where constructors weren't being bound correctly by extracting method binding logic into a dedicated `bindMethod` function that properly handles both instance and static methods against the class object. Optimize if-statement compilation by restructuring jump instruction emission to eliminate unnecessary CODE_JUMP instructions when no else branch exists, reducing bytecode size for conditional blocks. Update the superclass constructor test to verify inherited field access through super constructor chains, confirming correct field initialization across multiple inheritance levels.
2013-12-18 16:37:41 +01:00
patchJump(compiler, ifJump);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
block(compiler);
feat: refactor method_call benchmark to use inheritance and fix constructor binding Refactor the method_call benchmark across Lua, Python, Ruby, and Wren to use proper inheritance with super calls instead of direct state manipulation. This ensures all languages benchmark super method invocations consistently. Fix a bug in the Wren compiler where constructors weren't being bound correctly by extracting method binding logic into a dedicated `bindMethod` function that properly handles both instance and static methods against the class object. Optimize if-statement compilation by restructuring jump instruction emission to eliminate unnecessary CODE_JUMP instructions when no else branch exists, reducing bytecode size for conditional blocks. Update the superclass constructor test to verify inherited field access through super constructor chains, confirming correct field initialization across multiple inheritance levels.
2013-12-18 16:37:41 +01:00
// Patch the jump over the else.
patchJump(compiler, elseJump);
}
else
{
patchJump(compiler, ifJump);
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
}
return;
}
if (match(compiler, TOKEN_RETURN))
{
// Compile the return value.
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: Implicitly return null if there is a newline or } after the
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
// "return".
expression(compiler);
emit(compiler, CODE_RETURN);
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
return;
}
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
if (match(compiler, TOKEN_WHILE)) return whileStatement(compiler);
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
// Expression statement.
expression(compiler);
emit(compiler, CODE_POP);
}
// Compiles a class definition. Assumes the "class" token has already been
// consumed.
static void classDefinition(Compiler* compiler)
{
// Create a variable to store the class in.
// TODO: Allow anonymous classes?
int symbol = declareVariable(compiler);
// Load the superclass (if there is one).
if (match(compiler, TOKEN_IS))
{
parsePrecedence(compiler, false, PREC_CALL);
emit(compiler, CODE_SUBCLASS);
}
else
{
// Create the empty class.
emit(compiler, CODE_CLASS);
}
// Store a placeholder for the number of fields argument. We don't know
// the value until we've compiled all the methods to see which fields are
// used.
int numFieldsInstruction = emit(compiler, 255);
// Set up a symbol table for the class's fields. We'll initially compile
// them to slots starting at zero. When the method is bound to the close
// the bytecode will be adjusted by [wrenBindMethod] to take inherited
// fields into account.
SymbolTable* previousFields = compiler->fields;
SymbolTable fields;
initSymbolTable(&fields);
compiler->fields = &fields;
// Compile the method definitions.
consume(compiler, TOKEN_LEFT_BRACE, "Expect '}' after class body.");
while (!match(compiler, TOKEN_RIGHT_BRACE))
{
Code instruction = CODE_METHOD_INSTANCE;
bool isConstructor = false;
if (match(compiler, TOKEN_STATIC))
{
instruction = CODE_METHOD_STATIC;
// TODO: Need to handle fields inside static methods correctly.
// Currently, they're compiled as instance fields, which will be wrong
// wrong wrong given that the receiver is actually the class obj.
}
else if (peek(compiler) == TOKEN_NEW)
{
// If the method name is "new", it's a constructor.
isConstructor = true;
}
SignatureFn signature = rules[compiler->parser->current.type].method;
nextToken(compiler->parser);
if (signature == NULL)
{
error(compiler, "Expect method definition.");
break;
}
method(compiler, instruction, isConstructor, signature);
consume(compiler, TOKEN_LINE,
"Expect newline after definition in class.");
}
// Update the class with the number of fields.
compiler->fn->bytecode[numFieldsInstruction] = fields.count;
compiler->fields = previousFields;
// Store it in its name.
defineVariable(compiler, symbol);
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
static void variableDefinition(Compiler* compiler)
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
// TODO: Variable should not be in scope until after initializer.
int symbol = declareVariable(compiler);
// Compile the initializer.
if (match(compiler, TOKEN_EQ))
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
expression(compiler);
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
}
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
else
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
{
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
// Default initialize it to null.
null(compiler, false);
}
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
defineVariable(compiler, symbol);
}
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: implement for-in loop syntax with iterate/iteratorValue protocol and break support Add `for (variable in sequence)` syntax to the Wren language, introducing the `in` keyword token and compiler support for iterating over sequences using `iterate` and `iteratorValue` methods. Includes new benchmark files for Lua, Python, Ruby, and Wren, plus test cases covering single-expression bodies, block bodies, closures over loop variables, and break statements. Also fixes a missing POP() in CLOSE_UPVALUE and moves the while test file to a subdirectory.
2013-12-25 06:04:11 +01:00
// Compiles a "definition". These are the statements that bind new variables.
// They can only appear at the top level of a block and are prohibited in places
// like the non-curly body of an if or while.
void definition(Compiler* compiler)
{
if (match(compiler, TOKEN_CLASS)) return classDefinition(compiler);
if (match(compiler, TOKEN_VAR)) return variableDefinition(compiler);
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
block(compiler);
feat: replace AST-based parser with single-pass bytecode compiler and VM Replace the recursive-descent parser that built an AST (Node* tree) with a single-pass compiler that emits bytecode directly into a Block structure. Remove the parser.c/parser.h files and their AST node types (NodeLiteral, NodeCall, NodeBinaryOp, etc.), add new compiler.c/compiler.h with a Compiler struct and Pratt-style precedence parsing that outputs CODE_CONSTANT and CODE_END instructions, and introduce vm.c/vm.h containing a Fiber-based interpreter with a stack, interpret() loop, and printValue() for OBJ_INT. Update main.c to call compile() then interpret() instead of parse(), remove TOKEN_INDENT/TOKEN_OUTDENT from token.h and dumpTokens(), and adjust the Xcode project to include the new source files while excluding the old parser.
2013-10-22 21:16:39 +02:00
}
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Parses [source] to a "function" (a chunk of top-level code) for execution by
// [vm].
feat: add public C API header wren.h and refactor internal types to WrenVM prefix
2013-11-25 16:47:02 +01:00
ObjFn* wrenCompile(WrenVM* vm, const char* source)
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
{
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
Parser parser;
parser.vm = vm;
parser.source = source;
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
parser.hasError = false;
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
// Ignore leading newlines.
feat: replace int-based booleans with stdbool.h bool type across core source files Migrate all boolean-typed parameters, return values, struct fields, and local variables from `int` (with 0/1 convention) to C99 `bool` (`true`/`false`). Update corresponding comments, macro definitions, and documentation strings to use `true`/`false` terminology instead of "non-zero"/"zero". Add `#include <stdbool.h>` to `src/main.c`, `src/wren_compiler.c`, and `src/wren_value.h`. Remove the stale TODO comment in `src/wren_common.h` that originally proposed this change.
2013-12-15 08:41:23 +01:00
parser.skipNewlines = true;
refactor: remove heap-allocated token list and inline lexer state into compiler struct Eliminate the separate lexer module and its dynamic token allocation by moving token types, lexer state, and parsing functions directly into compiler.c. The compiler now receives a raw source string and length, lexes tokens on the fly into a stack-allocated Token struct, and discards the Buffer abstraction, token linked list, and associated memory management routines.
2013-10-22 22:25:39 +02:00
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
parser.tokenStart = source;
parser.currentChar = source;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
parser.currentLine = 1;
// Zero-init the current token. This will get copied to previous when
// advance() is called below.
feat: disallow field references outside class definitions with explicit error Add a check in the `field()` function to detect when a field is referenced outside of a class context. When `compiler->fields` is NULL (indicating no enclosing class), emit a clear error message "Cannot reference a field outside of a class definition." and assign a dummy field index of 255 to allow continued parsing with minimal cascading errors. Also includes minor cleanup: remove two stale TODO comments about EOF checking and better error messages in `nextToken()` and `consume()`, and add a new test file `test/field/outside_class.wren` that verifies the error is produced for top-level field assignment.
2013-11-29 19:53:56 +01:00
parser.current.type = TOKEN_ERROR;
refactor: replace integer offset fields with direct char* pointers in token and lexer structs Change Token struct to store `const char* start` pointer and `int length` instead of integer start/end offsets. Update Lexer struct similarly, replacing `tokenStart` and `currentChar` offsets with direct pointers into the source string. Adjust Local struct's `length` field type from `size_t` to `int` for consistency. Simplify error reporting in `endCompiler` to use `printf` with `%.*s` format specifier and the new pointer/length fields, eliminating manual character-by-character printing loops.
2013-12-02 00:22:24 +01:00
parser.current.start = source;
parser.current.length = 0;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
parser.current.line = 0;
// Read the first token.
nextToken(&parser);
feat: add ObjBlock type, block compilation, and newline skipping in parser Introduce a new ObjBlock object type with OBJ_BLOCK tag, replacing the placeholder OBJ_NUM in makeBlock(). Refactor the compiler to separate Parser and Compiler structs, adding skipNewlines and hasError fields to Parser and a parent pointer to Compiler for nested block support. Implement compileBlock() function that compiles a block up to a specified end token, and update callBlock() in the VM to push empty slots for local variables. Adjust token reading functions to accept Parser instead of Compiler, and remove the MAX_TOKEN enum entry.
2013-10-24 21:39:01 +02:00
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
Compiler compiler;
refactor: replace `Foo.new` constructor syntax with `new Foo` and simplify superclass constructor calls Remove the `this new` constructor declaration syntax in favor of a `new` keyword that creates instances directly. Superclass constructors are now invoked via bare `super(args)` instead of `super.new(args)`, eliminating the semantic weirdness of calling a constructor on a partially-constructed instance. Update the compiler to treat `new` as a keyword token, replace the `isMethod` flag with `methodName`/`methodLength` fields, and change the VM's `NEW` opcode to pop the class from the stack before creating the instance. Add a default `new` native method on Object that returns `this`. Update all benchmarks and tests to use the new syntax.
2013-12-19 16:02:27 +01:00
initCompiler(&compiler, &parser, NULL, NULL, 0);
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
refactor: replace hardcoded pinned array with linked list in WrenVM GC root tracking Replace the fixed-size `MAX_PINNED` array of `Obj*` pointers with a dynamically allocated linked list of `PinnedObj` nodes, removing the 16-object limit and eliminating the `numPinned` counter. Update `pinObj` to accept a caller-provided `PinnedObj` stack variable and link it at the head of the list, and simplify `unpinObj` to pop the head without requiring the object argument. Adjust `wrenNewVM` to initialize the new `pinned` pointer to NULL, and modify `collectGarbage` to iterate the linked list instead of the array.
2013-11-29 18:14:19 +01:00
PinnedObj pinned;
pinObj(vm, (Obj*)compiler.fn, &pinned);
feat: implement mark-sweep garbage collector with object linked list and pinned roots Replace manual malloc/free for Obj allocations with GC-managed heap. Add Obj->next linked list, markObj traversal for all object types, and sweep phase in VM. Introduce MAX_PINNED stack for temporary GC roots during compilation. Refactor all object constructors (newBool, newFunction, newClass, newNum, newString) to accept VM pointer and register allocations. Move function constant registration earlier in compiler to ensure GC reachability. Add DEBUG_GC_STRESS and TRACE_MEMORY macros in new common.h header.
2013-11-12 17:32:35 +01:00
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
for (;;)
{
feat: allow statement bodies for flow control in compiler and add return tests Replace the block() function's expression-only body with a call to definition(), enabling flow control statements like `if (foo) return bar` and `if (i > 2) break` without requiring curly braces. Add new test files for return statements after if, else, while, and inside functions/methods, and update existing break tests to use the simplified syntax.
2013-12-24 19:27:48 +01:00
definition(&compiler);
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
// If there is no newline, it must be the end of the block on the same line.
if (!match(&compiler, TOKEN_LINE))
{
fix: improve nested block comment parsing and add error messages to consume function
2013-11-14 08:06:53 +01:00
consume(&compiler, TOKEN_EOF, "Expect end of file.");
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
break;
}
if (match(&compiler, TOKEN_EOF)) break;
}
feat: implement closures with upvalue support and statement/expression separation Add closure objects that wrap functions with captured upvalues, enabling first-class closures that close over local variables from enclosing scopes. Introduce new bytecode instructions (CODE_CLOSURE, CODE_LOAD_UPVALUE, CODE_STORE_UPVALUE, CODE_CLOSE_UPVALUE, CODE_RETURN) and a CompilerUpvalue tracking system with a MAX_UPVALUES limit of 256. Refactor the grammar to strictly separate statements from expressions, fixing a bug where block expressions incorrectly popped locals while temporaries remained on the stack. Rename internal functions (wrenCallFunction, wrenDebugDumpInstruction, wrenDebugDumpStack) and add Upvalue allocation, closure creation, and debug dump support for the new instructions.
2013-12-04 16:43:50 +01:00
emit(&compiler, CODE_NULL);
emit(&compiler, CODE_RETURN);
refactor: unify compiler end-of-chunk code path into single endCompiler function Extract the common bytecode finalization logic into a dedicated endCompiler function that emits CODE_END and handles parent compiler function loading, replacing duplicated inline code. Update VM method definition opcodes to pop function from stack instead of reading from constants, enabling future closure support. Add TODO comments outlining planned closure capture mechanism and upvalue tracking requirements.
2013-12-01 23:59:56 +01:00
endCompiler(&compiler, -1);
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
refactor: replace hardcoded pinned array with linked list in WrenVM GC root tracking Replace the fixed-size `MAX_PINNED` array of `Obj*` pointers with a dynamically allocated linked list of `PinnedObj` nodes, removing the 16-object limit and eliminating the `numPinned` counter. Update `pinObj` to accept a caller-provided `PinnedObj` stack variable and link it at the head of the list, and simplify `unpinObj` to pop the head without requiring the object argument. Adjust `wrenNewVM` to initialize the new `pinned` pointer to NULL, and modify `collectGarbage` to iterate the linked list instead of the array.
2013-11-29 18:14:19 +01:00
unpinObj(vm);
feat: implement mark-sweep garbage collector with object linked list and pinned roots Replace manual malloc/free for Obj allocations with GC-managed heap. Add Obj->next linked list, markObj traversal for all object types, and sweep phase in VM. Introduce MAX_PINNED stack for temporary GC roots during compilation. Refactor all object constructors (newBool, newFunction, newClass, newNum, newString) to accept VM pointer and register allocations. Move function constant registration earlier in compiler to ensure GC reachability. Add DEBUG_GC_STRESS and TRACE_MEMORY macros in new common.h header.
2013-11-12 17:32:35 +01:00
feat: add `this` keyword support with method detection and error handling Introduce TOKEN_THIS token and `this` keyword parsing in the compiler, enabling the `this` expression for method contexts. Add `isMethod` flag to Compiler struct to track whether the current function is a method, and pass it through `initCompiler`. Implement `this_` parsing function that emits appropriate bytecode or errors when used outside a method. Remove the now-unnecessary `numLocals` field from ObjFn struct in vm.h. Add test cases for `this` at top level (expects error), in a method (valid usage), in a top-level function (expects error), and as a variable name (expects error).
2013-11-10 00:42:23 +01:00
return parser.hasError ? NULL : compiler.fn;
feat: restructure compiler to bottom-up order removing forward declarations Eliminate all forward declarations and reorder functions so each is defined before its first use, making recursion points explicit. Add semicolon counting to metrics script and include boolean equality test file.
2013-11-07 16:04:25 +01:00
}
feat: add wrenBindMethod to patch field indices and superclass dispatch in inherited methods When a class is defined, its superclass is not known until runtime since class definitions are imperative statements. This adds a new wrenBindMethod function that walks the bytecode of a method after it is bound to a class, adjusting LOAD_FIELD/STORE_FIELD indices to account for inherited fields and patching superclass dispatch targets. The newClass function now correctly accumulates numFields from the superclass chain, and the VM calls wrenBindMethod when binding methods to a class. Includes new test files for inherited field access and the is operator.
2013-12-11 01:13:25 +01:00
void wrenBindMethod(ObjClass* classObj, ObjFn* fn)
{
chore: remove author attribution from TODO comments across multiple source files Remove "(bob)" author prefix from all TODO comments in benchmark/method_call.wren, include/wren.h, src/main.c, src/wren_common.h, src/wren_compiler.c, src/wren_core.c, src/wren_value.c, src/wren_value.h, and src/wren_vm.c. Update the TODO_PATTERN regex in metrics script to match the new format without author parentheses. Also refactor test file counting in metrics to use os.walk with fnmatch for recursive directory traversal instead of glob.iglob.
2013-12-15 00:28:18 +01:00
// TODO: What about functions nested inside [fn]?
feat: add wrenBindMethod to patch field indices and superclass dispatch in inherited methods When a class is defined, its superclass is not known until runtime since class definitions are imperative statements. This adds a new wrenBindMethod function that walks the bytecode of a method after it is bound to a class, adjusting LOAD_FIELD/STORE_FIELD indices to account for inherited fields and patching superclass dispatch targets. The newClass function now correctly accumulates numFields from the superclass chain, and the VM calls wrenBindMethod when binding methods to a class. Includes new test files for inherited field access and the is operator.
2013-12-11 01:13:25 +01:00
int ip = 0;
for (;;)
{
Code instruction = fn->bytecode[ip++];
switch (instruction)
{
case CODE_LOAD_FIELD:
case CODE_STORE_FIELD:
feat: add optimized field load/store instructions for direct method access Introduce CODE_LOAD_FIELD_THIS and CODE_STORE_FIELD_THIS opcodes that bypass receiver stack operations when field access occurs directly within a method body. Refactor field() compiler to emit these faster instructions when compiler->methodName is set, and extract loadThis() helper to correctly resolve 'this' in nested functions. Update VM interpreter with dedicated dispatch cases that read receiver from frame->stackStart instead of popping, and add debug disassembly support. Add test cases for field closure capture and nested class field isolation, removing related TODO comments.
2013-12-23 20:17:40 +01:00
case CODE_LOAD_FIELD_THIS:
case CODE_STORE_FIELD_THIS:
feat: add wrenBindMethod to patch field indices and superclass dispatch in inherited methods When a class is defined, its superclass is not known until runtime since class definitions are imperative statements. This adds a new wrenBindMethod function that walks the bytecode of a method after it is bound to a class, adjusting LOAD_FIELD/STORE_FIELD indices to account for inherited fields and patching superclass dispatch targets. The newClass function now correctly accumulates numFields from the superclass chain, and the VM calls wrenBindMethod when binding methods to a class. Includes new test files for inherited field access and the is operator.
2013-12-11 01:13:25 +01:00
// Shift this class's fields down past the inherited ones.
fn->bytecode[ip++] += classObj->superclass->numFields;
break;
case CODE_END:
return;
default:
refactor: replace manual instruction argument counting with getNumArguments() in wrenBindMethod Remove the large switch statement that manually handled instruction argument counts for each opcode in wrenBindMethod(). Instead, use the existing getNumArguments() helper function to determine how many bytes each instruction consumes, reducing code duplication and making the method more maintainable when new instructions are added.
2013-12-25 07:10:30 +01:00
// Other instructions are unaffected, so just skip over them.
ip += getNumArguments(fn, ip - 1);
feat: add wrenBindMethod to patch field indices and superclass dispatch in inherited methods When a class is defined, its superclass is not known until runtime since class definitions are imperative statements. This adds a new wrenBindMethod function that walks the bytecode of a method after it is bound to a class, adjusting LOAD_FIELD/STORE_FIELD indices to account for inherited fields and patching superclass dispatch targets. The newClass function now correctly accumulates numFields from the superclass chain, and the VM calls wrenBindMethod when binding methods to a class. Includes new test files for inherited field access and the is operator.
2013-12-11 01:13:25 +01:00
break;
}
}
}
Reference in New Issue Copy Permalink