Commit Graph

114 Commits

Author SHA1 Message Date
Bob Nystrom
a8005feb21 chore: add Travis CI docs deployment job and update build dependencies
- Add automatic docs deployment stage for master branch pushes in .travis.yml
- Include python3-markdown, python3-pygments, python3-setuptools, and ruby-sass as build dependencies
- Switch from container-based to sudo-required builds with trusty dist for custom Pygments lexer installation
- Ensure build directory exists before generating docs in Makefile
2018-07-16 04:01:14 +00:00
Bob Nystrom
52d65d48df fix: ensure expression compilation consumes all input by checking for EOF token
When compiling a single expression, the parser now explicitly consumes the TOKEN_EOF token after parsing the expression. This ensures that any trailing garbage or unexpected tokens after the expression are caught and reported as errors, rather than being silently ignored. The comment for the non-expression path is also updated to clarify that it checks for end of file rather than end of block.
2018-04-28 23:54:33 +00:00
Bob Nystrom
be6c17558d fix: replace strncpy with memcpy and increase error message buffer size
Replace strncpy with memcpy in readBuiltInModule to avoid potential string termination issues
when copying module source code, and increase ERROR_MESSAGE_SIZE from 60 to 80 to provide
more room for compiler error messages with long variable names.
2018-04-28 23:38:09 +00:00
Bob Nystrom
c3aad8049e fix: mark field symbol tables during class compilation to prevent GC from freeing field strings 2018-04-28 19:13:03 +00:00
Bob Nystrom
2b165f9649 refactor: replace raw String struct with ObjString* in symbol tables and GC handling
Migrate the internal `String` struct to use the heap-allocated `ObjString*` throughout the compiler, debugger, VM, and value system. This unifies string representation, enables proper garbage collection of symbol table entries via new `wrenBlackenSymbolTable` function, and removes manual memory management for symbol names.
2018-04-28 17:22:42 +00:00
Bob Nystrom
883754094a fix: remove redundant braces and update comment for UTF-8 BOM skip in wrenCompile 2018-04-28 17:07:04 +00:00
Bob Nystrom
91ddcea168 feat: skip UTF-8 BOM in source input and add regression test
Add handling for UTF-8 byte order mark (BOM) at the start of source code
in wrenCompile function by checking for the three-byte sequence \xEF\xBB\xBF
and advancing the source pointer past it. Include a regression test file
(test/regression/520.wren) that contains a BOM to verify the fix works
correctly.
2018-04-28 17:00:59 +00:00
Bob Nystrom
5c3d883204 fix: guard signature string overflow when too many parameters are passed
Add a MAX_PARAMETERS bound to the loop in signatureParameterList to prevent
buffer overflow when a method signature has an excessive number of parameters.
A compile error is already reported in such cases, but the unbounded loop could
still write past the end of the name buffer. Also add regression test 494.wren
to cover this scenario.
2018-04-27 16:13:40 +00:00
Bob Nystrom
248e3a69f0 fix: pre-allocate slot zero in every compiled chunk to fix REPL local variable declarations
The compiler now always reserves slot zero for either the closure or method
receiver, eliminating a corner case where top-level REPL code lacked the
pre-allocated slot that function/method bodies expect. This ensures local
variables declared in REPL loops (e.g. `for (i in 1..2)`) get correct slot
indexes.

Additionally, inlines `wrenResetFiber` into `wrenNewFiber` and stores the
imported module's closure on the stack before invoking it, preventing a GC
from collecting the closure during import.
2018-04-27 15:20:49 +00:00
jclc
75e828913f refactor: replace manual BOM offset tracking with pointer arithmetic in wrenCompile 2018-04-07 03:04:32 +00:00
jclc
021f4d6eb3 chore: remove trailing blank line and whitespace in wrenCompile function 2018-04-04 02:14:05 +00:00
jclc
a296b6d546 fix: skip UTF-8 BOM when compiling Wren source in wrenCompile function 2018-04-04 02:11:59 +00:00
Michel Hermier
4e53cda68d refactor: replace custom String struct with ObjString* in symbol table and method names
Unify string representation across the codebase by switching SymbolTable and method name storage from a manually-managed `String` struct (char* buffer + length) to `ObjString*` pointers. This eliminates duplicate string allocations in the meta API's `metaGetModuleVariables`, reduces memory management overhead by leveraging the existing GC for symbol strings, and adds `wrenBlackenSymbolTable` to properly trace these GC-roots during collection. The change touches the compiler, debug dumper, VM runtime error formatting, and module variable initialization to use `->value` and `->length` fields directly from `ObjString`.
2018-03-28 16:12:50 +00:00
Bob Nystrom
88ad0fc138 feat: replace module import string operand with direct ObjModule reference in IMPORT_VARIABLE
Add CODE_END_MODULE opcode to store the current ObjModule in vm->lastModule after module body execution. Modify IMPORT_VARIABLE to load variables from vm->lastModule instead of using a constant string operand for the module name. Update opcode definitions, debug dump, compiler import logic, and interpreter dispatch accordingly. Refactor variable lookup into getModuleVariable helper. Adjust formatting in wren_primitive.c and wren_opt_meta.c.
2018-03-20 13:54:51 +00:00
Michel Hermier
28b3234fa5 fix: correct opcode argument counts and field offset patching in wrenBindMethodCode
The getNumArguments function had CODE_IMPORT_MODULE incorrectly listed as a 1-argument
opcode and CODE_IMPORT_VARIABLE as a 2-argument opcode. This swap caused incorrect
bytecode parsing during method binding. Additionally, wrenBindMethodCode had two bugs:
the instruction fetch was incrementing ip before reading the opcode, and field offset
patching was using ip++ instead of ip+1, corrupting subsequent bytecode. The super
instruction handling also had an incorrect ip+=2 skip that removed the symbol offset
before reading the constant index.
2018-01-11 10:56:03 +00:00
Bob Nystrom
3a1d93e2a8 fix: add unreachable return after switch in getNumArguments to fix compiler warning
The switch statement in getNumArguments covers all opcode cases, but the compiler
cannot prove exhaustiveness. Adding UNREACHABLE() followed by return 0 eliminates
the "control reaches end of non-void function" warning on strict compiler settings.
2017-11-01 05:07:07 +00:00
Bob Nystrom
981ea50407 feat: replace core library import calls with dedicated bytecode instructions
Replace the previous approach of desugaring import statements into calls to System.importModule() and System.getModuleVariable() with two new bytecode instructions: CODE_IMPORT_MODULE and CODE_IMPORT_VARIABLE. This eliminates the need for the corresponding primitive methods in the core library, adds proper debug dump support for the new opcodes, and updates the compiler to emit these instructions directly. The getNumArguments function is also updated to handle the new opcodes and the unreachable default case is removed.
2017-10-31 22:58:59 +00:00
Bob Nystrom
8bd5b85df5 fix: switch REPL eval to use fiber execution and add explicit stdout flush
Replace `Meta.eval()` to compile source into a fiber instead of a function, fixing issue #456 where top-level variable declarations failed in the REPL. Add `Stdout` import and flush stdout before reading stdin since `System.print()` no longer auto-flushes. Refactor token detection to use `lexFirst()` instead of full `lex()`, and add new `Meta.compile()` method returning a fiber. Internally rename `wrenNewString` to `wrenNewStringLength` with explicit length parameter, updating all call sites across compiler, core, and meta modules.
2017-10-13 14:58:57 +00:00
Bob Nystrom
8b2994b6e4 fix: rename ClassCompiler typedef to ClassInfo across compiler source
Rename the internal `ClassCompiler` struct typedef to `ClassInfo` to better reflect its purpose as a descriptor for the enclosing class context during compilation. Update all variable declarations, function signatures, and references throughout `wren_compiler.c` to use the new type name, including in `getEnclosingClass`, `field`, `super_`, and `declareMethod` functions. This resolves issue #469 by eliminating the misleading "Compiler" suffix from a type that tracks class metadata rather than compilation state.
2017-10-09 13:47:42 +00:00
Bob Nystrom
ffb15081f0 refactor: extract if-statement compilation into dedicated ifStatement helper 2017-10-08 17:47:31 +00:00
Bob Nystrom
bd31425b5f fix: detect missing closing brace in block parsing to prevent crash
The compiler previously relied on `match(TOKEN_RIGHT_BRACE)` in the while loop condition, which consumed the brace and made it impossible to detect a missing one. Changed to peek-based loop termination and added explicit `consume` call for the closing brace, ensuring proper error reporting when '}' is absent. Added regression test for issue #429.
2017-10-08 17:03:42 +00:00
Bob Nystrom
184126372c fix: set token type to TOKEN_ERROR after invalid character to prevent compiler loop
When the lexer encounters an invalid character, it now explicitly sets the
current token's type to TOKEN_ERROR and its length to 0. Previously, the
token retained the previous token's type, which could cause the compiler
to get stuck in an infinite loop in certain code paths.

Also improves the error message for non-ASCII invalid bytes by showing
the raw hex value instead of attempting to display the character, since
the lexer operates on raw bytes without UTF-8 decoding.

Adds regression test for issue #428 that was crashing the compiler with
an out-of-bounds memory access due to this loop.
2017-03-24 04:19:20 +00:00
Bob Nystrom
9b0d2aefb6 feat: add user_data API test and reorder WrenErrorFn vm parameter to first position
Add a new test suite for user data get/set operations on WrenVM, including
a Wren script that validates default NULL userData, retrieval after VM
creation, and mutation via wrenSetUserData. Also reorder the vm parameter
in WrenErrorFn callback signature and all its call sites to match the
convention used by other callback types, updating the typedef, the
reportError implementation, and all invocations in the compiler and
debug modules. Include the new source and header files in the Xcode
project build configuration.
2017-03-22 14:26:19 +00:00
scott
b2b187ab07 feat: add user-defined state data pointer and accessors to WrenVM
Add a `userData` void pointer field to `WrenConfiguration` and expose `wrenGetUserData`/`wrenSetUserData` functions for associating arbitrary state with a VM instance. Also update the `WrenErrorFn` signature to include the `WrenVM*` parameter, and propagate the VM pointer through all error callback invocations in the compiler, debug, and runtime paths.
2017-01-20 03:54:25 +00:00
Bob Nystrom
1b635d029a fix: replace strtol with strtoll for 64-bit hex literal parsing on 32-bit platforms
The previous implementation used strtol() for parsing hex literals, which only supports up to 32-bit values on 32-bit architectures. This caused hex literals larger than 0x7FFFFFFF to overflow or fail. Changed to strtoll() which guarantees 64-bit parsing regardless of platform word size.

Also updated the error message to include the actual sizeof(long int) for debugging, uncommented the previously disabled 64-bit hex literal examples in syntax.wren, and added a test case for 0xdeadbeef to verify the fix works correctly.
2017-01-13 05:53:21 +00:00
Bob Nystrom
409873c382 perf: replace O(n) linear scan with O(1) map lookup for constant deduplication in compiler
Replace the linear scan of the function's constant table in `addConstant()` with a hash map lookup using a new `ObjMap* constants` field on the `Compiler` struct. This avoids O(n) performance degradation when compiling functions with many constants, such as large literal-heavy functions. The map stores constant values as keys and their index as numeric values, lazily initialized on first insertion. Also adds OBJ_FN hashing support in `hashObject()` to allow bare function objects as map keys internally, and includes a regression test with 65,550 constant entries to validate the fix.
2016-08-16 14:43:34 +00:00
Bob Nystrom
3ff11eb9f3 feat: deduplicate constants in addConstant to reuse existing values
Add linear scan in addConstant() to check for duplicate constant values before appending. This reduces memory usage for repeated constants and enables support for functions with large numbers of repeated entries. Includes TODO noting O(n) complexity for future optimization.
2016-08-16 14:24:04 +00:00
Bob Nystrom
c726845c42 feat: implement expression evaluation and statement detection in REPL with Meta.compileExpression
Add executeInput() method to REPL that lexes input, determines if it's a statement or expression based on the first token, and evaluates accordingly using Meta.eval for statements or the new Meta.compileExpression for expressions. Extend Meta.compile_ foreign method with isExpression and printErrors parameters, update wrenCompile signature to support expression compilation mode, and add newline handling after equals in variable definitions.
2016-05-21 06:03:05 +00:00
Bob Nystrom
19fc5e7534 feat: always wrap functions in closures to simplify VM and improve invocation performance
Simplify the VM by ensuring all called objects are closures, removing the need for runtime checks between bare functions and closures. This eliminates the `IS_FN`/`IS_CLOSURE` branching in `validateFn`, `checkArity`, `bindMethod`, and `wrenNewFiber`, and changes `CallFrame.fn` from `Obj*` to `ObjClosure*`. The compiler now emits `CODE_CLOSURE` for all function declarations, even those without upvalues, trading a small allocation cost at declaration time for faster and more uniform invocation. Benchmarks show performance is neutral to slightly improved across all tested workloads.
2016-03-26 21:00:17 +00:00
Bob Nystrom
7e72462d90 feat: add WrenErrorType enum and route runtime errors through config errorFn callback
Replace direct fprintf(stderr) calls in wrenDebugPrintStackTrace with invocations of the host-provided errorFn callback, passing WREN_ERROR_RUNTIME for the error message and WREN_ERROR_STACK_TRACE for each stack frame. Introduce the WrenErrorType enum in wren.h with WREN_ERROR_COMPILE, WREN_ERROR_RUNTIME, and WREN_ERROR_STACK_TRACE values, update the WrenErrorFn typedef to accept a type parameter, and modify reportError in cli/vm.c to switch on the error type for formatted output. Adjust wrenDebugPrintStackTrace signature to take WrenVM* instead of ObjFiber* to access the config, and update all call sites accordingly.
2016-03-17 14:17:03 +00:00
Bob Nystrom
63e462bb84 chore: remove unused stdarg.h, stdio.h, and stdlib.h includes from wren_compiler.c and wren_value.c 2016-03-17 14:04:09 +00:00
Bob Nystrom
d9ec3627cf feat: add WrenErrorFn callback to WrenConfiguration for custom error reporting
- Add `WrenErrorFn` typedef in wren.h accepting module, line, and message parameters
- Include `errorFn` slot in `WrenConfiguration` struct with documentation comment
- Initialize `errorFn` to NULL in `wrenInitConfiguration()` in wren_vm.c
- Replace direct `fprintf(stderr, ...)` in wren_compiler.c with call to `parser->vm->config.errorFn()`
- Add guard in wren_compiler.c to skip error reporting when `errorFn` is NULL
- Implement `reportError()` in cli/vm.c that writes formatted errors to stderr
- Wire `reportError` into config during `initVM()` in cli/vm.c
2016-03-16 15:04:03 +00:00
Bob Nystrom
f16e5c775f feat: report undefined variable errors on the line where they are used
Store the line number of first use in implicitly declared variables as a numeric value, then synthesize a token at that line when reporting the error, ensuring bounded error message length even for pathologically long variable names.
2016-03-16 15:00:28 +00:00
Bob Nystrom
5d42ce9ad6 refactor: extract duplicate method detection into declareMethod helper and rename instanceMethods buffer to methods 2016-03-15 14:15:55 +00:00
Bob Nystrom
1d546cf840 feat: prevent duplicate method definitions in class compilation
Add tracking of static and instance method names during class compilation to detect and report duplicate method definitions. Introduce `hasMethod` helper function and `staticMethods`/`instanceMethods` buffers in the `ClassCompiler` struct, along with a `name` field for identification. Include a new test case `duplicate_methods.wren` verifying error generation for duplicate instance and static methods, while allowing same-named methods across static/instance boundaries.
2016-03-15 13:54:49 +00:00
Bob Nystrom
c33dc205c3 chore: remove unused DUP opcode from compiler, debug, opcodes, and VM interpreter 2016-03-14 05:11:21 +00:00
Stephen Belanger
7913424c1c feat: replace generic BufferFind with dedicated hasMethod for duplicate detection
Replace the generic `wrenIntBufferFind` function with a specialized `hasMethod` helper that performs exact method symbol matching in the compiler. This change removes the generic `BufferFind` macro from the buffer utility system and introduces a focused implementation in `wren_compiler.c` that directly compares method symbols using `memcmp`. The error message for duplicate methods is also simplified to exclude the module name. Additionally, a new test file `duplicate_methods.wren` is added to verify error handling for duplicate instance and static method definitions.
2016-03-13 03:14:19 +00:00
Bob Nystrom
c30ee8a297 refactor: consolidate compile error formatting into single printError helper
Replace multiple fprintf calls in lexError and compileError with a unified printError function that formats the entire error message into a fixed-size buffer before outputting it as a single fprintf call. This centralizes error formatting logic and prepares for future integration of a host-provided error callback by eliminating scattered fprintf invocations.
2016-03-12 05:30:54 +00:00
Bob Nystrom
51d1bb6b3a refactor: move shebang detection into default case and remove separate '#' branch 2016-03-12 01:23:31 +00:00
Bob Nystrom
0f18db0097 refactor: move compiler function state directly into ObjFn to eliminate copying on completion 2016-03-04 15:49:34 +00:00
Bob Nystrom
53f4cb15c9 fix: remove wrenSetCompiler and fix compiler root not being popped on error 2016-03-04 01:48:47 +00:00
Bob Nystrom
1ef9b77842 refactor: rename allowAssignment to canAssign and simplify parsePrecedence signature in wren_compiler.c 2016-03-04 00:30:48 +00:00
Stephen Belanger
b9b9611a8c fix: replace SymbolTable with IntBuffer for per-scope duplicate method detection 2016-02-28 20:53:34 +00:00
Bob Nystrom
06dad18184 docs: fix grammar and formatting in Wren VM comments and code 2016-02-28 00:05:50 +00:00
Bob Nystrom
0fde331974 refactor: replace import opcodes with core method calls for module loading
Remove CODE_LOAD_MODULE and CODE_IMPORT_VARIABLE opcodes, replacing them with calls to System.importModule(_) and System.getModuleVariable(_,_) primitives. This simplifies the interpreter loop, improves performance by reducing bytecode dispatch, and prepares for moving module loading logic into Wren source code for embedder flexibility.
2016-02-27 06:43:54 +00:00
Bob Nystrom
aca69b34e6 refactor: merge block() into statement() and remove redundant curly-block parsing
The block() function duplicated the curly-block parsing logic that statement() already handled for single-expression bodies. By removing block() and replacing its single call site in loopBody() with statement(), the compiler eliminates unnecessary indirection and simplifies the control flow parsing. The statement() comment is updated to clarify that it excludes variable binding statements like "var" and "class" which are not allowed in if-branch positions.
2016-02-25 15:04:48 +00:00
Bob Nystrom
c9390c624b fix: add missing code block marker in list docs and simplify parens increment in compiler
- Add missing `:::wren` code block marker before the `removeAt` example in the list documentation to ensure proper syntax highlighting.
- Simplify the `parens` array increment in `readString` by combining the assignment and post-increment into a single expression.
2016-02-24 14:52:12 +00:00
Stephen Belanger
ed94957e91 feat: add duplicate method detection in class compiler with error reporting
Introduce a `methods` symbol table and `name` field to `ClassCompiler` struct to track method signatures during compilation. When a method is defined, check for existing entries in the table and emit a compile error if a duplicate is found, preventing silent method overrides. Initialize the class name string before class definition processing to support error messages referencing the class name.
2016-02-24 06:07:08 +00:00
Bob Nystrom
5d04ed2aed fix: add overflow check for jump distance in patchJump and emit error on too much code
Add a MAX_JUMP constant set to 2^16 and validate jump distance in patchJump to prevent silent overflow when bytecode exceeds the maximum representable offset. Also remove the TODO comment in endLoop since the forward jump error will be caught by patchJump. Include test cases jump_too_far.wren and loop_too_far.wren to verify the compiler correctly reports "Too much code to jump over." for both conditional and loop constructs.
2016-02-19 14:57:38 +00:00
Bob Nystrom
cdf3b64ab2 refactor: replace classSlot int with Variable struct in defineMethod and method
Replace the integer `classSlot` parameter with a `Variable` struct in both `defineMethod()` and `method()` functions, removing duplicate code that handled top-level vs local class loading by delegating to the new `loadVariable()` helper.
2016-02-12 15:43:44 +00:00