fix: add tab character to whitespace skipping in nextToken parser
The nextToken function in wren_compiler.c previously only skipped spaces and carriage returns when advancing past whitespace, causing tab characters to be treated as non-whitespace tokens. This change adds '\t' to the whitespace check in both the initial case statement and the peekChar loop, ensuring tabs are properly skipped like other whitespace characters. A new test file test/whitespace.wren is added to verify correct handling of tabs in various indentation patterns and inline code positions.
This commit is contained in:
@@ -0,0 +1,26 @@
|
||||
// No indent
|
||||
IO.print("ok") // expect: ok
|
||||
|
||||
// Indent with space
|
||||
IO.print("ok") // expect: ok
|
||||
|
||||
// Indent with tab
|
||||
IO.print("ok") // expect: ok
|
||||
|
||||
// Indent with space then tab
|
||||
IO.print("ok") // expect: ok
|
||||
|
||||
// Indent with tab then space
|
||||
IO.print("ok") // expect: ok
|
||||
|
||||
// Indent with mixed tab and space
|
||||
IO.print("ok") // expect: ok
|
||||
|
||||
// Space in with code
|
||||
IO . print ( "ok" ) // expect: ok
|
||||
|
||||
// Tab in with code
|
||||
IO . print ( "ok" ) // expect: ok
|
||||
|
||||
// Tab and space mixed in with code
|
||||
IO . print ( "ok" ) // expect: ok
|
||||
Reference in New Issue
Block a user