feat: add extended language validators and fix tokenizer crash bugs

Expand nimcheck with validators and tokenizers for C, C++, C#, Go, Rust,
Ruby, CSS, SQL, Markdown, Dockerfile, Makefile, Kotlin, Lua, Swift,
TypeScript, and XML. Register all flavors in the validator factory and
improve auto-detection scoring for the new languages.

Fix infinite tokenizer loops that caused OOM kills: closeBracket now
advances position, finishTokenizeStep guards stalled tokenization, and
Jinja/JS tokenizers no longer double-advance on brackets.

Fix block-balance false positives in Lua (for/do) and Ruby (postfix
unless), SQL trailing-comma detection across whitespace, and Makefile
tab literals in test fixtures.
This commit is contained in:
2026-07-15 06:49:45 +02:00
parent df2b327a5d
commit 5a7c24f936
109 changed files with 8811 additions and 121 deletions
+66
View File
@@ -0,0 +1,66 @@
module EnumerableExtensions
refine Array do
def second
self[1]
end
def middle
self[size / 2]
end
end
end
using EnumerableExtensions
class Singleton
private_class_method :new
@instance = nil
def self.instance
@instance ||= new
end
end
module_function
def log(level, message)
warn "[#{level.upcase}] #{message}"
end
require "ostruct"
require "set"
require "delegate"
Point = Struct.new(:x, :y, keyword_init: true)
p = Point.new(x: 10, y: 20)
class Stack
include Enumerable
def initialize
@items = []
end
def push(item)
@items.push(item)
end
def pop
@items.pop
end
def each(&block)
@items.each(&block)
end
end
s = Stack.new
s.push(1)
s.push(2)
s.each { |i| puts i }
$global_count = 0
DEFAULT_TIMEOUT = 30
puts "Global: #{$global_count}, Timeout: #{DEFAULT_TIMEOUT}"
+12
View File
@@ -0,0 +1,12 @@
class Broken
attr_accessor :x
def missing_end
if x > 0
puts "positive"
def wrong_syntax
x = 42
{x + 1
end
+80
View File
@@ -0,0 +1,80 @@
#!/usr/bin/env ruby
require "json"
require "net/http"
require "uri"
class User
attr_accessor :id, :name, :email, :active
def initialize(id:, name:, email:, active: true)
@id = id
@name = name
@email = email
@active = active
end
def to_s
"#{@name} <#{@email}>"
end
def active?
@active
end
def deactivate!
@active = false
end
end
class UserRepository
def initialize
@users = {}
end
def add(user)
@users[user.id] = user
end
def find(id)
@users[id]
end
def find_by_email(email)
@users.values.find { |u| u.email == email }
end
def active_users
@users.values.select(&:active?)
end
def each(&block)
@users.values.each(&block)
end
def to_json(*args)
@users.values.map(&:to_h).to_json(*args)
end
end
def fetch_users(url)
uri = URI.parse(url)
response = Net::HTTP.get_response(uri)
raise "HTTP error: #{response.code}" unless response.is_a?(Net::HTTPSuccess)
JSON.parse(response.body)
rescue StandardError => e
warn "Failed to fetch users: #{e.message}"
[]
end
def process_items(items)
items.map { |item| yield item }
end
repo = UserRepository.new
repo.add(User.new(id: 1, name: "Alice", email: "alice@example.com"))
repo.each do |user|
puts user.to_s
end
puts "Active: #{repo.active_users.size}"