feat: add extended language validators and fix tokenizer crash bugs
Expand nimcheck with validators and tokenizers for C, C++, C#, Go, Rust, Ruby, CSS, SQL, Markdown, Dockerfile, Makefile, Kotlin, Lua, Swift, TypeScript, and XML. Register all flavors in the validator factory and improve auto-detection scoring for the new languages. Fix infinite tokenizer loops that caused OOM kills: closeBracket now advances position, finishTokenizeStep guards stalled tokenization, and Jinja/JS tokenizers no longer double-advance on brackets. Fix block-balance false positives in Lua (for/do) and Ruby (postfix unless), SQL trailing-comma detection across whitespace, and Makefile tab literals in test fixtures.
This commit is contained in:
+30
@@ -0,0 +1,30 @@
|
||||
# Nimcheck
|
||||
|
||||
---
|
||||
|
||||
> A fast syntax validator.
|
||||
|
||||
## Code Block
|
||||
|
||||
```nim
|
||||
proc hello(): string =
|
||||
result = "Hello"
|
||||
```
|
||||
|
||||
```python
|
||||
def hello() -> str:
|
||||
return "Hello"
|
||||
```
|
||||
|
||||
| A | B | C |
|
||||
|---|---|---|
|
||||
| 1 | 2 | 3 |
|
||||
|
||||
- [x] Done
|
||||
- [ ] Pending
|
||||
|
||||
~~strikethrough~~
|
||||
|
||||
Footnotes[^1]
|
||||
|
||||
[^1]: A footnote.
|
||||
Vendored
+26
@@ -0,0 +1,26 @@
|
||||
# Nimcheck
|
||||
|
||||
> A fast syntax validator
|
||||
|
||||
## Features
|
||||
|
||||
- Fast
|
||||
- Accurate
|
||||
- 27+ languages
|
||||
|
||||
## Unclosed Code Block
|
||||
|
||||
```nim
|
||||
proc hello(): string =
|
||||
result = "Hello"
|
||||
|
||||
Still in code block without closing fence.
|
||||
|
||||
## Broken Link
|
||||
|
||||
[Broken](
|
||||
|
||||
## Broken Table
|
||||
|
||||
| A | B | C
|
||||
| 1 | 2 | 3 |
|
||||
Vendored
+131
@@ -0,0 +1,131 @@
|
||||
# Nimcheck: Universal Syntax Validator
|
||||
|
||||
[](https://nim-lang.org)
|
||||
[](LICENSE)
|
||||
|
||||
> **Fast, accurate syntax validation for 27+ languages.**
|
||||
> Built in Nim. Zero dependencies beyond the standard library.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Features](#features)
|
||||
2. [Installation](#installation)
|
||||
3. [Quick Start](#quick-start)
|
||||
4. [Usage](#usage)
|
||||
5. [Supported Languages](#supported-languages)
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-language** — validates Nim, Python, JavaScript, TypeScript, PHP, Ruby, Rust, Go, Lua, C, C++, C#, Java, Swift, Kotlin, Bash, HTML, XML, Jinja, JSON, YAML, TOML, CSS, SQL, Markdown, Dockerfile, Makefile
|
||||
- **Blazing fast** — compiles to native code via Nim
|
||||
- **Auto-detection** — identifies language from extension, shebang, or content analysis
|
||||
- **CI-friendly** — exit codes and structured output
|
||||
|
||||
### Detection Methods
|
||||
|
||||
| Priority | Method | Confidence |
|
||||
|----------|-------------|------------|
|
||||
| 1 | Explicit | 100% |
|
||||
| 2 | Extension | 90% |
|
||||
| 3 | Shebang | 95% |
|
||||
| 4 | Content | 70-95% |
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### From Source
|
||||
|
||||
```bash
|
||||
git clone https://github.com/retoor/nimcheck
|
||||
cd nimcheck
|
||||
make release
|
||||
```
|
||||
|
||||
### Via Nimble
|
||||
|
||||
```bash
|
||||
nimble install nimcheck
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
nimcheck src/ # validate all files in src/
|
||||
nimcheck --flavor nim # validate as Nim
|
||||
nimcheck --json # JSON output
|
||||
nimcheck --watch # watch mode
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Validate a single file
|
||||
|
||||
```bash
|
||||
nimcheck src/main.nim
|
||||
```
|
||||
|
||||
### Validate with explicit language
|
||||
|
||||
```bash
|
||||
nimcheck --flavor python script.py
|
||||
```
|
||||
|
||||
### Recursive directory scan
|
||||
|
||||
```bash
|
||||
nimcheck --recursive project/
|
||||
```
|
||||
|
||||
### JSON output for CI
|
||||
|
||||
```bash
|
||||
nimcheck --json --recursive src/ > report.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Supported Languages
|
||||
|
||||
| Language | Extensions |
|
||||
|-------------|-------------------------------------|
|
||||
| Nim | `.nim`, `.nims`, `.nimble` |
|
||||
| Python | `.py`, `.pyw`, `.pyx`, `.pxd` |
|
||||
| JavaScript | `.js`, `.mjs`, `.cjs` |
|
||||
| TypeScript | `.ts`, `.tsx` |
|
||||
| PHP | `.php`, `.phtml` |
|
||||
| HTML | `.html`, `.htm`, `.xhtml` |
|
||||
| XML | `.xml`, `.svg` |
|
||||
| JSON | `.json`, `.jsonc` |
|
||||
| YAML | `.yaml`, `.yml` |
|
||||
| TOML | `.toml` |
|
||||
| CSS | `.css` |
|
||||
| SQL | `.sql` |
|
||||
| Markdown | `.md`, `.markdown` |
|
||||
| Ruby | `.rb` |
|
||||
| Rust | `.rs` |
|
||||
| Go | `.go` |
|
||||
| Lua | `.lua` |
|
||||
| C | `.c`, `.h` |
|
||||
| C++ | `.cpp`, `.cxx`, `.hpp` |
|
||||
| C# | `.cs` |
|
||||
| Java | `.java` |
|
||||
| Swift | `.swift` |
|
||||
| Kotlin | `.kt`, `.kts` |
|
||||
| Bash | `.sh`, `.bash` |
|
||||
| Dockerfile | `Dockerfile` |
|
||||
| Makefile | `Makefile` |
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT © [Retoor](https://github.com/retoor)
|
||||
Reference in New Issue
Block a user