All source listed below is under MIT license if no LICENSE file stating different is available.

Nimcheck

A multi-language source code validation framework written in Nim. Nimcheck uses tokenizer-based validation to check syntax correctness across 10 languages and config formats, with auto-detection, comprehensive diagnostics, and a never-crash guarantee.

Features

10 languages supported: Nim, Bash, Python, JavaScript, PHP, HTML, Jinja, JSON, YAML, TOML
Tokenizer-based validation -- real tokenizers that track bracket depth, string context, comments, and nesting
Auto-detection -- detect language from file extension or content analysis
Comprehensive diagnostics -- structured error reports with position, severity, hints, and debug output
Never-crash guarantee -- every validation path is wrapped in try/except; invalid input returns errors gracefully, never panics
JSON output -- machine-readable diagnostics via toJson() on any validation result
Inspection API -- extract module structure (imports, definitions, functions/classes) as JSON
Debug mode -- compile with -d:nimcheckDebug for detailed tokenization logging
No external dependencies -- pure Nim standard library

Quick Start

import nimcheck

# Validate source code from a string
let r = validateSource("proc hello() = echo 'hi'", lfNim)
echo r.errors  # seq[ValidationError]

# Auto-detect language
let r2 = validateSource("print('hello')", lfUnknown)  # detects Python

# Validate a file
let r3 = validateFile("path/to/file.nim", lfNim)

# Get JSON diagnostics
echo r.toJson().pretty()

# Inspect source structure (returns JsonNode)
let info = inspectSource("class Foo: ...", lfPython)
echo info.pretty()  # formatted JSON with class/function info

# Get human-readable report
let report = reportSource("proc foo() =", lfNim)
echo report  # formatted error summary

Building

Using Makefile (recommended)

# Build the library binary
make build

# Run all tests
make test

# Build with debug mode
make build-debug
make debug       # build debug + run all tests

# Run a specific language test
make test-nim
make test-python
make test-bash

# Lint with full warnings
make lint

# Clean build artifacts
make clean

# Show all available targets
make help

Using nimble

# Run all tests
nimble test

# Run per-language tests
nimble test_nim
nimble test_python

# Build the library
nimble build

Manual compilation

# Build the library
nim c src/nimcheck.nim

# Build with debug mode
nim c -d:nimcheckDebug src/nimcheck.nim

# Run all tests
nim c -r tests/test_all.nim

CLI Usage

When compiled as a standalone binary (make build produces bin/nimcheck):

# Validate a file
bin/nimcheck validate --file=source.nim

# Validate source code directly
bin/nimcheck validate --source="print('hi')" --flavor=python

# Read from stdin
echo "var x = 1" | bin/nimcheck validate

# Inspect source structure (returns JSON)
bin/nimcheck inspect --file=source.py

# Detect language
bin/nimcheck detect --file=unknown.txt
bin/nimcheck detect --source="#!/usr/bin/env bash"

# Version and help
bin/nimcheck version
bin/nimcheck help

API Reference

Core Functions

Function	Description
`validateSource(source, flavor, options, sourcePath)`	Validate source code string; returns `ValidationResult`
`validateFile(filePath, flavor, options)`	Validate a file; returns `ValidationResult`
`inspectSource(source, flavor, options, sourcePath)`	Return structural diagnostics as `JsonNode`
`inspectFile(filePath, flavor, options)`	Return structural diagnostics for a file as `JsonNode`
`validateSourceJson(source, flavor)`	Validate source and return result as JSON string
`validateFileJson(filePath, flavor)`	Validate file and return result as JSON string
`reportSource(source, flavor)`	Get a human-readable validation report (string)
`reportFile(filePath, flavor)`	Get a human-readable validation report for a file (string)
`detectFlavor(source, filePath)`	Auto-detect language flavor
`supportedFlavors()`	List all registered language flavors (returns `seq[string]`)
`version()`	Get the Nimcheck version string
`enableDebug()` / `disableDebug()`	Toggle debug mode globally

Types

ValidationResult -- contains errors: seq[ValidationError], valid: bool, moduleInfo, debugOutput, detectionInfo
ValidationError -- contains code, message, severity, position, hint, context
ValidationOptions -- contains flavor, debugMode, maxErrors, validateAll
LanguageFlavor -- enum: lfUnknown, lfNim, lfBash, lfPython, lfJavaScript, lfPHP, lfHTML, lfJinja, lfJSON, lfYAML, lfTOML

JSON Output

Both ValidationResult and ValidationError have a .toJson() method returning a JsonNode. Use .pretty() for formatted output.

Error Codes

Code	Meaning
`E1000`	Unexpected/unmatched bracket
`E1001`	Unclosed bracket
`E1002`	Unmatched closing bracket
`E2000`	Unclosed string literal
`E2001`	Invalid string escape
`E2002`	Unclosed multi-line string
`E3000`	Unclosed comment
`E3001`	Invalid comment syntax
`E4000`	Invalid syntax
`E4001`	Reserved word misuse
`E5000`	Unclosed block/control structure
`E9999`	Internal error (never-crash fallback)

Supported Languages

Language	File Extensions	Tokenizer Features
Nim	`.nim`, `.nims`, `.nimble`	Block comments `#[ ]#` with nesting, triple-quoted strings, raw strings, doc comments
Bash	`.sh`, `.bash`	Shebang detection, heredocs, command substitution, variable expansion, fi/done/esac matching
Python	`.py`	F-strings with nesting, triple-quoted strings, decorators, async/await, match statements
JavaScript	`.js`, `.jsx`, `.mjs`, `.cjs`	Template literals with nesting, regex literals, arrow functions, ESM imports
PHP	`.php`	`<?php` tags, heredocs/nowdocs, variable variables
HTML	`.html`, `.htm`	Tag balancing, self-closing tags, attributes
Jinja	`.j2`, `.jinja`, `.html.j2`	`{% %}`, `{{ }}`, `{# #}` block detection, nested blocks
JSON	`.json`	Bracket balance, string validation, comma checking
YAML	`.yaml`, `.yml`	Indentation tracking, key-value checking
TOML	`.toml`	Table headers, bracket balance, string validation

Testing

The test suite covers 250+ test cases across all supported languages:

Per-language tests -- valid and invalid source for each of the 10 languages
Exhaustive tests -- edge cases: unclosed strings, unclosed comments, mismatched brackets, binary data, unicode bombs
Fuzz tests -- null bytes, binary data, deep nesting (100 levels), empty sources, concurrent validation
Config format tests -- JSON, YAML, TOML validation with invalid syntax checks
Mixed-file tests -- HTML+Jinja combined files
Never-crash guarantee -- every test verifies invalid input returns graceful errors, not crashes
Debug mode tests -- make debug runs all tests with debug logging enabled

Run all tests with:

make test         # or
nimble test       # or
nim c -r tests/test_all.nim

Development

Prerequisites

Nim >= 2.0.0
make (optional, for Makefile targets)

Project structure

src/
  nimcheck.nim          Public API entry point and CLI (isMainModule)
  nimcheck/
    core/
      types.nim           Core types: ValidationResult, ValidationError, LanguageFlavor
      tokenizerbase.nim   Abstract tokenizer base class
      validatorbase.nim   Abstract validator base class
      detector.nim        Language auto-detection
      debug.nim           Debug logging infrastructure
    tokenizers/
      nim_tokenizer.nim
      bash_tokenizer.nim
      python_tokenizer.nim
      javascript_tokenizer.nim
      php_tokenizer.nim
      html_tokenizer.nim
      jinja_tokenizer.nim
    languages/
      nim_validator.nim
      bash_validator.nim
      python_validator.nim
      javascript_validator.nim
      php_validator.nim
      html_validator.nim
      jinja_validator.nim
      config_validators.nim   JSON, YAML, TOML validators
    reporting/
      errors.nim          Error formatting and JSON output
tests/
  test_all.nim            Master test runner (imports all test modules)
  test_<lang>.nim         Per-language tests
  test_<lang>_exhaustive.nim  Exhaustive edge-case tests per language
  test_config.nim         JSON/YAML/TOML tests
  test_fuzz.nim           Fuzz and boundary tests
  test_common.nim         Shared test utilities
  test_mixed.nim          Mixed-language file tests
  fixtures/               Test data files per language

Building with warnings

make lint
# or manually:
nim c --hints:on --warnings:on src/nimcheck.nim

Adding a new language

See the detailed checklist in LESSONS_LEARNED.md (Section 14).

Architecture

The framework uses a layered architecture:

Tokenizers -- convert raw source into a stream of tokens, tracking bracket depth, string context, and comments
Validators -- analyze token streams for language-specific syntax rules
Detector -- identifies language from file extension, shebang, or content heuristics
Reporting -- formats errors as human-readable strings or structured JSON

Validators register themselves at import time via a side-effect factory pattern. The public API (nimcheck.nim) imports all validators and re-exports core types for consumer convenience.

Versioning

Current version: 0.1.0

Contributing

Contributions are welcome. Please open an issue to discuss your idea before submitting a pull request. For detailed instructions on adding support for a new language, see LESSONS_LEARNED.md Section 16.

License

This project is licensed under the MIT License -- see the LICENSE file for details.

Author

molodetz

.gitea/workflows
src
tests
.gitignore
CONTRIBUTING.md
LESSONS_LEARNED.md
LICENSE
Makefile
nimcheck.nimble
README.md