Nimcheck

Universal source code validation framework written in Nim. Tokenizer-based syntax validation across 10 languages and config formats, with auto-detection, structured JSON diagnostics, structural inspection, and a never-crash guarantee.

Version: 0.1.0 | Author: molodetz | License: MIT | Requires: Nim >= 2.0.0

---

Table of Contents

1. Architecture
2. Quick Start
3. API Reference
4. Data Types
5. JSON Output Structures
6. Error Code Reference
7. IO Table: Supported Languages and Per-Language Errors
8. CLI Usage
9. Building and Testing
10. Project Structure
11. Contributing

---

Architecture

Nimcheck uses a four-layer pipeline architecture. Each layer is independent and communicates through well-defined interfaces.

Source Code (string or file)
        |
        v
+-------------------+
|     Detector      |  LanguageFlavor from extension, shebang, or content analysis
+-------------------+
        |
        v
+-------------------+
|    Tokenizer      |  Converts raw source into a seq[Token] stream
+-------------------+
        |
        v
+-------------------+
|    Validator      |  Analyzes token stream for syntax errors and structure
+-------------------+
        |
        v
+-------------------+
|    Reporting      |  Formats errors as human-readable text or structured JSON
+-------------------+
        |
        v
  ValidationResult

Layer 1: Detector (src/nimcheck/core/detector.nim)

Language auto-detection with cascading priority:

Priority	Method	Source	Confidence
1 (highest)	Explicit	User-provided LanguageFlavor	1.0
2	File extension	70+ extension-to-flavor mappings	0.9
3	Shebang line	#! line interpreter matching (bash, python, node, ruby, lua, swift)	0.95
4	Content analysis	Keyword scoring, structural patterns, bracket usage	0.6-0.9

The detectFlavor() function returns a (LanguageFlavor, FlavorDetectionMethod, confidence: float) tuple.

Layer 2: Tokenizer (src/nimcheck/core/tokenizerbase.nim, per-language impls)

Each tokenizer converts raw source into a seq[Token]. Tokenizers track:

• Position: line, column, and byte offset for every token
• Bracket stack: (, [, {, < pushed and popped with position tracking
• String context: escape sequences, multi-line delimiters, interpolation
• Comment context: line comments, block comments with nesting support

Tokenizers emit errors for:

• Unclosed strings (ErrUnclosedString)
• Unclosed comments (ErrUnclosedComment)
• Unexpected characters (ErrUnexpectedToken)
• Unclosed block constructs (language-specific, e.g., ${}, $())

Layer 3: Validator (src/nimcheck/core/validatorbase.nim, per-language impls)

Each validator:

1. Creates its tokenizer and runs tokenize()
2. Collects tokenizer errors
3. Runs analyzeTokens() — language-specific structural analysis
4. Runs buildModuleInfo() — populates ModuleInfo with imports, functions, classes
5. Computes validity, warning/info counts, duration

Validators register themselves via a factory pattern at import time. The registry maps flavor strings (e.g., "python", "bash") to constructors.

Layer 4: Reporting (src/nimcheck/reporting/errors.nim)

Produces:

• Human-readable reports: formatReport() with position, severity, code, message, source context with carets, and hints
• JSON output: every type has a toJson() method producing JsonNode
• Error categorization: countBySeverity(), countByCode(), filterBySeverity()

---

Quick Start

Nim library usage

import nimcheck

# Validate source code from a string
let r = validateSource("proc hello() = echo 'hi'", lfNim)
echo r.valid           # bool
echo r.errors          # seq[ValidationError]
echo r.toJson().pretty()  # Full JSON output

# Validate a file (auto-detects language from extension)
let r2 = validateFile("src/main.nim")

# Auto-detect language
let r3 = validateSource("print('hello')", lfUnknown)
echo r3.flavor         # lfPython

# Inspect source structure (returns JsonNode with functions, classes, imports)
let info = inspectSource("class Foo:\n    def bar(self): pass", lfPython)
echo info.pretty()

# Human-readable report
echo reportSource("proc foo() =\n  let x = \"unclosed", lfNim)

CLI usage

# Build the binary
make build

# Validate a file
bin/nimcheck validate --file=source.nim

# Validate source directly
bin/nimcheck validate --source="print('hi')" --flavor=python

# Read from stdin
echo "var x = 1" | bin/nimcheck validate

# Inspect structure (JSON output)
bin/nimcheck inspect --file=source.py

# Detect language
bin/nimcheck detect --file=unknown.txt
bin/nimcheck detect --source="#!/usr/bin/env bash"

# Version and help
bin/nimcheck version
bin/nimcheck help

---

API Reference

Core Functions

All functions are exported from nimcheck.nim. Import once with import nimcheck.

Validation

Function	Signature	Returns	Description
validateSource	(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")	ValidationResult	Validate source code string. Auto-detects flavor if lfUnknown.
validateFile	(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())	ValidationResult	Validate a file from disk. Auto-detects flavor from file extension.
validateSourceJson	(source: string, flavor: LanguageFlavor = lfUnknown)	string	Validate and return pretty-printed JSON string.
validateFileJson	(filePath: string, flavor: LanguageFlavor = lfUnknown)	string	Validate file and return pretty-printed JSON string.

Inspection

Function	Signature	Returns	Description
inspectSource	(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")	JsonNode	Validate and return structural diagnostics (functions, classes, imports).
inspectFile	(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())	JsonNode	Inspect a file and return structural diagnostics.

Reporting

Function	Signature	Returns	Description
reportSource	(source: string, flavor: LanguageFlavor = lfUnknown)	string	Human-readable validation report for source code.
reportFile	(filePath: string, flavor: LanguageFlavor = lfUnknown)	string	Human-readable validation report for a file.

Detection and Utility

Function	Signature	Returns	Description
detectFlavor	(source: string, filePath: string = "")	LanguageFlavor	Auto-detect language flavor (convenience wrapper).
supportedFlavors	()	seq[string]	List all registered language flavors sorted alphabetically.
version	()	string	Version string ("0.1.0").
enableDebug	()	void	Enable debug mode globally.
disableDebug	()	void	Disable debug mode globally.

Core Detector (lower-level)

Function	Signature	Returns	Description
detectFlavor (core)	(source: string, filePath: string = "", explicitFlavor: LanguageFlavor = lfUnknown)	(LanguageFlavor, FlavorDetectionMethod, float)	Full detection with confidence score.

Factory

Function	Signature	Returns	Description
createValidator	(flavor: LanguageFlavor, source: string, options: ValidationOptions, sourcePath: string)	ValidatorBase	Factory: create the correct validator for a flavor.
registerValidator	(flavor: string, constructor: ValidatorConstructor)	void	Register a new validator constructor.

---

Data Types

LanguageFlavor (enum, pure)

All 28 flavors defined in src/nimcheck/core/types.nim. Flavors with validators (bold) are fully supported; the rest are detectable but have no dedicated validator.

Enum Value	String	Validator	File Extensions (auto-detect)
lfUnknown	"unknown"	No	--
lfNim	"nim"	NimValidator	.nim, .nims, .nimble
lfPython	"python"	PythonValidator	.py, .pyw, .pyx, .pxd
lfBash	"bash"	BashValidator	.sh, .bash
lfShell	"shell"	BashValidator (shared)	.zsh, .fish
lfJavaScript	"javascript"	JavaScriptValidator	.js, .mjs, .cjs
lfTypeScript	"typescript"	JavaScriptValidator (shared)	.ts, .tsx
lfPHP	"php"	PHPValidator	.php, .phtml
lfHTML	"html"	HTMLValidator	.html, .htm, .xhtml
lfXML	"xml"	HTMLValidator (shared)	.xml, .svg
lfJinja	"jinja"	JinjaValidator	.jinja, .jinja2, .j2
lfJSON	"json"	JSONValidator	.json, .jsonc
lfYAML	"yaml"	YAMLValidator	.yaml, .yml
lfTOML	"toml"	TOMLValidator	.toml
lfCSS	"css"	No	.css
lfSQL	"sql"	No	.sql
lfMarkdown	"markdown"	No	.md, .markdown
lfDockerfile	"dockerfile"	No	Dockerfile
lfMakefile	"makefile"	No	Makefile, makefile
lfRuby	"ruby"	No	.rb
lfRust	"rust"	No	.rs
lfGo	"go"	No	.go
lfLua	"lua"	No	.lua
lfC	"c"	No	.c, .h
lfCpp	"cpp"	No	.cpp, .cxx, .hpp
lfCSharp	"csharp"	No	.cs
lfJava	"java"	No	.java
lfSwift	"swift"	No	.swift
lfKotlin	"kotlin"	No	.kt, .kts

FlavorDetectionMethod (enum)

Value	Meaning
fdmExplicit	User explicitly provided the flavor
fdmExtension	Detected from file extension
fdmShebang	Detected from #! shebang line
fdmContent	Detected via content keyword analysis
fdmUnknown	Could not determine

TokenKind (enum)

Lexical token classifications used by all tokenizers.

Kind	Description
tkWhitespace	Spaces, tabs, carriage returns
tkNewline	\n
tkComment	Line (#, //) or block (/* */) comment
tkDocComment	Documentation comment (##, ///, """)
tkString	Single or double quoted string literal
tkRawString	Raw string (r"...")
tkMultilineString	Triple-quoted string ("""...""")
tkNumber	Integer, float, hex, octal, binary
tkIdentifier	Variable, function, type name
tkKeyword	Language-specific reserved word
tkOperator	+, -, *, /, &&, etc.
tkAssignment	=, :=, +=, etc.
tkPunctuation	., ,, ;, :
kOpenParen	(
kCloseParen	)
kOpenBracket	[
kCloseBracket	]
kOpenBrace	{
kCloseBrace	}
kAngleOpen	<
kAngleClose	>
tkInterpolation	String interpolation (${var}, f-string {expr})
tkDirective	Preprocessor / compiler directive (#include, shebang)
tkTemplateTag	Template language tag ({{ }}, {% %})
tkSpecial	Language-specific special token
tkError	Malformed or unrecognizable character
tkEndOfFile	End of input marker

Token

Token = object
  kind: TokenKind
  value: string
  position: SourcePosition
  range: SourceRange

SourcePosition

SourcePosition = object
  line: int      # 1-indexed line number
  column: int    # 1-indexed column number
  offset: int    # 0-indexed byte offset from start

SourceRange

SourceRange = object
  startPos: SourcePosition
  endPos: SourcePosition

ErrorSeverity (enum)

Value	String	Description
esInfo	"info"	Informational note
esWarning	"warning"	Non-fatal issue
esError	"error"	Validation failure
esCritical	"critical"	Internal framework failure

ValidationError

ValidationError = object
  severity: ErrorSeverity
  message: string
  code: string           # Error code, e.g. "E0002"
  position: SourcePosition
  range: SourceRange
  context: string        # Surrounding source context (optional)
  hint: string           # Fix suggestion (optional)

ValidationResult

ValidationResult = object
  valid: bool
  flavor: LanguageFlavor
  detectionMethod: FlavorDetectionMethod
  errors: seq[ValidationError]
  warnings: int
  infos: int
  moduleInfo: ModuleInfo
  durationMs: float
  debugOutput: string     # Empty unless compiled with -d:nimcheckDebug

ValidationOptions

ValidationOptions = object
  flavor: LanguageFlavor        # lfUnknown = auto-detect (default)
  debugMode: bool               # Enable per-validation debug logging (default: false)
  strictMode: bool              # Warnings become errors (default: false)
  maxErrors: int                # Stop after N errors; 0 = unlimited (default: 0)
  includeContext: bool          # Include surrounding source lines in error output (default: true)
  includeHints: bool            # Include fix suggestions (default: true)

Structural Types (populated by analyzeTokens())

VariableInfo = object
  name: string
  kind: string             # "var", "let", "const", "global", "local"
  typeAnnotation: string
  defaultValue: string
  position: SourcePosition
  mutable: bool
  exported: bool

FunctionInfo = object
  name: string
  kind: string             # "function", "method", "proc", "func", "lambda"
  parameters: seq[ParameterInfo]
  returnType: string
  isAsync: bool
  isExported: bool
  isGenerator: bool
  visibility: string       # "public", "private", "protected"
  position: SourcePosition
  docComment: string

ParameterInfo = object
  name: string
  typeAnnotation: string
  defaultValue: string
  isOptional: bool
  isVarargs: bool

ClassInfo = object
  name: string
  kind: string             # "class", "object", "struct", "interface"
  baseTypes: seq[string]
  fields: seq[VariableInfo]
  methods: seq[FunctionInfo]
  generics: seq[string]
  visibility: string
  position: SourcePosition
  docComment: string

ImportInfo = object
  module: string
  symbols: seq[string]     # Specific imported symbols
  isRelative: bool
  alias: string
  position: SourcePosition

ModuleInfo = object
  flavor: LanguageFlavor
  detectionMethod: FlavorDetectionMethod
  imports: seq[ImportInfo]
  variables: seq[VariableInfo]
  functions: seq[FunctionInfo]
  classes: seq[ClassInfo]
  linesOfCode: int
  totalTokens: int
  comments: int
  debugInfo: JsonNode      # Language-specific extra info

---

JSON Output Structures

Every type has a .toJson() method. The ValidationResult.toJson() aggregates all nested structures.

ValidationResult JSON

{
  "valid": true,
  "flavor": "python",
  "detectionMethod": "by_extension",
  "errors": [
    {
      "severity": "error",
      "message": "Unclosed string literal starting with '\"'",
      "code": "E0002",
      "position": {"line": 2, "column": 13, "offset": 42},
      "range": {
        "start": {"line": 2, "column": 13, "offset": 42},
        "end":   {"line": 2, "column": 13, "offset": 42}
      },
      "context": "",
      "hint": "Add a closing quote before end of file"
    }
  ],
  "warnings": 0,
  "infos": 0,
  "moduleInfo": {
    "flavor": "python",
    "detectionMethod": "by_extension",
    "imports": [],
    "variables": [],
    "functions": [
      {
        "name": "hello",
        "kind": "def",
        "parameters": [],
        "returnType": "",
        "isAsync": false,
        "isExported": false,
        "isGenerator": false,
        "visibility": "",
        "position": {"line": 1, "column": 5, "offset": 4},
        "docComment": ""
      }
    ],
    "classes": [],
    "linesOfCode": 3,
    "totalTokens": 12,
    "comments": 0,
    "debugInfo": null
  },
  "durationMs": 0.342,
  "debugOutput": ""
}

ValidationError JSON

{
  "severity": "error",
  "message": "Unclosed string literal starting with '\"'",
  "code": "E0002",
  "position": {"line": 2, "column": 13, "offset": 42},
  "range": {
    "start": {"line": 2, "column": 13, "offset": 42},
    "end":   {"line": 2, "column": 13, "offset": 42}
  },
  "context": "",
  "hint": "Add a closing quote before end of file"
}

ModuleInfo JSON

{
  "flavor": "nim",
  "detectionMethod": "by_extension",
  "imports": [
    {
      "module": "strutils",
      "symbols": [],
      "isRelative": false,
      "alias": "",
      "position": {"line": 1, "column": 8, "offset": 7}
    }
  ],
  "variables": [],
  "functions": [
    {
      "name": "add",
      "kind": "proc",
      "parameters": [],
      "returnType": "",
      "isAsync": false,
      "isExported": false,
      "isGenerator": false,
      "visibility": "",
      "position": {"line": 3, "column": 6, "offset": 28},
      "docComment": ""
    }
  ],
  "classes": [
    {
      "name": "MyType",
      "kind": "type",
      "baseTypes": [],
      "fields": [],
      "methods": [],
      "generics": [],
      "visibility": "",
      "position": {"line": 5, "column": 6, "offset": 52},
      "docComment": ""
    }
  ],
  "linesOfCode": 8,
  "totalTokens": 45,
  "comments": 2,
  "debugInfo": null
}

Human-Readable Report Format

=== Validation Report ===
  Flavor: python
  Detection: by_extension
  Valid: false
  Errors: 1
  Warnings: 0
  Duration: 0.34ms
  Lines of code: 3
  Tokens: 12

  --- Findings ---
  [E0002] (error) L2:13: Unclosed string literal starting with '"'
         |     x = "
         |            ^
         hint: Add a closing quote before end of file

  --- Summary ---
  INVALID - source has issues that need attention.

Error Formatter Output (single error)

  [E0002] (error) L2:13: Unclosed string literal starting with '"'
         |     x = "
         |            ^
         hint: Add a closing quote before end of file

---

Error Code Reference

Error codes follow a prefixed numbering scheme. The prefix encodes the category and severity.

General Errors (E0xxx)

Code	Constant	Description	Triggered By
E0001	ErrUnexpectedToken	Unexpected or unrecognizable character	Tokenizer encounter of non-ASCII, non-operator char
E0002	ErrUnclosedString	Unclosed string literal	Missing closing quote at EOF
E0003	ErrUnclosedComment	Unclosed block comment	Missing */, ]#, #} etc. at EOF
E0004	ErrUnclosedBlock	Unclosed block/control structure	Language-specific: unclosed if/for/class/etc.
E0005	ErrMismatchedBracket	Mismatched or unexpected closing bracket	) without (, </div> without <div>
E0006	ErrUnclosedBracket	Unclosed bracket at EOF	validateBrackets() finds remaining items on bracket stack
E0007	ErrInvalidSyntax	General invalid syntax	JSON parse error, TOML missing =, YAML empty key
E0008	ErrUnexpectedEOF	Unexpected end of file	Reserved for incomplete constructs not covered by other codes
E0009	ErrEmptySource	Source is empty (zero-length string)	validate() with source.len == 0
E0010	ErrEncodingError	Encoding error	Reserved for non-UTF-8 input
E0011	--	File not found	validateFile() with non-existent path

Language-Specific Errors (E1xxx)

Code	Constant	Description	Languages
E1001	ErrUnexpectedIndent	Unexpected or inconsistent indentation	Python
E1002	ErrMissingIndent	Missing indentation after block opener	Python
E1003	ErrDuplicateParam	Duplicate parameter name	(reserved)
E1004	ErrUndefinedVariable	Reference to undefined variable	(reserved)
E1005	ErrTypeMismatch	Type mismatch	(reserved)
E1006	ErrInvalidAssignment	Invalid assignment target	(reserved)
E1007	ErrMissingReturn	Function missing return statement	(reserved)
E1008	ErrInvalidDecorator	Invalid decorator usage	Python
E1009	ErrInvalidImport	Invalid import statement	(reserved)

Template Errors (E2xxx)

Code	Constant	Description	Languages
E2001	ErrUnclosedTemplateTag	Unclosed template expression/block tag	Jinja
E2002	ErrUnclosedBlockTag	Unclosed block tag with no matching end tag	Jinja
E2003	ErrInvalidFilter	Invalid filter in template expression	Jinja

Info Codes (I0xxx)

Code	Constant	Description
I0001	InfoLongLine	Line exceeds recommended length
I0002	InfoTrailingWhitespace	Trailing whitespace detected

Warning Codes (W0xxx / W1xxx)

Code	Constant	Description
W0001	WarnUnusedVariable	Variable declared but never used
W0002	WarnInconsistentNaming	Naming convention inconsistency
W0003	WarnMissingDoc	Public symbol missing documentation
W1001	--	Tab characters in YAML indentation

Internal Error

Code	Description	Triggered By
E9999	Internal framework error (never-crash fallback)	Any unhandled Exception in validate() or validateSource()

---

IO Table: Supported Languages and Per-Language Errors

Input/Output Contract

Every validation function follows the same contract:

Input	Output
source: string (required)	ValidationResult with valid: bool
flavor: LanguageFlavor = lfUnknown	ValidationResult.errors: seq[ValidationError]
options: ValidationOptions (optional)	ValidationResult.moduleInfo: ModuleInfo
sourcePath: string = "" (optional)	ValidationResult.durationMs: float

Guarantees:

• Never throws — all exceptions are caught and returned as E9999 errors
• Empty source returns E0009 (or valid: true for YAML/TOML where empty is valid)
• File not found returns E0011
• Unknown flavor with no detection possible returns an E0007 error
• Every error has a code, message, severity, position, and optional hint

Per-Language Tokenizer, Validator, and Error Table

Nim (.nim, .nims, .nimble)

Aspect	Details
Tokenizer	nim_tokenizer.nim — NimTokenizer
Validator	nim_validator.nim — NimValidator
Keywords	80+ (addr, and, as, block, break, case, const, converter, defer, discard, elif, else, enum, except, export, finally, for, from, func, if, import, in, include, iterator, let, macro, method, mixin, mod, nil, not, object, of, or, proc, ptr, raise, ref, return, template, try, tuple, type, using, var, when, while, with, yield, etc.)

Error Code	Condition	Severity	Hint
E0001	Unexpected character (non-ASCII, unrecognizable)	esWarning	Shows hex U+XXXX value
E0002	Unclosed string ("..." at EOF)	esError	"Add a closing quote"
E0002	Unclosed multi-line string (""" at EOF)	esError	--
E0002	Newline inside string literal	esError	--
E0002	Unclosed raw string (r"..." at EOF)	esError	--
E0002	Unclosed character literal ('x at EOF)	esError	--
E0003	Unclosed block comment (#[ without ]#)	esError	"Add closing ]#"
E0006	Unclosed bracket ((, [, { at EOF)	esError	"Add matching closing bracket"

Structural extraction: imports (from X import Y), functions (proc, func, method, template, macro, iterator, converter), types (type Foo).

Bash / Shell (.sh, .bash, .zsh, .fish)

Aspect	Details
Tokenizer	bash_tokenizer.nim — BashTokenizer
Validator	bash_validator.nim — BashValidator
Keywords	38+ (if, then, else, elif, fi, case, esac, for, while, until, do, done, in, function, select, time, return, break, continue, exit, declare, local, export, readonly, unset, eval, exec, let, source, shift, trap, type, typeset, ulimit, umask, alias, read, echo)

Error Code	Condition	Severity	Hint
E0002	Unclosed single-quoted string	esError	--
E0002	Unclosed double-quoted string	esError	--
E0004	Unclosed ${var} expansion	esError	--
E0004	Unclosed $() command substitution	esError	--
E0004	Unclosed if block (if without matching fi)	esError	"Add missing 'fi'"
E0004	Unclosed loop (for/while/until without done)	esError	"Add missing 'done'"
E0004	Unclosed case block (case without esac)	esError	"Add missing 'esac'"
E0006	Unclosed bracket ((, [, { at EOF)	esError	--

Special features: shebang detection (#!), heredoc detection (<<EOF), variable expansion ($var, ${var}), backtick command substitution, arithmetic expansion ((( ))).

Python (.py, .pyw, .pyx, .pxd)

Aspect	Details
Tokenizer	python_tokenizer.nim — PythonTokenizer
Validator	python_validator.nim — PythonValidator
Keywords	33 (False, None, True, and, as, assert, async, await, break, class, continue, def, del, elif, else, except, finally, for, from, global, if, import, in, is, lambda, nonlocal, not, or, pass, raise, return, try, while, with, yield)

Error Code	Condition	Severity	Hint
E0002	Unclosed string ("..." or '...' at EOF)	esError	--
E0002	Unclosed triple-quoted string (""" or ''' at EOF)	esError	--
E0002	Unclosed f-string (f"..." at EOF)	esError	--
E0002	Newline inside string literal	esError	--
E1001	Indentation mismatch at block boundary	esError	--
E0006	Unclosed bracket ((, [, { at EOF)	esError	--

Special features: f-string interpolation with nested braces, indentation tracking (INDENT/DEDENT tokens), decorator detection (@), line continuation (\).

Structural extraction: imports (import X, from X import Y), functions (def), classes (class).

JavaScript / TypeScript (.js, .mjs, .cjs, .ts, .tsx)

Aspect	Details
Tokenizer	javascript_tokenizer.nim — JavaScriptTokenizer
Validator	javascript_validator.nim — JavaScriptValidator
Keywords	60+ (async, await, break, case, catch, class, const, continue, debugger, default, delete, do, else, export, extends, false, finally, for, function, if, import, in, instanceof, let, new, null, of, return, super, switch, static, this, throw, true, try, typeof, var, void, while, with, yield + TypeScript: interface, type, declare, module, namespace, abstract, readonly, as, any, boolean, number, string, never, keyof, infer, unknown)

Error Code	Condition	Severity	Hint
E0002	Unclosed string ("..." or '...' at EOF)	esError	--
E0002	Unclosed template literal (`...` at EOF)	esError	--
E0002	Newline inside string literal	esError	--
E0006	Unclosed bracket ((, [, { at EOF)	esError	--

Special features: template literals with ${} interpolation nesting, regex literal detection, arrow function detection (=>), ESM import/export.

Structural extraction: imports/exports, functions (function, async), classes.

PHP (.php, .phtml)

Aspect	Details
Tokenizer	php_tokenizer.nim — PHPTokenizer
Validator	php_validator.nim — PHPValidator
Mode switching	Tracks inPhpMode flag — toggles on <?php/<? and ?>

Error Code	Condition	Severity	Hint
E0002	Unclosed string	esError	--
E0006	Unclosed bracket ((, [, { at EOF)	esError	--

Special features: PHP/HTML mode switching, <?php, <?=, ?> detection, variable names ($var, $obj->prop).

Structural extraction: functions (function, fn), classes (class), imports/requires (include, require, use, namespace).

HTML / XML (.html, .htm, .xhtml, .xml, .svg)

Aspect	Details
Tokenizer	html_tokenizer.nim — HTMLTokenizer
Validator	html_validator.nim — HTMLValidator
Void elements	14: area, base, br, col, embed, hr, img, input, link, meta, param, source, track, wbr

Error Code	Condition	Severity	Hint
E0003	Unclosed HTML comment (<!-- without -->)	esError	--
E0004	Unclosed HTML tag (e.g., <div> without </div>)	esError	"Add closing </tagname>"
E0005	Unexpected closing tag (mismatched nesting)	esWarning	"Check that tags are properly nested"

Special features: tag stack balancing, void element awareness (no closing tag needed for self-closing elements), attribute parsing with quoted values, <!DOCTYPE>, <?xml?> directives.

Jinja2 (.jinja, .jinja2, .j2, .html.j2)

Aspect	Details
Tokenizer	jinja_tokenizer.nim — JinjaTokenizer
Validator	jinja_validator.nim — JinjaValidator

Error Code	Condition	Severity	Hint
E0003	Unclosed Jinja comment ({# without #})	esError	--
E2001	Unclosed expression tag ({{ without }})	Implicit via tokenizer	--
E2001	Unclosed block tag ({% without %})	Implicit via tokenizer	--
E2002	Unclosed for block (no matching endfor)	esError	"Add {% endfor %}"
E2002	Unclosed if block (no matching endif)	esError	"Add {% endif %}"
E2002	Unclosed block (no matching endblock)	esError	"Add {% endblock %}"
E2002	Unclosed macro, filter, raw, autoescape, call, with	esError	"Add matching end{tag}"

Special features: tracks 12 block tag types (for, if, block, macro, filter, raw, autoescape, call, with), validates matching end tags, handles content between template syntax and plain text.

JSON (.json, .jsonc)

Aspect	Details
Validator	config_validators.nim — JSONValidator
Method	Uses Nim's built-in parseJson() for full validation

Error Code	Condition	Severity	Hint
E0007	JSON parse error (any JsonParsingError)	esError	Error message from parser
E0007	Any other exception during parse	esError	"JSON error: {message}"
E0009	Empty JSON source	esError	--

YAML (.yaml, .yml)

Aspect	Details
Validator	config_validators.nim — YAMLValidator
Method	Line-by-line structural analysis

Error Code	Condition	Severity	Hint
E0007	Empty mapping key (no key before :)	esError	--
E0007	General YAML structural error	esError	--
E0009	No content after stripping comments/blanks	esError	--
W1001	Tab characters used for indentation	esWarning	"Use spaces for YAML indentation"

Note: Empty YAML (only whitespace/comments) is valid: true.

TOML (.toml)

Aspect	Details
Validator	config_validators.nim — TOMLValidator
Method	Line-by-line structural analysis

Error Code	Condition	Severity	Hint
E0007	Missing = in key-value pair	esError	"TOML uses key = value format"
E0007	Missing key before =	esError	--
E0007	General TOML structural error	esError	--
E0009	No content after stripping comments/blanks	esError	--

---

CLI Usage

Full CLI Reference

Nimcheck v0.1.0 - Universal Source Code Validation Framework

Usage:
  nimcheck validate [--file=<path> | --source=<code>] [--flavor=<lang>]
  nimcheck inspect  [--file=<path> | --source=<code>] [--flavor=<lang>]
  nimcheck detect   [--file=<path> | --source=<code>]
  nimcheck help

Options:
  --file=<path>     Source file to process
  --source=<code>   Source code string to process
  --flavor=<lang>   Language flavor (auto-detect if omitted)
  --debug           Enable debug mode
  --json            Output as JSON (default for inspect)

Supported flavors:
  - bash
  - html
  - jinja
  - jinja2
  - js
  - json
  - jsonc
  - nim
  - nimble
  - nims
  - php
  - phtml
  - python
  - py
  - shell
  - sh
  - toml
  - ts
  - typescript
  - yaml
  - yml

Exit Codes

Exit Code	Meaning
0	Validation successful (or help/version requested)
Non-zero (Nim default)	Runtime error

Note: The validate command always exits 0 on the Nim level; the valid field in the output indicates pass/fail. This is by design — Nimcheck never crashes.

---

Building and Testing

Prerequisites

• Nim >= 2.0.0
• make (optional for Makefile targets)

Build

# Using Makefile
make build                  # Release build -> bin/nimcheck
make build-debug            # Debug build with -d:nimcheckDebug

# Using nimble
nimble build

# Manual
nim c src/nimcheck.nim
nim c -d:nimcheckDebug src/nimcheck.nim   # With debug output

Test

The test suite covers 250+ test cases across all supported languages.

make test                   # Run all tests
make test-nim               # Nim-specific tests only
make test-python            # Python-specific tests only
make test-bash              # Bash-specific tests only
make test-js                # JavaScript/TypeScript tests
make test-php               # PHP tests
make test-html              # HTML/XML tests
make test-jinja             # Jinja2 tests
make test-json              # JSON tests
make test-yaml              # YAML tests
make test-toml              # TOML tests
make test-mixed             # Mixed-language file tests
make debug                  # Build debug + run all tests

Test Categories

Suite	Files	Coverage
Per-language basic	test_<lang>.nim	Valid + invalid source for each language
Per-language exhaustive	test_<lang>_exhaustive.nim	Edge cases: unclosed strings, unclosed comments, mismatched brackets, binary data, Unicode
Config formats	test_config.nim + `test_<json	yaml
Fuzz tests	test_fuzz.nim	Null bytes, binary data, deep nesting (100 levels), empty sources, concurrent validation
Mixed files	test_mixed.nim	HTML+Jinja combined, other mixed-format files
Common	test_common.nim	Shared test fixtures for valid and invalid code per language

Lint

make lint
# or
nim c --hints:on --warnings:on src/nimcheck.nim

Clean

make clean
make rebuild    # clean + build

---

Project Structure

.
|-- Makefile
|-- nimcheck.nimble
|-- README.md
|-- LESSONS_LEARNED.md
|-- CONTRIBUTING.md
|-- LICENSE
|-- bin/                          # Build output
|   `-- nimcheck                  # Compiled binary
|-- src/
|   |-- nimcheck.nim              # Public API entry point + CLI (isMainModule)
|   `-- nimcheck/
|       |-- core/
|       |   |-- types.nim         # All core types, toJson() methods, error code constants
|       |   |-- tokenizerbase.nim # Abstract TokenizerBase with position tracking, bracket stack, error recording
|       |   |-- validatorbase.nim # Abstract ValidatorBase with pipeline, factory, registry
|       |   |-- detector.nim      # Language auto-detection (extension, shebang, content scoring)
|       |   |-- debug.nim         # Debug logging (compiled only with -d:nimcheckDebug)
|       |   `-- init_validators.nim
|       |-- tokenizers/
|       |   |-- nim_tokenizer.nim       # Nim: """ strings, r"..." raw strings, #[ ]# nested comments, ## doc comments
|       |   |-- bash_tokenizer.nim      # Bash: heredocs, $var/${var}, $(), backticks, (( ))
|       |   |-- python_tokenizer.nim    # Python: f-strings, """ strings, decorators, indentation tracking
|       |   |-- javascript_tokenizer.nim # JS/TS: template literals, regex literals, arrow functions
|       |   |-- php_tokenizer.nim       # PHP: <?php/?> mode switching, $variables
|       |   |-- html_tokenizer.nim      # HTML/XML: tag balancing, void elements, attributes
|       |   `-- jinja_tokenizer.nim     # Jinja2: {{ }}, {% %}, {# #} block detection
|       |-- languages/
|       |   |-- nim_validator.nim       # Analyzes Nim: imports, procs, types
|       |   |-- bash_validator.nim      # Analyzes Bash: if/fi, for/done, case/esac matching
|       |   |-- python_validator.nim    # Analyzes Python: imports, def, class
|       |   |-- javascript_validator.nim # Analyzes JS/TS: imports, functions, classes
|       |   |-- php_validator.nim       # Analyzes PHP: functions, classes, includes
|       |   |-- html_validator.nim      # Analyzes HTML/XML: tag balancing
|       |   |-- jinja_validator.nim     # Analyzes Jinja2: block tag matching (12 tag types)
|       |   `-- config_validators.nim   # JSON (via parseJson), YAML, TOML validators
|       `-- reporting/
|           `-- errors.nim       # Human-readable and JSON error formatting
`-- tests/
    |-- test_all.nim             # Master test runner (imports all)
    |-- test_nim.nim             # Nim basic tests
    |-- test_nim_exhaustive.nim  # Nim edge cases
    |-- test_bash.nim
    |-- test_bash_exhaustive.nim
    |-- test_python.nim
    |-- test_python_exhaustive.nim
    |-- test_javascript.nim
    |-- test_javascript_exhaustive.nim
    |-- test_php.nim
    |-- test_php_exhaustive.nim
    |-- test_html.nim
    |-- test_html_exhaustive.nim
    |-- test_jinja.nim
    |-- test_jinja_exhaustive.nim
    |-- test_json_exhaustive.nim
    |-- test_yaml_exhaustive.nim
    |-- test_toml_exhaustive.nim
    |-- test_config.nim          # Config format tests
    |-- test_mixed.nim           # Mixed-language file tests
    |-- test_fuzz.nim            # Fuzz and boundary tests
    |-- test_common.nim          # Shared test fixtures
    `-- fixtures/                # Per-language test data files
        |-- nim/
        |-- bash/
        |-- python/
        |-- javascript/
        |-- php/
        |-- html/
        |-- jinja/
        |-- json/
        |-- yaml/
        |-- toml/
        `-- mixed/

---

Contributing

Contributions are welcome. Open an issue to discuss your idea before submitting a pull request.

Adding a New Language

See LESSONS_LEARNED.md for the complete checklist (Section 14). The minimum steps are:

1. Create src/nimcheck/tokenizers/<lang>_tokenizer.nim inheriting from TokenizerBase
2. Implement the tokenize() method with language-specific lexing
3. Create src/nimcheck/languages/<lang>_validator.nim inheriting from ValidatorBase
4. Implement createTokenizer(), analyzeTokens(), and buildModuleInfo()
5. Add registerValidator("flavor", constructor) in the validator's init() proc
6. Import the validator in src/nimcheck.nim
7. Add extension mappings to detector.nim
8. Add test fixtures and test files under tests/
9. Update the test master runner tests/test_all.nim

Design Principles

• KISS: Keep validators simple — tokenize, analyze brackets, report
• Never crash: Every validation path in a try/except
• No external deps: Pure Nim standard library
• DRY: Common logic in tokenizerbase.nim and validatorbase.nim

---

License

MIT -- see LICENSE for details.

Author

molodetz