From 96a5dc834a64f6a4b0f8705929caf8e2feddc5d7 Mon Sep 17 00:00:00 2001 From: retoor Date: Wed, 15 Jul 2026 06:59:03 +0200 Subject: [PATCH] docs: align README with project documentation rules Shrink README to a concise product overview (capabilities, quick start, build, test, structure, doc links). Move API reference, types, JSON schemas, and CLI detail to docs/API.md; move error catalogue to docs/ERRORS.md; move pipeline layers to docs/ARCHITECTURE.md. Add CLAUDE.md with README policy and sync task. --- CLAUDE.md | 43 ++ CONTRIBUTING.md | 2 +- README.md | 1100 +++--------------------------------------- docs/API.md | 459 ++++++++++++++++++ docs/ARCHITECTURE.md | 75 +++ docs/ERRORS.md | 267 ++++++++++ docs/STATUS.md | 15 +- 7 files changed, 922 insertions(+), 1039 deletions(-) create mode 100644 CLAUDE.md create mode 100644 docs/API.md create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/ERRORS.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..0a4f88f --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,43 @@ +# Nimcheck — project notes + +## Standing task + +Keep `README.md`, this file, and the test suite aligned with the current +implementation. Update all that apply together when any one changes. + +## README policy + +`README.md` is product-facing: concise overview, capabilities, quick start, build, +test, structure, documentation links. No badges, no contribution appeals, no +embedded API reference (that lives under `docs/`). + +## Configuration + +No `.env.json` or environment-based configuration. Nimcheck runs with defaults +after `nimble build`; no setup step required. + +## Testing + +- Framework: Nim `unittest` via `tests/test_all.nim` +- Run: `make test` or `nimble test` +- Single language: `nim c -r tests/test_.nim` +- Lint: `make lint` +- Coverage: `make coverage` (optional; requires nimcoverage) + +## Repository + +- Default branch: `master` +- CI: `.gitea/workflows/ci.yml` — see `docs/GITEA.md` +- Remote: `https://retoor.molodetz.nl/retoor/nimcheck.git` + +## Documentation map + +| File | Role | +|------|------| +| `README.md` | Overview and quick start | +| `docs/API.md` | API, types, JSON, CLI | +| `docs/ERRORS.md` | Error codes and per-language triggers | +| `docs/ARCHITECTURE.md` | Pipeline layers | +| `docs/STATUS.md` | Release status | +| `docs/GITEA.md` | CI and branch policy | +| `CONTRIBUTING.md` | Development workflow | \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1565ff6..b43894f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing to Nimcheck -Thanks for helping improve Nimcheck. This project is intentionally small: pure Nim stdlib, tokenizer-based validation, no external parser deps. +Pure Nim stdlib, tokenizer-based validation, no external parser dependencies. --- diff --git a/README.md b/README.md index 9f4fece..317e07a 100644 --- a/README.md +++ b/README.md @@ -1,1091 +1,119 @@ - # Nimcheck -Universal source code validation framework written in Nim. Tokenizer-based syntax validation across **28 language flavors** (40+ registry aliases), with auto-detection, structured JSON diagnostics, structural inspection, and a never-crash guarantee. +Universal source code validation framework for Nim. Tokenizer-based structural +analysis across 28 language flavors, with extension, shebang, and content +auto-detection, structured JSON diagnostics, and exception-safe validation +paths. -Version: **0.1.0** | Author: **molodetz** | License: **MIT** | Requires: **Nim >= 2.0.0** -Default branch: **`master`** | Remote: `https://retoor.molodetz.nl/retoor/nimcheck.git` +Author: retoor · Version 0.1.0 · License MIT · Nim >= 2.0.0 -**Docs:** [Project status & what's done](docs/STATUS.md) · [Gitea CI setup](docs/GITEA.md) · [Contributing](CONTRIBUTING.md) +## Capabilities ---- +- Detector — resolves `LanguageFlavor` from explicit input, file extension, + shebang, or content scoring with false-positive filters. +- Tokenizer — per-language lexers; position tracking, bracket stack, string and + comment context. +- Validator — structural analysis per flavor; factory registry with 40+ aliases. +- Reporting — `formatReport()` and `toJson()` on all result types. +- CLI — `validate`, `inspect`, and `detect` subcommands. +- Library — `import nimcheck` for `validateSource`, `validateFile`, `inspectSource`. -## Table of Contents - -1. [Architecture](#architecture) -2. [Quick Start](#quick-start) -3. [API Reference](#api-reference) -4. [Data Types](#data-types) -5. [JSON Output Structures](#json-output-structures) -6. [Error Code Reference](#error-code-reference) -7. [IO Table: Supported Languages and Per-Language Errors](#io-table-supported-languages-and-per-language-errors) -8. [CLI Usage](#cli-usage) -9. [Building and Testing](#building-and-testing) -10. [Project Structure](#project-structure) -11. [Contributing](#contributing) -12. [Documentation Index](#documentation-index) - ---- +Flavors with validators: Nim, Python, Bash, JavaScript, TypeScript, PHP, HTML, +XML, Jinja, JSON, YAML, TOML, C, C++, C#, Java, Go, Rust, Ruby, Lua, Swift, +Kotlin, CSS, SQL, Markdown, Dockerfile, Makefile. See `supportedFlavors()` or +`bin/nimcheck help` for registered aliases. ## Architecture -Nimcheck uses a four-layer pipeline architecture. Each layer is independent and communicates through well-defined interfaces. - ``` -Source Code (string or file) - | - v -+-------------------+ -| Detector | LanguageFlavor from extension, shebang, or content analysis -+-------------------+ - | - v -+-------------------+ -| Tokenizer | Converts raw source into a seq[Token] stream -+-------------------+ - | - v -+-------------------+ -| Validator | Analyzes token stream for syntax errors and structure -+-------------------+ - | - v -+-------------------+ -| Reporting | Formats errors as human-readable text or structured JSON -+-------------------+ - | - v - ValidationResult +Source → Detector → Tokenizer → Validator → Reporting → ValidationResult ``` -### Layer 1: Detector (`src/nimcheck/core/detector.nim`) +Layer detail: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). -Language auto-detection with cascading priority: +## Quick start -| Priority | Method | Source | Confidence | -|----------|--------|--------|------------| -| 1 (highest) | Explicit | User-provided `LanguageFlavor` | 1.0 | -| 2 | File extension | 70+ extension-to-flavor mappings | 0.9 | -| 3 | Shebang line | `#!` line interpreter matching (bash, python, node, ruby, lua, swift) | 0.95 | -| 4 | Content analysis | Keyword scoring, structural patterns, bracket usage | 0.6-0.9 | - -The `detectFlavor()` function returns a `(LanguageFlavor, FlavorDetectionMethod, confidence: float)` tuple. - -### Layer 2: Tokenizer (`src/nimcheck/core/tokenizerbase.nim`, per-language impls) - -Each tokenizer converts raw source into a `seq[Token]`. Tokenizers track: - -- **Position**: line, column, and byte offset for every token -- **Bracket stack**: `(`, `[`, `{`, `<` pushed and popped with position tracking -- **String context**: escape sequences, multi-line delimiters, interpolation -- **Comment context**: line comments, block comments with nesting support - -Tokenizers emit errors for: -- Unclosed strings (`ErrUnclosedString`) -- Unclosed comments (`ErrUnclosedComment`) -- Unexpected characters (`ErrUnexpectedToken`) -- Unclosed block constructs (language-specific, e.g., `${}`, `$()`) - -### Layer 3: Validator (`src/nimcheck/core/validatorbase.nim`, per-language impls) - -Each validator: - -1. Creates its tokenizer and runs `tokenize()` -2. Collects tokenizer errors -3. Runs `analyzeTokens()` - language-specific structural analysis -4. Runs `buildModuleInfo()` - populates `ModuleInfo` with imports, functions, classes -5. Computes validity, warning/info counts, duration - -Validators register themselves via a factory pattern at import time. The registry maps flavor strings (e.g., `"python"`, `"bash"`) to constructors. - -### Layer 4: Reporting (`src/nimcheck/reporting/errors.nim`) - -Produces: -- **Human-readable reports**: `formatReport()` with position, severity, code, message, source context with carets, and hints -- **JSON output**: every type has a `toJson()` method producing `JsonNode` -- **Error categorization**: `countBySeverity()`, `countByCode()`, `filterBySeverity()` - ---- - -## Quick Start - -### Nim library usage +### Library ```nim import nimcheck -# Validate source code from a string let r = validateSource("proc hello() = echo 'hi'", lfNim) -echo r.valid # bool -echo r.errors # seq[ValidationError] -echo r.toJson().pretty() # Full JSON output +echo r.valid +echo r.toJson().pretty() -# Validate a file (auto-detects language from extension) let r2 = validateFile("src/main.nim") - -# Auto-detect language -let r3 = validateSource("print('hello')", lfUnknown) -echo r3.flavor # lfPython - -# Inspect source structure (returns JsonNode with functions, classes, imports) let info = inspectSource("class Foo:\n def bar(self): pass", lfPython) -echo info.pretty() - -# Human-readable report -echo reportSource("proc foo() =\n let x = \"unclosed", lfNim) ``` -### CLI usage +### CLI ```bash -# Build the binary make build - -# Validate a file bin/nimcheck validate --file=source.nim - -# Validate source directly bin/nimcheck validate --source="print('hi')" --flavor=python - -# Read from stdin -echo "var x = 1" | bin/nimcheck validate - -# Inspect structure (JSON output) bin/nimcheck inspect --file=source.py - -# Detect language bin/nimcheck detect --file=unknown.txt -bin/nimcheck detect --source="#!/usr/bin/env bash" - -# Version and help -bin/nimcheck version -bin/nimcheck help +echo "var x = 1" | bin/nimcheck validate ``` ---- +CLI reference: [docs/API.md](docs/API.md) (or `bin/nimcheck help`). -## API Reference - -### Core Functions - -All functions are exported from `nimcheck.nim`. Import once with `import nimcheck`. - -#### Validation - -| Function | Signature | Returns | Description | -|----------|-----------|---------|-------------| -| `validateSource` | `(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")` | `ValidationResult` | Validate source code string. Auto-detects flavor if `lfUnknown`. | -| `validateFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())` | `ValidationResult` | Validate a file from disk. Auto-detects flavor from file extension. | -| `validateSourceJson` | `(source: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Validate and return pretty-printed JSON string. | -| `validateFileJson` | `(filePath: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Validate file and return pretty-printed JSON string. | - -#### Inspection - -| Function | Signature | Returns | Description | -|----------|-----------|---------|-------------| -| `inspectSource` | `(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")` | `JsonNode` | Validate and return structural diagnostics (functions, classes, imports). | -| `inspectFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())` | `JsonNode` | Inspect a file and return structural diagnostics. | - -#### Reporting - -| Function | Signature | Returns | Description | -|----------|-----------|---------|-------------| -| `reportSource` | `(source: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Human-readable validation report for source code. | -| `reportFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Human-readable validation report for a file. | - -#### Detection and Utility - -| Function | Signature | Returns | Description | -|----------|-----------|---------|-------------| -| `detectFlavor` | `(source: string, filePath: string = "")` | `LanguageFlavor` | Auto-detect language flavor (convenience wrapper). | -| `supportedFlavors` | `()` | `seq[string]` | List all registered language flavors sorted alphabetically. | -| `version` | `()` | `string` | Version string (`"0.1.0"`). | -| `enableDebug` | `()` | `void` | Enable debug mode globally. | -| `disableDebug` | `()` | `void` | Disable debug mode globally. | - -#### Core Detector (lower-level) - -| Function | Signature | Returns | Description | -|----------|-----------|---------|-------------| -| `detectFlavor` (core) | `(source: string, filePath: string = "", explicitFlavor: LanguageFlavor = lfUnknown)` | `(LanguageFlavor, FlavorDetectionMethod, float)` | Full detection with confidence score. | - -#### Factory - -| Function | Signature | Returns | Description | -|----------|-----------|---------|-------------| -| `createValidator` | `(flavor: LanguageFlavor, source: string, options: ValidationOptions, sourcePath: string)` | `ValidatorBase` | Factory: create the correct validator for a flavor. | -| `registerValidator` | `(flavor: string, constructor: ValidatorConstructor)` | `void` | Register a new validator constructor. | - ---- - -## Data Types - -### LanguageFlavor (enum, pure) - -All 28 flavors in `src/nimcheck/core/types.nim` have dedicated validators (v0.1.0). Registry aliases (e.g. `py`, `js`, `md`) map to the same validators — see `supportedFlavors()` or `bin/nimcheck help`. - -| Enum Value | String | Validator module | File Extensions (auto-detect) | -|------------|--------|------------------|-------------------------------| -| `lfUnknown` | `"unknown"` | -- | -- | -| **`lfNim`** | `"nim"` | `nim_validator.nim` | `.nim`, `.nims`, `.nimble` | -| **`lfPython`** | `"python"` | `python_validator.nim` | `.py`, `.pyw`, `.pyx`, `.pxd` | -| **`lfBash`** | `"bash"` | `bash_validator.nim` | `.sh`, `.bash` | -| **`lfShell`** | `"shell"` | `bash_validator.nim` (shared) | `.zsh`, `.fish` | -| **`lfJavaScript`** | `"javascript"` | `javascript_validator.nim` | `.js`, `.mjs`, `.cjs` | -| **`lfTypeScript`** | `"typescript"` | `type_xml_validator.nim` | `.ts`, `.tsx` | -| **`lfPHP`** | `"php"` | `php_validator.nim` | `.php`, `.phtml` | -| **`lfHTML`** | `"html"` | `html_validator.nim` | `.html`, `.htm`, `.xhtml` | -| **`lfXML`** | `"xml"` | `type_xml_validator.nim` | `.xml`, `.xsd`, `.xslt`, `.svg` | -| **`lfJinja`** | `"jinja"` | `jinja_validator.nim` | `.jinja`, `.jinja2`, `.j2` | -| **`lfJSON`** | `"json"` | `config_validators.nim` | `.json`, `.jsonc` | -| **`lfYAML`** | `"yaml"` | `config_validators.nim` | `.yaml`, `.yml` | -| **`lfTOML`** | `"toml"` | `config_validators.nim` | `.toml` | -| **`lfCSS`** | `"css"` | `extended_validators.nim` | `.css` | -| **`lfSQL`** | `"sql"` | `extended_validators.nim` | `.sql` | -| **`lfMarkdown`** | `"markdown"` | `extended_validators.nim` | `.md`, `.markdown` | -| **`lfDockerfile`** | `"dockerfile"` | `extended_validators.nim` | `Dockerfile` | -| **`lfMakefile`** | `"makefile"` | `extended_validators.nim` | `Makefile`, `makefile` | -| **`lfRuby`** | `"ruby"` | `extended_validators.nim` | `.rb` | -| **`lfRust`** | `"rust"` | `extended_validators.nim` | `.rs` | -| **`lfGo`** | `"go"` | `extended_validators.nim` | `.go` | -| **`lfLua`** | `"lua"` | `lang_validators.nim` | `.lua` | -| **`lfC`** | `"c"` | `cfamily_validators.nim` | `.c`, `.h` | -| **`lfCpp`** | `"cpp"` | `cfamily_validators.nim` | `.cpp`, `.cxx`, `.hpp` | -| **`lfCSharp`** | `"csharp"` | `cfamily_validators.nim` | `.cs` | -| **`lfJava`** | `"java"` | `cfamily_validators.nim` | `.java` | -| **`lfSwift`** | `"swift"` | `lang_validators.nim` | `.swift` | -| **`lfKotlin`** | `"kotlin"` | `lang_validators.nim` | `.kt`, `.kts` | - -### FlavorDetectionMethod (enum) - -| Value | Meaning | -|-------|---------| -| `fdmExplicit` | User explicitly provided the flavor | -| `fdmExtension` | Detected from file extension | -| `fdmShebang` | Detected from `#!` shebang line | -| `fdmContent` | Detected via content keyword analysis | -| `fdmUnknown` | Could not determine | - -### TokenKind (enum) - -Lexical token classifications used by all tokenizers. - -| Kind | Description | -|------|-------------| -| `tkWhitespace` | Spaces, tabs, carriage returns | -| `tkNewline` | `\n` | -| `tkComment` | Line (`#`, `//`) or block (`/* */`) comment | -| `tkDocComment` | Documentation comment (`##`, `///`, `"""`) | -| `tkString` | Single or double quoted string literal | -| `tkRawString` | Raw string (`r"..."`) | -| `tkMultilineString` | Triple-quoted string (`"""..."""`) | -| `tkNumber` | Integer, float, hex, octal, binary | -| `tkIdentifier` | Variable, function, type name | -| `tkKeyword` | Language-specific reserved word | -| `tkOperator` | `+`, `-`, `*`, `/`, `&&`, etc. | -| `tkAssignment` | `=`, `:=`, `+=`, etc. | -| `tkPunctuation` | `.`, `,`, `;`, `:` | -| `kOpenParen` | `(` | -| `kCloseParen` | `)` | -| `kOpenBracket` | `[` | -| `kCloseBracket` | `]` | -| `kOpenBrace` | `{` | -| `kCloseBrace` | `}` | -| `kAngleOpen` | `<` | -| `kAngleClose` | `>` | -| `tkInterpolation` | String interpolation (`${var}`, f-string `{expr}`) | -| `tkDirective` | Preprocessor / compiler directive (`#include`, shebang) | -| `tkTemplateTag` | Template language tag (`{{ }}`, `{% %}`) | -| `tkSpecial` | Language-specific special token | -| `tkError` | Malformed or unrecognizable character | -| `tkEndOfFile` | End of input marker | - -### Token - -``` -Token = object - kind: TokenKind - value: string - position: SourcePosition - range: SourceRange -``` - -### SourcePosition - -``` -SourcePosition = object - line: int # 1-indexed line number - column: int # 1-indexed column number - offset: int # 0-indexed byte offset from start -``` - -### SourceRange - -``` -SourceRange = object - startPos: SourcePosition - endPos: SourcePosition -``` - -### ErrorSeverity (enum) - -| Value | String | Description | -|-------|--------|-------------| -| `esInfo` | `"info"` | Informational note | -| `esWarning` | `"warning"` | Non-fatal issue | -| `esError` | `"error"` | Validation failure | -| `esCritical` | `"critical"` | Internal framework failure | - -### ValidationError - -``` -ValidationError = object - severity: ErrorSeverity - message: string - code: string # Error code, e.g. "E0002" - position: SourcePosition - range: SourceRange - context: string # Surrounding source context (optional) - hint: string # Fix suggestion (optional) -``` - -### ValidationResult - -``` -ValidationResult = object - valid: bool - flavor: LanguageFlavor - detectionMethod: FlavorDetectionMethod - errors: seq[ValidationError] - warnings: int - infos: int - moduleInfo: ModuleInfo - durationMs: float - debugOutput: string # Empty unless compiled with -d:nimcheckDebug -``` - -### ValidationOptions - -``` -ValidationOptions = object - flavor: LanguageFlavor # lfUnknown = auto-detect (default) - debugMode: bool # Enable per-validation debug logging (default: false) - strictMode: bool # Warnings become errors (default: false) - maxErrors: int # Stop after N errors; 0 = unlimited (default: 0) - includeContext: bool # Include surrounding source lines in error output (default: true) - includeHints: bool # Include fix suggestions (default: true) -``` - -### Structural Types (populated by `analyzeTokens()`) - -``` -VariableInfo = object - name: string - kind: string # "var", "let", "const", "global", "local" - typeAnnotation: string - defaultValue: string - position: SourcePosition - mutable: bool - exported: bool - -FunctionInfo = object - name: string - kind: string # "function", "method", "proc", "func", "lambda" - parameters: seq[ParameterInfo] - returnType: string - isAsync: bool - isExported: bool - isGenerator: bool - visibility: string # "public", "private", "protected" - position: SourcePosition - docComment: string - -ParameterInfo = object - name: string - typeAnnotation: string - defaultValue: string - isOptional: bool - isVarargs: bool - -ClassInfo = object - name: string - kind: string # "class", "object", "struct", "interface" - baseTypes: seq[string] - fields: seq[VariableInfo] - methods: seq[FunctionInfo] - generics: seq[string] - visibility: string - position: SourcePosition - docComment: string - -ImportInfo = object - module: string - symbols: seq[string] # Specific imported symbols - isRelative: bool - alias: string - position: SourcePosition - -ModuleInfo = object - flavor: LanguageFlavor - detectionMethod: FlavorDetectionMethod - imports: seq[ImportInfo] - variables: seq[VariableInfo] - functions: seq[FunctionInfo] - classes: seq[ClassInfo] - linesOfCode: int - totalTokens: int - comments: int - debugInfo: JsonNode # Language-specific extra info -``` - ---- - -## JSON Output Structures - -Every type has a `.toJson()` method. The `ValidationResult.toJson()` aggregates all nested structures. - -### ValidationResult JSON - -```json -{ - "valid": true, - "flavor": "python", - "detectionMethod": "by_extension", - "errors": [ - { - "severity": "error", - "message": "Unclosed string literal starting with '\"'", - "code": "E0002", - "position": {"line": 2, "column": 13, "offset": 42}, - "range": { - "start": {"line": 2, "column": 13, "offset": 42}, - "end": {"line": 2, "column": 13, "offset": 42} - }, - "context": "", - "hint": "Add a closing quote before end of file" - } - ], - "warnings": 0, - "infos": 0, - "moduleInfo": { - "flavor": "python", - "detectionMethod": "by_extension", - "imports": [], - "variables": [], - "functions": [ - { - "name": "hello", - "kind": "def", - "parameters": [], - "returnType": "", - "isAsync": false, - "isExported": false, - "isGenerator": false, - "visibility": "", - "position": {"line": 1, "column": 5, "offset": 4}, - "docComment": "" - } - ], - "classes": [], - "linesOfCode": 3, - "totalTokens": 12, - "comments": 0, - "debugInfo": null - }, - "durationMs": 0.342, - "debugOutput": "" -} -``` - -### ValidationError JSON - -```json -{ - "severity": "error", - "message": "Unclosed string literal starting with '\"'", - "code": "E0002", - "position": {"line": 2, "column": 13, "offset": 42}, - "range": { - "start": {"line": 2, "column": 13, "offset": 42}, - "end": {"line": 2, "column": 13, "offset": 42} - }, - "context": "", - "hint": "Add a closing quote before end of file" -} -``` - -### ModuleInfo JSON - -```json -{ - "flavor": "nim", - "detectionMethod": "by_extension", - "imports": [ - { - "module": "strutils", - "symbols": [], - "isRelative": false, - "alias": "", - "position": {"line": 1, "column": 8, "offset": 7} - } - ], - "variables": [], - "functions": [ - { - "name": "add", - "kind": "proc", - "parameters": [], - "returnType": "", - "isAsync": false, - "isExported": false, - "isGenerator": false, - "visibility": "", - "position": {"line": 3, "column": 6, "offset": 28}, - "docComment": "" - } - ], - "classes": [ - { - "name": "MyType", - "kind": "type", - "baseTypes": [], - "fields": [], - "methods": [], - "generics": [], - "visibility": "", - "position": {"line": 5, "column": 6, "offset": 52}, - "docComment": "" - } - ], - "linesOfCode": 8, - "totalTokens": 45, - "comments": 2, - "debugInfo": null -} -``` - -### Human-Readable Report Format - -``` -=== Validation Report === - Flavor: python - Detection: by_extension - Valid: false - Errors: 1 - Warnings: 0 - Duration: 0.34ms - Lines of code: 3 - Tokens: 12 - - --- Findings --- - [E0002] (error) L2:13: Unclosed string literal starting with '"' - | x = " - | ^ - hint: Add a closing quote before end of file - - --- Summary --- - INVALID - source has issues that need attention. -``` - -### Error Formatter Output (single error) - -``` - [E0002] (error) L2:13: Unclosed string literal starting with '"' - | x = " - | ^ - hint: Add a closing quote before end of file -``` - ---- - -## Error Code Reference - -Error codes follow a prefixed numbering scheme. The prefix encodes the category and severity. - -### General Errors (E0xxx) - -| Code | Constant | Description | Triggered By | -|------|----------|-------------|-------------| -| `E0001` | `ErrUnexpectedToken` | Unexpected or unrecognizable character | Tokenizer encounter of non-ASCII, non-operator char | -| `E0002` | `ErrUnclosedString` | Unclosed string literal | Missing closing quote at EOF | -| `E0003` | `ErrUnclosedComment` | Unclosed block comment | Missing `*/`, `]#`, `#}` etc. at EOF | -| `E0004` | `ErrUnclosedBlock` | Unclosed block/control structure | Language-specific: unclosed `if`/`for`/`class`/etc. | -| `E0005` | `ErrMismatchedBracket` | Mismatched or unexpected closing bracket | `)` without `(`, `` without `
` | -| `E0006` | `ErrUnclosedBracket` | Unclosed bracket at EOF | `validateBrackets()` finds remaining items on bracket stack | -| `E0007` | `ErrInvalidSyntax` | General invalid syntax | JSON parse error, TOML missing `=`, YAML empty key | -| `E0008` | `ErrUnexpectedEOF` | Unexpected end of file | Reserved for incomplete constructs not covered by other codes | -| `E0009` | `ErrEmptySource` | Source is empty (zero-length string) | `validate()` with `source.len == 0` | -| `E0010` | `ErrEncodingError` | Encoding error | Reserved for non-UTF-8 input | -| `E0011` | -- | File not found | `validateFile()` with non-existent path | - -### Language-Specific Errors (E1xxx) - -| Code | Constant | Description | Languages | -|------|----------|-------------|-----------| -| `E1001` | `ErrUnexpectedIndent` | Unexpected or inconsistent indentation | Python | -| `E1002` | `ErrMissingIndent` | Missing indentation after block opener | Python | -| `E1003` | `ErrDuplicateParam` | Duplicate parameter name | (reserved) | -| `E1004` | `ErrUndefinedVariable` | Reference to undefined variable | (reserved) | -| `E1005` | `ErrTypeMismatch` | Type mismatch | (reserved) | -| `E1006` | `ErrInvalidAssignment` | Invalid assignment target | (reserved) | -| `E1007` | `ErrMissingReturn` | Function missing return statement | (reserved) | -| `E1008` | `ErrInvalidDecorator` | Invalid decorator usage | Python | -| `E1009` | `ErrInvalidImport` | Invalid import statement | (reserved) | - -### Template Errors (E2xxx) - -| Code | Constant | Description | Languages | -|------|----------|-------------|-----------| -| `E2001` | `ErrUnclosedTemplateTag` | Unclosed template expression/block tag | Jinja | -| `E2002` | `ErrUnclosedBlockTag` | Unclosed block tag with no matching end tag | Jinja | -| `E2003` | `ErrInvalidFilter` | Invalid filter in template expression | Jinja | - -### Info Codes (I0xxx) - -| Code | Constant | Description | -|------|----------|-------------| -| `I0001` | `InfoLongLine` | Line exceeds recommended length | -| `I0002` | `InfoTrailingWhitespace` | Trailing whitespace detected | - -### Warning Codes (W0xxx / W1xxx) - -| Code | Constant | Description | -|------|----------|-------------| -| `W0001` | `WarnUnusedVariable` | Variable declared but never used | -| `W0002` | `WarnInconsistentNaming` | Naming convention inconsistency | -| `W0003` | `WarnMissingDoc` | Public symbol missing documentation | -| `W1001` | -- | Tab characters in YAML indentation | - -### Internal Error - -| Code | Description | Triggered By | -|------|-------------|-------------| -| `E9999` | Internal framework error (never-crash fallback) | Any unhandled `Exception` in `validate()` or `validateSource()` | - ---- - -## IO Table: Supported Languages and Per-Language Errors - -### Input/Output Contract - -Every validation function follows the same contract: - -| Input | Output | -|-------|--------| -| `source: string` (required) | `ValidationResult` with `valid: bool` | -| `flavor: LanguageFlavor = lfUnknown` | `ValidationResult.errors: seq[ValidationError]` | -| `options: ValidationOptions` (optional) | `ValidationResult.moduleInfo: ModuleInfo` | -| `sourcePath: string = ""` (optional) | `ValidationResult.durationMs: float` | - -**Guarantees**: -- Never throws - all exceptions are caught and returned as `E9999` errors -- Empty source returns `E0009` (or `valid: true` for YAML/TOML where empty is valid) -- File not found returns `E0011` -- Unknown flavor with no detection possible returns an `E0007` error -- Every error has a `code`, `message`, `severity`, `position`, and optional `hint` - -### Per-Language Tokenizer, Validator, and Error Table - -#### Nim (`.nim`, `.nims`, `.nimble`) - -| Aspect | Details | -|--------|---------| -| Tokenizer | `nim_tokenizer.nim` - `NimTokenizer` | -| Validator | `nim_validator.nim` - `NimValidator` | -| Keywords | 80+ (addr, and, as, block, break, case, const, converter, defer, discard, elif, else, enum, except, export, finally, for, from, func, if, import, in, include, iterator, let, macro, method, mixin, mod, nil, not, object, of, or, proc, ptr, raise, ref, return, template, try, tuple, type, using, var, when, while, with, yield, etc.) | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0001` | Unexpected character (non-ASCII, unrecognizable) | `esWarning` | Shows hex `U+XXXX` value | -| `E0002` | Unclosed string (`"..."` at EOF) | `esError` | "Add a closing quote" | -| `E0002` | Unclosed multi-line string (`"""` at EOF) | `esError` | -- | -| `E0002` | Newline inside string literal | `esError` | -- | -| `E0002` | Unclosed raw string (`r"..."` at EOF) | `esError` | -- | -| `E0002` | Unclosed character literal (`'x` at EOF) | `esError` | -- | -| `E0003` | Unclosed block comment (`#[` without `]#`) | `esError` | "Add closing `]#`" | -| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | "Add matching closing bracket" | - -Structural extraction: imports (`from X import Y`), functions (`proc`, `func`, `method`, `template`, `macro`, `iterator`, `converter`), types (`type Foo`). - -#### Bash / Shell (`.sh`, `.bash`, `.zsh`, `.fish`) - -| Aspect | Details | -|--------|---------| -| Tokenizer | `bash_tokenizer.nim` - `BashTokenizer` | -| Validator | `bash_validator.nim` - `BashValidator` | -| Keywords | 38+ (if, then, else, elif, fi, case, esac, for, while, until, do, done, in, function, select, time, return, break, continue, exit, declare, local, export, readonly, unset, eval, exec, let, source, shift, trap, type, typeset, ulimit, umask, alias, read, echo) | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0002` | Unclosed single-quoted string | `esError` | -- | -| `E0002` | Unclosed double-quoted string | `esError` | -- | -| `E0004` | Unclosed `${var}` expansion | `esError` | -- | -| `E0004` | Unclosed `$()` command substitution | `esError` | -- | -| `E0004` | Unclosed `if` block (`if` without matching `fi`) | `esError` | "Add missing 'fi'" | -| `E0004` | Unclosed loop (`for`/`while`/`until` without `done`) | `esError` | "Add missing 'done'" | -| `E0004` | Unclosed `case` block (`case` without `esac`) | `esError` | "Add missing 'esac'" | -| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- | - -Special features: shebang detection (`#!`), heredoc detection (`<`), ESM import/export. - -Structural extraction: imports/exports, functions (`function`, `async`), classes. - -#### PHP (`.php`, `.phtml`) - -| Aspect | Details | -|--------|---------| -| Tokenizer | `php_tokenizer.nim` - `PHPTokenizer` | -| Validator | `php_validator.nim` - `PHPValidator` | -| Mode switching | Tracks `inPhpMode` flag - toggles on `` | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0002` | Unclosed string | `esError` | -- | -| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- | - -Special features: PHP/HTML mode switching, `` detection, variable names (`$var`, `$obj->prop`). - -Structural extraction: functions (`function`, `fn`), classes (`class`), imports/requires (`include`, `require`, `use`, `namespace`). - -#### HTML / XML (`.html`, `.htm`, `.xhtml`, `.xml`, `.svg`) - -| Aspect | Details | -|--------|---------| -| Tokenizer | `html_tokenizer.nim` - `HTMLTokenizer` | -| Validator | `html_validator.nim` - `HTMLValidator` | -| Void elements | 14: area, base, br, col, embed, hr, img, input, link, meta, param, source, track, wbr | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0003` | Unclosed HTML comment (``) | `esError` | -- | -| `E0004` | Unclosed HTML tag (e.g., `
` without `
`) | `esError` | "Add closing ``" | -| `E0005` | Unexpected closing tag (mismatched nesting) | `esWarning` | "Check that tags are properly nested" | - -Special features: tag stack balancing, void element awareness (no closing tag needed for self-closing elements), attribute parsing with quoted values, ``, `` directives. - -#### Jinja2 (`.jinja`, `.jinja2`, `.j2`, `.html.j2`) - -| Aspect | Details | -|--------|---------| -| Tokenizer | `jinja_tokenizer.nim` - `JinjaTokenizer` | -| Validator | `jinja_validator.nim` - `JinjaValidator` | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0003` | Unclosed Jinja comment (`{#` without `#}`) | `esError` | -- | -| `E2001` | Unclosed expression tag (`{{` without `}}`) | Implicit via tokenizer | -- | -| `E2001` | Unclosed block tag (`{%` without `%}`) | Implicit via tokenizer | -- | -| `E2002` | Unclosed `for` block (no matching `endfor`) | `esError` | "Add `{% endfor %}`" | -| `E2002` | Unclosed `if` block (no matching `endif`) | `esError` | "Add `{% endif %}`" | -| `E2002` | Unclosed `block` (no matching `endblock`) | `esError` | "Add `{% endblock %}`" | -| `E2002` | Unclosed `macro`, `filter`, `raw`, `autoescape`, `call`, `with` | `esError` | "Add matching `end{tag}`" | - -Special features: tracks 12 block tag types (`for`, `if`, `block`, `macro`, `filter`, `raw`, `autoescape`, `call`, `with`), validates matching end tags, handles content between template syntax and plain text. - -#### JSON (`.json`, `.jsonc`) - -| Aspect | Details | -|--------|---------| -| Validator | `config_validators.nim` - `JSONValidator` | -| Method | Uses Nim's built-in `parseJson()` for full validation | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0007` | JSON parse error (any `JsonParsingError`) | `esError` | Error message from parser | -| `E0007` | Any other exception during parse | `esError` | "JSON error: {message}" | -| `E0009` | Empty JSON source | `esError` | -- | - -#### YAML (`.yaml`, `.yml`) - -| Aspect | Details | -|--------|---------| -| Validator | `config_validators.nim` - `YAMLValidator` | -| Method | Line-by-line structural analysis | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0007` | Empty mapping key (no key before `:`) | `esError` | -- | -| `E0007` | General YAML structural error | `esError` | -- | -| `E0009` | No content after stripping comments/blanks | `esError` | -- | -| `W1001` | Tab characters used for indentation | `esWarning` | "Use spaces for YAML indentation" | - -Note: Empty YAML (only whitespace/comments) is `valid: true`. - -#### TOML (`.toml`) - -| Aspect | Details | -|--------|---------| -| Validator | `config_validators.nim` - `TOMLValidator` | -| Method | Line-by-line structural analysis | - -| Error Code | Condition | Severity | Hint | -|------------|-----------|----------|------| -| `E0007` | Missing `=` in key-value pair | `esError` | "TOML uses `key = value` format" | -| `E0007` | Missing key before `=` | `esError` | -- | -| `E0007` | General TOML structural error | `esError` | -- | -| `E0009` | No content after stripping comments/blanks | `esError` | -- | - ---- - -## CLI Usage - -### Full CLI Reference - -``` -Nimcheck v0.1.0 - Universal Source Code Validation Framework - -Usage: - nimcheck validate [--file= | --source=] [--flavor=] - nimcheck inspect [--file= | --source=] [--flavor=] - nimcheck detect [--file= | --source=] - nimcheck help - -Options: - --file= Source file to process - --source= Source code string to process - --flavor= Language flavor (auto-detect if omitted) - --debug Enable debug mode - --json Output as JSON (default for inspect) - -Supported flavors: - (run `bin/nimcheck help` or `supportedFlavors()` for the full list — 40+ aliases) -``` - -### Exit Codes - -| Exit Code | Meaning | -|-----------|---------| -| 0 | Validation successful (or help/version requested) | -| Non-zero (Nim default) | Runtime error | - -Note: The `validate` command always exits 0 on the Nim level; the `valid` field in the output indicates pass/fail. This is by design - Nimcheck never crashes. - ---- - -## Building and Testing - -### Prerequisites +## Requirements - Nim >= 2.0.0 -- `make` (optional for Makefile targets) +- `make` (optional; Makefile wraps build and test targets) -### Build +No configuration file. No environment variables. Zero setup beyond installing Nim. + +## Building ```bash -# Using Makefile -make build # Release build -> bin/nimcheck -make build-debug # Debug build with -d:nimcheckDebug - -# Using nimble -nimble build - -# Manual -nim c src/nimcheck.nim -nim c -d:nimcheckDebug src/nimcheck.nim # With debug output +make build # release binary → bin/nimcheck +make build-debug # -d:nimcheckDebug +nimble build # equivalent via nimble ``` -### Test - -The test suite covers **400+ test cases** across all supported languages (basic, exhaustive, fuzz). +## Testing ```bash -make test # Run all tests -make test-nim # Nim-specific tests only -make test-python # Python-specific tests only -make test-bash # Bash-specific tests only -make test-js # JavaScript/TypeScript tests -make test-php # PHP tests -make test-html # HTML/XML tests -make test-jinja # Jinja2 tests -make test-json # JSON tests -make test-yaml # YAML tests -make test-toml # TOML tests -make test-mixed # Mixed-language file tests -make debug # Build debug + run all tests +make test # full suite (~11 s) +make lint # compile with all hints and warnings +nim c -r tests/test_go.nim # single language ``` -### Test Categories +Suites: per-language basic and exhaustive tests, config formats, mixed files, +fuzz (null bytes, deep nesting, concurrent calls). See `tests/test_all.nim`. -| Suite | Files | Coverage | -|-------|-------|----------| -| Per-language basic | `test_.nim` | Valid + invalid source for each language | -| Per-language exhaustive | `test__exhaustive.nim` | Edge cases: unclosed strings, unclosed comments, mismatched brackets, binary data, Unicode | -| Config formats | `test_config.nim` + `test__exhaustive.nim` | JSON/YAML/TOML structural validation | -| Fuzz tests | `test_fuzz.nim` | Null bytes, binary data, deep nesting (100 levels), empty sources, concurrent validation | -| Mixed files | `test_mixed.nim` | HTML+Jinja combined, other mixed-format files | -| Common | `test_common.nim` | Shared test fixtures for valid and invalid code per language | - -### Lint - -```bash -make lint -# or -nim c --hints:on --warnings:on src/nimcheck.nim -``` - -### Clean - -```bash -make clean -make rebuild # clean + build -``` - ---- - -## Project Structure +## Project structure ``` . -|-- Makefile -|-- nimcheck.nimble -|-- README.md -|-- LICENSE -|-- bin/ # Build output -| `-- nimcheck # Compiled binary -|-- src/ -| |-- nimcheck.nim # Public API entry point + CLI (isMainModule) -| `-- nimcheck/ -| |-- core/ -| | |-- types.nim # All core types, toJson() methods, error code constants -| | |-- tokenizerbase.nim # Abstract TokenizerBase with position tracking, bracket stack, error recording -| | |-- validatorbase.nim # Abstract ValidatorBase with pipeline, factory, registry -| | |-- detector.nim # Language auto-detection (extension, shebang, content scoring) -| | |-- debug.nim # Debug logging (compiled only with -d:nimcheckDebug) -| | `-- init_validators.nim -| |-- tokenizers/ -| | |-- nim_tokenizer.nim # Nim: """ strings, r"..." raw strings, #[ ]# nested comments, ## doc comments -| | |-- bash_tokenizer.nim # Bash: heredocs, $var/${var}, $(), backticks, (( )) -| | |-- python_tokenizer.nim # Python: f-strings, """ strings, decorators, indentation tracking -| | |-- javascript_tokenizer.nim # JS/TS: template literals, regex literals, arrow functions -| | |-- php_tokenizer.nim # PHP: mode switching, $variables -| | |-- html_tokenizer.nim # HTML/XML: tag balancing, void elements, attributes -| | |-- jinja_tokenizer.nim # Jinja2: {{ }}, {% %}, {# #} block detection -| | |-- extended_tokenizers.nim # Go, Rust, Ruby, CSS, SQL, Markdown, Dockerfile, Makefile -| | |-- cfamily_tokenizers.nim # C, C++, Java, C# -| | |-- lang_tokenizers.nim # Kotlin, Lua, Swift -| | `-- type_xml_tokenizer.nim # TypeScript, XML -| |-- languages/ -| | |-- nim_validator.nim -| | |-- bash_validator.nim -| | |-- python_validator.nim -| | |-- javascript_validator.nim -| | |-- php_validator.nim -| | |-- html_validator.nim -| | |-- jinja_validator.nim -| | |-- config_validators.nim # JSON, YAML, TOML -| | |-- extended_validators.nim # Go, Rust, Ruby, CSS, SQL, Markdown, Dockerfile, Makefile -| | |-- cfamily_validators.nim # C, C++, Java, C# -| | |-- lang_validators.nim # Kotlin, Lua, Swift -| | `-- type_xml_validator.nim # TypeScript, XML -| `-- reporting/ -| `-- errors.nim # Human-readable and JSON error formatting -`-- tests/ - |-- test_all.nim # Master test runner (imports all) - |-- test_nim.nim # Nim basic tests - |-- test_nim_exhaustive.nim # Nim edge cases - |-- test_bash.nim - |-- test_bash_exhaustive.nim - |-- test_python.nim - |-- test_python_exhaustive.nim - |-- test_javascript.nim - |-- test_javascript_exhaustive.nim - |-- test_php.nim - |-- test_php_exhaustive.nim - |-- test_html.nim - |-- test_html_exhaustive.nim - |-- test_jinja.nim - |-- test_jinja_exhaustive.nim - |-- test_json_exhaustive.nim - |-- test_yaml_exhaustive.nim - |-- test_toml_exhaustive.nim - |-- test_config.nim # Config format tests - |-- test_mixed.nim # Mixed-language file tests - |-- test_fuzz.nim # Fuzz and boundary tests - |-- test_common.nim # Shared test fixtures - `-- fixtures/ # Per-language test data files - |-- nim/ - |-- bash/ - |-- python/ - |-- javascript/ - |-- php/ - |-- html/ - |-- jinja/ - |-- json/ - |-- yaml/ - |-- toml/ - |-- c/, cpp/, csharp/, go/, rust/, ruby/, css/, sql/, markdown/ - |-- dockerfile/, makefile/, kotlin/, lua/, swift/, typescript/, xml/ - `-- mixed/ -|-- docs/ -| |-- STATUS.md # What's done, how to verify, future work -| `-- GITEA.md # CI / act_runner / branch policy -|-- CONTRIBUTING.md # Dev guide, adding languages, PR checklist -`-- .gitea/workflows/ci.yml # Gitea Actions pipeline +├── Makefile +├── nimcheck.nimble +├── src/nimcheck.nim # public API and CLI +├── src/nimcheck/ +│ ├── core/ # types, tokenizerbase, validatorbase, detector +│ ├── tokenizers/ # per-language lexers +│ ├── languages/ # per-language validators +│ └── reporting/ # error formatting +├── tests/ # test_all.nim + per-language suites +├── tests/fixtures/ # valid / invalid / edge_cases per language +├── docs/ # reference documentation +└── .gitea/workflows/ci.yml # Gitea Actions pipeline ``` ---- +## Documentation -## Contributing - -See **[CONTRIBUTING.md](CONTRIBUTING.md)** for the full guide (setup, adding languages, PR checklist). - -Design principles: KISS tokenizers, never crash, pure Nim stdlib, shared logic in `tokenizerbase.nim` / `validatorbase.nim`. - ---- - -## Documentation Index - -| Document | Purpose | -|----------|---------| -| [README.md](README.md) | Architecture, API reference, error codes, CLI | -| [docs/STATUS.md](docs/STATUS.md) | **What's done**, verification steps, optional future work | -| [docs/GITEA.md](docs/GITEA.md) | **Gitea Actions** setup, `master` branch policy, troubleshooting | -| [CONTRIBUTING.md](CONTRIBUTING.md) | How to develop, add languages, submit PRs | - ---- +| Document | Contents | +|----------|----------| +| [docs/API.md](docs/API.md) | Function reference, types, JSON schemas, CLI | +| [docs/ERRORS.md](docs/ERRORS.md) | Error codes, per-language triggers | +| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Pipeline layers and detection order | +| [docs/STATUS.md](docs/STATUS.md) | Release status, verification, changelog | +| [docs/GITEA.md](docs/GITEA.md) | CI setup, default branch `master` | +| [CONTRIBUTING.md](CONTRIBUTING.md) | Development workflow, adding languages | ## License -MIT -- see [LICENSE](LICENSE) for details. - -## Author - -molodetz +MIT — see [LICENSE](LICENSE). \ No newline at end of file diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..484b5e3 --- /dev/null +++ b/docs/API.md @@ -0,0 +1,459 @@ +# API Reference + +Function signatures, data types, and JSON output schemas for the Nimcheck library. + +### Core Functions + +All functions are exported from `nimcheck.nim`. Import once with `import nimcheck`. + +#### Validation + +| Function | Signature | Returns | Description | +|----------|-----------|---------|-------------| +| `validateSource` | `(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")` | `ValidationResult` | Validate source code string. Auto-detects flavor if `lfUnknown`. | +| `validateFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())` | `ValidationResult` | Validate a file from disk. Auto-detects flavor from file extension. | +| `validateSourceJson` | `(source: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Validate and return pretty-printed JSON string. | +| `validateFileJson` | `(filePath: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Validate file and return pretty-printed JSON string. | + +#### Inspection + +| Function | Signature | Returns | Description | +|----------|-----------|---------|-------------| +| `inspectSource` | `(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")` | `JsonNode` | Validate and return structural diagnostics (functions, classes, imports). | +| `inspectFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())` | `JsonNode` | Inspect a file and return structural diagnostics. | + +#### Reporting + +| Function | Signature | Returns | Description | +|----------|-----------|---------|-------------| +| `reportSource` | `(source: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Human-readable validation report for source code. | +| `reportFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Human-readable validation report for a file. | + +#### Detection and Utility + +| Function | Signature | Returns | Description | +|----------|-----------|---------|-------------| +| `detectFlavor` | `(source: string, filePath: string = "")` | `LanguageFlavor` | Auto-detect language flavor (convenience wrapper). | +| `supportedFlavors` | `()` | `seq[string]` | List all registered language flavors sorted alphabetically. | +| `version` | `()` | `string` | Version string (`"0.1.0"`). | +| `enableDebug` | `()` | `void` | Enable debug mode globally. | +| `disableDebug` | `()` | `void` | Disable debug mode globally. | + +#### Core Detector (lower-level) + +| Function | Signature | Returns | Description | +|----------|-----------|---------|-------------| +| `detectFlavor` (core) | `(source: string, filePath: string = "", explicitFlavor: LanguageFlavor = lfUnknown)` | `(LanguageFlavor, FlavorDetectionMethod, float)` | Full detection with confidence score. | + +#### Factory + +| Function | Signature | Returns | Description | +|----------|-----------|---------|-------------| +| `createValidator` | `(flavor: LanguageFlavor, source: string, options: ValidationOptions, sourcePath: string)` | `ValidatorBase` | Factory: create the correct validator for a flavor. | +| `registerValidator` | `(flavor: string, constructor: ValidatorConstructor)` | `void` | Register a new validator constructor. | + +--- + +## Data Types + +### LanguageFlavor (enum, pure) + +All 28 flavors in `src/nimcheck/core/types.nim` have dedicated validators (v0.1.0). Registry aliases (e.g. `py`, `js`, `md`) map to the same validators — see `supportedFlavors()` or `bin/nimcheck help`. + +| Enum Value | String | Validator module | File Extensions (auto-detect) | +|------------|--------|------------------|-------------------------------| +| `lfUnknown` | `"unknown"` | -- | -- | +| **`lfNim`** | `"nim"` | `nim_validator.nim` | `.nim`, `.nims`, `.nimble` | +| **`lfPython`** | `"python"` | `python_validator.nim` | `.py`, `.pyw`, `.pyx`, `.pxd` | +| **`lfBash`** | `"bash"` | `bash_validator.nim` | `.sh`, `.bash` | +| **`lfShell`** | `"shell"` | `bash_validator.nim` (shared) | `.zsh`, `.fish` | +| **`lfJavaScript`** | `"javascript"` | `javascript_validator.nim` | `.js`, `.mjs`, `.cjs` | +| **`lfTypeScript`** | `"typescript"` | `type_xml_validator.nim` | `.ts`, `.tsx` | +| **`lfPHP`** | `"php"` | `php_validator.nim` | `.php`, `.phtml` | +| **`lfHTML`** | `"html"` | `html_validator.nim` | `.html`, `.htm`, `.xhtml` | +| **`lfXML`** | `"xml"` | `type_xml_validator.nim` | `.xml`, `.xsd`, `.xslt`, `.svg` | +| **`lfJinja`** | `"jinja"` | `jinja_validator.nim` | `.jinja`, `.jinja2`, `.j2` | +| **`lfJSON`** | `"json"` | `config_validators.nim` | `.json`, `.jsonc` | +| **`lfYAML`** | `"yaml"` | `config_validators.nim` | `.yaml`, `.yml` | +| **`lfTOML`** | `"toml"` | `config_validators.nim` | `.toml` | +| **`lfCSS`** | `"css"` | `extended_validators.nim` | `.css` | +| **`lfSQL`** | `"sql"` | `extended_validators.nim` | `.sql` | +| **`lfMarkdown`** | `"markdown"` | `extended_validators.nim` | `.md`, `.markdown` | +| **`lfDockerfile`** | `"dockerfile"` | `extended_validators.nim` | `Dockerfile` | +| **`lfMakefile`** | `"makefile"` | `extended_validators.nim` | `Makefile`, `makefile` | +| **`lfRuby`** | `"ruby"` | `extended_validators.nim` | `.rb` | +| **`lfRust`** | `"rust"` | `extended_validators.nim` | `.rs` | +| **`lfGo`** | `"go"` | `extended_validators.nim` | `.go` | +| **`lfLua`** | `"lua"` | `lang_validators.nim` | `.lua` | +| **`lfC`** | `"c"` | `cfamily_validators.nim` | `.c`, `.h` | +| **`lfCpp`** | `"cpp"` | `cfamily_validators.nim` | `.cpp`, `.cxx`, `.hpp` | +| **`lfCSharp`** | `"csharp"` | `cfamily_validators.nim` | `.cs` | +| **`lfJava`** | `"java"` | `cfamily_validators.nim` | `.java` | +| **`lfSwift`** | `"swift"` | `lang_validators.nim` | `.swift` | +| **`lfKotlin`** | `"kotlin"` | `lang_validators.nim` | `.kt`, `.kts` | + +### FlavorDetectionMethod (enum) + +| Value | Meaning | +|-------|---------| +| `fdmExplicit` | User explicitly provided the flavor | +| `fdmExtension` | Detected from file extension | +| `fdmShebang` | Detected from `#!` shebang line | +| `fdmContent` | Detected via content keyword analysis | +| `fdmUnknown` | Could not determine | + +### TokenKind (enum) + +Lexical token classifications used by all tokenizers. + +| Kind | Description | +|------|-------------| +| `tkWhitespace` | Spaces, tabs, carriage returns | +| `tkNewline` | `\n` | +| `tkComment` | Line (`#`, `//`) or block (`/* */`) comment | +| `tkDocComment` | Documentation comment (`##`, `///`, `"""`) | +| `tkString` | Single or double quoted string literal | +| `tkRawString` | Raw string (`r"..."`) | +| `tkMultilineString` | Triple-quoted string (`"""..."""`) | +| `tkNumber` | Integer, float, hex, octal, binary | +| `tkIdentifier` | Variable, function, type name | +| `tkKeyword` | Language-specific reserved word | +| `tkOperator` | `+`, `-`, `*`, `/`, `&&`, etc. | +| `tkAssignment` | `=`, `:=`, `+=`, etc. | +| `tkPunctuation` | `.`, `,`, `;`, `:` | +| `kOpenParen` | `(` | +| `kCloseParen` | `)` | +| `kOpenBracket` | `[` | +| `kCloseBracket` | `]` | +| `kOpenBrace` | `{` | +| `kCloseBrace` | `}` | +| `kAngleOpen` | `<` | +| `kAngleClose` | `>` | +| `tkInterpolation` | String interpolation (`${var}`, f-string `{expr}`) | +| `tkDirective` | Preprocessor / compiler directive (`#include`, shebang) | +| `tkTemplateTag` | Template language tag (`{{ }}`, `{% %}`) | +| `tkSpecial` | Language-specific special token | +| `tkError` | Malformed or unrecognizable character | +| `tkEndOfFile` | End of input marker | + +### Token + +``` +Token = object + kind: TokenKind + value: string + position: SourcePosition + range: SourceRange +``` + +### SourcePosition + +``` +SourcePosition = object + line: int # 1-indexed line number + column: int # 1-indexed column number + offset: int # 0-indexed byte offset from start +``` + +### SourceRange + +``` +SourceRange = object + startPos: SourcePosition + endPos: SourcePosition +``` + +### ErrorSeverity (enum) + +| Value | String | Description | +|-------|--------|-------------| +| `esInfo` | `"info"` | Informational note | +| `esWarning` | `"warning"` | Non-fatal issue | +| `esError` | `"error"` | Validation failure | +| `esCritical` | `"critical"` | Internal framework failure | + +### ValidationError + +``` +ValidationError = object + severity: ErrorSeverity + message: string + code: string # Error code, e.g. "E0002" + position: SourcePosition + range: SourceRange + context: string # Surrounding source context (optional) + hint: string # Fix suggestion (optional) +``` + +### ValidationResult + +``` +ValidationResult = object + valid: bool + flavor: LanguageFlavor + detectionMethod: FlavorDetectionMethod + errors: seq[ValidationError] + warnings: int + infos: int + moduleInfo: ModuleInfo + durationMs: float + debugOutput: string # Empty unless compiled with -d:nimcheckDebug +``` + +### ValidationOptions + +``` +ValidationOptions = object + flavor: LanguageFlavor # lfUnknown = auto-detect (default) + debugMode: bool # Enable per-validation debug logging (default: false) + strictMode: bool # Warnings become errors (default: false) + maxErrors: int # Stop after N errors; 0 = unlimited (default: 0) + includeContext: bool # Include surrounding source lines in error output (default: true) + includeHints: bool # Include fix suggestions (default: true) +``` + +### Structural Types (populated by `analyzeTokens()`) + +``` +VariableInfo = object + name: string + kind: string # "var", "let", "const", "global", "local" + typeAnnotation: string + defaultValue: string + position: SourcePosition + mutable: bool + exported: bool + +FunctionInfo = object + name: string + kind: string # "function", "method", "proc", "func", "lambda" + parameters: seq[ParameterInfo] + returnType: string + isAsync: bool + isExported: bool + isGenerator: bool + visibility: string # "public", "private", "protected" + position: SourcePosition + docComment: string + +ParameterInfo = object + name: string + typeAnnotation: string + defaultValue: string + isOptional: bool + isVarargs: bool + +ClassInfo = object + name: string + kind: string # "class", "object", "struct", "interface" + baseTypes: seq[string] + fields: seq[VariableInfo] + methods: seq[FunctionInfo] + generics: seq[string] + visibility: string + position: SourcePosition + docComment: string + +ImportInfo = object + module: string + symbols: seq[string] # Specific imported symbols + isRelative: bool + alias: string + position: SourcePosition + +ModuleInfo = object + flavor: LanguageFlavor + detectionMethod: FlavorDetectionMethod + imports: seq[ImportInfo] + variables: seq[VariableInfo] + functions: seq[FunctionInfo] + classes: seq[ClassInfo] + linesOfCode: int + totalTokens: int + comments: int + debugInfo: JsonNode # Language-specific extra info +``` + +--- + +## JSON Output Structures + +Every type has a `.toJson()` method. The `ValidationResult.toJson()` aggregates all nested structures. + +### ValidationResult JSON + +```json +{ + "valid": true, + "flavor": "python", + "detectionMethod": "by_extension", + "errors": [ + { + "severity": "error", + "message": "Unclosed string literal starting with '\"'", + "code": "E0002", + "position": {"line": 2, "column": 13, "offset": 42}, + "range": { + "start": {"line": 2, "column": 13, "offset": 42}, + "end": {"line": 2, "column": 13, "offset": 42} + }, + "context": "", + "hint": "Add a closing quote before end of file" + } + ], + "warnings": 0, + "infos": 0, + "moduleInfo": { + "flavor": "python", + "detectionMethod": "by_extension", + "imports": [], + "variables": [], + "functions": [ + { + "name": "hello", + "kind": "def", + "parameters": [], + "returnType": "", + "isAsync": false, + "isExported": false, + "isGenerator": false, + "visibility": "", + "position": {"line": 1, "column": 5, "offset": 4}, + "docComment": "" + } + ], + "classes": [], + "linesOfCode": 3, + "totalTokens": 12, + "comments": 0, + "debugInfo": null + }, + "durationMs": 0.342, + "debugOutput": "" +} +``` + +### ValidationError JSON + +```json +{ + "severity": "error", + "message": "Unclosed string literal starting with '\"'", + "code": "E0002", + "position": {"line": 2, "column": 13, "offset": 42}, + "range": { + "start": {"line": 2, "column": 13, "offset": 42}, + "end": {"line": 2, "column": 13, "offset": 42} + }, + "context": "", + "hint": "Add a closing quote before end of file" +} +``` + +### ModuleInfo JSON + +```json +{ + "flavor": "nim", + "detectionMethod": "by_extension", + "imports": [ + { + "module": "strutils", + "symbols": [], + "isRelative": false, + "alias": "", + "position": {"line": 1, "column": 8, "offset": 7} + } + ], + "variables": [], + "functions": [ + { + "name": "add", + "kind": "proc", + "parameters": [], + "returnType": "", + "isAsync": false, + "isExported": false, + "isGenerator": false, + "visibility": "", + "position": {"line": 3, "column": 6, "offset": 28}, + "docComment": "" + } + ], + "classes": [ + { + "name": "MyType", + "kind": "type", + "baseTypes": [], + "fields": [], + "methods": [], + "generics": [], + "visibility": "", + "position": {"line": 5, "column": 6, "offset": 52}, + "docComment": "" + } + ], + "linesOfCode": 8, + "totalTokens": 45, + "comments": 2, + "debugInfo": null +} +``` + +### Human-Readable Report Format + +``` +=== Validation Report === + Flavor: python + Detection: by_extension + Valid: false + Errors: 1 + Warnings: 0 + Duration: 0.34ms + Lines of code: 3 + Tokens: 12 + + --- Findings --- + [E0002] (error) L2:13: Unclosed string literal starting with '"' + | x = " + | ^ + hint: Add a closing quote before end of file + + --- Summary --- + INVALID - source has issues that need attention. +``` + +### Error Formatter Output (single error) + +``` + [E0002] (error) L2:13: Unclosed string literal starting with '"' + | x = " + | ^ + hint: Add a closing quote before end of file +``` + +--- + +## CLI Usage + +``` +Nimcheck v0.1.0 - Universal Source Code Validation Framework + +Usage: + nimcheck validate [--file= | --source=] [--flavor=] + nimcheck inspect [--file= | --source=] [--flavor=] + nimcheck detect [--file= | --source=] + nimcheck help + +Options: + --file= Source file to process + --source= Source code string to process + --flavor= Language flavor (auto-detect if omitted) + --debug Enable debug mode + --json Output as JSON (default for inspect) +``` + +Run `bin/nimcheck help` for the current registered flavor list. + +The `validate` command exits 0 at the process level; inspect `valid` in the +output or JSON for pass/fail. Validation paths do not raise to the caller. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..0a8b245 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,75 @@ +# Architecture + +Nimcheck uses a four-layer pipeline architecture. Each layer is independent and communicates through well-defined interfaces. + +``` +Source Code (string or file) + | + v ++-------------------+ +| Detector | LanguageFlavor from extension, shebang, or content analysis ++-------------------+ + | + v ++-------------------+ +| Tokenizer | Converts raw source into a seq[Token] stream ++-------------------+ + | + v ++-------------------+ +| Validator | Analyzes token stream for syntax errors and structure ++-------------------+ + | + v ++-------------------+ +| Reporting | Formats errors as human-readable text or structured JSON ++-------------------+ + | + v + ValidationResult +``` + +### Layer 1: Detector (`src/nimcheck/core/detector.nim`) + +Language auto-detection with cascading priority: + +| Priority | Method | Source | Confidence | +|----------|--------|--------|------------| +| 1 (highest) | Explicit | User-provided `LanguageFlavor` | 1.0 | +| 2 | File extension | 70+ extension-to-flavor mappings | 0.9 | +| 3 | Shebang line | `#!` line interpreter matching (bash, python, node, ruby, lua, swift) | 0.95 | +| 4 | Content analysis | Keyword scoring, structural patterns, bracket usage | 0.6-0.9 | + +The `detectFlavor()` function returns a `(LanguageFlavor, FlavorDetectionMethod, confidence: float)` tuple. + +### Layer 2: Tokenizer (`src/nimcheck/core/tokenizerbase.nim`, per-language impls) + +Each tokenizer converts raw source into a `seq[Token]`. Tokenizers track: + +- **Position**: line, column, and byte offset for every token +- **Bracket stack**: `(`, `[`, `{`, `<` pushed and popped with position tracking +- **String context**: escape sequences, multi-line delimiters, interpolation +- **Comment context**: line comments, block comments with nesting support + +Tokenizers emit errors for: +- Unclosed strings (`ErrUnclosedString`) +- Unclosed comments (`ErrUnclosedComment`) +- Unexpected characters (`ErrUnexpectedToken`) +- Unclosed block constructs (language-specific, e.g., `${}`, `$()`) + +### Layer 3: Validator (`src/nimcheck/core/validatorbase.nim`, per-language impls) + +Each validator: + +1. Creates its tokenizer and runs `tokenize()` +2. Collects tokenizer errors +3. Runs `analyzeTokens()` - language-specific structural analysis +4. Runs `buildModuleInfo()` - populates `ModuleInfo` with imports, functions, classes +5. Computes validity, warning/info counts, duration + +Validators register themselves via a factory pattern at import time. The registry maps flavor strings (e.g., `"python"`, `"bash"`) to constructors. + +### Layer 4: Reporting (`src/nimcheck/reporting/errors.nim`) + +Produces: +- **Human-readable reports**: `formatReport()` with position, severity, code, message, source context with carets, and hints diff --git a/docs/ERRORS.md b/docs/ERRORS.md new file mode 100644 index 0000000..86efb03 --- /dev/null +++ b/docs/ERRORS.md @@ -0,0 +1,267 @@ +# Error Codes + +## Error Code Reference + +Error codes follow a prefixed numbering scheme. The prefix encodes the category and severity. + +### General Errors (E0xxx) + +| Code | Constant | Description | Triggered By | +|------|----------|-------------|-------------| +| `E0001` | `ErrUnexpectedToken` | Unexpected or unrecognizable character | Tokenizer encounter of non-ASCII, non-operator char | +| `E0002` | `ErrUnclosedString` | Unclosed string literal | Missing closing quote at EOF | +| `E0003` | `ErrUnclosedComment` | Unclosed block comment | Missing `*/`, `]#`, `#}` etc. at EOF | +| `E0004` | `ErrUnclosedBlock` | Unclosed block/control structure | Language-specific: unclosed `if`/`for`/`class`/etc. | +| `E0005` | `ErrMismatchedBracket` | Mismatched or unexpected closing bracket | `)` without `(`, `
` without `
` | +| `E0006` | `ErrUnclosedBracket` | Unclosed bracket at EOF | `validateBrackets()` finds remaining items on bracket stack | +| `E0007` | `ErrInvalidSyntax` | General invalid syntax | JSON parse error, TOML missing `=`, YAML empty key | +| `E0008` | `ErrUnexpectedEOF` | Unexpected end of file | Reserved for incomplete constructs not covered by other codes | +| `E0009` | `ErrEmptySource` | Source is empty (zero-length string) | `validate()` with `source.len == 0` | +| `E0010` | `ErrEncodingError` | Encoding error | Reserved for non-UTF-8 input | +| `E0011` | -- | File not found | `validateFile()` with non-existent path | + +### Language-Specific Errors (E1xxx) + +| Code | Constant | Description | Languages | +|------|----------|-------------|-----------| +| `E1001` | `ErrUnexpectedIndent` | Unexpected or inconsistent indentation | Python | +| `E1002` | `ErrMissingIndent` | Missing indentation after block opener | Python | +| `E1003` | `ErrDuplicateParam` | Duplicate parameter name | (reserved) | +| `E1004` | `ErrUndefinedVariable` | Reference to undefined variable | (reserved) | +| `E1005` | `ErrTypeMismatch` | Type mismatch | (reserved) | +| `E1006` | `ErrInvalidAssignment` | Invalid assignment target | (reserved) | +| `E1007` | `ErrMissingReturn` | Function missing return statement | (reserved) | +| `E1008` | `ErrInvalidDecorator` | Invalid decorator usage | Python | +| `E1009` | `ErrInvalidImport` | Invalid import statement | (reserved) | + +### Template Errors (E2xxx) + +| Code | Constant | Description | Languages | +|------|----------|-------------|-----------| +| `E2001` | `ErrUnclosedTemplateTag` | Unclosed template expression/block tag | Jinja | +| `E2002` | `ErrUnclosedBlockTag` | Unclosed block tag with no matching end tag | Jinja | +| `E2003` | `ErrInvalidFilter` | Invalid filter in template expression | Jinja | + +### Info Codes (I0xxx) + +| Code | Constant | Description | +|------|----------|-------------| +| `I0001` | `InfoLongLine` | Line exceeds recommended length | +| `I0002` | `InfoTrailingWhitespace` | Trailing whitespace detected | + +### Warning Codes (W0xxx / W1xxx) + +| Code | Constant | Description | +|------|----------|-------------| +| `W0001` | `WarnUnusedVariable` | Variable declared but never used | +| `W0002` | `WarnInconsistentNaming` | Naming convention inconsistency | +| `W0003` | `WarnMissingDoc` | Public symbol missing documentation | +| `W1001` | -- | Tab characters in YAML indentation | + +### Internal Error + +| Code | Description | Triggered By | +|------|-------------|-------------| +| `E9999` | Internal framework error (never-crash fallback) | Any unhandled `Exception` in `validate()` or `validateSource()` | + +--- + +## IO Table: Supported Languages and Per-Language Errors + +### Input/Output Contract + +Every validation function follows the same contract: + +| Input | Output | +|-------|--------| +| `source: string` (required) | `ValidationResult` with `valid: bool` | +| `flavor: LanguageFlavor = lfUnknown` | `ValidationResult.errors: seq[ValidationError]` | +| `options: ValidationOptions` (optional) | `ValidationResult.moduleInfo: ModuleInfo` | +| `sourcePath: string = ""` (optional) | `ValidationResult.durationMs: float` | + +**Guarantees**: +- Never throws - all exceptions are caught and returned as `E9999` errors +- Empty source returns `E0009` (or `valid: true` for YAML/TOML where empty is valid) +- File not found returns `E0011` +- Unknown flavor with no detection possible returns an `E0007` error +- Every error has a `code`, `message`, `severity`, `position`, and optional `hint` + +### Per-Language Tokenizer, Validator, and Error Table + +#### Nim (`.nim`, `.nims`, `.nimble`) + +| Aspect | Details | +|--------|---------| +| Tokenizer | `nim_tokenizer.nim` - `NimTokenizer` | +| Validator | `nim_validator.nim` - `NimValidator` | +| Keywords | 80+ (addr, and, as, block, break, case, const, converter, defer, discard, elif, else, enum, except, export, finally, for, from, func, if, import, in, include, iterator, let, macro, method, mixin, mod, nil, not, object, of, or, proc, ptr, raise, ref, return, template, try, tuple, type, using, var, when, while, with, yield, etc.) | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0001` | Unexpected character (non-ASCII, unrecognizable) | `esWarning` | Shows hex `U+XXXX` value | +| `E0002` | Unclosed string (`"..."` at EOF) | `esError` | "Add a closing quote" | +| `E0002` | Unclosed multi-line string (`"""` at EOF) | `esError` | -- | +| `E0002` | Newline inside string literal | `esError` | -- | +| `E0002` | Unclosed raw string (`r"..."` at EOF) | `esError` | -- | +| `E0002` | Unclosed character literal (`'x` at EOF) | `esError` | -- | +| `E0003` | Unclosed block comment (`#[` without `]#`) | `esError` | "Add closing `]#`" | +| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | "Add matching closing bracket" | + +Structural extraction: imports (`from X import Y`), functions (`proc`, `func`, `method`, `template`, `macro`, `iterator`, `converter`), types (`type Foo`). + +#### Bash / Shell (`.sh`, `.bash`, `.zsh`, `.fish`) + +| Aspect | Details | +|--------|---------| +| Tokenizer | `bash_tokenizer.nim` - `BashTokenizer` | +| Validator | `bash_validator.nim` - `BashValidator` | +| Keywords | 38+ (if, then, else, elif, fi, case, esac, for, while, until, do, done, in, function, select, time, return, break, continue, exit, declare, local, export, readonly, unset, eval, exec, let, source, shift, trap, type, typeset, ulimit, umask, alias, read, echo) | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0002` | Unclosed single-quoted string | `esError` | -- | +| `E0002` | Unclosed double-quoted string | `esError` | -- | +| `E0004` | Unclosed `${var}` expansion | `esError` | -- | +| `E0004` | Unclosed `$()` command substitution | `esError` | -- | +| `E0004` | Unclosed `if` block (`if` without matching `fi`) | `esError` | "Add missing 'fi'" | +| `E0004` | Unclosed loop (`for`/`while`/`until` without `done`) | `esError` | "Add missing 'done'" | +| `E0004` | Unclosed `case` block (`case` without `esac`) | `esError` | "Add missing 'esac'" | +| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- | + +Special features: shebang detection (`#!`), heredoc detection (`<`), ESM import/export. + +Structural extraction: imports/exports, functions (`function`, `async`), classes. + +#### PHP (`.php`, `.phtml`) + +| Aspect | Details | +|--------|---------| +| Tokenizer | `php_tokenizer.nim` - `PHPTokenizer` | +| Validator | `php_validator.nim` - `PHPValidator` | +| Mode switching | Tracks `inPhpMode` flag - toggles on `` | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0002` | Unclosed string | `esError` | -- | +| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- | + +Special features: PHP/HTML mode switching, `` detection, variable names (`$var`, `$obj->prop`). + +Structural extraction: functions (`function`, `fn`), classes (`class`), imports/requires (`include`, `require`, `use`, `namespace`). + +#### HTML / XML (`.html`, `.htm`, `.xhtml`, `.xml`, `.svg`) + +| Aspect | Details | +|--------|---------| +| Tokenizer | `html_tokenizer.nim` - `HTMLTokenizer` | +| Validator | `html_validator.nim` - `HTMLValidator` | +| Void elements | 14: area, base, br, col, embed, hr, img, input, link, meta, param, source, track, wbr | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0003` | Unclosed HTML comment (``) | `esError` | -- | +| `E0004` | Unclosed HTML tag (e.g., `
` without `
`) | `esError` | "Add closing ``" | +| `E0005` | Unexpected closing tag (mismatched nesting) | `esWarning` | "Check that tags are properly nested" | + +Special features: tag stack balancing, void element awareness (no closing tag needed for self-closing elements), attribute parsing with quoted values, ``, `` directives. + +#### Jinja2 (`.jinja`, `.jinja2`, `.j2`, `.html.j2`) + +| Aspect | Details | +|--------|---------| +| Tokenizer | `jinja_tokenizer.nim` - `JinjaTokenizer` | +| Validator | `jinja_validator.nim` - `JinjaValidator` | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0003` | Unclosed Jinja comment (`{#` without `#}`) | `esError` | -- | +| `E2001` | Unclosed expression tag (`{{` without `}}`) | Implicit via tokenizer | -- | +| `E2001` | Unclosed block tag (`{%` without `%}`) | Implicit via tokenizer | -- | +| `E2002` | Unclosed `for` block (no matching `endfor`) | `esError` | "Add `{% endfor %}`" | +| `E2002` | Unclosed `if` block (no matching `endif`) | `esError` | "Add `{% endif %}`" | +| `E2002` | Unclosed `block` (no matching `endblock`) | `esError` | "Add `{% endblock %}`" | +| `E2002` | Unclosed `macro`, `filter`, `raw`, `autoescape`, `call`, `with` | `esError` | "Add matching `end{tag}`" | + +Special features: tracks 12 block tag types (`for`, `if`, `block`, `macro`, `filter`, `raw`, `autoescape`, `call`, `with`), validates matching end tags, handles content between template syntax and plain text. + +#### JSON (`.json`, `.jsonc`) + +| Aspect | Details | +|--------|---------| +| Validator | `config_validators.nim` - `JSONValidator` | +| Method | Uses Nim's built-in `parseJson()` for full validation | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0007` | JSON parse error (any `JsonParsingError`) | `esError` | Error message from parser | +| `E0007` | Any other exception during parse | `esError` | "JSON error: {message}" | +| `E0009` | Empty JSON source | `esError` | -- | + +#### YAML (`.yaml`, `.yml`) + +| Aspect | Details | +|--------|---------| +| Validator | `config_validators.nim` - `YAMLValidator` | +| Method | Line-by-line structural analysis | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0007` | Empty mapping key (no key before `:`) | `esError` | -- | +| `E0007` | General YAML structural error | `esError` | -- | +| `E0009` | No content after stripping comments/blanks | `esError` | -- | +| `W1001` | Tab characters used for indentation | `esWarning` | "Use spaces for YAML indentation" | + +Note: Empty YAML (only whitespace/comments) is `valid: true`. + +#### TOML (`.toml`) + +| Aspect | Details | +|--------|---------| +| Validator | `config_validators.nim` - `TOMLValidator` | +| Method | Line-by-line structural analysis | + +| Error Code | Condition | Severity | Hint | +|------------|-----------|----------|------| +| `E0007` | Missing `=` in key-value pair | `esError` | "TOML uses `key = value` format" | +| `E0007` | Missing key before `=` | `esError` | -- | +| `E0007` | General TOML structural error | `esError` | -- | +| `E0009` | No content after stripping comments/blanks | `esError` | -- | + diff --git a/docs/STATUS.md b/docs/STATUS.md index 9eb7f93..10c6a21 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -106,7 +106,7 @@ bin/nimcheck detect --file=unknown.txt bin/nimcheck inspect --file=main.go --json ``` -See [README.md](../README.md) for full API and JSON schemas. +See [docs/API.md](API.md) for API and JSON schemas; [README.md](../README.md) for overview. --- @@ -153,4 +153,15 @@ These are **not blockers** for v0.1.0: | Lint | `make lint` | | Clean | `make clean` | | CI locally | Same as lint + test | -| Push | `git push origin master` | \ No newline at end of file +| Push | `git push origin master` | + +## Documentation layout + +| File | Purpose | +|------|---------| +| `README.md` | Product overview (concise) | +| `docs/API.md` | API reference, types, JSON, CLI | +| `docs/ERRORS.md` | Error codes, per-language table | +| `docs/ARCHITECTURE.md` | Pipeline design | +| `docs/GITEA.md` | CI setup | +| `CONTRIBUTING.md` | Dev workflow | \ No newline at end of file