docs: align README with project documentation rules
Shrink README to a concise product overview (capabilities, quick start, build, test, structure, doc links). Move API reference, types, JSON schemas, and CLI detail to docs/API.md; move error catalogue to docs/ERRORS.md; move pipeline layers to docs/ARCHITECTURE.md. Add CLAUDE.md with README policy and sync task.
This commit is contained in:
parent
7d6e7d925e
commit
96a5dc834a
43
CLAUDE.md
Normal file
43
CLAUDE.md
Normal file
@ -0,0 +1,43 @@
|
||||
# Nimcheck — project notes
|
||||
|
||||
## Standing task
|
||||
|
||||
Keep `README.md`, this file, and the test suite aligned with the current
|
||||
implementation. Update all that apply together when any one changes.
|
||||
|
||||
## README policy
|
||||
|
||||
`README.md` is product-facing: concise overview, capabilities, quick start, build,
|
||||
test, structure, documentation links. No badges, no contribution appeals, no
|
||||
embedded API reference (that lives under `docs/`).
|
||||
|
||||
## Configuration
|
||||
|
||||
No `.env.json` or environment-based configuration. Nimcheck runs with defaults
|
||||
after `nimble build`; no setup step required.
|
||||
|
||||
## Testing
|
||||
|
||||
- Framework: Nim `unittest` via `tests/test_all.nim`
|
||||
- Run: `make test` or `nimble test`
|
||||
- Single language: `nim c -r tests/test_<lang>.nim`
|
||||
- Lint: `make lint`
|
||||
- Coverage: `make coverage` (optional; requires nimcoverage)
|
||||
|
||||
## Repository
|
||||
|
||||
- Default branch: `master`
|
||||
- CI: `.gitea/workflows/ci.yml` — see `docs/GITEA.md`
|
||||
- Remote: `https://retoor.molodetz.nl/retoor/nimcheck.git`
|
||||
|
||||
## Documentation map
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `README.md` | Overview and quick start |
|
||||
| `docs/API.md` | API, types, JSON, CLI |
|
||||
| `docs/ERRORS.md` | Error codes and per-language triggers |
|
||||
| `docs/ARCHITECTURE.md` | Pipeline layers |
|
||||
| `docs/STATUS.md` | Release status |
|
||||
| `docs/GITEA.md` | CI and branch policy |
|
||||
| `CONTRIBUTING.md` | Development workflow |
|
||||
@ -1,6 +1,6 @@
|
||||
# Contributing to Nimcheck
|
||||
|
||||
Thanks for helping improve Nimcheck. This project is intentionally small: pure Nim stdlib, tokenizer-based validation, no external parser deps.
|
||||
Pure Nim stdlib, tokenizer-based validation, no external parser dependencies.
|
||||
|
||||
---
|
||||
|
||||
|
||||
459
docs/API.md
Normal file
459
docs/API.md
Normal file
@ -0,0 +1,459 @@
|
||||
# API Reference
|
||||
|
||||
Function signatures, data types, and JSON output schemas for the Nimcheck library.
|
||||
|
||||
### Core Functions
|
||||
|
||||
All functions are exported from `nimcheck.nim`. Import once with `import nimcheck`.
|
||||
|
||||
#### Validation
|
||||
|
||||
| Function | Signature | Returns | Description |
|
||||
|----------|-----------|---------|-------------|
|
||||
| `validateSource` | `(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")` | `ValidationResult` | Validate source code string. Auto-detects flavor if `lfUnknown`. |
|
||||
| `validateFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())` | `ValidationResult` | Validate a file from disk. Auto-detects flavor from file extension. |
|
||||
| `validateSourceJson` | `(source: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Validate and return pretty-printed JSON string. |
|
||||
| `validateFileJson` | `(filePath: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Validate file and return pretty-printed JSON string. |
|
||||
|
||||
#### Inspection
|
||||
|
||||
| Function | Signature | Returns | Description |
|
||||
|----------|-----------|---------|-------------|
|
||||
| `inspectSource` | `(source: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions(), sourcePath: string = "")` | `JsonNode` | Validate and return structural diagnostics (functions, classes, imports). |
|
||||
| `inspectFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown, options: ValidationOptions = newValidationOptions())` | `JsonNode` | Inspect a file and return structural diagnostics. |
|
||||
|
||||
#### Reporting
|
||||
|
||||
| Function | Signature | Returns | Description |
|
||||
|----------|-----------|---------|-------------|
|
||||
| `reportSource` | `(source: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Human-readable validation report for source code. |
|
||||
| `reportFile` | `(filePath: string, flavor: LanguageFlavor = lfUnknown)` | `string` | Human-readable validation report for a file. |
|
||||
|
||||
#### Detection and Utility
|
||||
|
||||
| Function | Signature | Returns | Description |
|
||||
|----------|-----------|---------|-------------|
|
||||
| `detectFlavor` | `(source: string, filePath: string = "")` | `LanguageFlavor` | Auto-detect language flavor (convenience wrapper). |
|
||||
| `supportedFlavors` | `()` | `seq[string]` | List all registered language flavors sorted alphabetically. |
|
||||
| `version` | `()` | `string` | Version string (`"0.1.0"`). |
|
||||
| `enableDebug` | `()` | `void` | Enable debug mode globally. |
|
||||
| `disableDebug` | `()` | `void` | Disable debug mode globally. |
|
||||
|
||||
#### Core Detector (lower-level)
|
||||
|
||||
| Function | Signature | Returns | Description |
|
||||
|----------|-----------|---------|-------------|
|
||||
| `detectFlavor` (core) | `(source: string, filePath: string = "", explicitFlavor: LanguageFlavor = lfUnknown)` | `(LanguageFlavor, FlavorDetectionMethod, float)` | Full detection with confidence score. |
|
||||
|
||||
#### Factory
|
||||
|
||||
| Function | Signature | Returns | Description |
|
||||
|----------|-----------|---------|-------------|
|
||||
| `createValidator` | `(flavor: LanguageFlavor, source: string, options: ValidationOptions, sourcePath: string)` | `ValidatorBase` | Factory: create the correct validator for a flavor. |
|
||||
| `registerValidator` | `(flavor: string, constructor: ValidatorConstructor)` | `void` | Register a new validator constructor. |
|
||||
|
||||
---
|
||||
|
||||
## Data Types
|
||||
|
||||
### LanguageFlavor (enum, pure)
|
||||
|
||||
All 28 flavors in `src/nimcheck/core/types.nim` have dedicated validators (v0.1.0). Registry aliases (e.g. `py`, `js`, `md`) map to the same validators — see `supportedFlavors()` or `bin/nimcheck help`.
|
||||
|
||||
| Enum Value | String | Validator module | File Extensions (auto-detect) |
|
||||
|------------|--------|------------------|-------------------------------|
|
||||
| `lfUnknown` | `"unknown"` | -- | -- |
|
||||
| **`lfNim`** | `"nim"` | `nim_validator.nim` | `.nim`, `.nims`, `.nimble` |
|
||||
| **`lfPython`** | `"python"` | `python_validator.nim` | `.py`, `.pyw`, `.pyx`, `.pxd` |
|
||||
| **`lfBash`** | `"bash"` | `bash_validator.nim` | `.sh`, `.bash` |
|
||||
| **`lfShell`** | `"shell"` | `bash_validator.nim` (shared) | `.zsh`, `.fish` |
|
||||
| **`lfJavaScript`** | `"javascript"` | `javascript_validator.nim` | `.js`, `.mjs`, `.cjs` |
|
||||
| **`lfTypeScript`** | `"typescript"` | `type_xml_validator.nim` | `.ts`, `.tsx` |
|
||||
| **`lfPHP`** | `"php"` | `php_validator.nim` | `.php`, `.phtml` |
|
||||
| **`lfHTML`** | `"html"` | `html_validator.nim` | `.html`, `.htm`, `.xhtml` |
|
||||
| **`lfXML`** | `"xml"` | `type_xml_validator.nim` | `.xml`, `.xsd`, `.xslt`, `.svg` |
|
||||
| **`lfJinja`** | `"jinja"` | `jinja_validator.nim` | `.jinja`, `.jinja2`, `.j2` |
|
||||
| **`lfJSON`** | `"json"` | `config_validators.nim` | `.json`, `.jsonc` |
|
||||
| **`lfYAML`** | `"yaml"` | `config_validators.nim` | `.yaml`, `.yml` |
|
||||
| **`lfTOML`** | `"toml"` | `config_validators.nim` | `.toml` |
|
||||
| **`lfCSS`** | `"css"` | `extended_validators.nim` | `.css` |
|
||||
| **`lfSQL`** | `"sql"` | `extended_validators.nim` | `.sql` |
|
||||
| **`lfMarkdown`** | `"markdown"` | `extended_validators.nim` | `.md`, `.markdown` |
|
||||
| **`lfDockerfile`** | `"dockerfile"` | `extended_validators.nim` | `Dockerfile` |
|
||||
| **`lfMakefile`** | `"makefile"` | `extended_validators.nim` | `Makefile`, `makefile` |
|
||||
| **`lfRuby`** | `"ruby"` | `extended_validators.nim` | `.rb` |
|
||||
| **`lfRust`** | `"rust"` | `extended_validators.nim` | `.rs` |
|
||||
| **`lfGo`** | `"go"` | `extended_validators.nim` | `.go` |
|
||||
| **`lfLua`** | `"lua"` | `lang_validators.nim` | `.lua` |
|
||||
| **`lfC`** | `"c"` | `cfamily_validators.nim` | `.c`, `.h` |
|
||||
| **`lfCpp`** | `"cpp"` | `cfamily_validators.nim` | `.cpp`, `.cxx`, `.hpp` |
|
||||
| **`lfCSharp`** | `"csharp"` | `cfamily_validators.nim` | `.cs` |
|
||||
| **`lfJava`** | `"java"` | `cfamily_validators.nim` | `.java` |
|
||||
| **`lfSwift`** | `"swift"` | `lang_validators.nim` | `.swift` |
|
||||
| **`lfKotlin`** | `"kotlin"` | `lang_validators.nim` | `.kt`, `.kts` |
|
||||
|
||||
### FlavorDetectionMethod (enum)
|
||||
|
||||
| Value | Meaning |
|
||||
|-------|---------|
|
||||
| `fdmExplicit` | User explicitly provided the flavor |
|
||||
| `fdmExtension` | Detected from file extension |
|
||||
| `fdmShebang` | Detected from `#!` shebang line |
|
||||
| `fdmContent` | Detected via content keyword analysis |
|
||||
| `fdmUnknown` | Could not determine |
|
||||
|
||||
### TokenKind (enum)
|
||||
|
||||
Lexical token classifications used by all tokenizers.
|
||||
|
||||
| Kind | Description |
|
||||
|------|-------------|
|
||||
| `tkWhitespace` | Spaces, tabs, carriage returns |
|
||||
| `tkNewline` | `\n` |
|
||||
| `tkComment` | Line (`#`, `//`) or block (`/* */`) comment |
|
||||
| `tkDocComment` | Documentation comment (`##`, `///`, `"""`) |
|
||||
| `tkString` | Single or double quoted string literal |
|
||||
| `tkRawString` | Raw string (`r"..."`) |
|
||||
| `tkMultilineString` | Triple-quoted string (`"""..."""`) |
|
||||
| `tkNumber` | Integer, float, hex, octal, binary |
|
||||
| `tkIdentifier` | Variable, function, type name |
|
||||
| `tkKeyword` | Language-specific reserved word |
|
||||
| `tkOperator` | `+`, `-`, `*`, `/`, `&&`, etc. |
|
||||
| `tkAssignment` | `=`, `:=`, `+=`, etc. |
|
||||
| `tkPunctuation` | `.`, `,`, `;`, `:` |
|
||||
| `kOpenParen` | `(` |
|
||||
| `kCloseParen` | `)` |
|
||||
| `kOpenBracket` | `[` |
|
||||
| `kCloseBracket` | `]` |
|
||||
| `kOpenBrace` | `{` |
|
||||
| `kCloseBrace` | `}` |
|
||||
| `kAngleOpen` | `<` |
|
||||
| `kAngleClose` | `>` |
|
||||
| `tkInterpolation` | String interpolation (`${var}`, f-string `{expr}`) |
|
||||
| `tkDirective` | Preprocessor / compiler directive (`#include`, shebang) |
|
||||
| `tkTemplateTag` | Template language tag (`{{ }}`, `{% %}`) |
|
||||
| `tkSpecial` | Language-specific special token |
|
||||
| `tkError` | Malformed or unrecognizable character |
|
||||
| `tkEndOfFile` | End of input marker |
|
||||
|
||||
### Token
|
||||
|
||||
```
|
||||
Token = object
|
||||
kind: TokenKind
|
||||
value: string
|
||||
position: SourcePosition
|
||||
range: SourceRange
|
||||
```
|
||||
|
||||
### SourcePosition
|
||||
|
||||
```
|
||||
SourcePosition = object
|
||||
line: int # 1-indexed line number
|
||||
column: int # 1-indexed column number
|
||||
offset: int # 0-indexed byte offset from start
|
||||
```
|
||||
|
||||
### SourceRange
|
||||
|
||||
```
|
||||
SourceRange = object
|
||||
startPos: SourcePosition
|
||||
endPos: SourcePosition
|
||||
```
|
||||
|
||||
### ErrorSeverity (enum)
|
||||
|
||||
| Value | String | Description |
|
||||
|-------|--------|-------------|
|
||||
| `esInfo` | `"info"` | Informational note |
|
||||
| `esWarning` | `"warning"` | Non-fatal issue |
|
||||
| `esError` | `"error"` | Validation failure |
|
||||
| `esCritical` | `"critical"` | Internal framework failure |
|
||||
|
||||
### ValidationError
|
||||
|
||||
```
|
||||
ValidationError = object
|
||||
severity: ErrorSeverity
|
||||
message: string
|
||||
code: string # Error code, e.g. "E0002"
|
||||
position: SourcePosition
|
||||
range: SourceRange
|
||||
context: string # Surrounding source context (optional)
|
||||
hint: string # Fix suggestion (optional)
|
||||
```
|
||||
|
||||
### ValidationResult
|
||||
|
||||
```
|
||||
ValidationResult = object
|
||||
valid: bool
|
||||
flavor: LanguageFlavor
|
||||
detectionMethod: FlavorDetectionMethod
|
||||
errors: seq[ValidationError]
|
||||
warnings: int
|
||||
infos: int
|
||||
moduleInfo: ModuleInfo
|
||||
durationMs: float
|
||||
debugOutput: string # Empty unless compiled with -d:nimcheckDebug
|
||||
```
|
||||
|
||||
### ValidationOptions
|
||||
|
||||
```
|
||||
ValidationOptions = object
|
||||
flavor: LanguageFlavor # lfUnknown = auto-detect (default)
|
||||
debugMode: bool # Enable per-validation debug logging (default: false)
|
||||
strictMode: bool # Warnings become errors (default: false)
|
||||
maxErrors: int # Stop after N errors; 0 = unlimited (default: 0)
|
||||
includeContext: bool # Include surrounding source lines in error output (default: true)
|
||||
includeHints: bool # Include fix suggestions (default: true)
|
||||
```
|
||||
|
||||
### Structural Types (populated by `analyzeTokens()`)
|
||||
|
||||
```
|
||||
VariableInfo = object
|
||||
name: string
|
||||
kind: string # "var", "let", "const", "global", "local"
|
||||
typeAnnotation: string
|
||||
defaultValue: string
|
||||
position: SourcePosition
|
||||
mutable: bool
|
||||
exported: bool
|
||||
|
||||
FunctionInfo = object
|
||||
name: string
|
||||
kind: string # "function", "method", "proc", "func", "lambda"
|
||||
parameters: seq[ParameterInfo]
|
||||
returnType: string
|
||||
isAsync: bool
|
||||
isExported: bool
|
||||
isGenerator: bool
|
||||
visibility: string # "public", "private", "protected"
|
||||
position: SourcePosition
|
||||
docComment: string
|
||||
|
||||
ParameterInfo = object
|
||||
name: string
|
||||
typeAnnotation: string
|
||||
defaultValue: string
|
||||
isOptional: bool
|
||||
isVarargs: bool
|
||||
|
||||
ClassInfo = object
|
||||
name: string
|
||||
kind: string # "class", "object", "struct", "interface"
|
||||
baseTypes: seq[string]
|
||||
fields: seq[VariableInfo]
|
||||
methods: seq[FunctionInfo]
|
||||
generics: seq[string]
|
||||
visibility: string
|
||||
position: SourcePosition
|
||||
docComment: string
|
||||
|
||||
ImportInfo = object
|
||||
module: string
|
||||
symbols: seq[string] # Specific imported symbols
|
||||
isRelative: bool
|
||||
alias: string
|
||||
position: SourcePosition
|
||||
|
||||
ModuleInfo = object
|
||||
flavor: LanguageFlavor
|
||||
detectionMethod: FlavorDetectionMethod
|
||||
imports: seq[ImportInfo]
|
||||
variables: seq[VariableInfo]
|
||||
functions: seq[FunctionInfo]
|
||||
classes: seq[ClassInfo]
|
||||
linesOfCode: int
|
||||
totalTokens: int
|
||||
comments: int
|
||||
debugInfo: JsonNode # Language-specific extra info
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JSON Output Structures
|
||||
|
||||
Every type has a `.toJson()` method. The `ValidationResult.toJson()` aggregates all nested structures.
|
||||
|
||||
### ValidationResult JSON
|
||||
|
||||
```json
|
||||
{
|
||||
"valid": true,
|
||||
"flavor": "python",
|
||||
"detectionMethod": "by_extension",
|
||||
"errors": [
|
||||
{
|
||||
"severity": "error",
|
||||
"message": "Unclosed string literal starting with '\"'",
|
||||
"code": "E0002",
|
||||
"position": {"line": 2, "column": 13, "offset": 42},
|
||||
"range": {
|
||||
"start": {"line": 2, "column": 13, "offset": 42},
|
||||
"end": {"line": 2, "column": 13, "offset": 42}
|
||||
},
|
||||
"context": "",
|
||||
"hint": "Add a closing quote before end of file"
|
||||
}
|
||||
],
|
||||
"warnings": 0,
|
||||
"infos": 0,
|
||||
"moduleInfo": {
|
||||
"flavor": "python",
|
||||
"detectionMethod": "by_extension",
|
||||
"imports": [],
|
||||
"variables": [],
|
||||
"functions": [
|
||||
{
|
||||
"name": "hello",
|
||||
"kind": "def",
|
||||
"parameters": [],
|
||||
"returnType": "",
|
||||
"isAsync": false,
|
||||
"isExported": false,
|
||||
"isGenerator": false,
|
||||
"visibility": "",
|
||||
"position": {"line": 1, "column": 5, "offset": 4},
|
||||
"docComment": ""
|
||||
}
|
||||
],
|
||||
"classes": [],
|
||||
"linesOfCode": 3,
|
||||
"totalTokens": 12,
|
||||
"comments": 0,
|
||||
"debugInfo": null
|
||||
},
|
||||
"durationMs": 0.342,
|
||||
"debugOutput": ""
|
||||
}
|
||||
```
|
||||
|
||||
### ValidationError JSON
|
||||
|
||||
```json
|
||||
{
|
||||
"severity": "error",
|
||||
"message": "Unclosed string literal starting with '\"'",
|
||||
"code": "E0002",
|
||||
"position": {"line": 2, "column": 13, "offset": 42},
|
||||
"range": {
|
||||
"start": {"line": 2, "column": 13, "offset": 42},
|
||||
"end": {"line": 2, "column": 13, "offset": 42}
|
||||
},
|
||||
"context": "",
|
||||
"hint": "Add a closing quote before end of file"
|
||||
}
|
||||
```
|
||||
|
||||
### ModuleInfo JSON
|
||||
|
||||
```json
|
||||
{
|
||||
"flavor": "nim",
|
||||
"detectionMethod": "by_extension",
|
||||
"imports": [
|
||||
{
|
||||
"module": "strutils",
|
||||
"symbols": [],
|
||||
"isRelative": false,
|
||||
"alias": "",
|
||||
"position": {"line": 1, "column": 8, "offset": 7}
|
||||
}
|
||||
],
|
||||
"variables": [],
|
||||
"functions": [
|
||||
{
|
||||
"name": "add",
|
||||
"kind": "proc",
|
||||
"parameters": [],
|
||||
"returnType": "",
|
||||
"isAsync": false,
|
||||
"isExported": false,
|
||||
"isGenerator": false,
|
||||
"visibility": "",
|
||||
"position": {"line": 3, "column": 6, "offset": 28},
|
||||
"docComment": ""
|
||||
}
|
||||
],
|
||||
"classes": [
|
||||
{
|
||||
"name": "MyType",
|
||||
"kind": "type",
|
||||
"baseTypes": [],
|
||||
"fields": [],
|
||||
"methods": [],
|
||||
"generics": [],
|
||||
"visibility": "",
|
||||
"position": {"line": 5, "column": 6, "offset": 52},
|
||||
"docComment": ""
|
||||
}
|
||||
],
|
||||
"linesOfCode": 8,
|
||||
"totalTokens": 45,
|
||||
"comments": 2,
|
||||
"debugInfo": null
|
||||
}
|
||||
```
|
||||
|
||||
### Human-Readable Report Format
|
||||
|
||||
```
|
||||
=== Validation Report ===
|
||||
Flavor: python
|
||||
Detection: by_extension
|
||||
Valid: false
|
||||
Errors: 1
|
||||
Warnings: 0
|
||||
Duration: 0.34ms
|
||||
Lines of code: 3
|
||||
Tokens: 12
|
||||
|
||||
--- Findings ---
|
||||
[E0002] (error) L2:13: Unclosed string literal starting with '"'
|
||||
| x = "
|
||||
| ^
|
||||
hint: Add a closing quote before end of file
|
||||
|
||||
--- Summary ---
|
||||
INVALID - source has issues that need attention.
|
||||
```
|
||||
|
||||
### Error Formatter Output (single error)
|
||||
|
||||
```
|
||||
[E0002] (error) L2:13: Unclosed string literal starting with '"'
|
||||
| x = "
|
||||
| ^
|
||||
hint: Add a closing quote before end of file
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CLI Usage
|
||||
|
||||
```
|
||||
Nimcheck v0.1.0 - Universal Source Code Validation Framework
|
||||
|
||||
Usage:
|
||||
nimcheck validate [--file=<path> | --source=<code>] [--flavor=<lang>]
|
||||
nimcheck inspect [--file=<path> | --source=<code>] [--flavor=<lang>]
|
||||
nimcheck detect [--file=<path> | --source=<code>]
|
||||
nimcheck help
|
||||
|
||||
Options:
|
||||
--file=<path> Source file to process
|
||||
--source=<code> Source code string to process
|
||||
--flavor=<lang> Language flavor (auto-detect if omitted)
|
||||
--debug Enable debug mode
|
||||
--json Output as JSON (default for inspect)
|
||||
```
|
||||
|
||||
Run `bin/nimcheck help` for the current registered flavor list.
|
||||
|
||||
The `validate` command exits 0 at the process level; inspect `valid` in the
|
||||
output or JSON for pass/fail. Validation paths do not raise to the caller.
|
||||
75
docs/ARCHITECTURE.md
Normal file
75
docs/ARCHITECTURE.md
Normal file
@ -0,0 +1,75 @@
|
||||
# Architecture
|
||||
|
||||
Nimcheck uses a four-layer pipeline architecture. Each layer is independent and communicates through well-defined interfaces.
|
||||
|
||||
```
|
||||
Source Code (string or file)
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| Detector | LanguageFlavor from extension, shebang, or content analysis
|
||||
+-------------------+
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| Tokenizer | Converts raw source into a seq[Token] stream
|
||||
+-------------------+
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| Validator | Analyzes token stream for syntax errors and structure
|
||||
+-------------------+
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| Reporting | Formats errors as human-readable text or structured JSON
|
||||
+-------------------+
|
||||
|
|
||||
v
|
||||
ValidationResult
|
||||
```
|
||||
|
||||
### Layer 1: Detector (`src/nimcheck/core/detector.nim`)
|
||||
|
||||
Language auto-detection with cascading priority:
|
||||
|
||||
| Priority | Method | Source | Confidence |
|
||||
|----------|--------|--------|------------|
|
||||
| 1 (highest) | Explicit | User-provided `LanguageFlavor` | 1.0 |
|
||||
| 2 | File extension | 70+ extension-to-flavor mappings | 0.9 |
|
||||
| 3 | Shebang line | `#!` line interpreter matching (bash, python, node, ruby, lua, swift) | 0.95 |
|
||||
| 4 | Content analysis | Keyword scoring, structural patterns, bracket usage | 0.6-0.9 |
|
||||
|
||||
The `detectFlavor()` function returns a `(LanguageFlavor, FlavorDetectionMethod, confidence: float)` tuple.
|
||||
|
||||
### Layer 2: Tokenizer (`src/nimcheck/core/tokenizerbase.nim`, per-language impls)
|
||||
|
||||
Each tokenizer converts raw source into a `seq[Token]`. Tokenizers track:
|
||||
|
||||
- **Position**: line, column, and byte offset for every token
|
||||
- **Bracket stack**: `(`, `[`, `{`, `<` pushed and popped with position tracking
|
||||
- **String context**: escape sequences, multi-line delimiters, interpolation
|
||||
- **Comment context**: line comments, block comments with nesting support
|
||||
|
||||
Tokenizers emit errors for:
|
||||
- Unclosed strings (`ErrUnclosedString`)
|
||||
- Unclosed comments (`ErrUnclosedComment`)
|
||||
- Unexpected characters (`ErrUnexpectedToken`)
|
||||
- Unclosed block constructs (language-specific, e.g., `${}`, `$()`)
|
||||
|
||||
### Layer 3: Validator (`src/nimcheck/core/validatorbase.nim`, per-language impls)
|
||||
|
||||
Each validator:
|
||||
|
||||
1. Creates its tokenizer and runs `tokenize()`
|
||||
2. Collects tokenizer errors
|
||||
3. Runs `analyzeTokens()` - language-specific structural analysis
|
||||
4. Runs `buildModuleInfo()` - populates `ModuleInfo` with imports, functions, classes
|
||||
5. Computes validity, warning/info counts, duration
|
||||
|
||||
Validators register themselves via a factory pattern at import time. The registry maps flavor strings (e.g., `"python"`, `"bash"`) to constructors.
|
||||
|
||||
### Layer 4: Reporting (`src/nimcheck/reporting/errors.nim`)
|
||||
|
||||
Produces:
|
||||
- **Human-readable reports**: `formatReport()` with position, severity, code, message, source context with carets, and hints
|
||||
267
docs/ERRORS.md
Normal file
267
docs/ERRORS.md
Normal file
@ -0,0 +1,267 @@
|
||||
# Error Codes
|
||||
|
||||
## Error Code Reference
|
||||
|
||||
Error codes follow a prefixed numbering scheme. The prefix encodes the category and severity.
|
||||
|
||||
### General Errors (E0xxx)
|
||||
|
||||
| Code | Constant | Description | Triggered By |
|
||||
|------|----------|-------------|-------------|
|
||||
| `E0001` | `ErrUnexpectedToken` | Unexpected or unrecognizable character | Tokenizer encounter of non-ASCII, non-operator char |
|
||||
| `E0002` | `ErrUnclosedString` | Unclosed string literal | Missing closing quote at EOF |
|
||||
| `E0003` | `ErrUnclosedComment` | Unclosed block comment | Missing `*/`, `]#`, `#}` etc. at EOF |
|
||||
| `E0004` | `ErrUnclosedBlock` | Unclosed block/control structure | Language-specific: unclosed `if`/`for`/`class`/etc. |
|
||||
| `E0005` | `ErrMismatchedBracket` | Mismatched or unexpected closing bracket | `)` without `(`, `</div>` without `<div>` |
|
||||
| `E0006` | `ErrUnclosedBracket` | Unclosed bracket at EOF | `validateBrackets()` finds remaining items on bracket stack |
|
||||
| `E0007` | `ErrInvalidSyntax` | General invalid syntax | JSON parse error, TOML missing `=`, YAML empty key |
|
||||
| `E0008` | `ErrUnexpectedEOF` | Unexpected end of file | Reserved for incomplete constructs not covered by other codes |
|
||||
| `E0009` | `ErrEmptySource` | Source is empty (zero-length string) | `validate()` with `source.len == 0` |
|
||||
| `E0010` | `ErrEncodingError` | Encoding error | Reserved for non-UTF-8 input |
|
||||
| `E0011` | -- | File not found | `validateFile()` with non-existent path |
|
||||
|
||||
### Language-Specific Errors (E1xxx)
|
||||
|
||||
| Code | Constant | Description | Languages |
|
||||
|------|----------|-------------|-----------|
|
||||
| `E1001` | `ErrUnexpectedIndent` | Unexpected or inconsistent indentation | Python |
|
||||
| `E1002` | `ErrMissingIndent` | Missing indentation after block opener | Python |
|
||||
| `E1003` | `ErrDuplicateParam` | Duplicate parameter name | (reserved) |
|
||||
| `E1004` | `ErrUndefinedVariable` | Reference to undefined variable | (reserved) |
|
||||
| `E1005` | `ErrTypeMismatch` | Type mismatch | (reserved) |
|
||||
| `E1006` | `ErrInvalidAssignment` | Invalid assignment target | (reserved) |
|
||||
| `E1007` | `ErrMissingReturn` | Function missing return statement | (reserved) |
|
||||
| `E1008` | `ErrInvalidDecorator` | Invalid decorator usage | Python |
|
||||
| `E1009` | `ErrInvalidImport` | Invalid import statement | (reserved) |
|
||||
|
||||
### Template Errors (E2xxx)
|
||||
|
||||
| Code | Constant | Description | Languages |
|
||||
|------|----------|-------------|-----------|
|
||||
| `E2001` | `ErrUnclosedTemplateTag` | Unclosed template expression/block tag | Jinja |
|
||||
| `E2002` | `ErrUnclosedBlockTag` | Unclosed block tag with no matching end tag | Jinja |
|
||||
| `E2003` | `ErrInvalidFilter` | Invalid filter in template expression | Jinja |
|
||||
|
||||
### Info Codes (I0xxx)
|
||||
|
||||
| Code | Constant | Description |
|
||||
|------|----------|-------------|
|
||||
| `I0001` | `InfoLongLine` | Line exceeds recommended length |
|
||||
| `I0002` | `InfoTrailingWhitespace` | Trailing whitespace detected |
|
||||
|
||||
### Warning Codes (W0xxx / W1xxx)
|
||||
|
||||
| Code | Constant | Description |
|
||||
|------|----------|-------------|
|
||||
| `W0001` | `WarnUnusedVariable` | Variable declared but never used |
|
||||
| `W0002` | `WarnInconsistentNaming` | Naming convention inconsistency |
|
||||
| `W0003` | `WarnMissingDoc` | Public symbol missing documentation |
|
||||
| `W1001` | -- | Tab characters in YAML indentation |
|
||||
|
||||
### Internal Error
|
||||
|
||||
| Code | Description | Triggered By |
|
||||
|------|-------------|-------------|
|
||||
| `E9999` | Internal framework error (never-crash fallback) | Any unhandled `Exception` in `validate()` or `validateSource()` |
|
||||
|
||||
---
|
||||
|
||||
## IO Table: Supported Languages and Per-Language Errors
|
||||
|
||||
### Input/Output Contract
|
||||
|
||||
Every validation function follows the same contract:
|
||||
|
||||
| Input | Output |
|
||||
|-------|--------|
|
||||
| `source: string` (required) | `ValidationResult` with `valid: bool` |
|
||||
| `flavor: LanguageFlavor = lfUnknown` | `ValidationResult.errors: seq[ValidationError]` |
|
||||
| `options: ValidationOptions` (optional) | `ValidationResult.moduleInfo: ModuleInfo` |
|
||||
| `sourcePath: string = ""` (optional) | `ValidationResult.durationMs: float` |
|
||||
|
||||
**Guarantees**:
|
||||
- Never throws - all exceptions are caught and returned as `E9999` errors
|
||||
- Empty source returns `E0009` (or `valid: true` for YAML/TOML where empty is valid)
|
||||
- File not found returns `E0011`
|
||||
- Unknown flavor with no detection possible returns an `E0007` error
|
||||
- Every error has a `code`, `message`, `severity`, `position`, and optional `hint`
|
||||
|
||||
### Per-Language Tokenizer, Validator, and Error Table
|
||||
|
||||
#### Nim (`.nim`, `.nims`, `.nimble`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `nim_tokenizer.nim` - `NimTokenizer` |
|
||||
| Validator | `nim_validator.nim` - `NimValidator` |
|
||||
| Keywords | 80+ (addr, and, as, block, break, case, const, converter, defer, discard, elif, else, enum, except, export, finally, for, from, func, if, import, in, include, iterator, let, macro, method, mixin, mod, nil, not, object, of, or, proc, ptr, raise, ref, return, template, try, tuple, type, using, var, when, while, with, yield, etc.) |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0001` | Unexpected character (non-ASCII, unrecognizable) | `esWarning` | Shows hex `U+XXXX` value |
|
||||
| `E0002` | Unclosed string (`"..."` at EOF) | `esError` | "Add a closing quote" |
|
||||
| `E0002` | Unclosed multi-line string (`"""` at EOF) | `esError` | -- |
|
||||
| `E0002` | Newline inside string literal | `esError` | -- |
|
||||
| `E0002` | Unclosed raw string (`r"..."` at EOF) | `esError` | -- |
|
||||
| `E0002` | Unclosed character literal (`'x` at EOF) | `esError` | -- |
|
||||
| `E0003` | Unclosed block comment (`#[` without `]#`) | `esError` | "Add closing `]#`" |
|
||||
| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | "Add matching closing bracket" |
|
||||
|
||||
Structural extraction: imports (`from X import Y`), functions (`proc`, `func`, `method`, `template`, `macro`, `iterator`, `converter`), types (`type Foo`).
|
||||
|
||||
#### Bash / Shell (`.sh`, `.bash`, `.zsh`, `.fish`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `bash_tokenizer.nim` - `BashTokenizer` |
|
||||
| Validator | `bash_validator.nim` - `BashValidator` |
|
||||
| Keywords | 38+ (if, then, else, elif, fi, case, esac, for, while, until, do, done, in, function, select, time, return, break, continue, exit, declare, local, export, readonly, unset, eval, exec, let, source, shift, trap, type, typeset, ulimit, umask, alias, read, echo) |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0002` | Unclosed single-quoted string | `esError` | -- |
|
||||
| `E0002` | Unclosed double-quoted string | `esError` | -- |
|
||||
| `E0004` | Unclosed `${var}` expansion | `esError` | -- |
|
||||
| `E0004` | Unclosed `$()` command substitution | `esError` | -- |
|
||||
| `E0004` | Unclosed `if` block (`if` without matching `fi`) | `esError` | "Add missing 'fi'" |
|
||||
| `E0004` | Unclosed loop (`for`/`while`/`until` without `done`) | `esError` | "Add missing 'done'" |
|
||||
| `E0004` | Unclosed `case` block (`case` without `esac`) | `esError` | "Add missing 'esac'" |
|
||||
| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- |
|
||||
|
||||
Special features: shebang detection (`#!`), heredoc detection (`<<EOF`), variable expansion (`$var`, `${var}`), backtick command substitution, arithmetic expansion (`(( ))`).
|
||||
|
||||
#### Python (`.py`, `.pyw`, `.pyx`, `.pxd`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `python_tokenizer.nim` - `PythonTokenizer` |
|
||||
| Validator | `python_validator.nim` - `PythonValidator` |
|
||||
| Keywords | 33 (False, None, True, and, as, assert, async, await, break, class, continue, def, del, elif, else, except, finally, for, from, global, if, import, in, is, lambda, nonlocal, not, or, pass, raise, return, try, while, with, yield) |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0002` | Unclosed string (`"..."` or `'...'` at EOF) | `esError` | -- |
|
||||
| `E0002` | Unclosed triple-quoted string (`"""` or `'''` at EOF) | `esError` | -- |
|
||||
| `E0002` | Unclosed f-string (`f"..."` at EOF) | `esError` | -- |
|
||||
| `E0002` | Newline inside string literal | `esError` | -- |
|
||||
| `E1001` | Indentation mismatch at block boundary | `esError` | -- |
|
||||
| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- |
|
||||
|
||||
Special features: f-string interpolation with nested braces, indentation tracking (INDENT/DEDENT tokens), decorator detection (`@`), line continuation (`\`).
|
||||
|
||||
Structural extraction: imports (`import X`, `from X import Y`), functions (`def`), classes (`class`).
|
||||
|
||||
#### JavaScript / TypeScript (`.js`, `.mjs`, `.cjs`, `.ts`, `.tsx`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `javascript_tokenizer.nim` - `JavaScriptTokenizer` |
|
||||
| Validator | `javascript_validator.nim` - `JavaScriptValidator` |
|
||||
| Keywords | 60+ (async, await, break, case, catch, class, const, continue, debugger, default, delete, do, else, export, extends, false, finally, for, function, if, import, in, instanceof, let, new, null, of, return, super, switch, static, this, throw, true, try, typeof, var, void, while, with, yield + TypeScript: interface, type, declare, module, namespace, abstract, readonly, as, any, boolean, number, string, never, keyof, infer, unknown) |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0002` | Unclosed string (`"..."` or `'...'` at EOF) | `esError` | -- |
|
||||
| `E0002` | Unclosed template literal (`` `...` `` at EOF) | `esError` | -- |
|
||||
| `E0002` | Newline inside string literal | `esError` | -- |
|
||||
| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- |
|
||||
|
||||
Special features: template literals with `${}` interpolation nesting, regex literal detection, arrow function detection (`=>`), ESM import/export.
|
||||
|
||||
Structural extraction: imports/exports, functions (`function`, `async`), classes.
|
||||
|
||||
#### PHP (`.php`, `.phtml`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `php_tokenizer.nim` - `PHPTokenizer` |
|
||||
| Validator | `php_validator.nim` - `PHPValidator` |
|
||||
| Mode switching | Tracks `inPhpMode` flag - toggles on `<?php`/`<?` and `?>` |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0002` | Unclosed string | `esError` | -- |
|
||||
| `E0006` | Unclosed bracket (`(`, `[`, `{` at EOF) | `esError` | -- |
|
||||
|
||||
Special features: PHP/HTML mode switching, `<?php`, `<?=`, `?>` detection, variable names (`$var`, `$obj->prop`).
|
||||
|
||||
Structural extraction: functions (`function`, `fn`), classes (`class`), imports/requires (`include`, `require`, `use`, `namespace`).
|
||||
|
||||
#### HTML / XML (`.html`, `.htm`, `.xhtml`, `.xml`, `.svg`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `html_tokenizer.nim` - `HTMLTokenizer` |
|
||||
| Validator | `html_validator.nim` - `HTMLValidator` |
|
||||
| Void elements | 14: area, base, br, col, embed, hr, img, input, link, meta, param, source, track, wbr |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0003` | Unclosed HTML comment (`<!--` without `-->`) | `esError` | -- |
|
||||
| `E0004` | Unclosed HTML tag (e.g., `<div>` without `</div>`) | `esError` | "Add closing `</tagname>`" |
|
||||
| `E0005` | Unexpected closing tag (mismatched nesting) | `esWarning` | "Check that tags are properly nested" |
|
||||
|
||||
Special features: tag stack balancing, void element awareness (no closing tag needed for self-closing elements), attribute parsing with quoted values, `<!DOCTYPE>`, `<?xml?>` directives.
|
||||
|
||||
#### Jinja2 (`.jinja`, `.jinja2`, `.j2`, `.html.j2`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Tokenizer | `jinja_tokenizer.nim` - `JinjaTokenizer` |
|
||||
| Validator | `jinja_validator.nim` - `JinjaValidator` |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0003` | Unclosed Jinja comment (`{#` without `#}`) | `esError` | -- |
|
||||
| `E2001` | Unclosed expression tag (`{{` without `}}`) | Implicit via tokenizer | -- |
|
||||
| `E2001` | Unclosed block tag (`{%` without `%}`) | Implicit via tokenizer | -- |
|
||||
| `E2002` | Unclosed `for` block (no matching `endfor`) | `esError` | "Add `{% endfor %}`" |
|
||||
| `E2002` | Unclosed `if` block (no matching `endif`) | `esError` | "Add `{% endif %}`" |
|
||||
| `E2002` | Unclosed `block` (no matching `endblock`) | `esError` | "Add `{% endblock %}`" |
|
||||
| `E2002` | Unclosed `macro`, `filter`, `raw`, `autoescape`, `call`, `with` | `esError` | "Add matching `end{tag}`" |
|
||||
|
||||
Special features: tracks 12 block tag types (`for`, `if`, `block`, `macro`, `filter`, `raw`, `autoescape`, `call`, `with`), validates matching end tags, handles content between template syntax and plain text.
|
||||
|
||||
#### JSON (`.json`, `.jsonc`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Validator | `config_validators.nim` - `JSONValidator` |
|
||||
| Method | Uses Nim's built-in `parseJson()` for full validation |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0007` | JSON parse error (any `JsonParsingError`) | `esError` | Error message from parser |
|
||||
| `E0007` | Any other exception during parse | `esError` | "JSON error: {message}" |
|
||||
| `E0009` | Empty JSON source | `esError` | -- |
|
||||
|
||||
#### YAML (`.yaml`, `.yml`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Validator | `config_validators.nim` - `YAMLValidator` |
|
||||
| Method | Line-by-line structural analysis |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0007` | Empty mapping key (no key before `:`) | `esError` | -- |
|
||||
| `E0007` | General YAML structural error | `esError` | -- |
|
||||
| `E0009` | No content after stripping comments/blanks | `esError` | -- |
|
||||
| `W1001` | Tab characters used for indentation | `esWarning` | "Use spaces for YAML indentation" |
|
||||
|
||||
Note: Empty YAML (only whitespace/comments) is `valid: true`.
|
||||
|
||||
#### TOML (`.toml`)
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Validator | `config_validators.nim` - `TOMLValidator` |
|
||||
| Method | Line-by-line structural analysis |
|
||||
|
||||
| Error Code | Condition | Severity | Hint |
|
||||
|------------|-----------|----------|------|
|
||||
| `E0007` | Missing `=` in key-value pair | `esError` | "TOML uses `key = value` format" |
|
||||
| `E0007` | Missing key before `=` | `esError` | -- |
|
||||
| `E0007` | General TOML structural error | `esError` | -- |
|
||||
| `E0009` | No content after stripping comments/blanks | `esError` | -- |
|
||||
|
||||
@ -106,7 +106,7 @@ bin/nimcheck detect --file=unknown.txt
|
||||
bin/nimcheck inspect --file=main.go --json
|
||||
```
|
||||
|
||||
See [README.md](../README.md) for full API and JSON schemas.
|
||||
See [docs/API.md](API.md) for API and JSON schemas; [README.md](../README.md) for overview.
|
||||
|
||||
---
|
||||
|
||||
@ -153,4 +153,15 @@ These are **not blockers** for v0.1.0:
|
||||
| Lint | `make lint` |
|
||||
| Clean | `make clean` |
|
||||
| CI locally | Same as lint + test |
|
||||
| Push | `git push origin master` |
|
||||
| Push | `git push origin master` |
|
||||
|
||||
## Documentation layout
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `README.md` | Product overview (concise) |
|
||||
| `docs/API.md` | API reference, types, JSON, CLI |
|
||||
| `docs/ERRORS.md` | Error codes, per-language table |
|
||||
| `docs/ARCHITECTURE.md` | Pipeline design |
|
||||
| `docs/GITEA.md` | CI setup |
|
||||
| `CONTRIBUTING.md` | Dev workflow |
|
||||
Loading…
Reference in New Issue
Block a user