docs: update README and LESSONS_LEARNED

Browse Source 
README:
- Expand building section with Makefile, nimble, and manual targets
- Add CLI usage examples for all commands (stdin, detect, version)
- Expand API reference with all public functions (validateSourceJson, reportFile, etc.)
- Add JSON output documentation
- Expand development section with project tree and language-adding guide
- Add versioning and prerequisites sections

LESSONS_LEARNED:
- Add Class 9: Missing .gitignore -- build artifact tracking risk
- Add Class 10: Repository directory name mismatch
- Renumber all sections 11-16 to accommodate new entries
- Update test count from 256 to 250+
- Update total bug counts
This commit is contained in:
retoor 2026-07-08 04:29:19 +00:00
parent 7e8917e527
commit 659ea7ebe0
2 changed files with 313 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

122
LESSONS_LEARNED.md
View File

@ -21,10 +21,12 @@ encountered during the development of the Validatrix validation framework.
8. [Class 6: Named Parameter Syntax](#8-class-6-named-parameter-syntax)
9. [Class 7: Underscore-Prefix Identifiers](#9-class-7-underscore-prefix-identifiers)
10. [Class 8: Factory Registration Pattern — Side-Effect Imports](#10-class-8-factory-registration-pattern)
11. [Root-Cause Analysis: Why These Bugs Happen](#11-root-cause-analysis)
12. [Systemic Anti-Patterns](#12-systemic-anti-patterns)
13. [Verification Protocol for Future Work](#13-verification-protocol)
14. [Checklist for New Tokenizer/Validator Modules](#14-checklist-for-new-modules)
11. [Class 9: Missing `.gitignore` — Build Artifacts Risk](#11-class-9-missing-gitignore)
12. [Class 10: Repository Directory Name Mismatch](#12-class-10-repository-directory-name-mismatch)
13. [Root-Cause Analysis: Why These Bugs Happen](#13-root-cause-analysis)
14. [Systemic Anti-Patterns](#14-systemic-anti-patterns)
15. [Verification Protocol for Future Work](#15-verification-protocol)
16. [Checklist for New Tokenizer/Validator Modules](#16-checklist-for-new-modules)
---
@ -36,7 +38,7 @@ paste propagation amplified every defect across every language. Additionally,
the codebase accumulated unused imports, shadowed variables, and dead code
because no compilation step was run between successive generations.
**Total bugs found and fixed:** 14 distinct classes, ~90 individual file hits.
**Total bugs found and fixed:** 16 distinct classes, ~95 individual file hits.
**Categorization:**
| Class | Description | Files Affected | Root Cause |
@ -49,8 +51,10 @@ because no compilation step was run between successive generations.
| 6 | Named param syntax (`hint:` vs `hint =`) | 1 (config_validators) | Nim named-argument syntax confusion |
| 7 | Underscore-prefix identifiers | 1 (config_validators) | Nim doesn't allow `_` prefix |
| 8 | Zero-content `init_validators.nim` | 1 | Dead code left from architecture refactor |
| 9 | Missing `.gitignore` | 0 (repo-wide) | No `.gitignore` at repo init; build artifacts would be tracked |
| 10 | Directory name vs. project name mismatch | 1 (repo root) | Project directory `nimcheck` never renamed after project was renamed to `validatrix` |
**256 tests pass. Zero compiler warnings.**
**250+ tests pass. Zero compiler warnings.**
---
@ -435,7 +439,105 @@ side-effect pattern.
---
## 11. Root-Cause Analysis
## 11. Class 9: Missing `.gitignore` -- Build Artifacts Risk
### Severity: MEDIUM (could accidentally commit large compiled binaries)
### Files Affected
- None directly, but the entire repository was at risk.
### The Pattern
The repository was initialised (`git init`) without a `.gitignore` file.
Build artifacts existed on disk:
```
bin/validatrix 1.2 MB compiled binary
src/validatrix.out 1.2 MB build output
tests/test_all 3.4 MB compiled test binary
tests/test_bash 1.2 MB compiled test binary
tests/test_nim 1.1 MB compiled test binary
tests/test_php 1.2 MB compiled test binary
dpc.log 400 B agent log
__pycache__/ 12 KB Python cache
```
Without a `.gitignore`, `git add .` would stage all of these, bloating
the repository with ~9.5 MB of binary artifacts that should never be
version-controlled.
### Root Cause
- No `.gitignore` was created at `git init` time.
- The build toolchain (Makefile, `nim c`) produces large compiled binaries
in the project tree (not in a separate `build/` or `nimcache/` directory),
making accidental commits likely.
- The `dpc.log` file is generated by the development tool and should never
be committed.
### Prevention
- **Create `.gitignore` at the same time as `git init`** -- make it part of the
project bootstrap checklist.
- **Run `git status` before every commit** to spot unexpected files.
- **Use `git add --dry-run`** to preview what would be staged.
- **Standard ignores to include** for any Nim project:
```
bin/
*.out
*.log
__pycache__/
*.pyc
*.o
*.exe
nimcache/
```
### Fix Applied
Created `.gitignore` excluding: `bin/`, `*.out`, `*.log`, `__pycache__/`,
`*.pyc`, compiled test binaries, OS junk files, and editor swap files.
Verified with `git add --dry-run` that only source files would be staged.
---
## 12. Class 10: Repository Directory Name Mismatch
### Severity: LOW (cosmetic, but causes confusion)
### Files Affected
- Repository root directory (`/home/retoor/projects/nimcheck/`)
### The Pattern
The project was originally named `nimcheck` (or similar), and the repository
directory was created with that name. During development the project was
renamed to **Validatrix** -- the `.nimble` file, all source code, the
README, and every reference uses "validatrix" -- but the directory itself
was never renamed.
This means:
- `cd` paths are `/home/retoor/projects/nimcheck/` but everything inside
says `validatrix`
- Git remote URLs (when added) would reference `nimcheck`
- New developers cloning the repo get `nimcheck/` as the checkout directory
### Root Cause
- Directory rename was simply forgotten after the project rename.
- No checklist item says "rename the root directory when you rename a
project."
- The `git init` happened in the old directory before the rename.
### Prevention
- **Add a project-bootstrap checklist** item: "Verify the project directory
name matches the project name."
- **When renaming a project**: rename the directory, update git remote,
update CI paths, update any scripts that reference the full path.
- **Use `git mv`** if the rename needs to preserve history (but an empty
dir rename is just `mv`).
### Fix Applied
Documented here. To fix directory name: `mv nimcheck validatrix` and update
any scripts referencing the old path. Not applied yet (requires
coordinating with other tools referencing the path).
---
## 13. Root-Cause Analysis
### Why Did All These Bugs Happen?
@ -468,7 +570,7 @@ side-effect pattern.
---
## 12. Systemic Anti-Patterns
## 14. Systemic Anti-Patterns
### 12a. Massive types.nim
`types.nim` is 456 lines with 19+ object types, 12+ toJson converter procs,
@ -513,7 +615,7 @@ in some places, leading to inconsistent reporting.
---
## 13. Verification Protocol
## 15. Verification Protocol
### Mandatory for Every Change
@ -548,7 +650,7 @@ in some places, leading to inconsistent reporting.
```
- Add `--warning[ResultShadowed]:on` to project config.
## 14. Checklist for New Tokenizer/Validator Modules
## 16. Checklist for New Tokenizer/Validator Modules
When adding a new language:

279
README.md
View File

@ -5,14 +5,14 @@ A multi-language source code validation framework written in Nim. Validatrix use
## Features
- **10 languages supported**: Nim, Bash, Python, JavaScript, PHP, HTML, Jinja, JSON, YAML, TOML
- **Tokenizer-based validation** — real tokenizers that track bracket depth, string context, comments, and nesting
- **Auto-detection** — detect language from file extension or content analysis
- **Comprehensive diagnostics** — structured error reports with position, severity, hints, and debug output
- **Never-crash guarantee** — every validation path is wrapped in try/except; invalid input returns errors gracefully, never panics
- **JSON output** — machine-readable diagnostics via `toJson()` on any validation result
- **Inspection API** — extract module structure (imports, definitions, functions/classes) as JSON
- **Debug mode** — compile with `-d:validatrixDebug` for detailed tokenization logging
- **No external dependencies** — pure Nim standard library
- **Tokenizer-based validation** -- real tokenizers that track bracket depth, string context, comments, and nesting
- **Auto-detection** -- detect language from file extension or content analysis
- **Comprehensive diagnostics** -- structured error reports with position, severity, hints, and debug output
- **Never-crash guarantee** -- every validation path is wrapped in try/except; invalid input returns errors gracefully, never panics
- **JSON output** -- machine-readable diagnostics via `toJson()` on any validation result
- **Inspection API** -- extract module structure (imports, definitions, functions/classes) as JSON
- **Debug mode** -- compile with `-d:validatrixDebug` for detailed tokenization logging
- **No external dependencies** -- pure Nim standard library
## Quick Start
@ -32,20 +32,96 @@ let r3 = validateFile("path/to/file.nim", lfNim)
# Get JSON diagnostics
echo r.toJson().pretty()
# Inspect source structure
# Inspect source structure (returns JsonNode)
let info = inspectSource("class Foo: ...", lfPython)
echo $info # JSON with class/function info
echo info.pretty() # formatted JSON with class/function info
# Get human-readable report
let report = reportSource("proc foo() =", lfNim)
echo report # formatted error summary
```
## Building
### Using Makefile (recommended)
```bash
# Build the library binary
make build
# Run all tests
make test
# Build with debug mode
make build-debug
make debug # build debug + run all tests
# Run a specific language test
make test-nim
make test-python
make test-bash
# Lint with full warnings
make lint
# Clean build artifacts
make clean
# Show all available targets
make help
```
### Using nimble
```bash
# Run all tests
nimble test
# Run per-language tests
nimble test_nim
nimble test_python
# Build the library
nimble build
```
### Manual compilation
```bash
# Build the library
nim c src/validatrix.nim
# Build with debug mode
nim c -d:validatrixDebug src/validatrix.nim
# Run all tests
nim c -r tests/test_all.nim
```
## CLI Usage
When compiled as a standalone binary:
When compiled as a standalone binary (`make build` produces `bin/validatrix`):
```
validatrix validate --file=source.nim
validatrix validate --source="print('hi')" --flavor=python
validatrix inspect --file=source.py
validatrix detect --file=unknown.txt
```bash
# Validate a file
bin/validatrix validate --file=source.nim
# Validate source code directly
bin/validatrix validate --source="print('hi')" --flavor=python
# Read from stdin
echo "var x = 1" | bin/validatrix validate
# Inspect source structure (returns JSON)
bin/validatrix inspect --file=source.py
# Detect language
bin/validatrix detect --file=unknown.txt
bin/validatrix detect --source="#!/usr/bin/env bash"
# Version and help
bin/validatrix version
bin/validatrix help
```
## API Reference
@ -54,19 +130,29 @@ validatrix detect --file=unknown.txt
| Function | Description |
|---|---|
| `validateSource(source, flavor, options, sourcePath)` | Validate source code string |
| `validateFile(filePath, flavor, options)` | Validate a file |
| `inspectSource(source, flavor, options, sourcePath)` | Return structural diagnostics as JSON |
| `inspectFile(filePath, flavor, options)` | Return structural diagnostics for a file as JSON |
| `validateSource(source, flavor, options, sourcePath)` | Validate source code string; returns `ValidationResult` |
| `validateFile(filePath, flavor, options)` | Validate a file; returns `ValidationResult` |
| `inspectSource(source, flavor, options, sourcePath)` | Return structural diagnostics as `JsonNode` |
| `inspectFile(filePath, flavor, options)` | Return structural diagnostics for a file as `JsonNode` |
| `validateSourceJson(source, flavor)` | Validate source and return result as JSON string |
| `validateFileJson(filePath, flavor)` | Validate file and return result as JSON string |
| `reportSource(source, flavor)` | Get a human-readable validation report (string) |
| `reportFile(filePath, flavor)` | Get a human-readable validation report for a file (string) |
| `detectFlavor(source, filePath)` | Auto-detect language flavor |
| `supportedFlavors()` | List all registered language flavors |
| `supportedFlavors()` | List all registered language flavors (returns `seq[string]`) |
| `version()` | Get the Validatrix version string |
| `enableDebug()` / `disableDebug()` | Toggle debug mode globally |
### Types
- **`ValidationResult`** — contains `errors: seq[ValidationError]`, `valid: bool`, `moduleInfo`, `debugOutput`, `detectionInfo`
- **`ValidationError`** — contains `code`, `message`, `severity`, `position`, `hint`, `context`
- **`ValidationOptions`** — contains `flavor`, `debugMode`, `maxErrors`, `validateAll`
- **`LanguageFlavor`** — enum: `lfUnknown`, `lfNim`, `lfBash`, `lfPython`, `lfJavaScript`, `lfPHP`, `lfHTML`, `lfJinja`, `lfJSON`, `lfYAML`, `lfTOML`
- **`ValidationResult`** -- contains `errors: seq[ValidationError]`, `valid: bool`, `moduleInfo`, `debugOutput`, `detectionInfo`
- **`ValidationError`** -- contains `code`, `message`, `severity`, `position`, `hint`, `context`
- **`ValidationOptions`** -- contains `flavor`, `debugMode`, `maxErrors`, `validateAll`
- **`LanguageFlavor`** -- enum: `lfUnknown`, `lfNim`, `lfBash`, `lfPython`, `lfJavaScript`, `lfPHP`, `lfHTML`, `lfJinja`, `lfJSON`, `lfYAML`, `lfTOML`
### JSON Output
Both `ValidationResult` and `ValidationError` have a `.toJson()` method returning a `JsonNode`. Use `.pretty()` for formatted output.
## Error Codes
@ -100,64 +186,101 @@ validatrix detect --file=unknown.txt
| YAML | `.yaml`, `.yml` | Indentation tracking, key-value checking |
| TOML | `.toml` | Table headers, bracket balance, string validation |
## Building
```bash
# Build the library
nim c src/validatrix.nim
# Build with debug mode
nim c -d:validatrixDebug src/validatrix.nim
# Run all tests
nim c -r tests/test_all.nim
# Build standalone binary
nim c -d:release src/validatrix.nim
```
## Testing
The test suite covers **256 test cases** across all supported languages:
The test suite covers **250+ test cases** across all supported languages:
- **25-30 tests per language** including edge cases
- **Fuzz tests** — null bytes, binary data, deep nesting (100 levels), unicode bombs, empty sources
- **Mixed-file validators** — HTML+Jinja combined files
- **Config formats** — JSON, YAML, TOML validation
- **Concurrent validation** — thread safety under parallel calls
- **Never-crash guarantee** — every test verifies invalid input returns graceful errors, not crashes
- **Per-language tests** -- valid and invalid source for each of the 10 languages
- **Exhaustive tests** -- edge cases: unclosed strings, unclosed comments, mismatched brackets, binary data, unicode bombs
- **Fuzz tests** -- null bytes, binary data, deep nesting (100 levels), empty sources, concurrent validation
- **Config format tests** -- JSON, YAML, TOML validation with invalid syntax checks
- **Mixed-file tests** -- HTML+Jinja combined files
- **Never-crash guarantee** -- every test verifies invalid input returns graceful errors, not crashes
- **Debug mode tests** -- `make debug` runs all tests with debug logging enabled
Run all tests with:
```bash
make test # or
nimble test # or
nim c -r tests/test_all.nim
```
## Development
### Prerequisites
- Nim >= 2.0.0
- `make` (optional, for Makefile targets)
### Project structure
```
src/
validatrix.nim Public API entry point and CLI (isMainModule)
validatrix/
core/
types.nim Core types: ValidationResult, ValidationError, LanguageFlavor
tokenizerbase.nim Abstract tokenizer base class
validatorbase.nim Abstract validator base class
detector.nim Language auto-detection
debug.nim Debug logging infrastructure
tokenizers/
nim_tokenizer.nim
bash_tokenizer.nim
python_tokenizer.nim
javascript_tokenizer.nim
php_tokenizer.nim
html_tokenizer.nim
jinja_tokenizer.nim
languages/
nim_validator.nim
bash_validator.nim
python_validator.nim
javascript_validator.nim
php_validator.nim
html_validator.nim
jinja_validator.nim
config_validators.nim JSON, YAML, TOML validators
reporting/
errors.nim Error formatting and JSON output
tests/
test_all.nim Master test runner (imports all test modules)
test_<lang>.nim Per-language tests
test_<lang>_exhaustive.nim Exhaustive edge-case tests per language
test_config.nim JSON/YAML/TOML tests
test_fuzz.nim Fuzz and boundary tests
test_common.nim Shared test utilities
test_mixed.nim Mixed-language file tests
fixtures/ Test data files per language
```
### Building with warnings
```bash
make lint
# or manually:
nim c --hints:on --warnings:on src/validatrix.nim
```
### Adding a new language
See the detailed checklist in [LESSONS_LEARNED.md](LESSONS_LEARNED.md) (Section 14).
## Architecture
```
src/validatrix/
validatrix.nim # Public API entry point
core/
types.nim # Core types: ValidationResult, ValidationError, LanguageFlavor
tokenizerbase.nim # Abstract tokenizer base class
validatorbase.nim # Abstract validator base class
detector.nim # Language auto-detection
debug.nim # Debug logging infrastructure
tokenizers/
nim_tokenizer.nim
bash_tokenizer.nim
python_tokenizer.nim
javascript_tokenizer.nim
php_tokenizer.nim
html_tokenizer.nim
jinja_tokenizer.nim
languages/
nim_validator.nim
bash_validator.nim
python_validator.nim
javascript_validator.nim
php_validator.nim
html_validator.nim
jinja_validator.nim
config_validators.nim # JSON, YAML, TOML validators
reporting/
errors.nim # Error formatting and JSON output
```
The framework uses a layered architecture:
1. **Tokenizers** -- convert raw source into a stream of tokens, tracking bracket depth, string context, and comments
2. **Validators** -- analyze token streams for language-specific syntax rules
3. **Detector** -- identifies language from file extension, shebang, or content heuristics
4. **Reporting** -- formats errors as human-readable strings or structured JSON
Validators register themselves at import time via a side-effect factory pattern. The public API (`validatrix.nim`) imports all validators and re-exports core types for consumer convenience.
## Versioning
Current version: **0.1.0**
## License
@ -165,4 +288,4 @@ MIT
## Author
molodetz — DevPlace Code
molodetz -- DevPlace Code