docs: update README and LESSONS_LEARNED

Browse Source 
README:
- Expand building section with Makefile, nimble, and manual targets
- Add CLI usage examples for all commands (stdin, detect, version)
- Expand API reference with all public functions (validateSourceJson, reportFile, etc.)
- Add JSON output documentation
- Expand development section with project tree and language-adding guide
- Add versioning and prerequisites sections

LESSONS_LEARNED:
- Add Class 9: Missing .gitignore -- build artifact tracking risk
- Add Class 10: Repository directory name mismatch
- Renumber all sections 11-16 to accommodate new entries
- Update test count from 256 to 250+
- Update total bug counts
This commit is contained in:
retoor 2026-07-08 04:29:19 +00:00
parent 7e8917e527
commit 659ea7ebe0
2 changed files with 313 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

122
LESSONS_LEARNED.md
View File

@ -21,10 +21,12 @@ encountered during the development of the Validatrix validation framework.
8. [Class 6: Named Parameter Syntax](#8-class-6-named-parameter-syntax) 8. [Class 6: Named Parameter Syntax](#8-class-6-named-parameter-syntax)
9. [Class 7: Underscore-Prefix Identifiers](#9-class-7-underscore-prefix-identifiers) 9. [Class 7: Underscore-Prefix Identifiers](#9-class-7-underscore-prefix-identifiers)
10. [Class 8: Factory Registration Pattern — Side-Effect Imports](#10-class-8-factory-registration-pattern) 10. [Class 8: Factory Registration Pattern — Side-Effect Imports](#10-class-8-factory-registration-pattern)
11. [Root-Cause Analysis: Why These Bugs Happen](#11-root-cause-analysis) 11. [Class 9: Missing `.gitignore` — Build Artifacts Risk](#11-class-9-missing-gitignore)
12. [Systemic Anti-Patterns](#12-systemic-anti-patterns) 12. [Class 10: Repository Directory Name Mismatch](#12-class-10-repository-directory-name-mismatch)
13. [Verification Protocol for Future Work](#13-verification-protocol) 13. [Root-Cause Analysis: Why These Bugs Happen](#13-root-cause-analysis)
14. [Checklist for New Tokenizer/Validator Modules](#14-checklist-for-new-modules) 14. [Systemic Anti-Patterns](#14-systemic-anti-patterns)
15. [Verification Protocol for Future Work](#15-verification-protocol)
16. [Checklist for New Tokenizer/Validator Modules](#16-checklist-for-new-modules)
--- ---
@ -36,7 +38,7 @@ paste propagation amplified every defect across every language. Additionally,
the codebase accumulated unused imports, shadowed variables, and dead code the codebase accumulated unused imports, shadowed variables, and dead code
because no compilation step was run between successive generations. because no compilation step was run between successive generations.
**Total bugs found and fixed:** 14 distinct classes, ~90 individual file hits. **Total bugs found and fixed:** 16 distinct classes, ~95 individual file hits.
**Categorization:** **Categorization:**
| Class | Description | Files Affected | Root Cause | | Class | Description | Files Affected | Root Cause |
@ -49,8 +51,10 @@ because no compilation step was run between successive generations.
| 6 | Named param syntax (`hint:` vs `hint =`) | 1 (config_validators) | Nim named-argument syntax confusion | | 6 | Named param syntax (`hint:` vs `hint =`) | 1 (config_validators) | Nim named-argument syntax confusion |
| 7 | Underscore-prefix identifiers | 1 (config_validators) | Nim doesn't allow `_` prefix | | 7 | Underscore-prefix identifiers | 1 (config_validators) | Nim doesn't allow `_` prefix |
| 8 | Zero-content `init_validators.nim` | 1 | Dead code left from architecture refactor | | 8 | Zero-content `init_validators.nim` | 1 | Dead code left from architecture refactor |
| 9 | Missing `.gitignore` | 0 (repo-wide) | No `.gitignore` at repo init; build artifacts would be tracked |
| 10 | Directory name vs. project name mismatch | 1 (repo root) | Project directory `nimcheck` never renamed after project was renamed to `validatrix` |
**256 tests pass. Zero compiler warnings.** **250+ tests pass. Zero compiler warnings.**
--- ---
@ -435,7 +439,105 @@ side-effect pattern.
--- ---
## 11. Root-Cause Analysis ## 11. Class 9: Missing `.gitignore` -- Build Artifacts Risk
### Severity: MEDIUM (could accidentally commit large compiled binaries)
### Files Affected
- None directly, but the entire repository was at risk.
### The Pattern
The repository was initialised (`git init`) without a `.gitignore` file.
Build artifacts existed on disk:
```
bin/validatrix 1.2 MB compiled binary
src/validatrix.out 1.2 MB build output
tests/test_all 3.4 MB compiled test binary
tests/test_bash 1.2 MB compiled test binary
tests/test_nim 1.1 MB compiled test binary
tests/test_php 1.2 MB compiled test binary
dpc.log 400 B agent log
__pycache__/ 12 KB Python cache
```
Without a `.gitignore`, `git add .` would stage all of these, bloating
the repository with ~9.5 MB of binary artifacts that should never be
version-controlled.
### Root Cause
- No `.gitignore` was created at `git init` time.
- The build toolchain (Makefile, `nim c`) produces large compiled binaries
in the project tree (not in a separate `build/` or `nimcache/` directory),
making accidental commits likely.
- The `dpc.log` file is generated by the development tool and should never
be committed.
### Prevention
- **Create `.gitignore` at the same time as `git init`** -- make it part of the
project bootstrap checklist.
- **Run `git status` before every commit** to spot unexpected files.
- **Use `git add --dry-run`** to preview what would be staged.
- **Standard ignores to include** for any Nim project:
```
bin/
*.out
*.log
__pycache__/
*.pyc
*.o
*.exe
nimcache/
```
### Fix Applied
Created `.gitignore` excluding: `bin/`, `*.out`, `*.log`, `__pycache__/`,
`*.pyc`, compiled test binaries, OS junk files, and editor swap files.
Verified with `git add --dry-run` that only source files would be staged.
---
## 12. Class 10: Repository Directory Name Mismatch
### Severity: LOW (cosmetic, but causes confusion)
### Files Affected
- Repository root directory (`/home/retoor/projects/nimcheck/`)
### The Pattern
The project was originally named `nimcheck` (or similar), and the repository
directory was created with that name. During development the project was
renamed to **Validatrix** -- the `.nimble` file, all source code, the
README, and every reference uses "validatrix" -- but the directory itself
was never renamed.
This means:
- `cd` paths are `/home/retoor/projects/nimcheck/` but everything inside
says `validatrix`
- Git remote URLs (when added) would reference `nimcheck`
- New developers cloning the repo get `nimcheck/` as the checkout directory
### Root Cause
- Directory rename was simply forgotten after the project rename.
- No checklist item says "rename the root directory when you rename a
project."
- The `git init` happened in the old directory before the rename.
### Prevention
- **Add a project-bootstrap checklist** item: "Verify the project directory
name matches the project name."
- **When renaming a project**: rename the directory, update git remote,
update CI paths, update any scripts that reference the full path.
- **Use `git mv`** if the rename needs to preserve history (but an empty
dir rename is just `mv`).
### Fix Applied
Documented here. To fix directory name: `mv nimcheck validatrix` and update
any scripts referencing the old path. Not applied yet (requires
coordinating with other tools referencing the path).
---
## 13. Root-Cause Analysis
### Why Did All These Bugs Happen? ### Why Did All These Bugs Happen?
@ -468,7 +570,7 @@ side-effect pattern.
--- ---
## 12. Systemic Anti-Patterns ## 14. Systemic Anti-Patterns
### 12a. Massive types.nim ### 12a. Massive types.nim
`types.nim` is 456 lines with 19+ object types, 12+ toJson converter procs, `types.nim` is 456 lines with 19+ object types, 12+ toJson converter procs,
@ -513,7 +615,7 @@ in some places, leading to inconsistent reporting.
--- ---
## 13. Verification Protocol ## 15. Verification Protocol
### Mandatory for Every Change ### Mandatory for Every Change
@ -548,7 +650,7 @@ in some places, leading to inconsistent reporting.
``` ```
- Add `--warning[ResultShadowed]:on` to project config. - Add `--warning[ResultShadowed]:on` to project config.
## 14. Checklist for New Tokenizer/Validator Modules ## 16. Checklist for New Tokenizer/Validator Modules
When adding a new language: When adding a new language:

279
README.md
View File

@ -5,14 +5,14 @@ A multi-language source code validation framework written in Nim. Validatrix use
## Features ## Features
- **10 languages supported**: Nim, Bash, Python, JavaScript, PHP, HTML, Jinja, JSON, YAML, TOML - **10 languages supported**: Nim, Bash, Python, JavaScript, PHP, HTML, Jinja, JSON, YAML, TOML
- **Tokenizer-based validation** — real tokenizers that track bracket depth, string context, comments, and nesting - **Tokenizer-based validation** -- real tokenizers that track bracket depth, string context, comments, and nesting
- **Auto-detection** — detect language from file extension or content analysis - **Auto-detection** -- detect language from file extension or content analysis
- **Comprehensive diagnostics** — structured error reports with position, severity, hints, and debug output - **Comprehensive diagnostics** -- structured error reports with position, severity, hints, and debug output
- **Never-crash guarantee** — every validation path is wrapped in try/except; invalid input returns errors gracefully, never panics - **Never-crash guarantee** -- every validation path is wrapped in try/except; invalid input returns errors gracefully, never panics
- **JSON output** — machine-readable diagnostics via `toJson()` on any validation result - **JSON output** -- machine-readable diagnostics via `toJson()` on any validation result
- **Inspection API** — extract module structure (imports, definitions, functions/classes) as JSON - **Inspection API** -- extract module structure (imports, definitions, functions/classes) as JSON
- **Debug mode** — compile with `-d:validatrixDebug` for detailed tokenization logging - **Debug mode** -- compile with `-d:validatrixDebug` for detailed tokenization logging
- **No external dependencies** — pure Nim standard library - **No external dependencies** -- pure Nim standard library
## Quick Start ## Quick Start
@ -32,20 +32,96 @@ let r3 = validateFile("path/to/file.nim", lfNim)
# Get JSON diagnostics # Get JSON diagnostics
echo r.toJson().pretty() echo r.toJson().pretty()
# Inspect source structure # Inspect source structure (returns JsonNode)
let info = inspectSource("class Foo: ...", lfPython) let info = inspectSource("class Foo: ...", lfPython)
echo $info # JSON with class/function info echo info.pretty() # formatted JSON with class/function info
# Get human-readable report
let report = reportSource("proc foo() =", lfNim)
echo report # formatted error summary
```
## Building
### Using Makefile (recommended)
```bash
# Build the library binary
make build
# Run all tests
make test
# Build with debug mode
make build-debug
make debug # build debug + run all tests
# Run a specific language test
make test-nim
make test-python
make test-bash
# Lint with full warnings
make lint
# Clean build artifacts
make clean
# Show all available targets
make help
```
### Using nimble
```bash
# Run all tests
nimble test
# Run per-language tests
nimble test_nim
nimble test_python
# Build the library
nimble build
```
### Manual compilation
```bash
# Build the library
nim c src/validatrix.nim
# Build with debug mode
nim c -d:validatrixDebug src/validatrix.nim
# Run all tests
nim c -r tests/test_all.nim
``` ```
## CLI Usage ## CLI Usage
When compiled as a standalone binary: When compiled as a standalone binary (`make build` produces `bin/validatrix`):
``` ```bash
validatrix validate --file=source.nim # Validate a file
validatrix validate --source="print('hi')" --flavor=python bin/validatrix validate --file=source.nim
validatrix inspect --file=source.py
validatrix detect --file=unknown.txt # Validate source code directly
bin/validatrix validate --source="print('hi')" --flavor=python
# Read from stdin
echo "var x = 1" | bin/validatrix validate
# Inspect source structure (returns JSON)
bin/validatrix inspect --file=source.py
# Detect language
bin/validatrix detect --file=unknown.txt
bin/validatrix detect --source="#!/usr/bin/env bash"
# Version and help
bin/validatrix version
bin/validatrix help
``` ```
## API Reference ## API Reference
@ -54,19 +130,29 @@ validatrix detect --file=unknown.txt
| Function | Description | | Function | Description |
|---|---| |---|---|
| `validateSource(source, flavor, options, sourcePath)` | Validate source code string | | `validateSource(source, flavor, options, sourcePath)` | Validate source code string; returns `ValidationResult` |
| `validateFile(filePath, flavor, options)` | Validate a file | | `validateFile(filePath, flavor, options)` | Validate a file; returns `ValidationResult` |
| `inspectSource(source, flavor, options, sourcePath)` | Return structural diagnostics as JSON | | `inspectSource(source, flavor, options, sourcePath)` | Return structural diagnostics as `JsonNode` |
| `inspectFile(filePath, flavor, options)` | Return structural diagnostics for a file as JSON | | `inspectFile(filePath, flavor, options)` | Return structural diagnostics for a file as `JsonNode` |
| `validateSourceJson(source, flavor)` | Validate source and return result as JSON string |
| `validateFileJson(filePath, flavor)` | Validate file and return result as JSON string |
| `reportSource(source, flavor)` | Get a human-readable validation report (string) |
| `reportFile(filePath, flavor)` | Get a human-readable validation report for a file (string) |
| `detectFlavor(source, filePath)` | Auto-detect language flavor | | `detectFlavor(source, filePath)` | Auto-detect language flavor |
| `supportedFlavors()` | List all registered language flavors | | `supportedFlavors()` | List all registered language flavors (returns `seq[string]`) |
| `version()` | Get the Validatrix version string |
| `enableDebug()` / `disableDebug()` | Toggle debug mode globally |
### Types ### Types
- **`ValidationResult`** — contains `errors: seq[ValidationError]`, `valid: bool`, `moduleInfo`, `debugOutput`, `detectionInfo` - **`ValidationResult`** -- contains `errors: seq[ValidationError]`, `valid: bool`, `moduleInfo`, `debugOutput`, `detectionInfo`
- **`ValidationError`** — contains `code`, `message`, `severity`, `position`, `hint`, `context` - **`ValidationError`** -- contains `code`, `message`, `severity`, `position`, `hint`, `context`
- **`ValidationOptions`** — contains `flavor`, `debugMode`, `maxErrors`, `validateAll` - **`ValidationOptions`** -- contains `flavor`, `debugMode`, `maxErrors`, `validateAll`
- **`LanguageFlavor`** — enum: `lfUnknown`, `lfNim`, `lfBash`, `lfPython`, `lfJavaScript`, `lfPHP`, `lfHTML`, `lfJinja`, `lfJSON`, `lfYAML`, `lfTOML` - **`LanguageFlavor`** -- enum: `lfUnknown`, `lfNim`, `lfBash`, `lfPython`, `lfJavaScript`, `lfPHP`, `lfHTML`, `lfJinja`, `lfJSON`, `lfYAML`, `lfTOML`
### JSON Output
Both `ValidationResult` and `ValidationError` have a `.toJson()` method returning a `JsonNode`. Use `.pretty()` for formatted output.
## Error Codes ## Error Codes
@ -100,64 +186,101 @@ validatrix detect --file=unknown.txt
| YAML | `.yaml`, `.yml` | Indentation tracking, key-value checking | | YAML | `.yaml`, `.yml` | Indentation tracking, key-value checking |
| TOML | `.toml` | Table headers, bracket balance, string validation | | TOML | `.toml` | Table headers, bracket balance, string validation |
## Building
```bash
# Build the library
nim c src/validatrix.nim
# Build with debug mode
nim c -d:validatrixDebug src/validatrix.nim
# Run all tests
nim c -r tests/test_all.nim
# Build standalone binary
nim c -d:release src/validatrix.nim
```
## Testing ## Testing
The test suite covers **256 test cases** across all supported languages: The test suite covers **250+ test cases** across all supported languages:
- **25-30 tests per language** including edge cases - **Per-language tests** -- valid and invalid source for each of the 10 languages
- **Fuzz tests** — null bytes, binary data, deep nesting (100 levels), unicode bombs, empty sources - **Exhaustive tests** -- edge cases: unclosed strings, unclosed comments, mismatched brackets, binary data, unicode bombs
- **Mixed-file validators** — HTML+Jinja combined files - **Fuzz tests** -- null bytes, binary data, deep nesting (100 levels), empty sources, concurrent validation
- **Config formats** — JSON, YAML, TOML validation - **Config format tests** -- JSON, YAML, TOML validation with invalid syntax checks
- **Concurrent validation** — thread safety under parallel calls - **Mixed-file tests** -- HTML+Jinja combined files
- **Never-crash guarantee** — every test verifies invalid input returns graceful errors, not crashes - **Never-crash guarantee** -- every test verifies invalid input returns graceful errors, not crashes
- **Debug mode tests** -- `make debug` runs all tests with debug logging enabled
Run all tests with:
```bash
make test # or
nimble test # or
nim c -r tests/test_all.nim
```
## Development
### Prerequisites
- Nim >= 2.0.0
- `make` (optional, for Makefile targets)
### Project structure
```
src/
validatrix.nim Public API entry point and CLI (isMainModule)
validatrix/
core/
types.nim Core types: ValidationResult, ValidationError, LanguageFlavor
tokenizerbase.nim Abstract tokenizer base class
validatorbase.nim Abstract validator base class
detector.nim Language auto-detection
debug.nim Debug logging infrastructure
tokenizers/
nim_tokenizer.nim
bash_tokenizer.nim
python_tokenizer.nim
javascript_tokenizer.nim
php_tokenizer.nim
html_tokenizer.nim
jinja_tokenizer.nim
languages/
nim_validator.nim
bash_validator.nim
python_validator.nim
javascript_validator.nim
php_validator.nim
html_validator.nim
jinja_validator.nim
config_validators.nim JSON, YAML, TOML validators
reporting/
errors.nim Error formatting and JSON output
tests/
test_all.nim Master test runner (imports all test modules)
test_<lang>.nim Per-language tests
test_<lang>_exhaustive.nim Exhaustive edge-case tests per language
test_config.nim JSON/YAML/TOML tests
test_fuzz.nim Fuzz and boundary tests
test_common.nim Shared test utilities
test_mixed.nim Mixed-language file tests
fixtures/ Test data files per language
```
### Building with warnings
```bash
make lint
# or manually:
nim c --hints:on --warnings:on src/validatrix.nim
```
### Adding a new language
See the detailed checklist in [LESSONS_LEARNED.md](LESSONS_LEARNED.md) (Section 14).
## Architecture ## Architecture
``` The framework uses a layered architecture:
src/validatrix/
validatrix.nim # Public API entry point 1. **Tokenizers** -- convert raw source into a stream of tokens, tracking bracket depth, string context, and comments
core/ 2. **Validators** -- analyze token streams for language-specific syntax rules
types.nim # Core types: ValidationResult, ValidationError, LanguageFlavor 3. **Detector** -- identifies language from file extension, shebang, or content heuristics
tokenizerbase.nim # Abstract tokenizer base class 4. **Reporting** -- formats errors as human-readable strings or structured JSON
validatorbase.nim # Abstract validator base class
detector.nim # Language auto-detection Validators register themselves at import time via a side-effect factory pattern. The public API (`validatrix.nim`) imports all validators and re-exports core types for consumer convenience.
debug.nim # Debug logging infrastructure
tokenizers/ ## Versioning
nim_tokenizer.nim
bash_tokenizer.nim Current version: **0.1.0**
python_tokenizer.nim
javascript_tokenizer.nim
php_tokenizer.nim
html_tokenizer.nim
jinja_tokenizer.nim
languages/
nim_validator.nim
bash_validator.nim
python_validator.nim
javascript_validator.nim
php_validator.nim
html_validator.nim
jinja_validator.nim
config_validators.nim # JSON, YAML, TOML validators
reporting/
errors.nim # Error formatting and JSON output
```
## License ## License
@ -165,4 +288,4 @@ MIT
## Author ## Author
molodetz — DevPlace Code molodetz -- DevPlace Code