156 lines
6.1 KiB
Markdown
156 lines
6.1 KiB
Markdown
|
# Nimcheck — Project Status
|
|||
|
|
|
|||
|
|
**Version:** 0.1.0
|
|||
|
|
**Default branch:** `master`
|
|||
|
|
**Remote:** `https://retoor.molodetz.nl/retoor/nimcheck.git`
|
|||
|
|
**Last verified:** 2026-07-15 — `nimble build` and `nimble test` pass (~11s, no crashes)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Summary
|
|||
|
|
|
|||
|
|
Nimcheck is **feature-complete for v0.1.0**. All planned languages have tokenizers, validators, fixtures, and tests. Critical crash bugs (infinite tokenizer loops → OOM) are fixed. The project is ready to use as a library, CLI, or CI gate.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## What Is Done
|
|||
|
|
|
|||
|
|
### Core framework
|
|||
|
|
|
|||
|
|
| Component | Status | Notes |
|
|||
|
|
|-----------|--------|-------|
|
|||
|
|
| Tokenizer base (`tokenizerbase.nim`) | Done | Position tracking, bracket stack, `closeBracket` advance, `finishTokenizeStep` stall guard |
|
|||
|
|
| Validator base (`validatorbase.nim`) | Done | Pipeline, factory, registry for all flavors |
|
|||
|
|
| Detector (`detector.nim`) | Done | Extension, shebang, content scoring; false-positive filters (entropy, margin) |
|
|||
|
|
| Public API (`nimcheck.nim`) | Done | `validateSource`, `validateFile`, `inspectSource`, `detectFlavor`, CLI |
|
|||
|
|
| Error reporting (`reporting/errors.nim`) | Done | Human + JSON output |
|
|||
|
|
| Never-crash guarantee | Done | All paths wrapped; fuzz-tested |
|
|||
|
|
|
|||
|
|
### Languages with full validator support (28 flavors, 40+ registry aliases)
|
|||
|
|
|
|||
|
|
| Group | Flavors | Tokenizer module | Validator module |
|
|||
|
|
|-------|---------|------------------|------------------|
|
|||
|
|
| Nim | `nim`, `nims`, `nimble` | `nim_tokenizer.nim` | `nim_validator.nim` |
|
|||
|
|
| Shell | `bash`, `sh`, `shell` | `bash_tokenizer.nim` | `bash_validator.nim` |
|
|||
|
|
| Python | `python`, `py` | `python_tokenizer.nim` | `python_validator.nim` |
|
|||
|
|
| JS / TS | `javascript`, `js`, `typescript`, `ts` | `javascript_tokenizer.nim`, `type_xml_tokenizer.nim` | `javascript_validator.nim`, `type_xml_validator.nim` |
|
|||
|
|
| PHP | `php`, `phtml` | `php_tokenizer.nim` | `php_validator.nim` |
|
|||
|
|
| HTML / XML | `html`, `htm`, `xml`, `xsd`, `xslt`, `svg` | `html_tokenizer.nim`, `type_xml_tokenizer.nim` | `html_validator.nim`, `type_xml_validator.nim` |
|
|||
|
|
| Jinja | `jinja`, `jinja2`, `j2` | `jinja_tokenizer.nim` | `jinja_validator.nim` |
|
|||
|
|
| Config | `json`, `jsonc`, `yaml`, `yml`, `toml` | (parser / line-based) | `config_validators.nim` |
|
|||
|
|
| C family | `c`, `h`, `cpp`, `cxx`, `hpp`, `java`, `csharp`, `cs` | `cfamily_tokenizers.nim` | `cfamily_validators.nim` |
|
|||
|
|
| Systems / scripting | `go`, `rust`, `ruby`, `lua`, `swift`, `kotlin`, `kt` | `extended_tokenizers.nim`, `lang_tokenizers.nim` | `extended_validators.nim`, `lang_validators.nim` |
|
|||
|
|
| Web / data / ops | `css`, `sql`, `markdown`, `md`, `dockerfile`, `makefile` | `extended_tokenizers.nim` | `extended_validators.nim` |
|
|||
|
|
|
|||
|
|
### Tests
|
|||
|
|
|
|||
|
|
| Suite | Count | Status |
|
|||
|
|
|-------|-------|--------|
|
|||
|
|
| Per-language basic + exhaustive | 30+ modules | All pass |
|
|||
|
|
| Config (`test_config.nim`) | JSON/YAML/TOML | Pass |
|
|||
|
|
| Mixed files (`test_mixed.nim`) | HTML+Jinja etc. | Pass |
|
|||
|
|
| Fuzz (`test_fuzz.nim`) | Null bytes, 10k strings, 100-level nesting, concurrency | Pass |
|
|||
|
|
| Fixtures | `tests/fixtures/<lang>/` valid, invalid, edge_cases | Present for all new langs |
|
|||
|
|
|
|||
|
|
### CI / Gitea
|
|||
|
|
|
|||
|
|
| Item | Status |
|
|||
|
|
|------|--------|
|
|||
|
|
| Workflow `.gitea/workflows/ci.yml` | Done — lint, test matrix (Nim 2.0.8 + stable), build artifact |
|
|||
|
|
| Default branch triggers | `master` (primary), `main` and `develop` also accepted |
|
|||
|
|
| act_runner requirement | `ubuntu-latest` label — see [GITEA.md](GITEA.md) |
|
|||
|
|
|
|||
|
|
### Bug fixes shipped in v0.1.0
|
|||
|
|
|
|||
|
|
1. **`closeBracket()` did not advance `pos`** — caused infinite loops on `)`, `]`, `}` in CLike tokenizers → OOM kill.
|
|||
|
|
2. **Jinja lone `{`** — text collector stalled without advancing.
|
|||
|
|
3. **JS/TS double-advance** — caller + `closeBracket` both advanced.
|
|||
|
|
4. **`finishTokenizeStep`** — safety net if tokenizer position stalls.
|
|||
|
|
5. **Lua `for … do`** — `do` incorrectly opened nested block.
|
|||
|
|
6. **Ruby postfix `unless`** — modifier form incorrectly opened block.
|
|||
|
|
7. **Makefile test tabs** — `\t` in triple-quoted strings was backslash+t, not TAB.
|
|||
|
|
8. **SQL trailing comma** — `,\n)` not detected (whitespace between comma and paren).
|
|||
|
|
9. **`detectFlavor` scoring** — C++, Kotlin, Swift, CSS, Markdown samples now detect correctly.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## How To Verify (local)
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# From repo root
|
|||
|
|
nimble build # or: make build
|
|||
|
|
nimble test # or: make test (~11 seconds)
|
|||
|
|
make lint # strict compile check
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Expected: exit code 0, no `[FAILED]` lines.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## How To Use
|
|||
|
|
|
|||
|
|
### Library
|
|||
|
|
|
|||
|
|
```nim
|
|||
|
|
import nimcheck
|
|||
|
|
let r = validateFile("src/main.nim")
|
|||
|
|
echo r.valid
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### CLI
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
make build
|
|||
|
|
bin/nimcheck validate --file=script.py
|
|||
|
|
bin/nimcheck detect --file=unknown.txt
|
|||
|
|
bin/nimcheck inspect --file=main.go --json
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
See [README.md](../README.md) for full API and JSON schemas.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## What Is NOT Done (optional / future)
|
|||
|
|
|
|||
|
|
These are **not blockers** for v0.1.0:
|
|||
|
|
|
|||
|
|
| Item | Priority | Notes |
|
|||
|
|
|------|----------|-------|
|
|||
|
|
| Compiler-grade semantics | Low | Validators are tokenizer/structure checks, not full parsers |
|
|||
|
|
| `finishTokenizeStep` on every tokenizer | Low | Defense in depth; core paths covered |
|
|||
|
|
| Semantic analysis (undefined vars, types) | Future | Error codes E1004–E1009 reserved |
|
|||
|
|
| Coverage job in CI | Optional | Requires `nimcoverage`; manual dispatch only |
|
|||
|
|
| Rename `master` → `main` | Optional | Repo uses `master`; CI supports both |
|
|||
|
|
| Published nimble package | Future | `nimble install` from git works today |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Repository Hygiene
|
|||
|
|
|
|||
|
|
**Ignored by `.gitignore`:** compiled test binaries, `bin/`, `*.log`, `debug_*`, `test_hang*`, editor temp files.
|
|||
|
|
|
|||
|
|
**Do not commit:** local debug scripts, `test_run*.log`, choosenim caches.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Release Checklist (when bumping version)
|
|||
|
|
|
|||
|
|
1. Update `version` in `nimcheck.nimble` and `nimcheckVersion` in `src/nimcheck.nim`
|
|||
|
|
2. Update version line in `README.md` and this file
|
|||
|
|
3. `nimble build && nimble test`
|
|||
|
|
4. Tag: `git tag v0.2.0 && git push origin v0.2.0`
|
|||
|
|
5. Confirm Gitea Actions green on `master`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Quick Reference
|
|||
|
|
|
|||
|
|
| Task | Command |
|
|||
|
|
|------|---------|
|
|||
|
|
| Build | `make build` / `nimble build` |
|
|||
|
|
| All tests | `make test` / `nimble test` |
|
|||
|
|
| One language | `nim c -r tests/test_go.nim` |
|
|||
|
|
| Lint | `make lint` |
|
|||
|
|
| Clean | `make clean` |
|
|||
|
|
| CI locally | Same as lint + test |
|
|||
|
|
| Push | `git push origin master` |
|