- Replace basic -O3 with -Ofast and add PGO instrumentation (fprofile-generate) and usage (fprofile-use) stages in the default 'build' target
- Introduce new 'build_quick' target for compilation without PGO
- Add numerous performance-oriented flags: fipa-pta, fipa-cp-clone, fdevirtualize-speculatively, fno-plt, fno-semantic-interposition, fomit-frame-pointer, fmerge-all-constants, fno-stack-protector, DNDEBUG, and floating-point relaxation flags
- Update linker flags to use flto=auto and fuse-linker-plugin
- Include training runs on spam, not_spam, and optional books directories during PGO instrumentation phase
- Add swift_isspam package with optimized main.swift using unsafe memory flags
- Extend CI workflow to install Swift 5.9.2 and include sisspam in build_all and benchmark
- Update Makefile with build_swift target and publish step for sisspam binary
- Add sisspam to .gitignore and README with author credit
- Enable Swift timing in bench.py alongside existing language implementations
Rewrite the isspam.c program to use memory-mapped files, a compact trie data structure for forbidden word lookup, and pthread-based parallel processing across up to 16 threads. Remove the old fixed-size buffer and linear forbidden word array in favor of a SIMD-accelerated scanning approach with AVX2 instructions. Update the Makefile to enable aggressive x86-64 optimization flags including -march=native, -flto, -ffast-math, and -mavx2 for maximum performance on supported hardware.
The Python benchmark using isspam.py was removed from the benchmark
run because its execution time was excessively slow compared to the
C++ and Rust implementations, skewing overall timing results.
Introduce a new `analyze_simd` function in main.rs using AVX2 256-bit registers for parallel text scanning, replacing the previous sequential analysis path. Add a stub `uring` module for future async I/O integration. Remove `core_affinity` and `memmap2` crate dependencies from Cargo.toml and Cargo.lock, along with their associated `libc`, `num_cpus`, `hermit-abi`, and `winapi` transitive dependencies. Comment out `mimalloc` and `io-uring` crate references. Uncomment file reading timing instrumentation and add a `bench_rust_only` Makefile target for direct benchmark execution.
The CI workflow was failing because the Rust toolchain installed via rustup
was not available in subsequent steps. Adding `. "$HOME/.cargo/env"` ensures
the cargo and rustc binaries are on PATH before the build commands execute.
Move Stats processing methods into dedicated parser module with separate muncher and for_loops implementations, update .gitignore to exclude test_books directory, and simplify README to remove outdated benchmark instructions
The change shifts `fs::read_to_string` from inside each spawned async task into the main synchronous loop before spawning. This eliminates redundant sequential file reads across concurrent tasks, yielding a ~30ms improvement on NVMe SSD workloads. The `Stats::process` method now accepts a pre-read `&str` instead of a file path, and failed file counts are tracked directly in the main loop.
Extract Stats struct into its own module with public fields and AddAssign/Display impls; introduce Trie data structure with insert and contains methods; refactor main.rs to use thread-local LazyCell<Trie> populated from the same forbidden word set; update README with trie benchmark results showing 1588ms vs 4737ms original.
Implement a character-level muncher parser as an alternative to the existing for-loop approach for text statistics. The muncher processes text by tracking whitespace and dot states to count words, sentences, capitalized words, and numeric characters. Benchmark results show the muncher completes in 504ms versus 5033ms for for-loops, though with different counts for capitalized words (16% vs 2%) and forbidden words (0% vs 2%) due to implementation differences. The README now includes both benchmark comparisons and data integrity notes documenting the statistical discrepancies between the two methods.
Add rayon 1.10.0 as a dependency in Cargo.toml and refactor Stats processing into a dedicated method that uses `par_iter()` for parallel file processing. Configure release profile with thin LTO, disabled debug symbols, and abort-on-panic for maximum runtime performance. Update README with build and benchmark integration instructions for the jest_rust project.
Remove the intermediate jisspam subdirectory within jest_rust, moving Cargo.lock, .gitignore, and test_files (spam1.txt, spam2.txt, spam3.txt, not_spam.txt) up one level. Update Makefile build_jest target to compile directly from jest_rust instead of jest_rust/jisspam and copy binary to parent directory.
Add new Rust-based spam detection project under jest_rust/jisspam with Cargo.toml, main.rs implementing forbidden word analysis, and test files for spam/not-spam classification. Include build_jest target in Makefile, add jisspam binary to .gitignore, and integrate its execution into bench.py benchmark suite alongside existing C, CPP, and Python implementations.
The `-ltbb` flag was removed from the `build_borded_cpp` target's g++ command
in the Makefile, eliminating the dependency on Intel Threading Building Blocks
for the borded C++ executable compilation.
- Swapped benchmark order in bench.py to run Rust before C for consistent measurement
- Added new main3.cpp source with parallel TBB-based implementation using async I/O and CRC32
- Updated Makefile to link borded_cpp_exec against libtbb and compile from main3.cpp
- Modified CMakeLists.txt to add main3.cpp target with TBB linkage and disable -Werror
Replace substring-based bad word detection with exact match in check_word, and create main2.cpp as a variant that removes the AnalysisResult struct entirely while preserving all other logic.
Add a new Makefile target to copy the Python isspam script into the project root, and extend the benchmark script to measure execution time for the Python version alongside existing C and C++ implementations.
Implement a new Python script `isspam.py` that analyzes text files for spam indicators by counting total words, capitalized words, sentences, numbers, and occurrences of a predefined list of 40 forbidden words. The script supports multi-file input via command-line arguments, reads files in 1024-byte chunks, and uses threading for concurrent analysis.
Add borded_cpp_exec to .gitignore, include build_borded_cpp in Makefile all target, and extend bench.py with Borded CPP timing. Change borded_cpp Dockerfile base from alpine to gcc:latest with apt-based dependencies.
Add a new `build_borded_cpp` make target that compiles the borded_cpp
version of isspam using C++23 with `-Ofast` optimization. Also reduce
the CMake minimum required version from 3.30 to 3.25 in the borded_cpp
CMakeLists.txt to improve compatibility with older CMake installations.
Introduce a Docker-based development setup for the borded_cpp project. The Dockerfile uses Alpine Linux with build-base, SQLite, Jansson, and CMake. The compose.yml mounts the project directory and a shared books volume, runs doit.sh which cleans, configures, and builds the project via CMake.
Replace the `import std;` module directive with explicit C++17 standard library includes
and substitute `std::println` calls with `std::cout` equivalents to ensure compatibility
with older compiler toolchains. Also add an overloaded `operator<<` for `AnalysisResult`
to enable formatted output via iostream, and convert `std::wifstream::open` to accept a
`std::string` instead of `std::string_view` for broader C++17 support.
Implement the first version of the borded spam parser as a C++26 project. The commit introduces a complete project skeleton including a `.gitignore` file for CMake and IDE artifacts, a `CMakeLists.txt` with strict warning flags and MSVC compatibility, and the core `src/main.cpp` containing the spam detection logic. The parser reads text files, tokenizes words, counts sentences and capitalized words, and flags occurrences from a predefined list of 35 forbidden spam-related terms (e.g., "crypto", "bitcoin", "recovery", "hack"). Results are aggregated via an `AnalysisResult` struct with a custom `operator+` for combining multiple file analyses.
Add a new C++ implementation of the isspam text analysis tool in retoor_c/isspam.cpp, mirroring the C version's functionality with thread-based file analysis. Update the Makefile with a build_cpp target and add the new binary to .gitignore. Modify bench.py to include timing for the C++ executable. Fix the C tokenizer's delimiter string from punctuation-only to include all standard whitespace characters (\f\v\r\n\t).
Replace the fork-based parallel execution in main() with POSIX threads, adding a new analyze_file_thread wrapper function. The change removes per-file printf output and eliminates the separate child process for each argument, instead creating a thread per file, joining all threads, and freeing the allocated thread array. This improves resource sharing and avoids process overhead for concurrent file analysis.
The diff shows a single character change in the Makefile's publish target, where the wget command for downloading the publish script had a space between the double dash and the quiet flag (`-- quiet`), which has been corrected to the proper format (`--quiet`). This fix ensures the wget command executes correctly without flag parsing errors.
- Bump .build-trigger timestamp from 2014-12-02 15:22 to 15:26 to force rebuild
- Add --quiet flag to both wget calls in publish target to suppress download progress output
- Remove duplicate `source $HOME/.cargo/env && make` step that was redundant with later build commands
- Move cargo environment sourcing to `build_risspam` step where it is actually needed for compilation
- Keep `make build` and `make run` steps without explicit cargo env to rely on default PATH setup
The __pycache__ directory was being tracked by git, causing compiled Python bytecode files like env.cpython-312.pyc to appear in the repository. This change adds __pycache__ to .gitignore and removes the existing pyc file from version control.
Rewrite `get_sentences` to return a `usize` count instead of `Vec<&str>`, removing the trailing-dot cleanup logic. Replace `get_words` and `get_capitalized_words` with a single `get_words` function that mutates output counters for total words, capitalized words, and forbidden words, iterating directly over the content. Add release profile optimizations with thin LTO and abort-on-panic in Cargo.toml.
Add fork() call in retoor_c/isspam.c main loop to spawn child process for each input file, allowing parallel analysis. Update bench.py to pass -p flag to Rust binary risspam for enabling parallel execution mode. Rebuild isspam binary to reflect the C source changes.
Add 'books' to .gitignore to exclude it from version control, and include generated Rust build fingerprint files for crossbeam-deque, crossbeam-epoch, crossbeam-utils, either, and rayon dependencies under 12bitfloat_rust/risspam/target/release/.fingerprint/
- Add 'benchmark' target to Makefile that extracts books archive and runs bench.py
- Reorder includes in isspam.c (rstr.h before rstring_list.h)
- Replace file_exists() with case-insensitive stricmp() for forbidden word matching
- Remove unused flags (show_capitalized, show_sentences, etc.) and stripws() function
- Add memory usage comment and fix trailing newline in rstr.h header guard
Refactor `get_capitalized_words` and `get_forbidden_words` to accept pre-extracted word lists instead of raw content strings, eliminating duplicate calls to `get_words` and `get_sentences`. Move word extraction into the caller `analyze` to compute word count once and reuse the list across all analysis steps, reducing memory allocations and freeing overhead.
- Update Cargo.lock version from 3 to 4
- Add rayon 1.10.0 with core, crossbeam-deque, crossbeam-epoch, crossbeam-utils, and either dependencies
- Register rayon as a dependency in the risspam package
- Update compiled release binary for risspam and top-level risspam binary
- Add rayon dependency to Cargo.toml and import parallel prelude in both isspam_v3.rs and risspam/src/main.rs
- Change get_numbers signature to take &str instead of &str and return Vec<&str> instead of Vec<String>, removing unnecessary allocation
- Guard word_count_per_sentence calculation against empty sentences list to prevent division by zero
- Enable file reading from command-line arguments in isspam_v1.rs by uncommenting and fixing the arg loop with skip(1)
- Remove embedded SPAM1 static test data from isspam_v1.rs and create new isspam_v2.rs with additional analysis functions
Improve readability of the README by inserting blank lines between major sections
and reformatting the Valgrind status paragraph to separate the Rust variant comment
from the date line, ensuring consistent markdown structure throughout the document.
- Document that repository contains both Rust (risspam) and C (isspam) implementations
- Update build and run examples to distinguish between the two versions
- Clarify stdin usage works only for isspam and valgrind status applies to isspam only
- Add echo messages for both build targets to indicate which project is being compiled
- Update isspam.c path to reflect new retoor_c/ subdirectory structure
- Keep binary output location unchanged at project root
The CI workflow was sourcing the cargo binary directory ($HOME/.cargo/bin) instead of the environment setup script ($HOME/.cargo/env), which prevented proper initialization of Rust toolchain paths during build steps.
Add curl to apt install list and install Rust nightly via rustup in the CI workflow. Update Makefile all target to include build_risspam before run_risspam, ensuring the Rust-based risspam binary is compiled before execution.
Implement core spam analysis functions including content cleaning, sentence splitting, capitalized word detection, and forbidden word filtering across three Rust source files with nightly let_chains feature
Add build_risspam target that compiles the Rust risspam binary from
12bitfloat_rust/risspam using cargo run --release, and add run_risspam
target with spam and not_spam test execution via run_spam_risspam and
run_not_spam_risspam recipes.
Introduce a global `total` counter incremented on each `analyze()` call, printing the file sequence number. Extend the analysis block to compute and display capitalized and forbidden word percentages when forbidden words exist. Add a new Python script `totals.py` that reads the application output and computes average percentages across all processed files.
The commit updates the README example output to reflect a spam file analysis with
detailed forbidden word breakdown and increased memory usage. It also fixes a
potential division by zero in `analyze()` by guarding `sentences->count` with a
ternary operator that defaults to 1 when zero, ensuring stable output for edge
cases. The binary `isspam` is rebuilt to include this fix.
- Remove the apostrophe flag from sprintf format specifier in rmalloc_lld_format to avoid locale-dependent thousands separator, which caused unexpected commas in output
- Increase static buffer size in rmalloc_stats from 200 to 300 bytes to prevent potential buffer overflow when formatting memory statistics with larger values
Introduce four boolean flags (show_capitalized, show_sentences, show_words, show_numbers) that control whether their respective analysis results are printed via sld(), replacing the previously commented-out calls. Add a file_exists() utility function that checks if a given path points to an existing readable file. Also append "publish" to .gitignore.
Add a new code block in the README under the "Build" section showing how to run the build with valgrind for memory checking. The block includes a comment that valgrind must be installed and shows the `make valgrind` command. This helps developers easily find and use the memory check build target.
Add a new "Example output" section to the README that demonstrates the expected output format when running the isspam tool on a non-spam file, including metrics for capitalized words, sentences, word counts, and memory usage statistics.
Add the complete source tree for the isspam spam detection utility, including the main C implementation (isspam.c), custom memory allocator (rmalloc.h), string builder (rstr.h), dynamic string list (rstring_list.h), build configuration (Makefile, .clang-format), CI workflow (.gitea/workflows/build.yaml), and sample test data (spam/examole_spam3.txt, not_spam/correct1.txt). The core logic defines a list of forbidden words and provides functions for tokenizing input, removing preserved characters, and checking content against the spam keyword set.
retoor
12bitfloat
12bitfloat
JestDotty
BordedDev