The timeout passed to asyncio.wait_for for the run_in_executor call fetching rants was raised to 15 seconds to accommodate slower responses from the underlying dr.get_rants API call.
The diff shows a single change in `src/drstats/sync.py` where `loop.run_in_executor(get_rants)` is corrected to `loop.run_in_executor(None, get_rants)`, adding the required `None` argument for the executor parameter to match the asyncio API signature.
The rants_of_user and posts_of_user views were missing a GROUP BY clause on the username column, causing the GROUP_CONCAT aggregations to produce incorrect concatenated results across all rows instead of per-user groupings. This fix adds the required GROUP BY username to both view definitions, ensuring each user's rants and comments are properly aggregated into separate rows.
The workflow previously only triggered on push events. This change adds a
scheduled trigger using a cron expression that runs every 30 minutes,
enabling periodic execution of the export statistics job independently
of code pushes.
- Increment active user count from 101 to 102 and total contributions from 2550 to 2557 in export/0_dataset.txt
- Refresh all export graphs (comments per day/hour/weekday, score per user/period) with data spanning 2024-11-07 to 2024-11-23
- Update view-comment_stats_per_hour.json with adjusted counts for hours 10, 11, 13, 14
- Rebuild dist/drstats-1.3.37 wheel and tarball, and update drstats.db binary
- Delete the sync_excempt Makefile target that was stubbed with a timeout warning
- Change export_stats to call dr.stats instead of dr.stats_all
- Remove extraneous blank lines in Makefile for cleaner formatting
This commit extends the README with a detailed analysis of a specific user (neo), including rank, contributions, ownership metrics, upvote statistics, and post length data. The new section provides anonymized insights into individual user behavior within the forum dataset, complementing the existing community-level statistics.
Rename the popularity rank column from 'rank' to 'rank_appreciation' in the SQL view
contributions_extended_ranked to make its meaning clearer. Update the corresponding
reference in dataset.py's statistics text generation to use the new column name.
Update the SQL view definition to replace the ambiguous `upvote_ratio` column name with the more descriptive `upvotes_per_post_on_average`, and update the ranked view to order by the new column name instead of the old one.
Update PKG-INFO metadata to remove UNKNOWN placeholders for Home-page and Platform fields, and add explicit Requires-Dist entries for aiohttp, dataset, matplotlib, and requests. Also trim trailing blank lines from the description section.
The Makefile's `all` target was missing the `sync` step, which has now been restored alongside a fix in `src/drstats/sync.py` where `asyncio.wait_for` incorrectly called `asyncio.run_in_executor` instead of the proper `loop.run_in_executor` to avoid event loop context errors during async execution.
The __main__.py and __init__.py files were inadvertently removed during a previous cleanup operation. This commit restores both files to their original state, ensuring the package can be properly executed as a module and imported as a package.
Remove all compiled Python cache files from the drstats package directory to prevent stale bytecode issues. Refactor the sync_rants function by extracting the rant processing logic into a private _sync_rants helper and wrap it with asyncio.wait_for using a 5-second timeout to handle rate-limited API responses gracefully, breaking the loop on timeout exceptions.
The previous code assumed every rant returned from the API would have a "comments" key, causing a KeyError crash when iterating over rants that had zero comments. This fix uses `.get("comments", [])` to safely default to an empty list, allowing the sync loop to continue without interruption.
Add raw query results for user contributions, mention counts, individual user posts, and view-comment statistics exported as text and JSON files under the export/ directory.
Introduce a new `get_views()` function in db.py that queries the sqlite_schema for user-defined views, and add a loop in dataset.py to export each view's query results as a JSON file. Also create a persistent `contributions_extended_ranked` view to replace the inline subquery in `get_contributions()`, and refactor `get_users()` to use a direct DISTINCT query instead of deduplicating in Python.
Implement rank calculation based on total score ordering and format score values with thousands separator for improved readability in the user statistics view.
Add two new Makefile targets for exporting statistics and mentions data from the dataset file, each prefixed with "===" markers in the dataset.py dump function to enable clean grep extraction. The export_statistics target filters lines containing "Statistics:" while export_mentions filters lines containing "times ment", both writing results to separate export files.
The clean step was removed from the default build pipeline to preserve historic statistics across builds, as deleted data still consumes git storage. Additionally, the dataset export filename was changed from dataset.txt to 0_dataset.txt, and the corresponding link in the README was updated to reflect this new path.
The CI export workflow previously redirected `make` output to `build_output.txt`.
This change removes the file redirection, allowing build output to flow directly
to the job's standard output stream.
Add .gitea/workflows/export.yaml for automated build on push, introduce `clean` target in Makefile to recreate export directory, remove stale `dump_text.py` from SOURCES.txt, and correct duplicate column definitions in the `user_stats` SQL view creation inside db.py.
Add new merge_images.py script that compiles all exported PNG graphs into a single composite image with 2-column layout and 480x320 per-image sizing. Introduce drstats/dataset.py with dump() function that outputs per-user contribution statistics and all rants/posts for LLM embedding. Rename the console script entry point from dr.rant_stats_all to dr.stats_all across setup.cfg, entry_points.txt, README.md, and PKG-INFO. Update Makefile to replace sync with sync_excempt target, add merge_images target, and add prerequisite reminders for export targets. Refactor db.py view creation queries to use compact string formatting. Redirect Duration class timing output and dataset.py printr calls to stderr.
Add dr.dataset console script entry point and implement dump function in new dataset module. Create SQL views for contributions, contributions_extended, rants_of_user, and posts_of_user in db.py to aggregate user activity metrics. Update Makefile with sync, export_stats, and export_dataset targets. Add timeout parameter to HTTP requests in devrant.py and remove demo code. Extend README with usage instructions for environment setup and available dr.* commands.
- Add credits section acknowledging Rohan Burke (coolq) for the devrant API wrapper
- Remove pirant dependency from setup.cfg and add matplotlib>=3.9.2
- Add new console scripts for sync and per-day/weekday/hour statistics
- Add new source files: db.py, devrant.py, sync.py, and updated statistics.py
- Include binary database file drstats.db and compiled Python cache files
Add dataset>=1.6.2 and pirant>=0.1.4.dev1 (via git URL) to install_requires in setup.cfg.
Rename the console_scripts entry point from 'utro = drstats.statistics:rant_stats' to 'dr.rant_stats = drstats.statistics:rant_stats'.
Update egg-info metadata files (PKG-INFO, requires.txt, entry_points.txt, SOURCES.txt, top_level.txt) to reflect new dependencies and package structure.
Initialize the dRStats repository with core project structure: add .gitignore excluding dist and venv directories, create Makefile for pip-based build and editable install, write README.md describing the devrant health monitoring purpose, define pyproject.toml using setuptools build backend, configure setup.cfg with metadata (name drstats, version 0.1.1, MIT license, aiohttp dependency) and console_scripts entry point mapping 'utro' to drstats.statistics:rant_stats, and include generated egg-info metadata files.
retoor