chore: replace aiosqlite with dataset and add bounded seen set to crawler

This commit is contained in:
2025-12-28 05:03:12 +00:00
parent a48bc9f97b
commit c409ec4541
10 changed files with 802 additions and 351 deletions
+71 -51
View File
@@ -1,42 +1,55 @@
# Princess Bot - Usage and Configuration Guide
# Princess Bot
Author: retoor <retoor@molodetz.nl>
An automated social media interaction bot for the devRant platform. Monitors a target user's posts and generates LLM-powered responses.
## Overview
Princess.py is an automated social media interaction bot designed to monitor and respond to specific user-generated content (rants and comments) on a platform. It fetches new posts made by a target user, generates witty or devastating responses using a language model, and keeps track of responded messages to avoid duplicates.
Princess Bot monitors rants and comments from a specified user on devRant, generates contextual responses using the Grok language model, and posts replies automatically. The bot maintains state to prevent duplicate responses.
The bot operates continuously, periodically checking for new content and replying accordingly.
## Architecture
---
The bot operates on a polling model with the following components:
## How It Works
| Component | Description |
|-----------|-------------|
| Api | devRant API client for authentication and content retrieval |
| GrokAPIClient | LLM integration for response generation |
| AsyncDataSet | Async SQLite wrapper for state persistence |
1. **Initialization**: The bot initializes with user credentials, target username, and API keys.
2. **Login**: It logs into the platform via the provided API.
3. **Content Monitoring**: It fetches recent rants and comments made by the target user.
4. **Response Generation**: For new content (not responded to before), it generates a response using a language model (GrokAPIClient).
5. **Response Posting**: It prints the content and the generated reply.
6. **Tracking**: It records responded messages in a local database to prevent duplicate responses.
7. **Loop**: It repeats this process every 60 seconds.
## Usage
---
### Quick Start
## Configuration
```bash
make
```
The script uses a `.env` file to manage sensitive credentials and configurable properties. Below are the supported environment variables:
This creates a virtual environment, installs dependencies, and starts the bot.
### Required Environment Variables
### Manual Setup
| Property | Description | Example |
|----------------------|----------------------------------------------------------|-------------------------------------------|
| `USERNAME` | Your platform username. | `my_username` |
| `PASSWORD` | Your platform password. | `my_password` |
| `TARGET` | The username of the user to monitor. | `target_user` |
| `LLM_KEY` | API key for the language model (Grok API). | `your-grok-api-key` |
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e ../../.
pip install -r requirements.txt
python princess.py
```
## Setup Instructions
### Configuration
1. **Create a `.env` file** in the same directory as `princess.py`.
2. **Add the required variables** with your credentials and target info:
Create a `.env` file with the following variables:
| Variable | Description |
|----------|-------------|
| `USERNAME` | devRant account username |
| `PASSWORD` | devRant account password |
| `TARGET` | Username of the user to monitor |
| `LLM_KEY` | API key for Grok language model |
Example:
```env
USERNAME=your_username
@@ -45,35 +58,42 @@ TARGET=target_username
LLM_KEY=your_grok_api_key
```
3. **Install dependencies** (if not already installed):
### Stopping
Press `Ctrl+C` to terminate the bot.
## Data Storage
Uses SQLite via AsyncDataSet with:
- Responded message tracking for deduplication
- Persistent state across restarts
## Requirements
- Python 3.10+
- python-dotenv
- aiosqlite
- aiohttp (via parent devranta package)
## Cleanup
```bash
pip install python-dotenv
make clean
```
4. **Run the script**:
Removes the virtual environment. Database file (`princess.db`) is preserved.
## File Structure
```bash
python princess.py
```
---
## Notes
- The bot stores responded messages in a local SQLite database (`princess.db`) to avoid duplicate responses.
- It runs indefinitely, checking for new content every 60 seconds.
- Make sure your API keys and credentials are kept secure and not shared publicly.
---
## Summary
Princess.py is a social media response bot that:
- Monitors a specific user's posts.
- Generates witty responses using a language model.
- Keeps track of responses to prevent duplicates.
- Runs continuously with minimal setup.
princess/
├── princess.py # Main bot implementation
├── ads.py # AsyncDataSet database wrapper
├── grk.py # Grok API client
├── requirements.txt # Dependencies
├── Makefile # Build automation
├── .env # Configuration (create manually)
├── .venv/ # Virtual environment (created on first run)
└── princess.db # SQLite database (created on first run)
```