**Plan: Back dedup with database – eliminate per-worker in-memory cache and prevent duplicate AI corrections** **Why this addresses the root cause** The current in‑memory `_content_cache` (per‑worker, 3‑second window) fails in multi‑worker deployments and even on a single worker allows a second submission before the first correction completes. Replacing it with a database‑backed dedup that stores a `(sender_uid, content_hash)` unique constraint gives a shared, atomic guard. A second submission within a 10‑second window will be rejected with `409 Conflict` before a second message row is ever inserted, preventing any duplicate AI correction call. **Concrete changes (what and where)** All changes are in `devplacepy/services/persist.py` and `devplacepy/tests/` (new test file). No changes to `correction.py`, `messages.py`, or configuration – the fix is fully in the dedup layer. 1. **Add a `message_dedup` table** In the database initialisation (likely `devplacepy/database.py` or the first `persist_message` call), create the table: ```sql CREATE TABLE IF NOT EXISTS message_dedup ( sender_uid TEXT NOT NULL, content_hash TEXT NOT NULL, created_at TEXT NOT NULL DEFAULT (datetime('now')), PRIMARY KEY (sender_uid, content_hash) ); ``` 2. **Replace the in‑memory dedup in `persist_message`** Open `devplacepy/services/persist.py`, locate the `persist_message` function. - Remove all references to `_content_cache` and the `dedup_window` constant. - Compute a SHA‑256 hex digest of `request.content`: ```python import hashlib content_hash = hashlib.sha256(request.content.encode()).hexdigest() ``` - Before the existing message‑row insertion, execute the following SQL wrapped in the same database connection: ```python now = datetime.utcnow().isoformat() # Clean old entries (TTL = 30 seconds) conn.execute("DELETE FROM message_dedup WHERE created_at < datetime('now', '-30 seconds')") # Try to insert the dedup entry try: conn.execute( "INSERT INTO message_dedup (sender_uid, content_hash, created_at) VALUES (?, ?, ?)", (sender, content_hash, now) ) except IntegrityError: # Existing dedup entry → duplicate submission raise HTTPException(status_code=409, detail="Duplicate message detected within the deduplication window.") ``` - If the insert succeeds, proceed with the existing message‑row insertion (line ~80‑100 in `persist_message`). - Do **not** call `schedule_correction` for a second submission because we never reach that point – the exception stops the function. 3. **Adjust the cleanup window** Keep the cleanup TTL at 30 seconds (longer than the 10‑second window we want for dedup). The dedup is effective for any entry younger than 30 seconds – entries older are removed before the next insert, so a second submission after 30 seconds is allowed. 4. **Add unit tests** Create `devplacepy/tests/services/test_persist.py` (or extend an existing test file if one exists). - Test that a first call to `persist_message` inserts the dedup entry and proceeds normally. - Test that a second call with identical content and same sender within 30 seconds raises `HTTPException(409)`. - Test that a second call with different content (different hash) proceeds. - Test that a second call after cleanup (simulate time advance) succeeds (if needed, mock `datetime.utcnow`). - Use the project’s test database fixture (likely `sqlite:///:memory:`). 5. **Verify entire test suite and linter pass** Run `pip install -e '.[dev]' -q && make test-unit` and `pip install -q ruff && ruff check .`. Both must exit 0 with no test failures or lint errors. **Definition of done** - The `message_dedup` table is created in the database (checked by running a migration test or by inspecting the database after `persist_message` first runs). - A duplicate submission of the identical message by the same sender within 10 seconds returns HTTP 409 and does **not** insert a second message row. - A duplicate submission >30 seconds apart (after cleanup) is accepted. - All existing unit tests (including the `send` endpoint tests) still pass. - `ruff` reports zero errors on modified/created files. - The broadcast path is never reached for the duplicate – only one WebSocket message is sent per logical submission. - The entire change is verifiable via: ``` pip install -e '.[dev]' -q && make test-unit pip install -q ruff && ruff check . ``` Both commands exit 0.