## Implementation Plan: AI Content Correction Markdown Preservation ### Overview The corruption occurs because the system prompt in `services/correction.py` (line 105) contains a potentially conflicting response-format instruction ("no code fences") adjacent to a content‑preservation instruction, and because `services/ai_modifier.py` (line 34) lacks any code‑block preservation instruction at all. The fix requires two prompt changes plus new tests. ### Changes #### 1. Harden the correction system prompt (`services/correction.py`) **File:** `services/correction.py` **Location:** `correct_text()` function, the system prompt (around line 105) **Current (abbreviated):** ```python system_prompt = f"""{_default_system_prompt} ... - no code fences - Preserve ... markdown """ ``` **Replace with:** ```python system_prompt = f"""{_default_system_prompt} ... - Do not modify content inside fenced code blocks (triple backticks). Leave the entire code block, including its opening language identifier (e.g., ```python), completely unchanged. - Do not add, remove, or alter any markdown code fences in the text. - Preserve all existing markdown structures (headings, lists, tables, links, images, etc.) exactly as written. """ ``` Remove the ambiguous `- no code fences` line entirely. #### 2. Harden the modifier system prompt (`services/ai_modifier.py`) **File:** `services/ai_modifier.py` **Location:** `modify_text()` function, system prompt (around line 34) **Current (the prompt lacks any preservation instruction):** Add the same three bullet points above, inserted after the existing role instruction. #### 3. Add end‑to‑end test for Markdown preservation **File:** `tests/e2e/post.py` (add a new test method or a separate file under `tests/e2e/profile/`) **Test case:** - Create a user with AI correction enabled. - Create a post containing: - A paragraph of normal prose. - A fenced code block with a language identifier (e.g., ````python\nprint("hello")\n````). - Trigger the correction pass (e.g., by calling the API endpoint or simulating the `schedule_correction` path). - Fetch the corrected post and assert: - The fenced code block is still present with the same language identifier. - The code inside the block is unchanged. - The prose outside the block may have been corrected (acceptable) but no code fences are added/removed. Also add a similar test for the modifier path. **Note:** If the test infrastructure requires mocking the AI gateway (to avoid live API calls), add a mock that returns a corrected version where the code block is preserved exactly. The assertion should still verify the prompt sent to the gateway contains the new preservation instructions (optional but recommended). #### 4. Update documentation (minor) **File:** `templates/docs/ai-correction.html` Lines 23–24 already claim code blocks are preserved; no change needed there. However, after fixing the prompt, the documentation will be accurate. Optionally add a note that the system prompt has been hardened. #### 5. Environment setup for verification The project’s check commands (`pip install -e '.[dev]' -q && make test-unit` and `pip install -q ruff && ruff check .`) may fail on systems with PEP 668 protection. Therefore, before running the checks, ensure a virtual environment is active: ```bash python3 -m venv .venv source .venv/bin/activate pip install --upgrade pip setuptools wheel pip install -e '.[dev]' -q make test-unit pip install -q ruff ruff check . ``` If the `make test-unit` target is not available or fails, fall back to direct pytest invocation: ```bash python -m pytest tests/unit/ -x -q ``` The verification commands above must pass. ### Definition of Done - [ ] The system prompt in `services/correction.py` no longer contains the ambiguous `- no code fences` line and explicitly instructs the model to leave fenced code blocks and their language identifiers unchanged. - [ ] The system prompt in `services/ai_modifier.py` includes the same explicit code‑block preservation instruction. - [ ] At least one new test (in `tests/e2e/` or `tests/api/`) verifies that content with a fenced code block survives the correction pipeline with the code block and language identifier intact. - [ ] All existing unit tests pass: `pip install -e '.[dev]' -q && make test-unit` returns exit code 0. - [ ] Lint passes: `pip install -q ruff && ruff check .` returns exit code 0. - [ ] The documentation at `templates/docs/ai-correction.html` accurately reflects the behaviour (no contradiction remains).