Fix #84: AI markdownifyer plugin corrupts valid Markdown structures #123

Open

typosaurus wants to merge 2 commits from typosaurus/ticket-84 into master

typosaurus commented

2026-07-23 04:20:48 +02:00

Collaborator

Resolves #84.

Plan

Implementation Plan: Protect Markdown Structure During AI Correction

Objective

Prevent the AI correction pipeline from corrupting valid Markdown structures (fenced code blocks, inline code, headings, lists, tables) by extracting code blocks before sending content to the LLM and reinserting them afterward.

Changes Required

1. Add Markdown structure preservation utility

File: devplacepy/services/markdown_preserve.py (new)

Implement a MarkdownPreserver class with:
- extract_blocks(text: str) -> tuple[str, list[str]]: Scans text for fenced code blocks (triple backticks, possibly with language identifier) and inline code spans (single backticks). Replaces each with a unique placeholder token (e.g., {%CODE_BLOCK_1%}). Returns modified text and list of original blocks.
- restore_blocks(text: str, blocks: list[str]) -> str: Replaces placeholders with original blocks.
Use regex that correctly handles nested backticks? Fenced code blocks can span multiple lines, but cannot contain the closing fence. Use re.finditer(r'```(?:\w+)?\n.*?```', text, re.DOTALL) and for inline code re.finditer(r'[^\n]+')`. Ensure placeholders do not conflict with existing content.
Consider also extracting other Markdown structures if needed? The ticket focuses on code blocks; start with fenced code and inline code. Add extensibility.

2. Modify correction pipeline to use preserver

File: devplacepy/services/correction.py

Import MarkdownPreserver from new module.
In correct_text (line ~123-140), before constructing prompt and calling gateway_complete:
- Create preserver instance.
- Call extract_blocks(text) to get preserved blocks and sanitized text.
- Pass only the sanitized text in the prompt (replace original content with sanitized text).
- After receiving AI response, call restore_blocks(ai_response, blocks) to reinsert preserved blocks.
In _run_correction (line ~187-216), the result from correct_text will now contain restored blocks; no further changes needed.
Update any prompt instructions if necessary (remove the previous instruction that LLM should preserve code blocks, since we now handle it server-side). Or keep as belt-and-suspenders.

File: devplacepy/services/ai_modifier.py

Apply the same extraction/restore logic in modify_text (line ~30-41) and _run_modification (line ~76-109), since it uses the same pattern.

3. Add structural validation check

File: devplacepy/services/correction.py (after restoration) and/or devplacepy/services/ai_modifier.py

After restoration, add a validation step:
- Count occurrences of backtick fences in original vs. restored text. If mismatch (e.g., number of opening ``` differs), log warning and possibly fall back to original? If the AI corrupted the structure despite extraction, the restoration will reinsert the original blocks, ensuring no corruption. The extraction already removes code blocks, so the AI cannot corrupt them. The validation is secondary.
- For now, it’s sufficient to rely on the extraction/restore mechanism.

4. Update existing tests and add new tests

File: tests/unit/services/correction.py (existing)

Modify the existing test (line 8-37) that mocks AI gateway and checks prompt content: after changes, the prompt should contain the sanitized text, not the original. Update assertion string accordingly.
Add new test test_correction_preserves_code_blocks:
- Input text with fenced code block (e.g., "some text\npython\nprint('hello')\n\nmore text").
- Mock gateway_complete to return a modified version that might corrupt code (e.g., "some text\nmodified\nmore text").
- After correction, assert that the output still contains the original code block (because it was extracted and restored). This verifies the restoration mechanism.
- Also test with inline code, multiple blocks, and no code blocks.

File: tests/unit/services/ai_modifier.py (existing)

Similarly update the existing test (line 8-38) to reflect the changed prompt.
Add analogous test for modify_text to ensure preservation works.

File: tests/unit/services/markdown_preserve.py (new)

Unit tests for extract_blocks and restore_blocks:
- Basic extraction and restoration round-trip.
- Edge cases: empty text, no code blocks, multiple blocks, language identifiers, nested backticks? Not allowed in Markdown, but ensure regex handles fences without language.
- Verify placeholders are unique and do not collide.
- Test that non-code-related Markdown structures (headings, lists) are not affected.

5. Documentation update

File: templates/docs/ai-modifier.html (line 35-36)

Update the claim that code blocks are never touched to reflect the new server-side protection (e.g., "Code blocks are extracted before AI processing and reinserted afterward, ensuring they remain unchanged").

File: devplacepy/services/CLAUDE.md (line 21-27)

Update documentation of the correction feature to describe the new extraction/restore mechanism.

6. Lint and code style

Ensure new code follows project style (PEP8, existing lint config). Run ruff check or equivalent to fix any issues.

Definition of Done

New module devplacepy/services/markdown_preserve.py implemented with extract_blocks and restore_blocks methods.
correction.py and ai_modifier.py updated to extract code blocks before AI call and restore after response.
Existing unit tests in tests/unit/services/correction.py and tests/unit/services/ai_modifier.py updated to reflect the new prompt content and to test code block preservation.
New unit tests in tests/unit/services/markdown_preserve.py cover extraction/restoration edge cases.
All unit tests pass with the command:
```
pip install -e '.[dev]' -q && python -m pytest
```
(No new failures, including those from earlier unrelated test suites – the change must not introduce regressions.)
Lint command passes with no new violations (run ruff check . or whatever the project uses; no fixable issues introduced).
templates/docs/ai-modifier.html and devplacepy/services/CLAUDE.md updated to accurately describe the new protection mechanism.

Opened automatically by Typosaurus.

Resolves #84. ## Plan ## Implementation Plan: Protect Markdown Structure During AI Correction ### Objective Prevent the AI correction pipeline from corrupting valid Markdown structures (fenced code blocks, inline code, headings, lists, tables) by extracting code blocks before sending content to the LLM and reinserting them afterward. ### Changes Required #### 1. Add Markdown structure preservation utility **File:** `devplacepy/services/markdown_preserve.py` (new) - Implement a `MarkdownPreserver` class with: - `extract_blocks(text: str) -> tuple[str, list[str]]`: Scans text for fenced code blocks (triple backticks, possibly with language identifier) and inline code spans (single backticks). Replaces each with a unique placeholder token (e.g., `{%CODE_BLOCK_1%}`). Returns modified text and list of original blocks. - `restore_blocks(text: str, blocks: list[str]) -> str`: Replaces placeholders with original blocks. - Use regex that correctly handles nested backticks? Fenced code blocks can span multiple lines, but cannot contain the closing fence. Use `re.finditer(r'```(?:\w+)?\n.*?```', text, re.DOTALL)` and for inline code `re.finditer(r'`[^`\n]+`')`. Ensure placeholders do not conflict with existing content. - Consider also extracting other Markdown structures if needed? The ticket focuses on code blocks; start with fenced code and inline code. Add extensibility. #### 2. Modify correction pipeline to use preserver **File:** `devplacepy/services/correction.py` - Import `MarkdownPreserver` from new module. - In `correct_text` (line ~123-140), before constructing prompt and calling `gateway_complete`: - Create preserver instance. - Call `extract_blocks(text)` to get preserved blocks and sanitized text. - Pass only the sanitized text in the prompt (replace original `content` with sanitized text). - After receiving AI response, call `restore_blocks(ai_response, blocks)` to reinsert preserved blocks. - In `_run_correction` (line ~187-216), the `result` from `correct_text` will now contain restored blocks; no further changes needed. - Update any prompt instructions if necessary (remove the previous instruction that LLM should preserve code blocks, since we now handle it server-side). Or keep as belt-and-suspenders. **File:** `devplacepy/services/ai_modifier.py` - Apply the same extraction/restore logic in `modify_text` (line ~30-41) and `_run_modification` (line ~76-109), since it uses the same pattern. #### 3. Add structural validation check **File:** `devplacepy/services/correction.py` (after restoration) and/or `devplacepy/services/ai_modifier.py` - After restoration, add a validation step: - Count occurrences of backtick fences in original vs. restored text. If mismatch (e.g., number of opening ` ``` ` differs), log warning and possibly fall back to original? If the AI corrupted the structure despite extraction, the restoration will reinsert the original blocks, ensuring no corruption. The extraction already removes code blocks, so the AI cannot corrupt them. The validation is secondary. - For now, it’s sufficient to rely on the extraction/restore mechanism. #### 4. Update existing tests and add new tests **File:** `tests/unit/services/correction.py` (existing) - Modify the existing test (line 8-37) that mocks AI gateway and checks prompt content: after changes, the prompt should contain the sanitized text, not the original. Update assertion string accordingly. - Add new test `test_correction_preserves_code_blocks`: - Input text with fenced code block (e.g., "some text\n```python\nprint('hello')\n```\nmore text"). - Mock `gateway_complete` to return a modified version that might corrupt code (e.g., "some text\nmodified\nmore text"). - After correction, assert that the output still contains the original code block (because it was extracted and restored). This verifies the restoration mechanism. - Also test with inline code, multiple blocks, and no code blocks. **File:** `tests/unit/services/ai_modifier.py` (existing) - Similarly update the existing test (line 8-38) to reflect the changed prompt. - Add analogous test for `modify_text` to ensure preservation works. **File:** `tests/unit/services/markdown_preserve.py` (new) - Unit tests for `extract_blocks` and `restore_blocks`: - Basic extraction and restoration round-trip. - Edge cases: empty text, no code blocks, multiple blocks, language identifiers, nested backticks? Not allowed in Markdown, but ensure regex handles fences without language. - Verify placeholders are unique and do not collide. - Test that non-code-related Markdown structures (headings, lists) are not affected. #### 5. Documentation update **File:** `templates/docs/ai-modifier.html` (line 35-36) - Update the claim that code blocks are never touched to reflect the new server-side protection (e.g., "Code blocks are extracted before AI processing and reinserted afterward, ensuring they remain unchanged"). **File:** `devplacepy/services/CLAUDE.md` (line 21-27) - Update documentation of the correction feature to describe the new extraction/restore mechanism. #### 6. Lint and code style - Ensure new code follows project style (PEP8, existing lint config). Run `ruff check` or equivalent to fix any issues. ### Definition of Done - [ ] New module `devplacepy/services/markdown_preserve.py` implemented with `extract_blocks` and `restore_blocks` methods. - [ ] `correction.py` and `ai_modifier.py` updated to extract code blocks before AI call and restore after response. - [ ] Existing unit tests in `tests/unit/services/correction.py` and `tests/unit/services/ai_modifier.py` updated to reflect the new prompt content and to test code block preservation. - [ ] New unit tests in `tests/unit/services/markdown_preserve.py` cover extraction/restoration edge cases. - [ ] All unit tests pass with the command: ``` pip install -e '.[dev]' -q && python -m pytest ``` (No new failures, including those from earlier unrelated test suites – the change must not introduce regressions.) - [ ] Lint command passes with no new violations (run `ruff check .` or whatever the project uses; no fixable issues introduced). - [ ] `templates/docs/ai-modifier.html` and `devplacepy/services/CLAUDE.md` updated to accurately describe the new protection mechanism. Opened automatically by Typosaurus.

typosaurus added 2 commits 2026-07-23 04:20:48 +02:00