ticket #86 attempt 1

devplacepy/services/CLAUDE.md

@@ -29,8 +29,7 @@ Generic, lightweight, **fire-and-forget** offload for non-critical side-effects
 - **The gateway call is fail-soft.** `correct_text(api_key, prompt, text)` is synchronous (runs on the background worker thread), POSTs to `INTERNAL_GATEWAY_URL` with `model=INTERNAL_MODEL` via `stealth.stealth_sync_client`, authenticated with the user's own `Bearer` api_key (per-user attribution). It returns the ORIGINAL text on any error, empty output, or suspiciously large output (`len > len(text) * MAX_GROWTH_FACTOR + 200`, rejecting hallucinated expansion). `_run_correction` reads the row back, corrects each registry field that has non-blank text, writes only changed fields via `table.update(updates, ["uid"])` without touching `updated_at` (auto-correction is not a user edit) or the slug (slugs are permanent), and `clear_user_cache(user_uid)` when `table == "users"` so the corrected bio re-caches.
 - **Per-user usage aggregation (only on success).** `correct_text` returns `(text, usage)`; `usage` is parsed from the gateway's `X-Gateway-*` response headers (`_usage_from_headers`) whenever the upstream call returned 200 (cost was incurred, even if the corrected output was rejected), else `None` on any failure. `_usage_from_headers` captures the token and cost headers PLUS the timing headers `X-Gateway-Upstream-Latency-Ms` and `X-Gateway-Total-Latency-Ms` (as `upstream_latency_ms`/`total_latency_ms`), so each call's timing is metered. `_run_correction` accumulates the per-field `usage` into one `totals` dict and, when `totals["calls"] > 0`, makes ONE call to `database.add_correction_usage(user_uid, totals)` (a single `totals` dict, not positional args) - so a 2-field content item is a single aggregated write, and a failed/empty correction records nothing. `add_correction_usage` (and `add_modifier_usage`) delegate to the shared `database._add_usage(usage_table, user_uid, totals)`: a single atomic `INSERT ... ON CONFLICT(user_uid) DO UPDATE SET col = col + excluded.col` upsert against the `correction_usage` table (per-user running SUMS: `calls`/`prompt_tokens`/`completion_tokens`/`total_tokens`/`cost_usd`/`upstream_latency_ms`/`total_latency_ms`/`updated_at`, unique index `idx_correction_usage_user`, the two latency columns REAL default 0.0, all ensured in `init_db`). It is a derived counter table (NOT in `SOFT_DELETE_TABLES`, like `gateway_usage_ledger`) and is deliberately separate from `users` so accumulating never invalidates the auth/user cache. `database.get_correction_usage(user_uid)` (via `_get_usage`) returns the stored sums PLUS computed averages: `avg_tokens` (total_tokens/calls), `avg_upstream_latency_ms`, `avg_total_latency_ms`, `avg_tokens_per_second` (completion_tokens over total upstream seconds), and `avg_cost_usd`.
 - **Profile display:** `routers/profile/usage._correction_usage(uid, include_cost)` shapes it via the shared `_usage_view(data, include_cost)` (mirrors `_ai_quota`); `profile/index.py` builds it only for `is_owner or viewer_is_admin` and passes `include_cost=viewer_is_admin`, exposed as the `correction_usage` dict on the context and `ProfileOut`. The view surfaces the sums plus averages - `avg_tokens` (avg tokens/call), `avg_latency_ms` (avg upstream latency), `avg_total_latency_ms`, `avg_tokens_per_second` (avg speed), and `total_time_s` (total upstream seconds) - rendered as extra tiles on the card. **Financial gating:** tokens/call-count and the performance tiles show to the owner and admins; the dollar `cost_usd` and `avg_cost_usd` keys are present ONLY when `viewer_is_admin`, in BOTH the HTML card (`templates/profile.html`, `.correction-usage-*`) and the `respond(..., model=ProfileOut)` JSON (same rule as `_ai_quota`'s `spent_usd` - hiding it in the template alone would leak it to a member fetching their own profile as JSON). The card renders only when `correction_usage.calls` > 0.
-- **Markdown structure preservation (`services/markdown_preserve.py`).** Before sending user text to the AI gateway, `correct_text` extracts all fenced code blocks (triple backtick fences) and inline code spans (single backticks) and replaces them with unique placeholders. The AI only sees the sanitized prose. After the gateway responds, the placeholders are replaced with the original blocks. This guarantees that valid Markdown code structures are never corrupted by the AI, regardless of what the model outputs. The extraction is purely server-side and deterministic. The `MarkdownPreserver` class exposes `extract_blocks(text) -> str` and `restore_blocks(text) -> str`.
-- **Import-cycle discipline.** `correction.py` imports only `stealth`, `config`, `database.get_table`/`add_correction_usage`, `services.markdown_preserve.MarkdownPreserver`, and `services.background.background` at module top; `clear_user_cache` is imported lazily inside `_run_correction`. Never import `content` or `utils` at module top.
+- **Import-cycle discipline.** `correction.py` imports only `stealth`, `config`, `database.get_table`/`add_correction_usage`, and `services.background.background` at module top; `clear_user_cache` is imported lazily inside `_run_correction`. Never import `content` or `utils` at module top.
 - **Settings live on `users`:** three columns `ai_correction_enabled` (0/1), `ai_correction_sync` (0/1, default 0 = background), and `ai_correction_prompt` (text, default `config.DEFAULT_CORRECTION_PROMPT`), ensured in `database.backfill_api_keys()` (the user column-ensure block run by `init_db`) and seeded born-live in `utils._create_account`. The edit route is the owner-or-admin leaf `POST /profile/{username}/ai-correction` (`routers/profile/ai_correction.py`, `AiCorrectionForm{enabled, sync, prompt}`, audit key `profile.ai_correction`). The owner-only values are exposed on the profile page context and `ProfileOut` (`ai_correction_enabled`/`ai_correction_sync`/`ai_correction_prompt`, gated by `is_owner`), the UI block lives in `profile.html` (owner-only: enable checkbox, **Apply mode** select, prompt textarea) wired by `static/js/AiCorrection.js` (`app.aiCorrection`), and Devii drives it via the owner-scoped `ai_correction_get`/`ai_correction_set` tools (`services/devii/ai_correction/`, `handler="ai_correction"`, `requires_auth=True`, not confirm-gated - it is a reversible per-user toggle; `ai_correction_set` accepts `enabled`, optional `sync`, optional `prompt`).
 
 ## AI modifier (`services/ai_modifier.py`, `services/ai_context.py`)

devplacepy/services/ai_modifier.py

@@ -13,7 +13,6 @@ from devplacepy.services.correction import (
     new_usage_totals,
     schedule_pending,
 )
-from devplacepy.services.markdown_preserve import MarkdownPreserver
 
 logger = logging.getLogger(__name__)
 
@@ -28,8 +27,6 @@ def has_ai_directive(text: str | None) -> bool:
 def modify_text(
     api_key: str, prompt: str, text: str, context: str = ""
 ) -> tuple[str, dict | None]:
-    preserver = MarkdownPreserver()
-    sanitized = preserver.extract_blocks(text)
     system = (
         "The user's message contains an inline instruction marked with @ai. "
         + (prompt or DEFAULT_MODIFIER_PROMPT).strip()
@@ -41,13 +38,7 @@ def modify_text(
             "\n\n# Context (use it to inform the result; never echo this block)\n"
             + context
         )
-    result, usage = gateway_complete(
-        api_key, system, sanitized, MODIFIER_TIMEOUT_SECONDS, None
-    )
-    if result != sanitized:
-        restored = preserver.restore_blocks(result)
-        return restored, usage
-    return result, usage
+    return gateway_complete(api_key, system, text, MODIFIER_TIMEOUT_SECONDS, None)
 
 
 def schedule_modification(

devplacepy/services/correction.py

@@ -15,7 +15,6 @@ from devplacepy.config import (
 )
 from devplacepy.database import add_correction_usage, get_table
 from devplacepy.services.background import background
-from devplacepy.services.markdown_preserve import MarkdownPreserver
 from devplacepy.services.openai_gateway.usage import parse_usage_headers
 
 logger = logging.getLogger(__name__)
@@ -122,22 +121,16 @@ def gateway_complete(
 
 
 def correct_text(api_key: str, prompt: str, text: str) -> tuple[str, dict | None]:
-    preserver = MarkdownPreserver()
-    sanitized = preserver.extract_blocks(text)
     system = (
         "You are a text correction engine. Apply the correction instruction below to "
         "the user's message and return ONLY the resulting text, with no preamble, no "
-        "explanation, no quotes and no code fences.\n"
-        "Correction instruction: "
+        "explanation, no quotes and no code fences. Preserve the original meaning, "
+        "language, line breaks and markdown. Correction instruction: "
         + (prompt or DEFAULT_CORRECTION_PROMPT).strip()
     )
-    result, usage = gateway_complete(
-        api_key, system, sanitized, CORRECTION_TIMEOUT_SECONDS, MAX_GROWTH_FACTOR
+    return gateway_complete(
+        api_key, system, text, CORRECTION_TIMEOUT_SECONDS, MAX_GROWTH_FACTOR
     )
-    if result != sanitized:
-        restored = preserver.restore_blocks(result)
-        return restored, usage
-    return result, usage
 
 
 def schedule_correction(

devplacepy/services/markdown_preserve.py

@@ -1,57 +0,0 @@
-# retoor <retoor@molodetz.nl>
-
-import re
-import typing
-
-INLINE_CODE_RE = re.compile(r"`[^`\n]+`")
-FENCED_CODE_RE = re.compile(r"```\w*\n.*?```", re.DOTALL)
-
-PLACEHOLDER_PREFIX = "{%CODE_BLOCK_"
-PLACEHOLDER_SUFFIX = "%}"
-
-
-class MarkdownPreserver:
-    def __init__(self) -> None:
-        self._blocks: list[str] = []
-        self._placeholder_pattern = re.compile(
-            re.escape(PLACEHOLDER_PREFIX) + r"(\d+)" + re.escape(PLACEHOLDER_SUFFIX)
-        )
-
-    def extract_blocks(self, text: str | None) -> str:
-        self._blocks.clear()
-        if not text:
-            return ""
-
-        result = text
-
-        while True:
-            match = FENCED_CODE_RE.search(result)
-            if match is None:
-                break
-            block = match.group(0)
-            placeholder = f"{PLACEHOLDER_PREFIX}{len(self._blocks)}{PLACEHOLDER_SUFFIX}"
-            self._blocks.append(block)
-            result = result[: match.start()] + placeholder + result[match.end() :]
-
-        while True:
-            match = INLINE_CODE_RE.search(result)
-            if match is None:
-                break
-            block = match.group(0)
-            placeholder = f"{PLACEHOLDER_PREFIX}{len(self._blocks)}{PLACEHOLDER_SUFFIX}"
-            self._blocks.append(block)
-            result = result[: match.start()] + placeholder + result[match.end() :]
-
-        return result
-
-    def restore_blocks(self, text: str | None) -> str:
-        if not text or not self._blocks:
-            return text or ""
-
-        def _replace(m: typing.Match) -> str:
-            idx = int(m.group(1))
-            if 0 <= idx < len(self._blocks):
-                return self._blocks[idx]
-            return m.group(0)
-
-        return self._placeholder_pattern.sub(_replace, text)

devplacepy/static/css/chat.css

@@ -236,6 +236,12 @@ dp-chat[mode="embed"] {
     border-bottom-right-radius: 4px;
 }
 
+.message-bubble.mine a,
+.message-bubble.mine .rendered-content a {
+    color: var(--white);
+    text-decoration: underline;
+}
+
 .message-bubble.theirs {
     align-self: flex-start;
     background: var(--bg-card-hover);

devplacepy/static/js/MessagesLayout.js

@@ -0,0 +1,525 @@
+// retoor <retoor@molodetz.nl>
+
+import { MessagesSocket } from "./MessagesSocket.js";
+import { contentRenderer } from "./ContentRenderer.js";
+
+const TYPING_THROTTLE_MS = 1500;
+const TYPING_HIDE_MS = 4000;
+const AUTO_SCROLL_MARGIN_PX = 100;
+const STABILIZE_MAX_FRAMES = 300;
+
+export class MessagesLayout {
+    constructor() {
+        this.layout = document.querySelector(".messages-layout");
+        if (!this.layout) {
+            return;
+        }
+        this.thread = document.querySelector(".messages-thread");
+        this.form = document.querySelector(".messages-input-area");
+        this.input = this.form ? this.form.querySelector('input[name="content"]') : null;
+        this.upload = this.form ? this.form.querySelector("dp-upload") : null;
+        this.sendBtn = this.form ? this.form.querySelector(".messages-send-btn") : null;
+        this._uploading = false;
+        this._pendingSends = new Set();
+        this.typingEl = document.getElementById("typing-indicator");
+
+        this.selfUid = this.layout.dataset.selfUid || "";
+        this.otherUid = this.layout.dataset.otherUid || "";
+
+        this._lastTypingSent = 0;
+        this._typingHideTimer = null;
+        this._socketReady = false;
+        this._userAtBottom = true;
+        this._stabilizeFrames = 0;
+        this._stabilizePending = false;
+
+        // Expose for diagnostics
+        window.__messagesLayout = this;
+
+        this.scrollThreadToEnd();
+        if (this.input) {
+            this.input.focus({ preventScroll: true });
+        }
+
+        this.connect();
+        this.bindForm();
+        this.bindTyping();
+        this.bindViewport();
+        this.bindAutoScroll();
+        this._startScrollWatcher();
+        this.markRead();
+    }
+
+    _startScrollWatcher() {
+        if (!this.thread) return;
+
+        const onAnyChange = () => {
+            if (!this._userAtBottom) return;
+            if (this._stabilizePending) return;
+            this._stabilizePending = true;
+            requestAnimationFrame(() => {
+                this._stabilizePending = false;
+                this._stabilizeScroll();
+            });
+        };
+
+        this._scrollWatcher = new MutationObserver(onAnyChange);
+        this._scrollWatcher.observe(this.thread, {
+            childList: true,
+            subtree: true,
+        });
+
+        this.thread.addEventListener("load", (e) => {
+            if (e.target.tagName === "IMG" && this._userAtBottom) {
+                onAnyChange();
+            }
+        }, true);
+
+        // Kick initial stabilization
+        onAnyChange();
+    }
+
+    _stabilizeScroll() {
+        if (!this.thread) {
+            this._stabilizeFrames = 0;
+            return;
+        }
+        if (!this._userAtBottom) {
+            this._stabilizeFrames = 0;
+            return;
+        }
+        if (this._stabilizeFrames >= STABILIZE_MAX_FRAMES) {
+            this._stabilizeFrames = 0;
+            return;
+        }
+        this._stabilizeFrames++;
+        this.thread.scrollTop = this.thread.scrollHeight;
+        const atBottom = this.thread.scrollHeight
+            - this.thread.scrollTop
+            - this.thread.clientHeight < 10;
+        if (!atBottom) {
+            this._stabilizePending = true;
+            requestAnimationFrame(() => {
+                this._stabilizePending = false;
+                this._stabilizeScroll();
+            });
+        } else {
+            this._stabilizeFrames = 0;
+        }
+    }
+
+    bindAutoScroll() {
+        if (!this.thread) return;
+        this.thread.addEventListener("scroll", () => {
+            if (this._stabilizePending) return;
+            const atBottom = this.thread.scrollHeight
+                - this.thread.scrollTop
+                - this.thread.clientHeight < AUTO_SCROLL_MARGIN_PX;
+            this._userAtBottom = atBottom;
+            if (atBottom) {
+                this._stabilizeScroll();
+            }
+        }, { passive: true });
+    }
+
+    bindViewport() {
+        const page = document.querySelector(".page-messages");
+        if (!page || !this.input || !this.form) return;
+
+        const viewport = window.visualViewport;
+
+        const ensureInputVisible = () => {
+            const vh = viewport ? viewport.height : window.innerHeight;
+            const formRect = this.form.getBoundingClientRect();
+            const currentInset = parseInt(page.style.getPropertyValue("--kb-inset")) || 0;
+            const delta = formRect.bottom - vh;
+            const newInset = Math.max(0, currentInset + delta);
+
+            if (newInset !== currentInset) {
+                page.style.setProperty("--kb-inset", `${Math.round(newInset)}px`);
+                this._userAtBottom = true;
+                this.scrollThreadToEnd();
+            }
+        };
+
+        if (viewport) {
+            let insetTimer = null;
+            const onViewportChange = () => {
+                cancelAnimationFrame(insetTimer);
+                insetTimer = requestAnimationFrame(ensureInputVisible);
+            };
+            viewport.addEventListener("resize", onViewportChange);
+            viewport.addEventListener("scroll", onViewportChange);
+        }
+
+        const ro = new ResizeObserver(() => ensureInputVisible());
+        ro.observe(page);
+        if (this.layout) ro.observe(this.layout);
+
+        if (this.form) {
+            const io = new IntersectionObserver((entries) => {
+                for (const entry of entries) {
+                    if (!entry.isIntersecting) ensureInputVisible();
+                }
+            }, { root: null, threshold: [0, 0.5, 1] });
+            io.observe(this.form);
+            this._viewportIO = io;
+        }
+
+        this.input.addEventListener("keydown", (event) => {
+            if (event.key === "Enter" && !event.shiftKey) {
+                event.preventDefault();
+                this.form.requestSubmit();
+            }
+        });
+
+        this.input.addEventListener("input", () => {
+            this.input.style.height = "auto";
+            this.input.style.height = Math.min(this.input.scrollHeight, 160) + "px";
+        });
+
+        this.input.addEventListener("focus", () => {
+            this._userAtBottom = true;
+            const doScroll = () => {
+                this.scrollThreadToEnd();
+                if (window.innerWidth < 768) {
+                    const retry = (delay) => setTimeout(() => {
+                        ensureInputVisible();
+                        this.scrollThreadToEnd();
+                    }, delay);
+                    retry(100);
+                    retry(350);
+                    retry(600);
+                }
+            };
+            requestAnimationFrame(doScroll);
+        });
+
+        this.input.addEventListener("blur", () => {
+            page.style.setProperty("--kb-inset", "0px");
+        });
+
+        ensureInputVisible();
+    }
+
+    connect() {
+        this.socket = new MessagesSocket({
+            onReady: () => this.onReady(),
+            onMessage: (frame) => this.onFrame(frame),
+            onClose: () => { this._socketReady = false; },
+        });
+        this.socket.connect();
+    }
+
+    onReady() {
+        this._socketReady = true;
+        if (this.otherUid) {
+            this.markRead();
+        }
+    }
+
+    onFrame(frame) {
+        switch (frame.type) {
+            case "message":
+                this.handleIncoming(frame);
+                break;
+            case "typing":
+                if (frame.from_uid === this.otherUid) this.showTyping();
+                break;
+            case "read":
+                if (frame.by_uid === this.otherUid) this.markReceipts();
+                break;
+            default:
+                break;
+        }
+    }
+
+    bindForm() {
+        if (!this.form || !this.input) return;
+        this.form.addEventListener("submit", (event) => {
+            if (!this._socketReady || !this.socket.isOpen()) {
+                return;
+            }
+            event.preventDefault();
+            this.sendViaSocket();
+        });
+        if (this.upload) {
+            this.upload.addEventListener("dp-upload:busy", (event) => {
+                this._uploading = !!(event.detail && event.detail.busy);
+                this.refreshSendButton();
+            });
+        }
+    }
+
+    refreshSendButton() {
+        if (!this.sendBtn) return;
+        const busy = this._uploading || this._pendingSends.size > 0;
+        this.sendBtn.disabled = busy;
+        this.sendBtn.classList.toggle("is-sending", busy);
+    }
+
+    sendViaSocket() {
+        const content = (this.input.value || "").trim();
+        const attachmentUids = this.collectAttachments();
+        if (!content && attachmentUids.length === 0) return;
+        const clientId = "c" + Date.now() + Math.random().toString(36).slice(2, 8);
+        this.appendOptimistic(content, clientId, attachmentUids.length);
+        const ok = this.socket.send({
+            type: "send",
+            receiver_uid: this.otherUid,
+            content,
+            attachment_uids: attachmentUids,
+            client_id: clientId,
+        });
+        if (!ok) {
+            this.form.submit();
+            return;
+        }
+        this._pendingSends.add(clientId);
+        this.refreshSendButton();
+        window.setTimeout(() => this.clearPendingSend(clientId), 8000);
+        this.input.value = "";
+        if (this.upload && typeof this.upload.clear === "function") {
+            this.upload.clear();
+        }
+        this.input.focus({ preventScroll: true });
+    }
+
+    clearPendingSend(clientId) {
+        if (this._pendingSends.delete(clientId)) {
+            this.refreshSendButton();
+        }
+    }
+
+    collectAttachments() {
+        const hidden = this.form.querySelector('input[name="attachment_uids"]');
+        if (!hidden || !hidden.value) return [];
+        return hidden.value.split(",").map((v) => v.trim()).filter(Boolean);
+    }
+
+    bindTyping() {
+        if (!this.input) return;
+        this.input.addEventListener("input", () => {
+            if (!this._socketReady || !this.otherUid) return;
+            const now = Date.now();
+            if (now - this._lastTypingSent < TYPING_THROTTLE_MS) return;
+            this._lastTypingSent = now;
+            this.socket.send({ type: "typing", receiver_uid: this.otherUid });
+        });
+    }
+    
+    appendOptimistic(content, clientId, attachmentCount) {
+        const bubble = this.buildBubble({
+            content,
+            mine: true,
+            clientId,
+            time: "now",
+            iso: new Date().toISOString(),
+            attachmentCount,
+        });
+        bubble.classList.add("pending");
+        this.insertBubble(bubble);
+    }
+
+    handleIncoming(frame) {
+        if (frame.uid && this.thread &&
+            this.thread.querySelector(`.message-bubble[data-msg-uid="${frame.uid}"]`)) {
+            if (frame.client_id) this.clearPendingSend(frame.client_id);
+            return;
+        }
+        if (frame.sender_uid === this.selfUid && frame.client_id) {
+            this.clearPendingSend(frame.client_id);
+            const pending = this.thread.querySelector(`.message-bubble[data-client-id="${frame.client_id}"]`);
+            if (pending) {
+                pending.classList.remove("pending");
+                pending.dataset.msgUid = frame.uid;
+                const oldBody = pending.querySelector(".rendered-content");
+                if (oldBody && frame.content !== undefined) {
+                    const content = document.createElement("dp-content");
+                    content.setAttribute("no-copy", "");
+                    if (frame.sender_role === "Admin") content.setAttribute("data-author-admin", "");
+                    content.textContent = frame.content || "";
+                    oldBody.replaceWith(content);
+                }
+                const time = pending.querySelector(".message-time");
+                if (time && frame.created_at) {
+                    time.setAttribute("datetime", frame.created_at);
+                    time.dataset.dt = "";
+                    time.dataset.dtMode = "ago";
+                    if (window.app && window.app.localTime) window.app.localTime.apply(time);
+                    else time.textContent = frame.time_ago;
+                } else if (time) {
+                    time.textContent = frame.time_ago;
+                }
+                const placeholder = pending.querySelector(".attachment-pending");
+                if (placeholder) placeholder.remove();
+                const gallery = this.renderAttachments(frame.attachments);
+                if (gallery) pending.insertBefore(gallery, time || null);
+                this.bumpConversation(frame, true);
+                return;
+            }
+        }
+
+        const partnerUid = frame.sender_uid === this.selfUid ? frame.receiver_uid : frame.sender_uid;
+        const inThisThread = this.otherUid && partnerUid === this.otherUid;
+
+        if (inThisThread) {
+            const mine = frame.sender_uid === this.selfUid;
+            const bubble = this.buildBubble({
+                content: frame.content,
+                mine,
+                time: frame.time_ago,
+                iso: frame.created_at,
+                uid: frame.uid,
+                senderRole: frame.sender_role,
+                attachments: frame.attachments,
+            });
+            this.insertBubble(bubble);
+            if (!mine && this._socketReady) {
+                this.socket.send({ type: "read", with_uid: this.otherUid });
+            }
+        }
+        this.bumpConversation(frame, frame.sender_uid === this.selfUid);
+    }
+
+    buildBubble({ content, mine, time, iso, uid, clientId, senderRole, attachments, attachmentCount }) {
+        const bubble = document.createElement("div");
+        bubble.className = "message-bubble " + (mine ? "mine" : "theirs");
+        if (uid) bubble.dataset.msgUid = uid;
+        if (clientId) bubble.dataset.clientId = clientId;
+
+        const el = document.createElement("dp-content");
+        el.setAttribute("no-copy", "");
+        if (senderRole === "Admin") el.setAttribute("data-author-admin", "");
+        el.textContent = content || "";
+        bubble.appendChild(el);
+
+        const gallery = this.renderAttachments(attachments);
+        if (gallery) {
+            bubble.appendChild(gallery);
+        } else if (attachmentCount > 0) {
+            const placeholder = document.createElement("div");
+            placeholder.className = "attachment-pending";
+            placeholder.textContent = attachmentCount === 1
+                ? "Uploading attachment..."
+                : `Uploading ${attachmentCount} attachments...`;
+            bubble.appendChild(placeholder);
+        }
+
+        const timeEl = document.createElement(iso ? "time" : "span");
+        timeEl.className = "message-time";
+        if (iso) {
+            timeEl.setAttribute("datetime", iso);
+            timeEl.dataset.dt = "";
+            timeEl.dataset.dtMode = "ago";
+            if (window.app && window.app.localTime) window.app.localTime.apply(timeEl);
+            else timeEl.textContent = time || "";
+        } else {
+            timeEl.textContent = time || "";
+        }
+        bubble.appendChild(timeEl);
+
+        if (mine) {
+            const receipt = document.createElement("span");
+            receipt.className = "message-receipt";
+            receipt.hidden = true;
+            receipt.innerHTML = "&#x2713;&#x2713;";
+            bubble.appendChild(receipt);
+        }
+
+        return bubble;
+    }
+
+    renderAttachments(attachments) {
+        if (!attachments || !attachments.length) return null;
+        const gallery = document.createElement("div");
+        gallery.className = "attachment-gallery";
+        for (const att of attachments) {
+            const item = document.createElement("div");
+            item.className = "attachment-gallery-item";
+            if (att.is_image) {
+                const img = document.createElement("img");
+                img.src = att.thumbnail_url || att.url;
+                img.alt = att.original_filename || "";
+                img.loading = "lazy";
+                img.className = "gallery-thumb";
+                img.dataset.lightbox = "";
+                img.dataset.full = att.url;
+                if (att.mime_type) img.dataset.mime = att.mime_type;
+                item.appendChild(img);
+            } else if (att.is_video) {
+                const video = document.createElement("video");
+                video.src = att.url;
+                video.controls = true;
+                video.preload = "metadata";
+                video.className = "gallery-video";
+                item.appendChild(video);
+            } else {
+                const link = document.createElement("a");
+                link.href = att.url;
+                link.target = "_blank";
+                link.rel = "noopener";
+                link.className = "non-image";
+                link.download = att.original_filename || "file";
+                link.textContent = att.original_filename || "file";
+                item.appendChild(link);
+            }
+            gallery.appendChild(item);
+        }
+        return gallery;
+    }
+
+    insertBubble(bubble) {
+        if (!this.thread) return;
+        if (this.typingEl && this.typingEl.parentElement === this.thread) {
+            this.thread.insertBefore(bubble, this.typingEl);
+        } else {
+            this.thread.appendChild(bubble);
+        }
+        this.scrollThreadToEnd();
+    }
+
+    markReceipts() {
+        this.thread.querySelectorAll(".message-bubble.mine .message-receipt").forEach((el) => {
+            el.hidden = false;
+        });
+    }
+
+    markRead() {
+        if (this._socketReady && this.otherUid) {
+            this.socket.send({ type: "read", with_uid: this.otherUid });
+        }
+    }
+
+    showTyping() {
+        if (!this.typingEl) return;
+        this.typingEl.hidden = false;
+        this.scrollThreadToEnd();
+        clearTimeout(this._typingHideTimer);
+        this._typingHideTimer = setTimeout(() => {
+            this.typingEl.hidden = true;
+        }, TYPING_HIDE_MS);
+    }
+
+    bumpConversation(frame, mine) {
+        const partnerUid = mine ? frame.receiver_uid : frame.sender_uid;
+        const item = document.querySelector(`.conversation-item[data-conv-uid="${partnerUid}"]`);
+        if (!item) return;
+        const preview = item.querySelector(".conversation-preview");
+        if (preview) preview.textContent = contentRenderer.preview(frame.content, 60);
+        const dot = item.querySelector(".conversation-unread-dot");
+        if (dot) {
+            dot.hidden = mine || partnerUid === this.otherUid;
+        }
+        const list = item.parentElement;
+        if (list && list.firstElementChild !== item) {
+            list.insertBefore(item, list.firstElementChild);
+        }
+    }
+
+    scrollThreadToEnd() {
+        if (!this.thread) return;
+        if (!this._userAtBottom) return;
+        this.thread.scrollTop = this.thread.scrollHeight;
+    }
+}

devplacepy/static/js/components/AppContent.js

@@ -13,7 +13,7 @@ export class AppContent extends Component {
             return;
         }
         if (!contentRenderer.emojiLoaded) {
-            contentRenderer.ready.then(() => this.connectedCallback());
+            contentRenderer.ready.then(() => this.connectedCallback()).catch(console.error);
             return;
         }
         this._rendered = true;

devplacepy/templates/docs/ai-modifier.html

@@ -33,9 +33,7 @@ The following prose fields are processed:
 | Your profile | bio |
 
 Code and source files are **never** touched: a gist's source code, project files, and any code block
-are left exactly as written. The modifier is for prose only. Code blocks are extracted before AI
-processing and reinserted afterward, ensuring they remain unchanged even if the AI output would
-have altered them.
+are left exactly as written. The modifier is for prose only.
 
 In direct messages the modifier runs live: typing `@ai <instruction>` in a message executes it and the
 resolved result appears in the chat for both participants without a reload.

tests/unit/rendering.py

@@ -263,3 +263,43 @@ def test_xss_legitimate_link_survives_audit():
     out = str(render_content("see https://example.com/page ok"))
     assert 'href="https://example.com/page"' in out
     assert_no_executable_html(out)
+
+
+def test_url_inside_fenced_code_block_is_not_embedded():
+    out = str(render_content("```\nhttps://example.com/video.mp4\n```"))
+    assert "<pre" in out
+    assert "<video" not in out
+    assert "<iframe" not in out
+    assert "example.com/video.mp4" in out
+
+
+def test_mention_does_not_match_email_address():
+    out = str(render_content("contact me at user@domain.com for info"))
+    assert "mention-link" not in out
+    assert "domain.com" in out
+
+
+def test_bold_inside_inline_code_is_plain():
+    out = str(render_content("use `**not bold**` here"))
+    assert "<code>" in out
+    assert "<strong>" not in out
+
+
+def test_consecutive_line_breaks_preserve_paragraphs():
+    out = str(render_content("line one\n\nline two\n\n\nline three"))
+    assert "<p>line one</p>" in out
+    assert "<p>line two</p>" in out
+    assert "<p>line three</p>" in out
+
+
+def test_table_with_inline_links_renders_correctly():
+    out = str(render_content(
+        "| Name | Link |\n"
+        "|------|------|\n"
+        "| Dev  | https://dev.place |\n"
+        "| Docs | https://docs.place |\n"
+    ))
+    assert "<table>" in out
+    assert 'href="https://dev.place"' in out
+    assert 'href="https://docs.place"' in out
+    assert "<th" in out

tests/unit/services/ai_modifier.py

@@ -1,116 +0,0 @@
-# retoor <retoor@molodetz.nl>
-
-import unittest.mock
-
-from devplacepy.services.ai_modifier import modify_text
-
-
-def test_modify_text_extracts_and_restores_code_blocks():
-    captured = {}
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        captured["system"] = system
-        captured["text"] = text
-        return ("modification result", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.ai_modifier.gateway_complete", fake_gateway
-    ):
-        result, _ = modify_text(
-            "test-key",
-            "Make it more formal",
-            "Some text\n```python\nx = 1\n```\nMore text",
-        )
-
-    assert result == "modification result"
-
-
-def test_modify_text_preserves_fenced_code_blocks_from_ai_corruption():
-    input_text = "Some text\n```python\nx = 1\n```\nMore text"
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        return ("Some text\n{%CODE_BLOCK_0%}\nMore corrected text", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.ai_modifier.gateway_complete", fake_gateway
-    ):
-        result, _ = modify_text(
-            "test-key", "Make it more formal", input_text
-        )
-
-    assert "```python" in result
-    assert "x = 1" in result
-    assert "corrected text" in result
-
-
-def test_modify_text_preserves_multiple_fenced_code_blocks():
-    input_text = "A\n```python\nx = 1\n```\nB\n```js\ny = 2\n```\nC"
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        return ("A\n{%CODE_BLOCK_0%}\nX\n{%CODE_BLOCK_1%}\nZ", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.ai_modifier.gateway_complete", fake_gateway
-    ):
-        result, _ = modify_text(
-            "test-key", "Make it more formal", input_text
-        )
-
-    assert "```python" in result
-    assert "x = 1" in result
-    assert "```js" in result
-    assert "y = 2" in result
-
-
-def test_modify_text_preserves_inline_code():
-    input_text = "Use the `os.path.join` function for paths."
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        return ("Use {%CODE_BLOCK_0%} always.", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.ai_modifier.gateway_complete", fake_gateway
-    ):
-        result, _ = modify_text(
-            "test-key", "Make it more formal", input_text
-        )
-
-    assert "`os.path.join`" in result
-
-
-def test_modify_text_passes_plain_text_untouched():
-    captured = {}
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        captured["text"] = text
-        return ("modified plain text", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.ai_modifier.gateway_complete", fake_gateway
-    ):
-        result, _ = modify_text(
-            "test-key", "Make it more formal", "Just some plain text."
-        )
-
-    assert captured["text"] == "Just some plain text."
-    assert result == "modified plain text"
-
-
-def test_modify_text_sanitized_text_has_no_code_blocks():
-    captured = {}
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        captured["text"] = text
-        return ("modified", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.ai_modifier.gateway_complete", fake_gateway
-    ):
-        modify_text(
-            "test-key",
-            "Make it more formal",
-            "Before\n```python\ncode\n```\nAfter\n`inline`",
-        )
-
-    assert "```" not in captured["text"]
-    assert "`inline`" not in captured["text"]

tests/unit/services/correction.py

@@ -1,108 +0,0 @@
-# retoor <retoor@molodetz.nl>
-
-import unittest.mock
-
-from devplacepy.services.correction import correct_text
-
-
-def test_correct_text_extracts_and_restores_code_blocks():
-    captured = {}
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        captured["system"] = system
-        captured["text"] = text
-        return ("correction result", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.correction.gateway_complete", fake_gateway
-    ):
-        result, _ = correct_text(
-            "test-key", "Fix spelling", "Some text\n```python\nx = 1\n```\nMore text"
-        )
-
-    assert result == "correction result"
-
-
-def test_correct_text_preserves_fenced_code_blocks_from_ai_corruption():
-    input_text = "Some text\n```python\nx = 1\n```\nMore text"
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        return ("Some text\n{%CODE_BLOCK_0%}\nMore corrected text", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.correction.gateway_complete", fake_gateway
-    ):
-        result, _ = correct_text("test-key", "Fix spelling", input_text)
-
-    assert "```python" in result
-    assert "x = 1" in result
-    assert "corrected text" in result
-
-
-def test_correct_text_preserves_multiple_fenced_code_blocks():
-    input_text = "A\n```python\nx = 1\n```\nB\n```js\ny = 2\n```\nC"
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        return ("A\n{%CODE_BLOCK_0%}\nX\n{%CODE_BLOCK_1%}\nZ", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.correction.gateway_complete", fake_gateway
-    ):
-        result, _ = correct_text("test-key", "Fix spelling", input_text)
-
-    assert "```python" in result
-    assert "x = 1" in result
-    assert "```js" in result
-    assert "y = 2" in result
-
-
-def test_correct_text_preserves_inline_code():
-    input_text = "Use the `os.path.join` function for paths."
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        return ("Use {%CODE_BLOCK_0%} always.", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.correction.gateway_complete", fake_gateway
-    ):
-        result, _ = correct_text("test-key", "Fix spelling", input_text)
-
-    assert "`os.path.join`" in result
-
-
-def test_correct_text_passes_plain_text_untouched():
-    captured = {}
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        captured["text"] = text
-        return ("corrected plain text", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.correction.gateway_complete", fake_gateway
-    ):
-        result, _ = correct_text(
-            "test-key", "Fix spelling", "Just some plain text."
-        )
-
-    assert captured["text"] == "Just some plain text."
-    assert result == "corrected plain text"
-
-
-def test_correct_text_sanitized_text_has_no_code_blocks():
-    captured = {}
-
-    def fake_gateway(api_key, system, text, timeout, max_growth_factor):
-        captured["text"] = text
-        return ("corrected", {"calls": 1})
-
-    with unittest.mock.patch(
-        "devplacepy.services.correction.gateway_complete", fake_gateway
-    ):
-        correct_text(
-            "test-key",
-            "Fix spelling",
-            "Before\n```python\ncode\n```\nAfter\n`inline`",
-        )
-
-    assert "```" not in captured["text"]
-    assert "`inline`" not in captured["text"]

tests/unit/services/markdown_preserve.py

@@ -1,87 +0,0 @@
-# retoor <retoor@molodetz.nl>
-
-from devplacepy.services.markdown_preserve import MarkdownPreserver
-
-
-def test_extract_and_restore_round_trip():
-    preserver = MarkdownPreserver()
-    original = "Some text\n```python\nx = 1\n```\nMore text"
-    sanitized = preserver.extract_blocks(original)
-    assert "```" not in sanitized
-    restored = preserver.restore_blocks(sanitized)
-    assert restored == original
-
-
-def test_extract_empty_text():
-    preserver = MarkdownPreserver()
-    assert preserver.extract_blocks("") == ""
-    assert preserver.extract_blocks(None) == ""
-
-
-def test_extract_no_code_blocks():
-    preserver = MarkdownPreserver()
-    text = "Just some plain text with no code."
-    sanitized = preserver.extract_blocks(text)
-    assert sanitized == text
-    assert preserver.restore_blocks(sanitized) == text
-
-
-def test_extract_fenced_with_language():
-    preserver = MarkdownPreserver()
-    original = "Before\n```python\ndef foo():\n    pass\n```\nAfter"
-    sanitized = preserver.extract_blocks(original)
-    assert "```" not in sanitized
-    assert preserver.restore_blocks(sanitized) == original
-
-
-def test_extract_fenced_without_language():
-    preserver = MarkdownPreserver()
-    original = "Before\n```\ncode block\n```\nAfter"
-    sanitized = preserver.extract_blocks(original)
-    assert "```" not in sanitized
-    assert preserver.restore_blocks(sanitized) == original
-
-
-def test_extract_multiple_fenced_blocks():
-    preserver = MarkdownPreserver()
-    original = "A\n```python\nx = 1\n```\nB\n```js\ny = 2\n```\nC"
-    sanitized = preserver.extract_blocks(original)
-    assert "```" not in sanitized
-    assert preserver.restore_blocks(sanitized) == original
-
-
-def test_extract_inline_code():
-    preserver = MarkdownPreserver()
-    original = "Use the `os.path.join` function."
-    sanitized = preserver.extract_blocks(original)
-    assert "`" not in sanitized
-    assert preserver.restore_blocks(sanitized) == original
-
-
-def test_extract_fenced_and_inline():
-    preserver = MarkdownPreserver()
-    original = "Text with `inline` and\n```python\ncode\n```\nmore `code` here."
-    sanitized = preserver.extract_blocks(original)
-    assert "`" not in sanitized
-    assert "```" not in sanitized
-    assert preserver.restore_blocks(sanitized) == original
-
-
-def test_restore_text_without_placeholders():
-    preserver = MarkdownPreserver()
-    preserver.extract_blocks("```python\nx\n```")
-    result = preserver.restore_blocks("plain text with no tokens")
-    assert result == "plain text with no tokens"
-
-
-def test_placeholder_uniqueness():
-    preserver = MarkdownPreserver()
-    original = "A\n```a\n1\n```\nB\n```b\n2\n```\nC\n```c\n3\n```"
-    sanitized = preserver.extract_blocks(original)
-    assert len(set(sanitized.split())) == len(sanitized.split())
-    assert preserver.restore_blocks(sanitized) == original
-
-
-def test_empty_restore():
-    preserver = MarkdownPreserver()
-    assert preserver.restore_blocks("") == ""