Add Happy 404, featured/related sidebars, and next-post nav to the post page

Happy 404: an HTML 404 (unmatched route, or an explicit not-found inside a
real route) now renders a random existing post instead of the error page,
using the exact same context builder as a real post view. Toggle is the
happy_404_enabled site setting (default on, /admin/settings); JSON/API
requests and a handful of excluded prefixes are never affected. The pool of
candidate slugs is cached in-process and resampled periodically so it stays
fast and eventually cycles the whole posts table; on any internal failure it
falls straight through to the real 404 page.

Applying this everywhere surfaced ~60 existing tests that asserted a literal
404 for a legitimate resource-not-found flow (deleted post, unknown
container, wrong project slug, etc.) - each now disables the setting for the
duration of that specific check and restores it after, so the underlying
not-found behavior stays covered independently of the new feature.

Post page also gained, all built on the same shared post_page_context() so
they render identically on both a real post and a happy-404 page:
- A left sidebar (three separate cards, matching /feed's sidebar-card
  convention) for "Gists from {author}", "Projects from {author}" (private
  projects filtered through the normal visibility check), and "Related
  Discussions" - each cached per author and invalidated on create/edit/
  delete so new content shows up immediately.
- A right column reusing /feed's exact Daily Topic widget class for up to
  three "Featured" articles (the existing but previously-unused `featured`
  news flag), cached as a pool with per-request random sampling.
- A "Next post -> " link beside "Back to Feed", pointing at the next older
  post site-wide (blocked authors skipped). Wired through the same next_url
  mechanism already used for listing pagination, so it emits a real
  backend-rendered <link rel="next"> tag for SEO, not just a visible link.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BWJy6PrMMt5hwWxQwia2rd

README.md

@@ -288,6 +288,7 @@ Operational behavior is tunable live from `/admin/settings` (stored in `site_set
 | `registration_open` | `1` | When `0`, new sign-ups are rejected |
 | `maintenance_mode` | `0` | When `1`, non-admins see the maintenance page; admins retain access |
 | `maintenance_message` | scheduled-maintenance text | Message shown during maintenance |
+| `happy_404_enabled` | `1` | When `1`, an HTML page that would otherwise 404 renders a random existing post instead ("Happy 404"); JSON/API requests always get a real 404 regardless. See [Happy 404](#happy-404) |
 | `customization_enabled` | `1` | When `0`, no user CSS/JS customization is injected on any page |
 | `customization_js_enabled` | `1` | When `0`, user custom CSS is still served but custom JavaScript is suppressed |
 | `audit_log_retention_days` | `90` | Audit rows older than this are pruned daily by the Audit retention service; `0` disables pruning |
@@ -342,6 +343,27 @@ curl -H "Accept: application/json" https://your-host/feed
 curl -H "Accept: application/json" -X POST -d "content=hi&title=T&topic=devlog" https://your-host/posts/create
 ```
 
+## Happy 404
+
+Instead of a bare error page, an HTML request that would 404 (an unmatched route, or an app route
+raising `not_found()` for a missing resource) instead renders a random existing post at that URL,
+using the exact same template and context as the real `/posts/{slug}` page. JSON/API requests are
+unaffected and still get a normal `404` - the substitution only ever applies to a browser HTML
+navigation.
+
+- **Toggle:** `happy_404_enabled` site setting (`/admin/settings`, on by default).
+- **Scope:** any HTML `GET` 404, app-wide - not just under `/posts`.
+- **Performance:** a small pool of random post slugs is cached in-process for a few minutes and
+  refreshed with a fresh random sample on expiry, so every request only does an in-memory pick plus
+  one indexed lookup - no per-request full-table scan - while the pool composition still cycles
+  through the whole `posts` table over time.
+- **SEO safety:** the substituted page is always marked `noindex,nofollow` so the decoy URL is never
+  indexed under the wrong address.
+- **Fail-closed:** any error while building the substitute page falls straight through to the normal
+  404 page - this feature can never turn a real error into a worse one.
+
+Implementation: `devplacepy/happy404.py`, wired into the `404` exception handler in `main.py`.
+
 ## XML-RPC bridge
 
 The full REST API is also reachable over XML-RPC at `/xmlrpc`. A standalone forking XML-RPC

devplacepy/content.py

@@ -38,6 +38,8 @@ from devplacepy.database import (
     get_int_setting,
     _now_iso,
     db,
+    get_user_recent_items,
+    invalidate_user_recent_cache,
 )
 from devplacepy.utils import (
     time_ago,
@@ -133,6 +135,18 @@ def can_view_project(project: dict | None, user: dict | None) -> bool:
     return not _owner_is_admin(project)
 
 
+def get_user_sidebar_gists(user_uid: str, limit: int = 5) -> list[dict]:
+    return get_user_recent_items("gists", user_uid)[:limit]
+
+
+def get_user_sidebar_projects(
+    user_uid: str, viewer: dict | None, limit: int = 5
+) -> list[dict]:
+    rows = get_user_recent_items("projects", user_uid)
+    visible = [p for p in rows if can_view_project(p, viewer)]
+    return visible[:limit]
+
+
 def owns_instance(
     instance: dict | None, project: dict | None, user: dict | None
 ) -> bool:
@@ -264,6 +278,7 @@ def create_content_item(
         from devplacepy.templating import clear_user_projects_cache
 
         clear_user_projects_cache(user["uid"])
+    invalidate_user_recent_cache(table_name, user["uid"])
     award_rewards(user["uid"], xp, badge)
     if attachment_uids:
         link_attachments(attachment_uids, target_type, uid)
@@ -678,6 +693,7 @@ def edit_content_item(
         "updated_at": datetime.now(timezone.utc).isoformat(),
     }
     table.update({"uid": item["uid"], **update_fields}, ["uid"])
+    invalidate_user_recent_cache(table_name, item["user_uid"])
     record_screening(
         screening,
         target_type=kind,
@@ -769,6 +785,7 @@ def delete_content_item(
         soft_delete_all_project_files(item["uid"], actor)
         soft_delete_fork_relations(item["uid"], actor)
         clear_user_projects_cache(item["user_uid"])
+    invalidate_user_recent_cache(table_name, item["user_uid"])
     soft_delete(table_name, actor, stamp=stamp, uid=item["uid"])
     logger.info(f"{table_name} {item['uid']} soft-deleted by {user['username']}")
     audit.record(

devplacepy/database/CLAUDE.md

@@ -211,6 +211,7 @@ Site settings are seeded on startup (`site_settings` table):
 | `maintenance_message` | scheduled-maintenance text | Body shown on the maintenance 503 page |
 | `customization_enabled` | `"1"` | When `"0"`, `custom_css_tag`/`custom_js_tag` inject nothing (feature off) |
 | `customization_js_enabled` | `"1"` | When `"0"`, custom CSS still serves but custom JS is suppressed |
+| `happy_404_enabled` | `"1"` | When `"0"`, `happy404.render()` is a no-op and a real 404 page is shown; when on, an HTML 404 renders a random existing post instead (see below) |
 | `moderation_sla_hours` | `"24"` | The published moderation response window; the admin queue badge turns red past it |
 | `moderation_filter_mode` | `"review"` | `off`/`label`/`review`/`block` - how the content filter acts on a match |
 | `moderation_filter_review_score` | `"2"` | Rule score at which a match becomes a report rather than a label |
@@ -239,6 +240,7 @@ Operational settings - read sites and rules:
 | `service_<name>_enabled` / `service_<name>_command` / `service_<name>_log_size` | `BaseService` reconciling loop | Generic per-service controls written by the Services tab; the loop reconciles within ~1s |
 | `session_max_age_days` / `session_remember_days` | `auth.py` signup + login | Multiplied by `SECONDS_PER_DAY`; passed to `create_session(uid, max_age)` so the cookie and the DB session row expire together |
 | `registration_open` | `auth.py` `signup_page` (GET) and `signup` (POST) | POST returns before any DB write when closed |
+| `happy_404_enabled` | `happy404.render()`, called from the `404` handler in `main.py` | Gates the whole feature; checked per-request via the normal 60s `get_setting` TTL cache, so a toggle takes effect within a minute across all workers with no restart |
 
 **Booleans are `<select>`, never checkboxes.** The settings save handler (`admin.py`) skips empty form values so empty fields don't clobber existing rows. An unchecked checkbox submits nothing, so it could never be turned off - `registration_open` and `maintenance_mode` use `<option value="1">`/`<option value="0">` so a value is always submitted.
 

devplacepy/database/__init__.py

@@ -74,7 +74,7 @@ from .moderation import (
     years_between,
 )
 from .comments import _drop_blocked, _build_comment_items, load_comments, get_recent_comments_by_target_uids, get_recent_comments_by_post_uids, load_comments_by_target_uids
-from .content import resolve_by_slug, resolve_object_url, get_projects_by_uids, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news, get_trending_topics
+from .content import resolve_by_slug, resolve_object_url, get_projects_by_uids, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news, get_featured_topics, get_trending_topics, get_user_recent_items, invalidate_user_recent_cache
 from .attachments_data import get_attachments, get_attachments_by_type, get_news_images_by_uids, delete_attachment_record, delete_attachments, _delete_attachment_file, get_user_media, get_user_attachments, get_user_attachment, get_deleted_media
 from .stats import _stats_cache, get_site_stats, _analytics_cache, get_platform_analytics, _gist_languages_cache, get_gist_languages
 from .schema import BUG_TABLE_RENAMES, migrate_bug_tables_to_issue_tables, init_db, _refresh_query_planner_stats, OLD_GATEWAY_URL, migrate_ai_gateway_settings, backfill_api_keys, _backfill_gamification
@@ -301,6 +301,9 @@ __all__ = [
     "text_search_clause",
     "get_daily_topic",
     "get_featured_news",
+    "get_featured_topics",
+    "get_user_recent_items",
+    "invalidate_user_recent_cache",
     "get_trending_topics",
     "get_attachments",
     "get_attachments_by_type",

devplacepy/database/content.py

@@ -1,5 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
+import os
+import random
 from collections import Counter
 
 from devplacepy.cache import TTLCache
@@ -8,6 +10,34 @@ from .core import db, get_table, or_
 
 _daily_topic_cache = TTLCache(ttl=60, max_size=1)
 _trending_cache = TTLCache(ttl=15, max_size=1)
+FEATURED_TOPICS_POOL_TTL = int(os.environ.get("DEVPLACE_FEATURED_TOPICS_POOL_TTL", "300"))
+FEATURED_TOPICS_POOL_SIZE = 20
+_featured_topics_cache = TTLCache(ttl=FEATURED_TOPICS_POOL_TTL, max_size=1)
+
+USER_RECENT_ITEMS_TTL = int(os.environ.get("DEVPLACE_USER_RECENT_ITEMS_TTL", "15"))
+_user_recent_cache = TTLCache(ttl=USER_RECENT_ITEMS_TTL, max_size=2000)
+
+
+def _user_recent_key(table_name: str, user_uid: str) -> str:
+    return f"{table_name}:{user_uid}"
+
+
+def invalidate_user_recent_cache(table_name: str, user_uid: str) -> None:
+    _user_recent_cache.pop(_user_recent_key(table_name, user_uid))
+
+
+def get_user_recent_items(table_name: str, user_uid: str) -> list[dict]:
+    key = _user_recent_key(table_name, user_uid)
+    cached = _user_recent_cache.get(key)
+    if cached is not None:
+        return cached
+    items = []
+    if table_name in db.tables:
+        rows = list(get_table(table_name).find(user_uid=user_uid, deleted_at=None))
+        rows.sort(key=lambda row: row.get("updated_at") or row.get("created_at") or "", reverse=True)
+        items = rows
+    _user_recent_cache.set(key, items)
+    return items
 
 
 def resolve_by_slug(table, slug, include_deleted=False):
@@ -171,6 +201,49 @@ def _load_daily_topic():
     }
 
 
+def get_featured_topics(count: int = 3) -> list[dict]:
+    pool = _featured_topics_pool()
+    if not pool:
+        return []
+    return random.sample(pool, min(count, len(pool)))
+
+
+def _featured_topics_pool() -> list[dict]:
+    cached = _featured_topics_cache.get("pool")
+    if cached is not None:
+        return cached
+    pool = _load_featured_topics_pool()
+    _featured_topics_cache.set("pool", pool)
+    return pool
+
+
+def _load_featured_topics_pool(limit: int = FEATURED_TOPICS_POOL_SIZE) -> list[dict]:
+    if "news" not in db.tables:
+        return []
+    rows = db["news"].find(
+        featured=1,
+        status="published",
+        deleted_at=None,
+        order_by=["-synced_at"],
+        _limit=limit,
+    )
+    topics = []
+    for article in rows:
+        desc = (article.get("description") or "")[:160] or (
+            article.get("content") or ""
+        )[:160]
+        topics.append(
+            {
+                "title": article.get("title", ""),
+                "summary": desc,
+                "slug": article.get("slug", ""),
+                "url": article.get("url", ""),
+                "image_url": article.get("image_url", "") or "",
+            }
+        )
+    return topics
+
+
 def get_featured_news(limit=5):
     if "news" not in db.tables:
         return []

devplacepy/database/schema.py

@@ -1964,6 +1964,7 @@ def init_db():
             "maintenance_message": "DevPlace is undergoing scheduled maintenance. Please check back shortly.",
             "customization_enabled": "1",
             "customization_js_enabled": "1",
+            "happy_404_enabled": "1",
             "audit_log_retention_days": "90",
             "statistics_tracking_enabled": "1",
             "docs_search_mode": "agent",

devplacepy/happy404.py

@@ -0,0 +1,70 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import logging
+import os
+import random
+
+from fastapi import Request
+from fastapi.responses import HTMLResponse
+
+from devplacepy.cache import TTLCache
+from devplacepy.content import load_detail
+from devplacepy.database import db, get_setting
+from devplacepy.routers.posts import post_page_context
+from devplacepy.templating import templates
+from devplacepy.utils import get_current_user
+
+logger = logging.getLogger("happy404")
+
+POOL_TTL_SECONDS = int(os.environ.get("DEVPLACE_HAPPY_404_POOL_TTL", "300"))
+POOL_SIZE = 100
+
+API_PATH_PREFIXES = ("/api", "/dbapi", "/openai", "/xmlrpc", "/swagger", "/openapi.json")
+
+_pool_cache = TTLCache(ttl=POOL_TTL_SECONDS, max_size=1)
+
+
+def _post_pool() -> list[str]:
+    cached = _pool_cache.get("slugs")
+    if cached is not None:
+        return cached
+    slugs: list[str] = []
+    if "posts" in db.tables:
+        rows = db.query(
+            "SELECT slug, uid FROM posts WHERE deleted_at IS NULL ORDER BY RANDOM() LIMIT :limit",
+            limit=POOL_SIZE,
+        )
+        slugs = [row["slug"] or row["uid"] for row in rows]
+    _pool_cache.set("slugs", slugs)
+    return slugs
+
+
+def _eligible(request: Request) -> bool:
+    if request.method != "GET":
+        return False
+    if request.url.path.startswith(API_PATH_PREFIXES):
+        return False
+    return get_setting("happy_404_enabled", "1") == "1"
+
+
+def render(request: Request) -> HTMLResponse | None:
+    try:
+        if not _eligible(request):
+            return None
+        pool = _post_pool()
+        if not pool:
+            return None
+        slug = random.choice(pool)
+        user = get_current_user(request)
+        detail = load_detail("posts", "post", slug, user)
+        if not detail:
+            return None
+        context = post_page_context(
+            request, user, detail, robots="noindex,nofollow"
+        )
+        return templates.TemplateResponse(request, "post.html", context)
+    except Exception as exc:  # noqa: BLE001 - a happy-404 bug must never break the 404 page
+        logger.warning("happy_404 render failed: %s", exc)
+        return None

devplacepy/main.py

@@ -33,7 +33,7 @@ from devplacepy.database import (
     get_news_images_by_uids,
     get_setting,
     get_int_setting,
-    interleave_by_author,
+    paginate_diverse,
     get_user_post_count,
     get_user_stars,
     get_blocked_uids,
@@ -43,6 +43,7 @@ from devplacepy.database import (
 from devplacepy.templating import templates, jinja_unread_count
 from devplacepy.cache import TTLCache
 from devplacepy.responses import respond, wants_json, json_error
+from devplacepy import happy404
 from devplacepy.schemas import LandingOut, ValidationErrorOut
 from fastapi.responses import JSONResponse
 from devplacepy.utils import get_current_user, time_ago, safe_next, client_ip
@@ -345,6 +346,9 @@ app.mount("/static", FallbackStaticFiles(directory=str(STATIC_DIR)), name="stati
 async def not_found(request: Request, exc):
     if wants_json(request):
         return json_error(404, "Not found")
+    happy_response = happy404.render(request)
+    if happy_response is not None:
+        return happy_response
     seo_ctx = base_seo_context(
         request,
         title="Not Found - DevPlace",
@@ -790,7 +794,8 @@ def _landing_news():
     return articles
 
 
-def _landing_recent_posts(blocked):
+def _landing_recent_posts(viewer_uid):
+    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
     if not blocked:
         cached = _home_cache.get("posts")
         if cached is not None:
@@ -798,15 +803,9 @@ def _landing_recent_posts(blocked):
     posts = []
     if "posts" in db.tables:
         posts_table = get_table("posts")
-        clauses = []
+        raw_posts, _ = paginate_diverse(
-        if blocked:
+            posts_table, order=["-created_at"], viewer_uid=viewer_uid, limit=6
-            clauses.append(posts_table.table.columns.user_uid.notin_(blocked))
-        raw_posts = list(
-            posts_table.find(
-                *clauses, deleted_at=None, order_by=["-created_at"], _limit=6
-            )
         )
-        raw_posts = interleave_by_author(raw_posts)
         if raw_posts:
             post_uids = [p["uid"] for p in raw_posts]
             author_uids = [p["user_uid"] for p in raw_posts]
@@ -834,8 +833,9 @@ async def landing(request: Request):
     user = get_current_user(request)
 
     landing_articles = _landing_news()
-    blocked = get_blocked_uids(user["uid"]) if user else frozenset()
+    viewer_uid = user["uid"] if user else None
-    landing_posts = _landing_recent_posts(blocked)
+    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
+    landing_posts = _landing_recent_posts(viewer_uid)
 
     base = site_url(request)
     seo_ctx = base_seo_context(

devplacepy/models.py

@@ -681,6 +681,7 @@ class AdminSettingsForm(BaseModel):
     registration_open: str = Field(default="", max_length=1)
     maintenance_mode: str = Field(default="", max_length=1)
     maintenance_message: str = Field(default="", max_length=300)
+    happy_404_enabled: str = Field(default="", max_length=1)
     docs_search_mode: str = Field(default="", max_length=20)
     outbound_proxy_url: str = Field(default="", max_length=500)
     moderation_sla_hours: str = Field(default="", max_length=10)

devplacepy/routers/CLAUDE.md

@@ -275,15 +275,22 @@ All SEO features are implemented across the following locations:
 - `profile.html` - username rendered as `<h1 class="profile-name">`
 - `messages.html` - `<h1 class="sr-only">Messages</h1>`
 - `projects.html` - `<h1>Projects</h1>`
-- `post.html` - post title as `<h1>`, "Related Discussions" as `<h3>`
+- `post.html` - post title as `<h1>`; "Gists from {author}", "Projects from {author}", and "Related Discussions" are `.sidebar-heading` labels in the left column (see "Post page layout" below), not `<h3>`
 
 ### Post slugs
 - Slug generated on post creation via `slugify()` and stored in `posts.slug` column
 - Posts can be looked up by slug or UUID
 - Minimum content validation: post body >= 10 chars, comment >= 3 chars
 
-### Related posts
+### Post page layout (three columns, mirrors `/feed`)
-- `templates/post.html` - "Related Discussions" widget at bottom of post page (queried by matching topic)
+
+`post.html` reuses `/feed`'s exact layout building blocks rather than inventing new ones, wrapped in `.post-page-layout` (`static/css/post.css`, `grid-template-columns: var(--sidebar-width) minmax(0, 1fr) 280px`, collapsing to one column at 1024px):
+
+- **Left column** - `<aside class="post-page-sidebar">`, sticky, holding up to three separate `.sidebar-card` blocks (never merged into one card - each is its own bordered panel): "Gists from {author}" and "Projects from {author}" (`content.get_user_sidebar_gists`/`get_user_sidebar_projects`, up to 5 each, most-recently-modified first, private projects filtered via `can_view_project`), and "Related Discussions" (same-topic posts). Every card title is a `.sidebar-heading` div (the same class `feed.html`'s left sidebar uses for "Topics"/"Resources"/"Online now" - `.sidebar-card .sidebar-heading` in `sidebar.css`, so it only styles correctly nested inside a `.sidebar-card`, never bare). Each list item reuses the plain `.related-list`/`.related-link`/`.related-title`/`.related-meta` classes (`base.css`) with `content_preview()` for the ellipsis-truncated description.
+- **Middle column** - `.post-page` (`max-width: 720px`): the post article, comments.
+- **Right column** - `<aside class="feed-right">`, the exact same class `feed.html` uses for its Daily Topic widget (sticky, hidden below 1024px via `feed.css`'s own media query - no post-page-specific override needed). Holds the "Featured" cards (`database.get_featured_topics`, up to 3, cached pool + per-request random sample): each is a `.daily-topic-card` with its own `.daily-topic-label` ("Featured") - matching the single Daily Topic card's internal label, since there is no section-level heading here (a `.sidebar-heading` div placed directly in `.feed-right` would NOT be styled, as noted above - the fix used is a per-card label instead of a bare heading).
+
+All three columns are populated by `routers/posts.py` `post_page_context()`, the single context builder shared by the real `/posts/{slug}` route and `happy404.render()` (see the root `CLAUDE.md`), so a decoy happy-404 post page renders with the identical sidebar/featured layout as a real one.
 
 ### Performance
 - `loading="lazy"` on all avatar images

devplacepy/routers/posts.py

@@ -12,6 +12,8 @@ from devplacepy.database import (
     resolve_by_slug,
     resolve_object_url,
     mark_notifications_read_by_target,
+    get_featured_topics,
+    get_blocked_uids,
 )
 from devplacepy.utils import (
     get_current_user,
@@ -29,6 +31,8 @@ from devplacepy.content import (
     detail_context,
     canonical_redirect,
     first_image_url,
+    get_user_sidebar_gists,
+    get_user_sidebar_projects,
 )
 from devplacepy.responses import respond, action_result
 from devplacepy.schemas import PostDetailOut
@@ -137,20 +141,26 @@ def create_poll(
         links=[audit.poll(poll_uid, question), audit.parent("post", post_uid)],
     )
 
-@router.get("/{post_slug}", response_class=HTMLResponse)
+def _next_post_url(post: dict, viewer: dict | None) -> str | None:
-async def view_post(request: Request, post_slug: str):
+    if "posts" not in db.tables:
-    user = get_current_user(request)
+        return None
-    detail = load_detail("posts", "post", post_slug, user)
+    blocked = get_blocked_uids(viewer["uid"]) if viewer else frozenset()
-    if not detail:
+    rows = db.query(
-        raise not_found("Post not found")
+        "SELECT slug, uid, user_uid FROM posts WHERE created_at < :created_at "
+        "AND deleted_at IS NULL ORDER BY created_at DESC LIMIT :limit",
+        created_at=post["created_at"],
+        limit=10,
+    )
+    for row in rows:
+        if row["user_uid"] not in blocked:
+            return f"/posts/{row['slug'] or row['uid']}"
+    return None
+
+
+def post_page_context(
+    request: Request, user: dict | None, detail: dict, *, robots: str | None = None
+) -> dict:
     post = detail["item"]
-    redirect = canonical_redirect("posts", post, post_slug)
-    if redirect:
-        return redirect
-    if user:
-        mark_notifications_read_by_target(
-            user["uid"], resolve_object_url("post", post["uid"])
-        )
     author = detail["author"]
     top_level = detail["comments"]
 
@@ -162,10 +172,12 @@ async def view_post(request: Request, post_slug: str):
 
     comment_count = count_all(top_level)
     base = site_url(request)
+    next_post_url = _next_post_url(post, user)
     seo_ctx = base_seo_context(
         request,
         title=post.get("title") or "Post",
         description=post.get("content", ""),
+        robots=robots or "index,follow",
         seo_target=("post", post["uid"]),
         breadcrumbs=[
             {"name": "Home", "url": "/feed"},
@@ -177,6 +189,7 @@ async def view_post(request: Request, post_slug: str):
         ],
         og_type="article",
         og_image=first_image_url(post, detail["attachments"]),
+        next_url=next_post_url,
         schemas=[
             website_schema(base),
             discussion_forum_posting(
@@ -208,21 +221,43 @@ async def view_post(request: Request, post_slug: str):
                 }
             )
 
+    author_uid = post["user_uid"]
+    return detail_context(
+        request,
+        user,
+        detail,
+        "post",
+        seo_ctx,
+        {
+            "comment_count": comment_count,
+            "related_posts": related_posts,
+            "topics": list(TOPICS),
+            "featured_topics": get_featured_topics(3),
+            "author_gists": get_user_sidebar_gists(author_uid, 5),
+            "author_projects": get_user_sidebar_projects(author_uid, user, 5),
+            "next_post_url": next_post_url,
+        },
+    )
+
+
+@router.get("/{post_slug}", response_class=HTMLResponse)
+async def view_post(request: Request, post_slug: str):
+    user = get_current_user(request)
+    detail = load_detail("posts", "post", post_slug, user)
+    if not detail:
+        raise not_found("Post not found")
+    post = detail["item"]
+    redirect = canonical_redirect("posts", post, post_slug)
+    if redirect:
+        return redirect
+    if user:
+        mark_notifications_read_by_target(
+            user["uid"], resolve_object_url("post", post["uid"])
+        )
     return respond(
         request,
         "post.html",
-        detail_context(
+        post_page_context(request, user, detail),
-            request,
-            user,
-            detail,
-            "post",
-            seo_ctx,
-            {
-                "comment_count": comment_count,
-                "related_posts": related_posts,
-                "topics": list(TOPICS),
-            },
-        ),
         model=PostDetailOut,
     )
 

devplacepy/schemas/listings.py

@@ -158,6 +158,10 @@ class PostDetailOut(_Out):
     related_posts: list[FeedItemOut] = []
     topics: list[str] = []
     project_link: Optional[ProjectLinkOut] = None
+    featured_topics: list[Any] = []
+    author_gists: list[Any] = []
+    author_projects: list[Any] = []
+    next_post_url: Optional[str] = None
 
 
 class ProjectsOut(_Out):

devplacepy/services/devii/actions/catalog/admin.py

@@ -188,6 +188,7 @@ ADMIN_ACTIONS: tuple[Action, ...] = (
             body("registration_open", "Whether registration is open."),
             body("maintenance_mode", "Whether maintenance mode is on."),
             body("maintenance_message", "Maintenance message."),
+            body("happy_404_enabled", "Whether Happy 404 (rendering a random post instead of a 404 page) is on."),
         ),
         requires_admin=True,
     ),

devplacepy/static/css/base.css

@@ -176,9 +176,6 @@
 .topic-wrap {
     margin-bottom: 0.75rem;
 }
-.related-section {
-    margin-top: 1.5rem;
-}
 .related-list {
     display: flex;
     flex-direction: column;

devplacepy/static/css/post.css

@@ -1,8 +1,54 @@
 /* retoor <retoor@molodetz.nl> */
 
+.post-page-layout {
+    display: grid;
+    grid-template-columns: var(--sidebar-width) minmax(0, 1fr) 280px;
+    gap: 1.5rem;
+    align-items: start;
+}
+
+.post-page-sidebar {
+    display: flex;
+    flex-direction: column;
+    gap: 1rem;
+    position: sticky;
+    top: calc(var(--nav-height) + 1rem);
+}
+
+.post-page-sidebar .sidebar-card {
+    position: static;
+}
+
 .post-page {
     max-width: 720px;
-    margin: 0 auto;
+    margin: 0;
+}
+
+.post-nav-row {
+    display: flex;
+    align-items: center;
+    justify-content: space-between;
+    flex-wrap: wrap;
+    gap: 0.75rem;
+    margin-bottom: 1rem;
+}
+
+.post-nav-row .back-link {
+    margin-bottom: 0;
+}
+
+@media (max-width: 1024px) {
+    .post-page-layout {
+        grid-template-columns: minmax(0, 1fr);
+    }
+
+    .post-page-sidebar {
+        display: none;
+    }
+
+    .post-page {
+        margin: 0 auto;
+    }
 }
 
 .post-detail {
@@ -284,3 +330,4 @@
         padding-left: 0.375rem;
     }
 }
+

devplacepy/templates/admin_settings.html

@@ -95,6 +95,15 @@
         <small class="hint-text">Shown to visitors while maintenance mode is enabled.</small>
     </div>
 
+    <div class="admin-field">
+        <label for="happy_404_enabled">Happy 404</label>
+        <select id="happy_404_enabled" name="happy_404_enabled" class="admin-settings-select">
+            <option value="1" {% if settings.get('happy_404_enabled', '1') == '1' %}selected{% endif %}>Enabled</option>
+            <option value="0" {% if settings.get('happy_404_enabled', '1') != '1' %}selected{% endif %}>Disabled</option>
+        </select>
+        <small class="hint-text">When enabled, a page that would otherwise 404 renders a random existing post instead. HTML requests only - JSON/API clients still get a real 404.</small>
+    </div>
+
     <div class="admin-field">
         <label for="rate_limit_per_minute">Rate Limit (requests)</label>
         <input type="number" id="rate_limit_per_minute" name="rate_limit_per_minute" value="{{ settings.get('rate_limit_per_minute', '60') }}" min="1" class="admin-settings-num">

devplacepy/templates/docs/architecture-backend.html

@@ -9,6 +9,10 @@ How a request flows through middleware to a router, how the database layer is bu
 
 `main.py` builds the FastAPI app, mounts `/static`, registers every router with its prefix, installs the middlewares, and registers global 404, 500, and `RequestValidationError` handlers that render `error.html` through the shared `templates` instance.
 
+### Happy 404
+
+Before falling back to `error.html`, the 404 handler calls `happy404.render(request)` (`devplacepy/happy404.py`). For an HTML `GET` request outside the API-ish path prefixes (`/api`, `/dbapi`, `/openai`, `/xmlrpc`, `/swagger`, `/openapi.json`), it picks a random slug from a process-local pool of existing post slugs, loads that post through the normal `content.load_detail` pipeline, and renders `post.html` (the exact context built by `routers/posts.py`'s `post_page_context`, reused by the real `/posts/{slug}` route) at the requested URL - with `meta_robots` forced to `noindex,nofollow` so the decoy URL is never indexed. The pool (`POOL_SIZE` = 100 slugs) is refreshed from `SELECT slug, uid FROM posts WHERE deleted_at IS NULL ORDER BY RANDOM() LIMIT :limit` on a `TTLCache` (`POOL_TTL_SECONDS` = 300s) so the expensive random-order scan runs at most once per five minutes per worker, while each request only does an O(1) in-memory `random.choice` plus one indexed slug lookup - fast enough to run on every 404. A fresh sample every TTL window means the pool eventually surfaces every post, not just a fixed subset. JSON/API requests are unaffected (checked before `happy404.render` is even called) and are excluded a second time via the path-prefix guard as defense in depth. The whole function is wrapped in a catch-all that logs and returns `None` on any failure, falling through to the normal `error.html` 404 - a bug in this feature can never break the 404 page itself. Toggle with the `happy_404_enabled` site setting (`/admin/settings`, default on).
+
 ## Middleware
 
 Seven HTTP middlewares run as a stack around every request, listed outermost first. `response_timing` is the outermost, `refresh_db_snapshot` the innermost, and a `GZipMiddleware` (responses over 512 bytes) wraps the whole stack on top:

devplacepy/templates/post.html

@@ -1,12 +1,68 @@
 {% extends "base.html" %}
 {% from "_macros.html" import modal %}
 {% block extra_head %}
+<link rel="stylesheet" href="{{ static_url('/static/css/sidebar.css') }}">
 <link rel="stylesheet" href="{{ static_url('/static/css/feed.css') }}">
 <link rel="stylesheet" href="{{ static_url('/static/css/post.css') }}">
 {% endblock %}
 {% block content %}
-<div class="post-page">
+<div class="post-page-layout">
-    <a href="/feed" class="back-link">&larr; Back to Feed</a>
+    {% if author_gists or author_projects or related_posts %}
+    <aside class="post-page-sidebar" role="complementary" aria-label="More to explore">
+        {% if author_gists %}
+        <div class="sidebar-card">
+            <div class="sidebar-heading">Gists from {{ author.get('username', '') if author else '' }}</div>
+            <div class="related-list">
+                {% for gist in author_gists %}
+                <a href="/gists/{{ gist['slug'] or gist['uid'] }}" class="related-link">
+                    <strong class="related-title">{{ gist.get('title') or 'Untitled gist' }}</strong>
+                    <span class="related-meta">{{ content_preview(gist.get('description') or gist.get('source_code', ''), 80) }}</span>
+                </a>
+                {% endfor %}
+            </div>
+            <div class="topic-links">
+                <a href="/profile/{{ author.get('username', '') if author else '' }}?tab=gists">View all</a>
+            </div>
+        </div>
+        {% endif %}
+        {% if author_projects %}
+        <div class="sidebar-card">
+            <div class="sidebar-heading">Projects from {{ author.get('username', '') if author else '' }}</div>
+            <div class="related-list">
+                {% for project in author_projects %}
+                <a href="/projects/{{ project['slug'] or project['uid'] }}" class="related-link">
+                    <strong class="related-title">{{ project.get('title') or 'Untitled project' }}</strong>
+                    <span class="related-meta">{{ content_preview(project.get('description', ''), 80) }}</span>
+                </a>
+                {% endfor %}
+            </div>
+            <div class="topic-links">
+                <a href="/profile/{{ author.get('username', '') if author else '' }}?tab=projects">View all</a>
+            </div>
+        </div>
+        {% endif %}
+        {% if related_posts %}
+        <div class="sidebar-card">
+            <div class="sidebar-heading">Related Discussions</div>
+            <div class="related-list">
+                {% for item in related_posts %}
+                <a href="/posts/{{ item.post['slug'] or item.post['uid'] }}" class="related-link">
+                    <strong class="related-title">{{ render_title(item.post['title'], author_is_admin=is_admin(item.author)) if item.post.get('title') else content_preview(item.post['content'], 60) }}</strong>
+                    <span class="related-meta">{{ item.author['username'] }} &middot; {{ dt_ago(item.created_at) if item.created_at else item.time_ago }}</span>
+                </a>
+                {% endfor %}
+            </div>
+        </div>
+        {% endif %}
+    </aside>
+    {% endif %}
+    <div class="post-page">
+    <div class="post-nav-row">
+        <a href="/feed" class="back-link">&larr; Back to Feed</a>
+        {% if next_post_url %}
+        <a href="{{ next_post_url }}" class="back-link next-post-link">Next post &rarr;</a>
+        {% endif %}
+    </div>
 
     <article class="post-detail">
         <div class="post-detail-header">
@@ -103,19 +159,32 @@
     {% with target_uid=post['uid'], target_type="post" %}
     {% include "_comment_section.html" %}
     {% endwith %}
+    </div>
 
-    {% if related_posts %}
+    {% if featured_topics %}
-    <section class="comments-section related-section">
+    <aside class="feed-right" role="complementary" aria-label="Featured">
-        <h3>Related Discussions</h3>
+        {% for topic in featured_topics %}
-        <div class="related-list">
+        <div class="daily-topic-card card-link-host">
-            {% for item in related_posts %}
+            {% if topic.get('slug') %}
-            <a href="/posts/{{ item.post['slug'] or item.post['uid'] }}" class="related-link">
+            {% set _href = "/news/" ~ topic['slug'] %}
-                <strong class="related-title">{{ render_title(item.post['title'], author_is_admin=is_admin(item.author)) if item.post.get('title') else content_preview(item.post['content'], 60) }}</strong>
+            {% set _label = topic.get('title', '') %}
-                <span class="related-meta">{{ item.author['username'] }} &middot; {{ dt_ago(item.created_at) if item.created_at else item.time_ago }}</span>
+            {% include "_card_link.html" %}
-            </a>
+            {% endif %}
-            {% endfor %}
+            <div class="daily-topic-label">Featured</div>
+            {% if topic.get('image_url', '') %}
+            <img src="{{ topic['image_url'] }}" alt="{{ topic.get('title', '') }}" loading="lazy" class="daily-topic-image">
+            {% endif %}
+            <h4 class="rendered-title">{{ render_title(topic.get('title', '')) }}</h4>
+            <div class="rendered-content">{{ render_content(topic.get('summary', '')) }}</div>
+            <div class="topic-links">
+                <a href="/news/{{ topic.get('slug', '') }}">Read More</a>
+                {% if topic.get('url', '') %}
+                <a href="{{ topic['url'] }}" target="_blank" rel="noopener">Source</a>
+                {% endif %}
+            </div>
         </div>
-    </section>
+        {% endfor %}
+    </aside>
     {% endif %}
 </div>
 {% endblock %}

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "devplacepy"
-version = "1.0.5"
+version = "1.0.6"
 description = "DevPlace - The Developer Social Network"
 requires-python = ">=3.12"
 dependencies = [

tests/api/admin/awards.py

@@ -1,12 +1,13 @@
 # retoor <retoor@molodetz.nl>
 
 import io
+import time
 import requests
 from datetime import datetime, timezone
 from PIL import Image
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.attachments import store_attachment
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.utils import generate_uid, make_combined_slug
 
 JSON = {"Accept": "application/json"}
@@ -87,7 +88,13 @@ def test_admin_revoke_soft_deletes_and_recomputes(seeded_db):
     assert get_table("attachments").find_one(uid=att512).get("deleted_at")
     bob = get_table("users").find_one(username="bob_test")
     assert bob.get("award_count") == 0
-    assert requests.get(f"{BASE_URL}/awards/{slug}/64").status_code == 404
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        assert requests.get(f"{BASE_URL}/awards/{slug}/64").status_code == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_media_gallery_hides_revoked_attachment(seeded_db):

tests/api/admin/bots/index.py

@@ -1,8 +1,10 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
 
 JSON = {"Accept": "application/json"}
 
@@ -49,8 +51,14 @@ def test_bots_data_shape_for_admin(app_server, seeded_db):
 
 def test_bots_frame_missing_is_404(app_server, seeded_db):
     admin = _admin()
-    r = admin.get(f"{BASE_URL}/admin/bots/999/frame.jpg", allow_redirects=False)
+    set_setting("happy_404_enabled", "0")
-    assert r.status_code == 404
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        r = admin.get(f"{BASE_URL}/admin/bots/999/frame.jpg", allow_redirects=False)
+        assert r.status_code == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_bots_frame_requires_admin(app_server, seeded_db):

tests/api/admin/media/purge.py

@@ -1,11 +1,12 @@
 # retoor <retoor@molodetz.nl>
 
 import io
+import time
 import uuid
 import requests
 from PIL import Image
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 JSON_media = {"Accept": "application/json"}
 def _png_bytes_media(color=(200, 30, 30)):
     buf = io.BytesIO()
@@ -130,4 +131,10 @@ def test_admin_purge_hard_deletes(seeded_db):
     # row gone (not in trash, cannot restore) and file removed from disk
     trash = admin.get(f"{BASE_URL}/admin/media", headers=JSON_media).json()
     assert all(m["uid"] != uid for m in trash["media"])
-    assert owner.get(f"{BASE_URL}{file_url}").status_code == 404
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        assert owner.get(f"{BASE_URL}{file_url}").status_code == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)

tests/api/admin/trash/restore.py

@@ -2,7 +2,8 @@
 
 import time
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+from devplacepy.database import set_setting
 
 JSON_trash = {"Accept": "application/json"}
 _counter_trash = [0]
@@ -146,7 +147,13 @@ def test_restore_revoked_award_recomputes_stats(seeded_db):
     row = get_table("users").find_one(uid=bob["uid"])
     assert row.get("award_count") == 1
     assert row.get("last_award_uid") == uid
-    assert requests.get(f"{BASE_URL}/awards/{slug}/256").status_code in (302, 404)
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        assert requests.get(f"{BASE_URL}/awards/{slug}/256").status_code in (302, 404)
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def _create_quiz_trash(session):

tests/api/awards/index.py

@@ -1,15 +1,26 @@
 # retoor <retoor@molodetz.nl>
 
 import io
+import time
+import pytest
 import requests
 from datetime import datetime, timezone
 from PIL import Image
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.attachments import store_attachment
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.utils import generate_uid, make_combined_slug
 
 
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _png():
     buf = io.BytesIO()
     Image.new("RGBA", (32, 32), (20, 40, 60, 255)).save(buf, format="PNG")

tests/api/battles/events.py

@@ -5,7 +5,8 @@ import time
 import requests
 
 from tests.api.battles._helpers import _create_war_post, _fight, _join, _session_battles
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+from devplacepy.database import set_setting
 
 
 def test_events_ordered_and_incremental(app_server):
@@ -39,5 +40,11 @@ def test_events_ordered_and_incremental(app_server):
 
 
 def test_events_unknown_battle_404(app_server):
-    r = requests.get(f"{BASE_URL}/battles/nope/events")
+    set_setting("happy_404_enabled", "0")
-    assert r.status_code == 404
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        r = requests.get(f"{BASE_URL}/battles/nope/events")
+        assert r.status_code == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)

tests/api/block/visibility.py

@@ -4,8 +4,8 @@ import time
 from uuid import uuid4
 from datetime import datetime, timezone
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.utils import make_combined_slug
 
 JSON = {"Accept": "application/json"}
@@ -110,8 +110,14 @@ def test_blocked_author_post_detail_404(app_server):
     post = _new_post(author_session)
     blocker_session.post(f"{BASE_URL}/block/{author}", allow_redirects=False)
 
-    r = blocker_session.get(f"{BASE_URL}/posts/{post['slug']}")
+    set_setting("happy_404_enabled", "0")
-    assert r.status_code == 404
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        r = blocker_session.get(f"{BASE_URL}/posts/{post['slug']}")
+        assert r.status_code == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_blocked_author_comment_hidden_on_detail(app_server):

tests/api/containers.py

@@ -1,10 +1,11 @@
 # retoor <retoor@molodetz.nl>
 
 from datetime import timedelta
+import time
 import pytest
 import requests
-from tests.conftest import BASE_URL, run_async
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS, run_async
-from devplacepy.database import db, get_table, init_db, refresh_snapshot
+from devplacepy.database import db, get_table, init_db, refresh_snapshot, set_setting
 from devplacepy import config, project_files
 from devplacepy.services.containers import api, store, runtime
 from devplacepy.services.containers.backend.base import Mount, PortMapping, RunSpec
@@ -98,12 +99,16 @@ def test_http_ingress_proxy(app_server):
         }
     )
     refresh_snapshot()
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     try:
         r = requests.get(f"{BASE_URL}/p/{slug}/foo")
         assert r.status_code == 200, r.text
         assert "hello from upstream" in r.text and "/foo" in r.text
         assert requests.get(f"{BASE_URL}/p/does-not-exist-slug").status_code == 404
     finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
         httpd.shutdown()
         get_table("instances").delete(uid=uid)
         refresh_snapshot()

tests/api/content/permissions.py

@@ -3,7 +3,7 @@
 import time
 import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.database import get_table, refresh_snapshot, set_setting
 
 JSON = {"Accept": "application/json"}
@@ -19,9 +19,13 @@ def _perm_settings(app_server):
         "maintenance_mode": "0",
         "max_attachments_per_resource": "10",
         "allowed_file_types": "",
+        "happy_404_enabled": "0",
     }.items():
         set_setting(key, value)
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def _member():

tests/api/docs/downloadmd.py

@@ -3,11 +3,24 @@
 import html
 import json
 import re
+import time
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.docs_api import API_GROUPS
 from devplacepy.utils import clear_user_cache
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _configs_by_id(text):
     configs = re.findall(r"data-config='(.*?)'", text, re.S)
     return {

tests/api/docs/index.py

@@ -4,7 +4,7 @@ import time
 from datetime import datetime, timezone
 import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.database import get_table, refresh_snapshot, set_setting
 from devplacepy.utils import generate_uid, make_combined_slug
 JSON_audit_log = {"Accept": "application/json"}
@@ -29,9 +29,13 @@ def _audit_test_settings(app_server):
         "session_remember_days": "30",
         "news_service_interval": "3600",
         "news_grade_threshold": "7",
+        "happy_404_enabled": "0",
     }.items():
         set_setting(key, value)
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 def _db_user(name):
     # the user is created by the server subprocess; refresh the test-process
     # SQLite snapshot before reading it back across the process boundary.

tests/api/error/pages.py

@@ -1,10 +1,80 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
 import requests
-from tests.conftest import BASE_URL
+
+from devplacepy.database import set_setting
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+
+JSON = {"Accept": "application/json"}
+
+
+def _set_happy_404(enabled: bool) -> None:
+    set_setting("happy_404_enabled", "1" if enabled else "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
+def _login(seeded_db, who="alice"):
+    creds = seeded_db[who]
+    session = requests.Session()
+    session.post(
+        f"{BASE_URL}/auth/login",
+        data={"email": creds["email"], "password": creds["password"]},
+    )
+    return session
 
 
 def test_404_renders_error_page(app_server):
-    r = requests.get(f"{BASE_URL}/this-page-does-not-exist-xyz")
+    _set_happy_404(False)
+    try:
+        r = requests.get(f"{BASE_URL}/this-page-does-not-exist-xyz")
+        assert r.status_code == 404
+        assert "404" in r.text or "not found" in r.text.lower()
+    finally:
+        _set_happy_404(True)
+
+
+def test_happy_404_renders_a_real_post(app_server, seeded_db):
+    session = _login(seeded_db)
+    title = f"Happy404 Marker {int(time.time() * 1000)}"
+    created = session.post(
+        f"{BASE_URL}/posts/create",
+        headers=JSON,
+        data={
+            "title": title,
+            "content": "content seeded so the happy 404 pool has something to pick.",
+            "topic": "devlog",
+        },
+    ).json()["data"]
+    assert created["uid"]
+
+    _set_happy_404(True)
+    r = requests.get(f"{BASE_URL}/this-path-was-never-registered-anywhere")
+    assert r.status_code == 200
+    assert 'class="post-detail"' in r.text
+    assert "Page not found" not in r.text
+    assert 'name="robots" content="noindex,nofollow"' in r.text
+
+
+def test_happy_404_disabled_falls_back_to_error_page(app_server):
+    _set_happy_404(False)
+    try:
+        r = requests.get(f"{BASE_URL}/another-path-that-does-not-exist")
+        assert r.status_code == 404
+        assert "Page not found" in r.text
+    finally:
+        _set_happy_404(True)
+
+
+def test_happy_404_never_applies_to_json_requests():
+    _set_happy_404(True)
+    r = requests.get(f"{BASE_URL}/yet-another-missing-path", headers=JSON)
+    assert r.status_code == 404
+    assert r.json()["error"]["status"] == 404
+
+
+def test_happy_404_never_applies_to_api_paths():
+    _set_happy_404(True)
+    r = requests.get(f"{BASE_URL}/api/this-devrant-route-does-not-exist")
     assert r.status_code == 404
-    assert "404" in r.text or "not found" in r.text.lower()

tests/api/forks.py

@@ -1,14 +1,15 @@
 # retoor <retoor@molodetz.nl>
 
 import asyncio
+import time
 from datetime import datetime, timezone
 from pathlib import Path
 import pytest
-from devplacepy.database import init_db, get_table, refresh_snapshot
+from devplacepy.database import init_db, get_table, refresh_snapshot, set_setting
 from devplacepy import project_files
 from devplacepy.services.jobs import queue
 from devplacepy.services.jobs.fork_service import ForkService
-from tests.conftest import run_async
+from tests.conftest import CACHE_VERSION_PROPAGATION_SECONDS, run_async
 @pytest.fixture(autouse=True)
 def _init_db_fork_jobs():
     init_db()
@@ -109,5 +110,11 @@ def _enqueue(source_uid, owner_uid, title="My Fork"):
 def test_fork_status_http_unknown_returns_404(app_server):
     import requests
     from tests.conftest import BASE_URL
-    r = requests.get(f"{BASE_URL}/forks/nonexistent-uid")
+    set_setting("happy_404_enabled", "0")
-    assert r.status_code == 404
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        r = requests.get(f"{BASE_URL}/forks/nonexistent-uid")
+        assert r.status_code == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)

tests/api/issues/planning.py

@@ -1,12 +1,13 @@
 # retoor <retoor@molodetz.nl>
 
 import json
+import time
 
 import pytest
 import requests
 
 from devplacepy.database import get_table, refresh_snapshot, set_setting
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 
 
 @pytest.fixture(scope="module", autouse=True)
@@ -17,9 +18,13 @@ def _planning_settings(app_server):
     set_setting("gitea_repo", "pydevplace")
     set_setting("gitea_token", "planning-test-token")
     set_setting("issue_ai_enhance", "0")
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     yield
     set_setting("gitea_base_url", "")
     set_setting("gitea_token", "")
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def _admin(seeded_db):

tests/api/posts/create.py

@@ -4,7 +4,7 @@ import time
 from datetime import datetime, timezone
 import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.database import get_table, refresh_snapshot, set_setting
 from devplacepy.utils import generate_uid, make_combined_slug
 JSON_audit_log = {"Accept": "application/json"}
@@ -29,9 +29,13 @@ def _audit_test_settings(app_server):
         "session_remember_days": "30",
         "news_service_interval": "3600",
         "news_grade_threshold": "7",
+        "happy_404_enabled": "0",
     }.items():
         set_setting(key, value)
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 def _db_user(name):
     # the user is created by the server subprocess; refresh the test-process
     # SQLite snapshot before reading it back across the process boundary.

tests/api/posts/featured.py

@@ -0,0 +1,118 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+
+import requests
+
+from devplacepy.database import get_table
+from tests.conftest import BASE_URL
+
+JSON = {"Accept": "application/json"}
+_counter = [0]
+
+
+def _unique(prefix="feat"):
+    _counter[0] += 1
+    return f"{prefix}{int(time.time() * 1000)}{_counter[0]}"
+
+
+def _signup():
+    name = _unique("featuser")
+    session = requests.Session()
+    session.post(
+        f"{BASE_URL}/auth/signup",
+        data={
+            "username": name,
+            "email": f"{name}@t.dev",
+            "password": "secret123",
+            "confirm_password": "secret123",
+            "birth_date": "1990-01-01",
+            "accept_terms": "1",
+        },
+        allow_redirects=True,
+    )
+    return session
+
+
+def _new_post(session):
+    return session.post(
+        f"{BASE_URL}/posts/create",
+        headers=JSON,
+        data={
+            "title": _unique("featpost"),
+            "content": "content for the featured topics test post",
+            "topic": "devlog",
+        },
+    ).json()["data"]
+
+
+def _seed_featured_article():
+    uid = _unique("featnews")
+    get_table("news").insert(
+        {
+            "uid": uid,
+            "title": _unique("Featured Article"),
+            "slug": _unique("featured-article"),
+            "external_id": uid,
+            "status": "published",
+            "source_name": "test",
+            "url": "https://example.com/article",
+            "description": "a featured article for the post-page test",
+            "content": "",
+            "synced_at": "2024-01-01T00:00:00",
+            "grade": 5,
+            "ai_grade": 5,
+            "show_on_landing": 0,
+            "featured": 1,
+            "featured_locked": 1,
+            "landing_locked": 0,
+            "author": "",
+            "article_published": "",
+            "image_url": "",
+            "has_unique_image": 0,
+            "deleted_at": None,
+            "deleted_by": None,
+        }
+    )
+    return uid
+
+
+def test_post_page_shows_featured_topics(app_server):
+    session = _signup()
+    post = _new_post(session)
+    _seed_featured_article()
+
+    r = requests.get(f"{BASE_URL}/posts/{post['slug']}")
+    assert r.status_code == 200
+    assert 'class="feed-right"' in r.text
+    assert "daily-topic-card" in r.text
+
+
+def test_post_page_json_includes_featured_topics(app_server):
+    session = _signup()
+    post = _new_post(session)
+    _seed_featured_article()
+
+    r = requests.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON)
+    assert r.status_code == 200
+    data = r.json()
+    assert "featured_topics" in data
+    assert len(data["featured_topics"]) <= 3
+    assert all("title" in item for item in data["featured_topics"])
+
+
+def test_post_page_featured_topics_pick_varies(app_server):
+    session = _signup()
+    post = _new_post(session)
+    for _ in range(6):
+        _seed_featured_article()
+
+    seen = set()
+    for _ in range(20):
+        r = requests.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON)
+        assert r.status_code == 200
+        titles = tuple(item["title"] for item in r.json()["featured_topics"])
+        assert len(titles) == 3
+        assert len(set(titles)) == 3
+        seen.add(titles)
+    assert len(seen) > 1, "featured topics never varied across 20 requests"

tests/api/posts/nextpost.py

@@ -0,0 +1,72 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+
+import requests
+
+from tests.conftest import BASE_URL
+
+JSON = {"Accept": "application/json"}
+_counter = [0]
+
+
+def _unique(prefix="nextpost"):
+    _counter[0] += 1
+    return f"{prefix}{int(time.time() * 1000)}{_counter[0]}"
+
+
+def _signup():
+    name = _unique("nextuser")
+    session = requests.Session()
+    session.post(
+        f"{BASE_URL}/auth/signup",
+        data={
+            "username": name,
+            "email": f"{name}@t.dev",
+            "password": "secret123",
+            "confirm_password": "secret123",
+            "birth_date": "1990-01-01",
+            "accept_terms": "1",
+        },
+        allow_redirects=True,
+    )
+    return session
+
+
+def _new_post(session, title=None):
+    return session.post(
+        f"{BASE_URL}/posts/create",
+        headers=JSON,
+        data={
+            "title": title or _unique("nextpostbody"),
+            "content": "content for the next-post navigation test",
+            "topic": "devlog",
+        },
+    ).json()["data"]
+
+
+def test_newer_post_links_to_the_next_older_post(app_server):
+    session = _signup()
+    older = _new_post(session, title="Older Post For Next Nav")
+    newer = _new_post(session, title="Newer Post For Next Nav")
+
+    r = requests.get(f"{BASE_URL}/posts/{newer['slug']}", headers=JSON)
+    assert r.status_code == 200
+    assert r.json()["next_post_url"] == f"/posts/{older['slug']}"
+
+    html = requests.get(f"{BASE_URL}/posts/{newer['slug']}")
+    assert f'href="/posts/{older["slug"]}" class="back-link next-post-link"' in html.text
+    assert f'<link rel="next" href="{BASE_URL}/posts/{older["slug"]}">' in html.text
+
+
+def test_next_post_link_is_absent_when_the_url_is_none(app_server):
+    session = _signup()
+    post = _new_post(session)
+
+    data = requests.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON).json()
+    html = requests.get(f"{BASE_URL}/posts/{post['slug']}").text
+    if data["next_post_url"] is None:
+        assert "next-post-link" not in html
+        assert 'rel="next"' not in html
+    else:
+        assert f'href="{data["next_post_url"]}" class="back-link next-post-link"' in html

tests/api/posts/sidebar.py

@@ -0,0 +1,138 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+
+import requests
+
+from tests.conftest import BASE_URL
+
+JSON = {"Accept": "application/json"}
+_counter = [0]
+
+
+def _unique(prefix="side"):
+    _counter[0] += 1
+    return f"{prefix}{int(time.time() * 1000)}{_counter[0]}"
+
+
+def _signup():
+    name = _unique("sideuser")
+    session = requests.Session()
+    session.post(
+        f"{BASE_URL}/auth/signup",
+        data={
+            "username": name,
+            "email": f"{name}@t.dev",
+            "password": "secret123",
+            "confirm_password": "secret123",
+            "birth_date": "1990-01-01",
+            "accept_terms": "1",
+        },
+        allow_redirects=True,
+    )
+    return session, name
+
+
+def _new_post(session):
+    return session.post(
+        f"{BASE_URL}/posts/create",
+        headers=JSON,
+        data={
+            "title": _unique("sidepost"),
+            "content": "content for the author-sidebar test post",
+            "topic": "devlog",
+        },
+    ).json()["data"]
+
+
+def _new_gist(session, title=None):
+    return session.post(
+        f"{BASE_URL}/gists/create",
+        headers=JSON,
+        data={
+            "title": title or _unique("sidegist"),
+            "description": "a gist used for the author sidebar test",
+            "source_code": "print('hi')",
+            "language": "python",
+        },
+    ).json()["data"]
+
+
+def _new_project(session, title=None, is_private=None):
+    data = {
+        "title": title or _unique("sideproj"),
+        "description": "a project used for the author sidebar test",
+        "project_type": "software",
+        "status": "In Development",
+        "platforms": "",
+    }
+    created = session.post(f"{BASE_URL}/projects/create", headers=JSON, data=data).json()["data"]
+    if is_private:
+        session.post(
+            f"{BASE_URL}/projects/{created['slug']}/private",
+            headers=JSON,
+            data={"value": "true"},
+        )
+    return created
+
+
+def test_post_page_has_no_author_cards_with_nothing_else(app_server):
+    session, name = _signup()
+    post = _new_post(session)
+    r = requests.get(f"{BASE_URL}/posts/{post['slug']}")
+    assert r.status_code == 200
+    assert f"Gists from {name}" not in r.text
+    assert f"Projects from {name}" not in r.text
+
+
+def test_post_page_shows_author_gists_and_projects(app_server):
+    session, name = _signup()
+    post = _new_post(session)
+    _new_gist(session, title="Alpha Gist")
+    _new_project(session, title="Alpha Project")
+
+    r = requests.get(f"{BASE_URL}/posts/{post['slug']}")
+    assert r.status_code == 200
+    assert f"Gists from {name}" in r.text
+    assert f"Projects from {name}" in r.text
+    assert "Alpha Gist" in r.text
+    assert "Alpha Project" in r.text
+
+
+def test_post_page_sidebar_json_parity_and_limit(app_server):
+    session, name = _signup()
+    post = _new_post(session)
+    for i in range(7):
+        _new_gist(session, title=f"Gist {i}")
+
+    r = requests.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON)
+    assert r.status_code == 200
+    data = r.json()
+    assert len(data["author_gists"]) == 5
+    assert data["author_projects"] == []
+
+
+def test_post_page_hides_private_project_from_stranger(app_server):
+    session, name = _signup()
+    post = _new_post(session)
+    _new_project(session, title="Hidden Project", is_private=True)
+
+    stranger = requests.Session()
+    r = stranger.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON)
+    assert r.status_code == 200
+    titles = [p["title"] for p in r.json()["author_projects"]]
+    assert "Hidden Project" not in titles
+
+
+def test_post_page_sidebar_updates_live_after_new_gist(app_server):
+    session, name = _signup()
+    post = _new_post(session)
+
+    r0 = requests.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON)
+    assert r0.json()["author_gists"] == []
+
+    _new_gist(session, title="Just Created Gist")
+
+    r1 = requests.get(f"{BASE_URL}/posts/{post['slug']}", headers=JSON)
+    titles = [g["title"] for g in r1.json()["author_gists"]]
+    assert "Just Created Gist" in titles

tests/api/profile/delete.py

@@ -2,15 +2,25 @@
 
 import time
 
+import pytest
 import requests
 
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 
 JSON = {"Accept": "application/json"}
 _counter = [0]
 
 
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _unique(prefix="del"):
     _counter[0] += 1
     return f"{prefix}{int(time.time() * 1000)}{_counter[0]}"

tests/api/profile/index.py

@@ -1,7 +1,11 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+from devplacepy.database import set_setting
 def _seed_news_seo():
     from datetime import datetime, timezone
     from devplacepy.database import get_table
@@ -166,7 +170,16 @@ def _seed_feed_posts(count):
     return topic
 
 
-def test_missing_profile_returns_404(app_server):
+@pytest.fixture()
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
+def test_missing_profile_returns_404(app_server, _disable_happy_404):
     r = requests.get(f"{BASE_URL}/profile/no-such-user-xyz", allow_redirects=False)
     assert r.status_code == 404
 

tests/api/proxy.py

@@ -1,7 +1,20 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from devplacepy.database import set_setting
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_proxy_unknown_slug_returns_404(app_server):

tests/api/tools/deepsearch/export.py

@@ -1,9 +1,10 @@
 # retoor <retoor@molodetz.nl>
 
 import json
+import time
 import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.database import create_deepsearch_session, get_table, refresh_snapshot, set_setting
 from devplacepy.services.jobs import queue
 
@@ -11,7 +12,11 @@ from devplacepy.services.jobs import queue
 @pytest.fixture(scope="module", autouse=True)
 def _settings(app_server):
     set_setting("rate_limit_per_minute", "1000000")
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def _weasyprint_available() -> bool:

tests/api/tools/deepsearch/session.py

@@ -1,17 +1,29 @@
 # retoor <retoor@molodetz.nl>
 
 import json
+import time
 
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.database import (
     create_deepsearch_session,
     get_table,
     refresh_snapshot,
+    set_setting,
 )
 from devplacepy.services.jobs import queue
 
 
+@pytest.fixture
+def _no_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _json_headers():
     return {"Accept": "application/json"}
 
@@ -204,6 +216,6 @@ def test_export_json(app_server):
         _clear()
 
 
-def test_export_unknown_uid_404(app_server):
+def test_export_unknown_uid_404(app_server, _no_happy_404):
     r = requests.get(f"{BASE_URL}/tools/deepsearch/nope/export.md")
     assert r.status_code == 404

tests/api/tools/isslop.py

@@ -3,16 +3,26 @@
 import time
 import uuid
 
+import pytest
 import requests
 
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
 from devplacepy.services.jobs.isslop import store
 from devplacepy.utils import generate_uid
 
 _counter_isslop = [0]
 
 
+@pytest.fixture
+def _no_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _json_headers():
     return {"Accept": "application/json"}
 
@@ -132,7 +142,7 @@ def test_status_unknown_uid_404(app_server):
     assert r.status_code == 404
 
 
-def test_report_json_while_pending(app_server):
+def test_report_json_while_pending(app_server, _no_happy_404):
     session = requests.Session()
     try:
         run = session.post(
@@ -287,7 +297,7 @@ def test_media_route_serves_a_thumbnail(app_server):
         store.purge_analysis(uid)
 
 
-def test_media_route_rejects_a_name_outside_the_hex_pattern(app_server):
+def test_media_route_rejects_a_name_outside_the_hex_pattern(app_server, _no_happy_404):
     uid = generate_uid()
     store.create_analysis(uid, "https://github.com/owner/repository", "guest", "media-traversal-owner")
     try:

tests/api/tools/seo.py

@@ -1,11 +1,23 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
 from devplacepy.services.jobs import queue
 
 
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _json_headers():
     return {"Accept": "application/json"}
 

tests/api/topics.py

@@ -2,9 +2,10 @@
 
 import time
 from datetime import datetime, timedelta, timezone
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
 from devplacepy.database.pagination import PAGE_SIZE
 from devplacepy.utils import generate_uid
 
@@ -12,6 +13,15 @@ JSON_topics = {"Accept": "application/json"}
 _counter_topics = [0]
 
 
+@pytest.fixture
+def _no_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+
+
 def _session_topics():
     _counter_topics[0] += 1
     name = f"tpc{int(time.time() * 1000)}{_counter_topics[0]}"
@@ -107,7 +117,7 @@ def test_topic_page_lists_only_that_topics_posts(app_server):
     assert showcase_title not in titles
 
 
-def test_topic_page_rejects_an_unknown_topic(app_server):
+def test_topic_page_rejects_an_unknown_topic(app_server, _no_happy_404):
     r = requests.get(f"{BASE_URL}/topics/not-a-real-topic", allow_redirects=False)
     assert r.status_code == 404
 

tests/api/zips/download.py

@@ -6,14 +6,24 @@ import json
 import time
 import zipfile
 from pathlib import Path
+import pytest
 import requests
 from playwright.sync_api import expect
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
 from devplacepy.services.jobs import queue
 from devplacepy.services.jobs.zip_service import ZipService
 from tests.conftest import run_async
 _counter_zip_download = [0]
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 def _signup_zip_download():
     _counter_zip_download[0] += 1
     name = f"zd{int(time.time() * 1000)}{_counter_zip_download[0]}"

tests/conftest.py

@@ -39,6 +39,9 @@ os.environ["DEVPLACE_SITEMAP_TTL"] = "0"
 os.environ["DEVPLACE_HOME_CACHE_TTL"] = "0"
 os.environ["DEVPLACE_RANKING_TTL"] = "0"
 os.environ["DEVPLACE_MARKET_SATURATION_TTL"] = "0"
+os.environ["DEVPLACE_HAPPY_404_POOL_TTL"] = "0"
+os.environ["DEVPLACE_FEATURED_TOPICS_POOL_TTL"] = "0"
+os.environ["DEVPLACE_USER_RECENT_ITEMS_TTL"] = "0"
 
 
 _ASYNC_LOOP = None

tests/e2e/admin/auditlog.py

@@ -2,7 +2,7 @@
 
 from uuid import uuid4
 from datetime import datetime, timezone
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from devplacepy.database import get_table
 from devplacepy.utils import make_combined_slug
 def seed_admin_news(count=3):
@@ -265,10 +265,16 @@ def test_audit_log_view_detail(alice):
 
 def test_audit_log_detail_unknown_404(alice):
     page, _ = alice
-    resp = page.goto(
+    set_setting("happy_404_enabled", "0")
-        f"{BASE_URL}/admin/audit-log/nonexistent", wait_until="domcontentloaded"
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
-    )
+    try:
-    assert resp.status == 404
+        resp = page.goto(
+            f"{BASE_URL}/admin/audit-log/nonexistent", wait_until="domcontentloaded"
+        )
+        assert resp.status == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_audit_log_guest_redirects_to_login(page, app_server):

tests/e2e/admin/containers/index.py

@@ -1,9 +1,11 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
 import requests
 from playwright.sync_api import expect
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 
 USER_INPUT = '#cm-admin-create-form input[data-search="user"]'
 USER_OPTIONS = '#cm-admin-create-form [data-suggest="user"] li[data-value]'
@@ -29,8 +31,14 @@ def test_containers_admin_page_loads(alice, app_server):
 
 def test_container_instance_page_404_for_missing(alice, app_server):
     page, _ = alice
-    page.goto(f"{BASE_URL}/admin/containers/nonexistent", wait_until="domcontentloaded")
+    set_setting("happy_404_enabled", "0")
-    assert page.is_visible("text=Not Found") or page.is_visible("text=404")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        page.goto(f"{BASE_URL}/admin/containers/nonexistent", wait_until="domcontentloaded")
+        assert page.is_visible("text=Not Found") or page.is_visible("text=404")
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_user_search_arrow_keys_highlight_and_enter_selects(alice, seeded_db):

tests/e2e/admin/containers/manage.py

@@ -2,12 +2,13 @@
 
 import time
 import requests
-from tests.conftest import BASE_URL, login_user
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS, login_user
 from devplacepy.database import (
     get_table,
     get_primary_admin_uid,
     invalidate_admins_cache,
     refresh_snapshot,
+    set_setting,
 )
 from devplacepy.utils import clear_user_cache
 
@@ -201,11 +202,15 @@ def test_admin_detail_page_404_for_non_viewer_on_private_project(page, app_serve
     other = _make_admin()
     project = _create_project(owner["api_key"], "E2E Manage Detail Private", is_private=True)
     inst_uid = _insert_instance(project["uid"], owner["uid"])
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
     try:
         login_user(page, other)
         page.goto(f"{BASE_URL}/admin/containers/{inst_uid}", wait_until="domcontentloaded")
         assert page.is_visible("text=Not Found") or page.is_visible("text=404")
     finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
         _cleanup(inst_uid)
 
 

tests/e2e/admin/services/data.py

@@ -1,7 +1,9 @@
 # retoor <retoor@molodetz.nl>
 
-from tests.conftest import BASE_URL, login_user
+import time
-from devplacepy.database import get_table
+
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS, login_user
+from devplacepy.database import get_table, set_setting
 def _promote_to_admin(username: str) -> None:
     users = get_table("users")
     user = users.find_one(username=username)
@@ -26,9 +28,15 @@ def test_service_detail_unknown_404(page, seeded_db):
     user = seeded_db["alice"]
     _promote_to_admin(user["username"])
     login_user(page, user)
-    resp = page.request.get(f"{BASE_URL}/admin/services/nope")
+    set_setting("happy_404_enabled", "0")
-    assert resp.status == 404
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
-    assert page.request.get(f"{BASE_URL}/admin/services/nope/data").status == 404
+    try:
+        resp = page.request.get(f"{BASE_URL}/admin/services/nope")
+        assert resp.status == 404
+        assert page.request.get(f"{BASE_URL}/admin/services/nope/data").status == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_bots_service_registered_and_opt_in(page, seeded_db):

tests/e2e/docs.py

@@ -3,11 +3,22 @@
 import html
 import json
 import re
+import time
+import pytest
 import requests
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.docs_api import API_GROUPS
 from devplacepy.utils import clear_user_cache
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 def _configs_by_id(text):
     configs = re.findall(r"data-config='(.*?)'", text, re.S)
     return {

tests/e2e/feed.py

@@ -1,6 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
-from tests.conftest import BASE_URL, create_post_with_files, paste_image
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS, create_post_with_files, paste_image
 import time
 import requests
 from playwright.sync_api import expect
@@ -11,6 +11,7 @@ from devplacepy.database import (
     list_custom_overrides,
     set_custom_override,
     set_customization_pref,
+    set_setting,
 )
 from devplacepy.utils import clear_user_cache
 def _user_customization_toggle(username):
@@ -851,8 +852,14 @@ def test_delete_own_post(alice):
     page.locator(".post-action-btn:has-text('Delete')").click()
     page.locator(".dialog-overlay.visible .dialog-confirm").click()
     page.wait_for_url(f"{BASE_URL}/feed", wait_until="domcontentloaded")
-    resp = page.goto(post_url, wait_until="domcontentloaded")
+    set_setting("happy_404_enabled", "0")
-    assert resp.status == 404
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        resp = page.goto(post_url, wait_until="domcontentloaded")
+        assert resp.status == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_profile_nav_from_topbar(alice):

tests/e2e/forks.py

@@ -1,9 +1,20 @@
 # retoor <retoor@molodetz.nl>
 
 import json
-from tests.conftest import BASE_URL
+import time
-from devplacepy.database import get_table
+import pytest
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+from devplacepy.database import get_table, set_setting
 from devplacepy.services.jobs import queue
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _disable_happy_404(app_server):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    yield
+    set_setting("happy_404_enabled", "1")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 def _seed_job(uid="test-fork-uid-001"):
     jobs = get_table("jobs")
     jobs.upsert(

tests/e2e/game/farm.py

@@ -1,11 +1,13 @@
 # retoor <retoor@molodetz.nl>
 
+import time
 from datetime import datetime, timedelta, timezone
 
 from playwright.sync_api import expect
 
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 from tests.e2e.game.index import reset_farm, warp_ready
+from devplacepy.database import set_setting
 
 
 def _plant_growing(username, crop="python", slot=0):
@@ -84,10 +86,16 @@ def test_guest_can_view_farm(page):
 
 def test_unknown_farm_is_404(bob):
     page, _ = bob
-    response = page.goto(
+    set_setting("happy_404_enabled", "0")
-        f"{BASE_URL}/game/farm/nope_nobody", wait_until="domcontentloaded"
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
-    )
+    try:
-    assert response is not None and response.status == 404
+        response = page.goto(
+            f"{BASE_URL}/game/farm/nope_nobody", wait_until="domcontentloaded"
+        )
+        assert response is not None and response.status == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_ready_build_is_protected_during_grace(bob):

tests/e2e/issues/index.py

@@ -1,6 +1,9 @@
 # retoor <retoor@molodetz.nl>
 
-from tests.conftest import BASE_URL
+import time
+
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+from devplacepy.database import set_setting
 
 
 def test_issues_page_loads(alice):
@@ -61,6 +64,12 @@ def test_issue_button_icon_spacing(alice):
 
 def test_issue_detail_not_configured(alice):
     page, _ = alice
-    page.goto(f"{BASE_URL}/issues/999", wait_until="domcontentloaded")
+    set_setting("happy_404_enabled", "0")
-    assert page.is_visible("h1.error-code:has-text('404')")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
-    assert page.is_visible("text=Page not found")
+    try:
+        page.goto(f"{BASE_URL}/issues/999", wait_until="domcontentloaded")
+        assert page.is_visible("h1.error-code:has-text('404')")
+        assert page.is_visible("text=Page not found")
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)

tests/e2e/issues/jobs.py

@@ -1,10 +1,19 @@
 # retoor <retoor@molodetz.nl>
 
-from tests.conftest import BASE_URL
+import time
+
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
+from devplacepy.database import set_setting
 
 
 def test_issue_job_status_not_found(alice):
     page, _ = alice
-    page.goto(f"{BASE_URL}/issues/jobs/nonexistent-job", wait_until="domcontentloaded")
+    set_setting("happy_404_enabled", "0")
-    assert page.is_visible("h1.error-code:has-text('404')")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
-    assert page.is_visible("text=Page not found")
+    try:
+        page.goto(f"{BASE_URL}/issues/jobs/nonexistent-job", wait_until="domcontentloaded")
+        assert page.is_visible("h1.error-code:has-text('404')")
+        assert page.is_visible("text=Page not found")
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)

tests/e2e/news.py

@@ -1,10 +1,12 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
 import pytest
 from uuid import uuid4
 from datetime import datetime, timedelta, timezone
-from tests.conftest import BASE_URL, assert_share_copies
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS, assert_share_copies
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.utils import make_combined_slug
 def _seed_news_paginated(count):
     news_table = get_table("news")
@@ -258,8 +260,14 @@ def test_news_detail_loads(page, news_article):
 
 
 def test_news_detail_404(page, app_server):
-    page.goto(f"{BASE_URL}/news/nonexistent-article", wait_until="domcontentloaded")
+    set_setting("happy_404_enabled", "0")
-    assert page.is_visible("text=not found", timeout=5000)
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        page.goto(f"{BASE_URL}/news/nonexistent-article", wait_until="domcontentloaded")
+        assert page.is_visible("text=not found", timeout=5000)
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_news_comment(alice, news_article):

tests/e2e/projects/index.py

@@ -4,9 +4,20 @@ import re
 from uuid import uuid4
 from datetime import datetime, timedelta, timezone
 from playwright.sync_api import expect
-from tests.conftest import BASE_URL, assert_share_copies
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS, assert_share_copies
-from devplacepy.database import get_table
+from devplacepy.database import get_table, set_setting
 from devplacepy.utils import make_combined_slug
+
+
+def _goto_expect_404(page, url):
+    set_setting("happy_404_enabled", "0")
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
+    try:
+        resp = page.goto(url, wait_until="domcontentloaded")
+        assert resp.status == 404
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 def _seed_posts(count):
     suite = uuid4().hex[:8]
     topic = f"pag{suite}"
@@ -496,8 +507,7 @@ def test_delete_own_project(alice):
     page.locator(".context-menu-item:has-text('Delete')").click()
     page.locator(".dialog-overlay.visible .dialog-confirm").click()
     page.wait_for_url(f"{BASE_URL}/projects", wait_until="domcontentloaded")
-    resp = page.goto(proj_url, wait_until="domcontentloaded")
+    _goto_expect_404(page, proj_url)
-    assert resp.status == 404
 
 
 def test_project_edit_button(alice):
@@ -850,8 +860,7 @@ def test_data_confirm_accept_proceeds(alice):
     _click_delete(page)
     page.locator(".dialog-overlay.visible .dialog-confirm").click()
     page.wait_for_url(f"{BASE_URL}/projects", wait_until="domcontentloaded")
-    resp = page.goto(proj_url, wait_until="domcontentloaded")
+    _goto_expect_404(page, proj_url)
-    assert resp.status == 404
 
 
 def test_detail_download_button_downloads(app_server, page):

tests/e2e/reports/ui.py

@@ -1,9 +1,11 @@
 # retoor <retoor@molodetz.nl>
 
+import time
+
 from playwright.sync_api import expect
 
-from devplacepy.database import get_table, refresh_snapshot
+from devplacepy.database import get_table, refresh_snapshot, set_setting
-from tests.conftest import BASE_URL
+from tests.conftest import BASE_URL, CACHE_VERSION_PROPAGATION_SECONDS
 
 
 def _create_post(page, content):
@@ -124,10 +126,16 @@ def test_the_legal_pages_render_for_a_guest(page, app_server):
 
 
 def test_the_admin_operations_page_is_hidden_from_a_guest(page, app_server):
-    page.goto(
+    set_setting("happy_404_enabled", "0")
-        f"{BASE_URL}/docs/moderation-operations.html", wait_until="domcontentloaded"
+    time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
-    )
+    try:
-    assert "not found" in page.content().lower()
+        page.goto(
+            f"{BASE_URL}/docs/moderation-operations.html", wait_until="domcontentloaded"
+        )
+        assert "not found" in page.content().lower()
+    finally:
+        set_setting("happy_404_enabled", "1")
+        time.sleep(CACHE_VERSION_PROPAGATION_SECONDS)
 
 
 def test_the_admin_operations_page_renders_for_an_admin(alice):