Render markdown in gist and project sidebar titles; add content visibility and GIF picker
DevPlace CI / test (push) Failing after 1h5m29s
DevPlace CI / test (push) Failing after 1h5m29s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -166,6 +166,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
|
||||
| `devplacepy/services/game/CLAUDE.md` | Code Farm idle game (economy invariants, raids, the one-pure-function rule) |
|
||||
| `devplacepy/services/opinionwar/CLAUDE.md` | Opinion Wars (week-long faction battles on posts: atomic fight/resolve transitions, cooldown-before-coins compensation, event seq allocation, relay-on-lock-owner) |
|
||||
| `devplacepy/services/quiz/CLAUDE.md` | Quizzes (the terminal publish lock, attempt atomicity, answer-key withholding, AI free-text grading, the best-attempt scoreboard) |
|
||||
| `devplacepy/services/giphy/CLAUDE.md` | GIF picker: GIPHY search through the server, admin-held API key, GIFs stored as ordinary attachments, cache and per-user caps |
|
||||
| `devplacepy/services/skills/CLAUDE.md` | Agent Skills: strict `SKILL.md` parser, SSRF-guarded fetcher, the bounded prompt index, per-user privacy, soft delete and account-deletion cascade, why a skill is data and never code |
|
||||
| `devplacepy/services/nodes/CLAUDE.md` | Community node network: the protocol specification (single source of truth), the no-open-proxy rule, the two transports and their trust levels, the decisions taken, and the gaps per phase |
|
||||
| `devplacepy/services/community/CLAUDE.md` | Community discovery: rsearch + feed detection, one-time structure learning, generated collectors, dry run, the failure floor, admin-only visibility |
|
||||
@@ -206,6 +207,7 @@ Routers in `devplacepy/routers/` are organised as a **directory tree that mirror
|
||||
| `/admin`, `/admin/services`, `/admin/containers` | admin/ package |
|
||||
| `/issues` | issues/ package - see `services/gitea/CLAUDE.md` |
|
||||
| `/gists`, `/news`, `/uploads`, `/media` | flat files |
|
||||
| `/giphy` | giphy.py - GIF picker backend (search, trending, attach), see `services/giphy/CLAUDE.md` |
|
||||
| `/openai` | openai_gateway.py - see `services/openai_gateway/CLAUDE.md` |
|
||||
| `/devii` | devii.py - see `services/devii/CLAUDE.md` |
|
||||
| `/zips`, `/forks` | see `services/jobs/CLAUDE.md` |
|
||||
@@ -310,6 +312,7 @@ The escape hatch is deliberately two-factor and must never be self-served: after
|
||||
- **Admin seniority: a junior admin cannot manage a more senior admin.** Every per-user mutation in `routers/admin/users.py` and `routers/admin/moderation.py` is gated by `is_senior_admin(actor, target)` (`routers/admin/_shared.py`) - blocks (audits `result="denied"`) when the target is an Admin who registered earlier. Server-side, so it also covers Devii's admin tools.
|
||||
- **Never pass a `respond()` context key that collides with a Jinja global.** `respond(request, template, ctx, model=XOut)` feeds the same `ctx` to both the Pydantic model (JSON) and the template. A key like `is_admin`/`avatar_url`/`is_self` holding a non-callable value shadows the global across the whole inheritance chain, turning `{% if is_admin(user) %}` into `False(user)` -> `TypeError`, a 500 that fires only for the branch that calls the global. Name viewer/permission flags distinctly (`viewer_is_admin`, not `is_admin`) in both schema and context.
|
||||
- **Project visibility (`is_private`) and read-only (`read_only`)** are owner-controlled flags on `projects`. Read access is gated by the single `content.can_view_project(project, user)` predicate at EVERY read surface - never re-implement the check inline. Predicate: `not is_private OR is_owner OR (is_admin AND owner is not an admin)` - a project hidden by a member stays visible to any admin, but one hidden by an admin is visible only to that owner admin. **Containers have their own, stricter isolation predicates** (`owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance`) - the primary administrator sees/manages every container; any other admin can VIEW others' containers only on public projects and can MANAGE only instances they own. Read-only is enforced as a single data-layer guard `project_files._guard_writable(project_uid)` at the top of every mutation entrypoint - add it to any NEW file-mutating function. Devii may flip read-only/visibility only after explicit confirmation (`CONFIRM_REQUIRED`). Full UI-level detail in `devplacepy/routers/projects/CLAUDE.md`.
|
||||
- **Content visibility (`published`/`hidden`/`draft`/`offline`) on posts, gists, projects and news is one specification, `devplacepy/database/visibility.py`.** Any new read of those tables applies `listed_clause`/`listed_sql` (or goes through `paginate`, which does it), and any single-item or target access uses `content.can_view_item`/`can_view_target`. NULL reads as published. Full detail in `devplacepy/database/CLAUDE.md`.
|
||||
- **Deletions ALWAYS require confirmation:** `confirmation_error` gates EVERY content delete tool via `CONFIRM_REQUIRED` (`delete_post`, `delete_comment`, `delete_gist`, `delete_project`, `project_delete_file`, `delete_media`, `delete_attachment`, `admin_delete_news`, container delete, and any `container_exec` matching `dispatcher.DESTRUCTIVE_COMMAND`). The first call is refused; the agent must show the exact target then pass `confirm=true`. **Load-bearing: every confirmation-gated tool MUST also declare a `confirm` boolean param in its catalog spec** - schemas set `additionalProperties: false`, so a gated tool without a declared `confirm` param can never receive it and loops forever.
|
||||
|
||||
## Every user-generated surface is reportable by construction (hard rule)
|
||||
|
||||
@@ -89,7 +89,7 @@ devplacepy/
|
||||
| `/admin/trash` | Admin **Trash**: review, restore, and permanently purge soft-deleted content (posts, comments, gists, projects, news, project files, attachments) across the platform |
|
||||
| `/admin/devii-tasks` | Admin **Devii tasks**: every scheduled task across all owners with its schedule, run count, expiry and failure streak, plus per-task disable and delete |
|
||||
| `/notifications` | Notification list, mark read, live unread counts (`/notifications/counts`) |
|
||||
| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, reconnect catch-up, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages: the original is delivered immediately, then a second `message` frame with `ai_processed` replaces the bubble for both participants when the rewrite lands (including across workers). An opened conversation loads its 500 most recent messages and can page older rows with `?before=`; switching threads does not drop the socket. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets |
|
||||
| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, reconnect catch-up, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages: the original is delivered immediately, then a second `message` frame with `ai_processed` replaces the bubble for both participants when the rewrite lands (including across workers). An opened conversation loads its 500 most recent messages and can page older rows with `?before=`; switching threads does not drop the socket. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. An administrator can delete the messages they themselves sent (`POST /messages/{uid}/delete`, soft delete, confirmation in the chat UI and in Devii); the bubble disappears live for both participants. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets |
|
||||
| `/votes` | Upvote/downvote on posts, comments, projects |
|
||||
| `/reactions` | Emoji reactions on posts, comments, gists, projects |
|
||||
| `/bookmarks` | Save/unsave content; `/bookmarks/saved` personal list |
|
||||
@@ -229,6 +229,10 @@ The rack's **control terminal** supports every current game API route. Try `help
|
||||
|
||||
The farm refreshes live over the pub/sub bus (a watered build appears on the owner's screen at once) and every plot countdown ticks client-side. Every endpoint also answers JSON, and Devii can play the game on the member's behalf via the `game_*` tools (`game_state`, `game_plant`, `game_harvest`, `game_plot_stop`, `game_buy_plot`, `game_upgrade_ci`, `game_fertilize`, `game_daily`, `game_claim_quest`, `game_water`, `game_steal`, `game_view_farm`, `game_leaderboard`, `game_upgrade_perk`, `game_upgrade_legacy`, `game_prestige`, `game_claim_grant`, `game_upgrade_mastery`, `game_buy_infrastructure`, `game_upgrade_defense`, `game_buy_cosmetic`, `game_equip_cosmetic`, `game_downgrade_defense`). See the API reference group **Code Farm** and the full player guide at `/docs/code-farm.html`.
|
||||
|
||||
## Content visibility
|
||||
|
||||
Posts, gists, projects and news carry a status: **published** (default), **hidden**, **offline** or **draft**. Hidden items drop out of every list, search, topic page, profile and the sitemap but still open by direct link (marked `noindex`). Offline and draft items are visible only to their owner and administrators; everyone else gets a 404 and cannot comment, vote, react or bookmark them. The field is optional and backwards compatible: omitted on create it defaults to published, omitted on edit the current value is kept, and old rows read as published. Owners set it with the Visibility field in the create and edit forms (and the `visibility` parameter of the API and the Devii create and edit tools); administrators set a news article's status from the admin news page (`POST /admin/news/{uid}/visibility`, Devii `admin_set_news_visibility`). The specification is `devplacepy/database/visibility.py` (a NULL value reads as published, so no data migration), and the user-facing page is `/docs/content-visibility.html`. The website offers published, hidden and offline; draft is accepted by the API and reserved for a future drafting flow.
|
||||
|
||||
## Engagement
|
||||
|
||||
- **Emoji reactions** - react with **any** emoji on posts, comments, gists, and projects, separate from voting and carrying no ranking weight. A short quick-pick palette covers the common reactions, and a `+` button next to it opens the full searchable emoji picker (every standard emoji, including skin tones), so a reaction is never limited to a preset list. Emoji already used on an item are shown as counted chips beside the palette.
|
||||
@@ -256,6 +260,12 @@ DevPlace is open in the sense that it does not editorialise technical opinion. I
|
||||
- **Account deletion.** Self-service at `/profile/{username}/delete`, reauthenticated with the account password. Sessions and tokens are revoked, the profile is anonymised immediately, and all content is removed under one deletion event that stays restorable for `account_deletion_grace_hours` (default 24) before `devplace accounts prune` (and the Moderation housekeeping service) purges it permanently. The devRant `DELETE /api/users/me` routes into the same cascade.
|
||||
- **Legal pages**: [Terms of Service](/docs/terms.html), [Community Guidelines](/docs/community-guidelines.html), [Privacy Policy](/docs/privacy.html), [How moderation works](/docs/content-moderation.html), [Notice and takedown](/docs/intellectual-property.html) and [Contact](/docs/contact.html). All six are indexed in the sitemap and reachable from the docs index; the footer of every page links Terms, Privacy, Community Guidelines and Contact. Contact details come from the `contact_email`, `contact_phone` and `contact_address` settings, so the in-product page and any app-store trader declaration have one source of truth.
|
||||
|
||||
## GIF picker (GIPHY)
|
||||
|
||||
A "GIF" button beside the attachment button in the post composer, every comment form and the message input opens a searchable GIPHY panel (trending first, "Powered by GIPHY" attribution). A picked GIF is downloaded by the server and stored as an ordinary attachment, so it renders in the attachment gallery, can be reported and deleted like any file, and counts against the normal attachment limits.
|
||||
|
||||
An administrator enters the GIPHY API key under **Admin > Services > Giphy** (also the content rating, default `pg`, and the API base URL); the key is stored in `site_settings`, never reaches a browser, and the button stays hidden until it is set. Searches run on the server, are cached for five minutes, and each member may make 30 per minute. Routes: `GET /giphy/status`, `/giphy/search?q=`, `/giphy/trending`, `POST /giphy/attach` (documented under Uploads at `/docs/uploads.html`); Devii has `search_gifs`, `trending_gifs` and `attach_gif`. Details: `devplacepy/services/giphy/CLAUDE.md`.
|
||||
|
||||
## Skills
|
||||
|
||||
Members can install [Agent Skills](https://agentskills.io) from a URL at `/skills` so Devii can use them. A skill is a `SKILL.md` (YAML frontmatter plus markdown instructions) with optional text files under `scripts/`, `references/` and `assets/`. Skills are private to their owner, count against a per-user cap, and only the name and a one-line description of enabled skills are added to Devii's system prompt; Devii loads the full text on demand. A skill is untrusted third-party text: it can never grant Devii a tool, bypass a confirmation or run a script. Deleting a skill is a soft delete and needs confirmation when Devii does it. Documented in the API docs under *Skills* and in `devplacepy/services/skills/CLAUDE.md`.
|
||||
|
||||
+140
-5
@@ -41,6 +41,13 @@ from devplacepy.database import (
|
||||
db,
|
||||
get_user_recent_items,
|
||||
invalidate_user_recent_cache,
|
||||
VISIBILITY_COLUMNS,
|
||||
TARGET_TABLES,
|
||||
item_visibility,
|
||||
is_listed,
|
||||
is_open,
|
||||
listed_clause,
|
||||
normalize_visibility,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
time_ago,
|
||||
@@ -67,7 +74,14 @@ from devplacepy.services.moderation.screening import (
|
||||
screen_fields,
|
||||
)
|
||||
|
||||
CREATE_METADATA_KEYS = ("project_type", "is_private", "language", "topic", "status")
|
||||
CREATE_METADATA_KEYS = (
|
||||
"project_type",
|
||||
"is_private",
|
||||
"language",
|
||||
"topic",
|
||||
"status",
|
||||
"visibility",
|
||||
)
|
||||
|
||||
BOOKMARKABLE_TYPES = {"post", "gist", "project", "news", "quiz"}
|
||||
REACTABLE_TYPES = {"post", "comment", "gist", "project", "quiz"}
|
||||
@@ -95,6 +109,26 @@ def is_owner(item: dict | None, user: dict | None) -> bool:
|
||||
return bool(item and user and item["user_uid"] == user["uid"])
|
||||
|
||||
|
||||
def can_view_item(table_name: str, item: dict | None, user: dict | None) -> bool:
|
||||
if not item:
|
||||
return False
|
||||
if is_open(table_name, item):
|
||||
return True
|
||||
if not user:
|
||||
return False
|
||||
return bool(item.get("user_uid") == user.get("uid") or is_admin(user))
|
||||
|
||||
|
||||
def can_view_target(target_type: str, target_uid: str, user: dict | None) -> bool:
|
||||
table_name = TARGET_TABLES.get(target_type)
|
||||
if not table_name:
|
||||
return True
|
||||
item = resolve_by_slug(get_table(table_name), target_uid)
|
||||
if not item:
|
||||
return True
|
||||
return can_view_item(table_name, item, user)
|
||||
|
||||
|
||||
def mature_hidden_by_default() -> bool:
|
||||
return get_int_setting("moderation_mature_default_hidden", 1) != 0
|
||||
|
||||
@@ -128,6 +162,8 @@ def _owner_is_admin(project: dict) -> bool:
|
||||
def can_view_project(project: dict | None, user: dict | None) -> bool:
|
||||
if not project:
|
||||
return False
|
||||
if not can_view_item("projects", project, user):
|
||||
return False
|
||||
if not project.get("is_private"):
|
||||
return True
|
||||
if is_owner(project, user):
|
||||
@@ -138,14 +174,17 @@ def can_view_project(project: dict | None, user: dict | None) -> bool:
|
||||
|
||||
|
||||
def get_user_sidebar_gists(user_uid: str, limit: int = 5) -> list[dict]:
|
||||
return get_user_recent_items("gists", user_uid)[:limit]
|
||||
rows = get_user_recent_items("gists", user_uid)
|
||||
return [g for g in rows if is_listed("gists", g)][:limit]
|
||||
|
||||
|
||||
def get_user_sidebar_projects(
|
||||
user_uid: str, viewer: dict | None, limit: int = 5
|
||||
) -> list[dict]:
|
||||
rows = get_user_recent_items("projects", user_uid)
|
||||
visible = [p for p in rows if can_view_project(p, viewer)]
|
||||
visible = [
|
||||
p for p in rows if is_listed("projects", p) and can_view_project(p, viewer)
|
||||
]
|
||||
return visible[:limit]
|
||||
|
||||
|
||||
@@ -246,6 +285,13 @@ def first_image_url(item: dict, attachments: list | None) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _with_visibility(table_name: str, fields: dict) -> dict:
|
||||
column = VISIBILITY_COLUMNS.get(table_name)
|
||||
if not column:
|
||||
return fields
|
||||
return {**fields, column: normalize_visibility(fields.get(column))}
|
||||
|
||||
|
||||
def create_content_item(
|
||||
table_name: str,
|
||||
target_type: str,
|
||||
@@ -258,6 +304,7 @@ def create_content_item(
|
||||
attachment_uids: list | None,
|
||||
request=None,
|
||||
) -> tuple[str, str]:
|
||||
fields = _with_visibility(table_name, fields)
|
||||
screening = screen_fields(table_name, fields)
|
||||
refuse_if_blocked(screening)
|
||||
uid = generate_uid()
|
||||
@@ -284,7 +331,10 @@ def create_content_item(
|
||||
award_rewards(user["uid"], xp, badge)
|
||||
if attachment_uids:
|
||||
link_attachments(attachment_uids, target_type, uid)
|
||||
create_mention_notifications(mention_text, user["uid"], f"/{table_name}/{slug}")
|
||||
if is_open(table_name, fields):
|
||||
create_mention_notifications(
|
||||
mention_text, user["uid"], f"/{table_name}/{slug}"
|
||||
)
|
||||
logger.info(f"{target_type} {uid} created by {user['username']}")
|
||||
label = fields.get("title") or slug
|
||||
links = [audit.target(target_type, uid, label)]
|
||||
@@ -667,6 +717,81 @@ def detail_context(
|
||||
return context
|
||||
|
||||
|
||||
def apply_visibility(
|
||||
request,
|
||||
table_name: str,
|
||||
target_type: str,
|
||||
user: dict,
|
||||
item: dict,
|
||||
value: str,
|
||||
) -> tuple[str, str]:
|
||||
column = VISIBILITY_COLUMNS[table_name]
|
||||
old_value = item_visibility(table_name, item)
|
||||
new_value = normalize_visibility(value)
|
||||
get_table(table_name).update({"uid": item["uid"], column: new_value}, ["uid"])
|
||||
owner_uid = item.get("user_uid")
|
||||
if owner_uid:
|
||||
invalidate_user_recent_cache(table_name, owner_uid)
|
||||
if table_name == "posts":
|
||||
clear_user_post_count(owner_uid)
|
||||
if table_name == "projects":
|
||||
from devplacepy.templating import clear_user_projects_cache
|
||||
|
||||
clear_user_projects_cache(owner_uid)
|
||||
label = item.get("title") or item["uid"]
|
||||
logger.info(
|
||||
f"{target_type} {item['uid']} visibility {old_value} -> {new_value} by {user['username']}"
|
||||
)
|
||||
audit.record(
|
||||
request,
|
||||
f"{target_type}.visibility.set",
|
||||
user=user,
|
||||
target_type=target_type,
|
||||
target_uid=item["uid"],
|
||||
target_label=label,
|
||||
old_value=old_value,
|
||||
new_value=new_value,
|
||||
summary=f"{user['username']} set {target_type} {label} to {new_value}",
|
||||
links=[audit.target(target_type, item["uid"], label)],
|
||||
)
|
||||
return old_value, new_value
|
||||
|
||||
|
||||
def set_visibility_item(
|
||||
request,
|
||||
table_name: str,
|
||||
target_type: str,
|
||||
user: dict,
|
||||
slug: str,
|
||||
value: str,
|
||||
redirect_fail: str,
|
||||
):
|
||||
from devplacepy.responses import action_result, wants_json, json_error
|
||||
|
||||
item = resolve_by_slug(get_table(table_name), slug)
|
||||
if not item or not is_admin(user):
|
||||
audit.record(
|
||||
request,
|
||||
f"{target_type}.visibility.set",
|
||||
user=user,
|
||||
result="denied",
|
||||
target_type=target_type,
|
||||
target_uid=item["uid"] if item else slug,
|
||||
target_label=item.get("title") if item else slug,
|
||||
summary=f"{user['username']} denied changing visibility of {target_type} {slug}",
|
||||
)
|
||||
if wants_json(request):
|
||||
return json_error(403, "Not allowed")
|
||||
return RedirectResponse(url=redirect_fail, status_code=302)
|
||||
_, new_value = apply_visibility(request, table_name, target_type, user, item, value)
|
||||
url = f"/{table_name}/{item.get('slug') or item['uid']}"
|
||||
return action_result(
|
||||
request,
|
||||
url,
|
||||
data={"uid": item["uid"], "visibility": new_value, "url": url},
|
||||
)
|
||||
|
||||
|
||||
def edit_content_item(
|
||||
request,
|
||||
table_name: str,
|
||||
@@ -681,6 +806,7 @@ def edit_content_item(
|
||||
kind = target_type or table_name.rstrip("s")
|
||||
table = get_table(table_name)
|
||||
item = resolve_by_slug(table, slug)
|
||||
visibility_value = update_fields.pop("visibility", None)
|
||||
if not is_owner(item, user):
|
||||
audit.record(
|
||||
request,
|
||||
@@ -703,6 +829,10 @@ def edit_content_item(
|
||||
}
|
||||
table.update({"uid": item["uid"], **update_fields}, ["uid"])
|
||||
invalidate_user_recent_cache(table_name, item["user_uid"])
|
||||
if visibility_value is not None and normalize_visibility(
|
||||
visibility_value
|
||||
) != item_visibility(table_name, item):
|
||||
apply_visibility(request, table_name, kind, user, item, visibility_value)
|
||||
record_screening(
|
||||
screening,
|
||||
target_type=kind,
|
||||
@@ -817,6 +947,8 @@ def load_detail(
|
||||
item = resolve_by_slug(get_table(table_name), slug)
|
||||
if not item:
|
||||
return None
|
||||
if not can_view_item(table_name, item, user):
|
||||
return None
|
||||
if user and item["user_uid"] in get_blocked_uids(user["uid"]):
|
||||
return None
|
||||
author = get_users_by_uids([item["user_uid"]]).get(item["user_uid"])
|
||||
@@ -898,7 +1030,10 @@ def enrich_items(
|
||||
|
||||
|
||||
def count_project_devlog(project_uid: str) -> int:
|
||||
return get_table("posts").count(project_uid=project_uid, deleted_at=None)
|
||||
posts = get_table("posts")
|
||||
listed = listed_clause(posts)
|
||||
clauses = [listed] if listed is not None else []
|
||||
return posts.count(*clauses, project_uid=project_uid, deleted_at=None)
|
||||
|
||||
|
||||
def get_project_devlog(
|
||||
|
||||
@@ -246,3 +246,22 @@ Operational settings - read sites and rules:
|
||||
**Booleans are `<select>`, never checkboxes.** The settings save handler (`admin.py`) skips empty form values so empty fields don't clobber existing rows. An unchecked checkbox submits nothing, so it could never be turned off - `registration_open` and `maintenance_mode` use `<option value="1">`/`<option value="0">` so a value is always submitted.
|
||||
|
||||
|
||||
|
||||
## Content visibility (`database/visibility.py`, the single specification)
|
||||
|
||||
Posts, gists, projects and news carry one status from `VISIBILITY_VALUES`: `published` (default), `hidden`, `draft`, `offline`. The module is the specification; routers, forms, schemas, templates, Devii and the docs import it and never repeat a literal.
|
||||
|
||||
| Status | Listed (feed, search, topics, profiles, sitemap, counters) | Opens by link | Who |
|
||||
|---|---|---|---|
|
||||
| `published` | yes | yes | everyone |
|
||||
| `hidden` | no | yes, `noindex,nofollow` | everyone with the link |
|
||||
| `offline` | no | no (404) | owner and administrators |
|
||||
| `draft` | no | no (404) | owner and administrators; accepted by every API but not offered by the UI (`VISIBILITY_CHOICES` is the three user-facing values) |
|
||||
|
||||
- **Column.** `VISIBILITY_COLUMNS = {posts, gists, projects: "visibility", news: "status"}`. News reuses its existing `status` column (it already held `published`/`draft`), so no news migration exists. `ensure_visibility_columns()` (called from `init_db`) adds the text column and `idx_<table>_visibility` ONLY to tables that already exist; it must never create the table (an empty `posts` table breaks every `"posts" in db.tables` guard and the first-signup schema). A NULL or unknown value reads as `published` everywhere (`normalize_visibility`, `item_visibility`), so legacy rows needed no backfill and `init_db` does no per-row work.
|
||||
- **Every read goes through one of three choke points, never an inline compare.** `paginate`/`paginate_diverse` auto-append `listed_clause(table)` for any table in `VISIBILITY_COLUMNS` (`include_unlisted_of=<uid>` ORs in that owner's own rows, used for an owner's own profile/listing); `listed_clause(table, own_uid)` for `count()`/`find()` call sites; `listed_sql(table, alias)` for raw SQL (it returns `1 = 1` when the table or column does not exist yet, so a fresh database never errors). The predicates `is_listed`/`is_open` work on a row dict. Single-item access is `content.can_view_item(table, item, user)` (open, or owner, or admin) and `content.can_view_target(target_type, uid, user)` for polymorphic targets; `load_detail` applies the first, so every detail page 404s without per-route code. `can_view_project` composes visibility first, then the `is_private` rule.
|
||||
- **Writes.** `content.apply_visibility` is the one writer (column update, cache invalidation, `<kind>.visibility.set` audit with old/new value). `edit_content_item` pops the optional `visibility` form field and calls `apply_visibility` when it changed; `content.set_visibility_item` exists only for the administrator news route (news has no edit form). `create_content_item` stamps the column (`_with_visibility`) and skips mention notifications for non-open items.
|
||||
- **Public counters** (`get_user_post_count`, `get_site_stats`, `get_post_counts_by_user_uids`, trending topics, topic hub counts, sitemap author counts, happy-404 pool) count listed rows only, so the owner's own profile count excludes their hidden and offline posts.
|
||||
- **Backwards compatibility (hard rule).** `visibility` is optional on every API: `VisibilityCreateMixin` defaults to `published`, `VisibilityEditMixin` defaults to `None` meaning keep the current value (never reset to published), NULL rows read as published, and the docs/Devii parameter descriptions say so (`tests/unit/docs_api/visibility.py`, `tests/api/content/visibility.py` legacy-client tests). Never make the field required.
|
||||
- **Outputs.** `VisibleOut` (`schemas/content.py`) gives `PostOut`/`GistOut`/`ProjectOut` a `visibility` that reads NULL as `published`. Forms use `VisibilityCreateMixin` (default `published`) and `VisibilityEditMixin` (`None` keeps the current value); `VisibilityForm{value}` backs the admin news route.
|
||||
- **Adding a table.** Put it in `VISIBILITY_COLUMNS` and `TARGET_TABLES`, add its form mixin and Out base, apply `listed_clause`/`listed_sql` to each new read, and extend `tests/api/content/visibility.py`. `tests/unit/database/visibility.py` asserts the spec is self-consistent.
|
||||
|
||||
@@ -7,6 +7,7 @@ from .settings import _settings_cache, get_setting, get_int_setting, set_setting
|
||||
from .users import get_users_by_uids, _admins_cache, invalidate_admins_cache, get_admin_uids, set_user_timezone, set_last_seen, get_online_users, get_primary_admin_uid, is_account_active, search_users_by_username
|
||||
from .relations import _relations_cache, get_user_relations, get_blocked_uids, get_muted_uids, get_silenced_uids, invalidate_user_relations
|
||||
from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, clear_user_post_count, build_pagination
|
||||
from .visibility import PUBLISHED, HIDDEN, DRAFT, OFFLINE, VISIBILITY_VALUES, VISIBILITY_CHOICES, DEFAULT_VISIBILITY, LISTED_VALUES, OPEN_VALUES, VISIBILITY_COLUMNS, TARGET_TABLES, VISIBILITY_LABELS, VISIBILITY_HINTS, normalize_visibility, visibility_column, item_visibility, is_listed, is_open, listed_clause, listed_sql, ensure_visibility_columns, visibility_options, visibility_label
|
||||
from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns, soft_delete, soft_delete_in, restore, purge, list_deleted, count_deleted, restore_event, purge_event
|
||||
from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_user_notes, get_polls_by_post_uids, get_poll_for_post
|
||||
from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage, AWARD_USAGE_KEY, add_award_usage, get_award_usage
|
||||
@@ -143,6 +144,29 @@ __all__ = [
|
||||
"clear_user_post_count",
|
||||
"build_pagination",
|
||||
"SOFT_DELETE_TABLES",
|
||||
"PUBLISHED",
|
||||
"HIDDEN",
|
||||
"DRAFT",
|
||||
"OFFLINE",
|
||||
"VISIBILITY_VALUES",
|
||||
"VISIBILITY_CHOICES",
|
||||
"DEFAULT_VISIBILITY",
|
||||
"LISTED_VALUES",
|
||||
"OPEN_VALUES",
|
||||
"VISIBILITY_COLUMNS",
|
||||
"TARGET_TABLES",
|
||||
"VISIBILITY_LABELS",
|
||||
"VISIBILITY_HINTS",
|
||||
"normalize_visibility",
|
||||
"visibility_column",
|
||||
"item_visibility",
|
||||
"is_listed",
|
||||
"is_open",
|
||||
"listed_clause",
|
||||
"listed_sql",
|
||||
"ensure_visibility_columns",
|
||||
"visibility_options",
|
||||
"visibility_label",
|
||||
"ensure_soft_delete_columns",
|
||||
"soft_delete",
|
||||
"soft_delete_in",
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .core import TTLCache, datetime, db, timedelta, timezone
|
||||
from .visibility import listed_sql
|
||||
|
||||
|
||||
def record_activity(user_uid: str, action: str) -> int:
|
||||
@@ -79,7 +80,7 @@ def get_activity_calendar(user_uid: str) -> dict:
|
||||
if sources:
|
||||
cutoff = (datetime.now(timezone.utc) - timedelta(days=364)).date().isoformat()
|
||||
union = " UNION ALL ".join(
|
||||
f"SELECT created_at FROM {table} WHERE user_uid = :u AND deleted_at IS NULL"
|
||||
f"SELECT created_at FROM {table} WHERE user_uid = :u AND deleted_at IS NULL AND {listed_sql(table)}"
|
||||
for table in sources
|
||||
)
|
||||
rows = db.query(
|
||||
|
||||
@@ -7,6 +7,7 @@ from collections import Counter
|
||||
from devplacepy.cache import TTLCache
|
||||
|
||||
from .core import db, get_table, or_
|
||||
from .visibility import listed_sql
|
||||
|
||||
_daily_topic_cache = TTLCache(ttl=60, max_size=1)
|
||||
_trending_cache = TTLCache(ttl=15, max_size=1)
|
||||
@@ -284,6 +285,7 @@ def get_trending_topics(limit: int = 6) -> list[dict]:
|
||||
return []
|
||||
rows = db.query(
|
||||
"SELECT topic FROM posts WHERE deleted_at IS NULL "
|
||||
f"AND {listed_sql('posts')} "
|
||||
"AND topic IS NOT NULL AND topic != '' "
|
||||
"ORDER BY created_at DESC LIMIT 200"
|
||||
)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .core import TTLCache, _in_clause, db, defaultdict
|
||||
from .visibility import listed_sql
|
||||
|
||||
|
||||
_comment_count_cache = TTLCache(ttl=15, max_size=10000)
|
||||
@@ -36,7 +37,7 @@ def get_post_counts_by_user_uids(user_uids):
|
||||
return {}
|
||||
placeholders, params = _in_clause(user_uids)
|
||||
rows = db.query(
|
||||
f"SELECT user_uid, COUNT(*) as c FROM posts WHERE user_uid IN ({placeholders}) AND deleted_at IS NULL GROUP BY user_uid",
|
||||
f"SELECT user_uid, COUNT(*) as c FROM posts WHERE user_uid IN ({placeholders}) AND deleted_at IS NULL AND {listed_sql('posts')} GROUP BY user_uid",
|
||||
**params,
|
||||
)
|
||||
return {r["user_uid"]: r["c"] for r in rows}
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from devplacepy.cache import TTLCache
|
||||
from .core import db, get_table
|
||||
from .relations import get_blocked_uids
|
||||
from .visibility import listed_clause
|
||||
|
||||
|
||||
PAGE_SIZE = 25
|
||||
@@ -19,10 +20,14 @@ def paginate(
|
||||
cursor_field="created_at",
|
||||
viewer_uid=None,
|
||||
limit=PAGE_SIZE,
|
||||
include_unlisted_of=None,
|
||||
**filters,
|
||||
):
|
||||
order = order or ["-" + cursor_field]
|
||||
clauses = list(clauses)
|
||||
listed = listed_clause(table, own_uid=include_unlisted_of)
|
||||
if listed is not None:
|
||||
clauses.append(listed)
|
||||
if table.has_column("deleted_at") and "deleted_at" not in filters:
|
||||
clauses.append(table.table.columns.deleted_at.is_(None))
|
||||
if viewer_uid and table.has_column("user_uid"):
|
||||
@@ -66,6 +71,7 @@ def paginate_diverse(
|
||||
uid_key="user_uid",
|
||||
viewer_uid=None,
|
||||
limit=PAGE_SIZE,
|
||||
include_unlisted_of=None,
|
||||
**filters,
|
||||
):
|
||||
rows, next_cursor = paginate(
|
||||
@@ -76,6 +82,7 @@ def paginate_diverse(
|
||||
cursor_field=cursor_field,
|
||||
viewer_uid=viewer_uid,
|
||||
limit=limit,
|
||||
include_unlisted_of=include_unlisted_of,
|
||||
**filters,
|
||||
)
|
||||
return interleave_by_author(rows, uid_key=uid_key), next_cursor
|
||||
@@ -91,7 +98,9 @@ def get_user_post_count(user_uid: str) -> int:
|
||||
return cached
|
||||
if "posts" not in db.tables:
|
||||
return 0
|
||||
count = get_table("posts").count(user_uid=user_uid, deleted_at=None)
|
||||
listed = listed_clause(get_table("posts"))
|
||||
clauses = [listed] if listed is not None else []
|
||||
count = get_table("posts").count(*clauses, user_uid=user_uid, deleted_at=None)
|
||||
_user_post_count_cache.set(user_uid, count)
|
||||
return count
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from devplacepy.services.nodes.spec import SPEC as NODES_SPEC
|
||||
from .core import AQUALITY_NEWS_GRADING_MODEL, AQUALITY_NEWS_GRADING_URL, DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, NEWS_GRADING_MODEL_DEFAULT, _drop_index, _index, _uid_index, db, defaultdict, get_table, logger
|
||||
from .settings import get_setting, set_setting
|
||||
from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns
|
||||
from .visibility import ensure_visibility_columns
|
||||
from .ranking import _authors_cache
|
||||
|
||||
|
||||
@@ -181,6 +182,8 @@ def init_db():
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
("client_id", ""),
|
||||
("deleted_at", ""),
|
||||
("deleted_by", ""),
|
||||
):
|
||||
if not messages.has_column(column):
|
||||
messages.create_column_by_example(column, example)
|
||||
@@ -1899,6 +1902,8 @@ def init_db():
|
||||
for table in SOFT_DELETE_TABLES:
|
||||
ensure_soft_delete_columns(table)
|
||||
|
||||
ensure_visibility_columns()
|
||||
|
||||
from .sync import ensure_export_indexes
|
||||
|
||||
ensure_export_indexes()
|
||||
|
||||
@@ -17,6 +17,7 @@ SOFT_DELETE_TABLES = [
|
||||
"reactions",
|
||||
"bookmarks",
|
||||
"notes",
|
||||
"messages",
|
||||
"skills",
|
||||
"communities",
|
||||
"node_devices",
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from .core import TTLCache, datetime, db, timedelta, timezone
|
||||
from .ranking import get_top_authors
|
||||
from .visibility import listed_clause
|
||||
|
||||
|
||||
_stats_cache = TTLCache(ttl=30, max_size=50)
|
||||
@@ -16,19 +17,20 @@ def get_site_stats() -> dict:
|
||||
.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
.isoformat()
|
||||
)
|
||||
def listed_count(name, **filters):
|
||||
if name not in db.tables:
|
||||
return 0
|
||||
listed = listed_clause(db[name])
|
||||
clauses = [listed] if listed is not None else []
|
||||
return db[name].count(*clauses, **filters)
|
||||
|
||||
stats = {
|
||||
"total_members": db["users"].count() if "users" in db.tables else 0,
|
||||
"posts_today": db["posts"].count(
|
||||
created_at={">=": today_start}, deleted_at=None
|
||||
)
|
||||
if "posts" in db.tables
|
||||
else 0,
|
||||
"total_projects": db["projects"].count(deleted_at=None)
|
||||
if "projects" in db.tables
|
||||
else 0,
|
||||
"total_gists": db["gists"].count(deleted_at=None)
|
||||
if "gists" in db.tables
|
||||
else 0,
|
||||
"posts_today": listed_count(
|
||||
"posts", created_at={">=": today_start}, deleted_at=None
|
||||
),
|
||||
"total_projects": listed_count("projects", deleted_at=None),
|
||||
"total_gists": listed_count("gists", deleted_at=None),
|
||||
}
|
||||
_stats_cache.set("site", stats)
|
||||
return stats
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from sqlalchemy import or_
|
||||
from .core import db, get_table, _index
|
||||
|
||||
PUBLISHED = "published"
|
||||
HIDDEN = "hidden"
|
||||
DRAFT = "draft"
|
||||
OFFLINE = "offline"
|
||||
|
||||
VISIBILITY_VALUES = (PUBLISHED, HIDDEN, DRAFT, OFFLINE)
|
||||
VISIBILITY_CHOICES = (PUBLISHED, HIDDEN, OFFLINE)
|
||||
DEFAULT_VISIBILITY = PUBLISHED
|
||||
LISTED_VALUES = (PUBLISHED,)
|
||||
OPEN_VALUES = (PUBLISHED, HIDDEN)
|
||||
|
||||
VISIBILITY_COLUMNS = {
|
||||
"posts": "visibility",
|
||||
"gists": "visibility",
|
||||
"projects": "visibility",
|
||||
"news": "status",
|
||||
}
|
||||
|
||||
TARGET_TABLES = {
|
||||
"post": "posts",
|
||||
"gist": "gists",
|
||||
"project": "projects",
|
||||
"news": "news",
|
||||
}
|
||||
|
||||
VISIBILITY_LABELS = {
|
||||
PUBLISHED: "Published",
|
||||
HIDDEN: "Hidden",
|
||||
DRAFT: "Draft",
|
||||
OFFLINE: "Offline",
|
||||
}
|
||||
|
||||
VISIBILITY_HINTS = {
|
||||
PUBLISHED: "Listed everywhere and open to everyone.",
|
||||
HIDDEN: "Not listed or searchable; anyone with the link can still open it.",
|
||||
DRAFT: "Only you can see it. Not published yet.",
|
||||
OFFLINE: "Taken offline for now. Only you can see it; publish it again at any time.",
|
||||
}
|
||||
|
||||
def normalize_visibility(value, fallback: str = DEFAULT_VISIBILITY) -> str:
|
||||
text = str(value or "").strip().lower()
|
||||
return text if text in VISIBILITY_VALUES else fallback
|
||||
|
||||
|
||||
def visibility_column(table_name: str) -> str | None:
|
||||
return VISIBILITY_COLUMNS.get(table_name)
|
||||
|
||||
|
||||
def item_visibility(table_name: str, item: dict | None) -> str:
|
||||
column = visibility_column(table_name)
|
||||
if not column or not item:
|
||||
return DEFAULT_VISIBILITY
|
||||
return normalize_visibility(item.get(column))
|
||||
|
||||
|
||||
def is_listed(table_name: str, item: dict | None) -> bool:
|
||||
return item_visibility(table_name, item) in LISTED_VALUES
|
||||
|
||||
|
||||
def is_open(table_name: str, item: dict | None) -> bool:
|
||||
return item_visibility(table_name, item) in OPEN_VALUES
|
||||
|
||||
|
||||
def listed_clause(table, own_uid: str | None = None):
|
||||
name = getattr(table, "name", None) or getattr(table.table, "name", None)
|
||||
column_name = visibility_column(name)
|
||||
if not column_name or not table.exists:
|
||||
return None
|
||||
columns = table.table.columns
|
||||
if column_name not in columns:
|
||||
return None
|
||||
column = columns[column_name]
|
||||
clause = or_(column.is_(None), column.in_(LISTED_VALUES))
|
||||
if own_uid and "user_uid" in columns:
|
||||
clause = or_(clause, columns.user_uid == own_uid)
|
||||
return clause
|
||||
|
||||
|
||||
def listed_sql(table_name: str, alias: str = "") -> str:
|
||||
column = visibility_column(table_name)
|
||||
if not column or table_name not in db.tables:
|
||||
return "1 = 1"
|
||||
if not get_table(table_name).has_column(column):
|
||||
return "1 = 1"
|
||||
prefix = f"{alias}." if alias else ""
|
||||
return f"COALESCE({prefix}{column}, '{DEFAULT_VISIBILITY}') = '{PUBLISHED}'"
|
||||
|
||||
|
||||
def ensure_visibility_columns() -> None:
|
||||
for table_name, column in VISIBILITY_COLUMNS.items():
|
||||
if table_name == "news" or table_name not in db.tables:
|
||||
continue
|
||||
table = get_table(table_name)
|
||||
if not table.has_column(column):
|
||||
table.create_column_by_example(column, "")
|
||||
_index(db, table_name, f"idx_{table_name}_visibility", [column])
|
||||
|
||||
|
||||
def visibility_options(values=VISIBILITY_CHOICES) -> list[dict]:
|
||||
return [
|
||||
{"value": v, "label": VISIBILITY_LABELS[v], "hint": VISIBILITY_HINTS[v]}
|
||||
for v in values
|
||||
]
|
||||
|
||||
|
||||
def visibility_label(value) -> str:
|
||||
return VISIBILITY_LABELS[normalize_visibility(value)]
|
||||
@@ -99,3 +99,71 @@ NON_BODY_ENDPOINTS = {
|
||||
"notifications-open",
|
||||
"admin-bots-frame",
|
||||
}
|
||||
|
||||
|
||||
VISIBILITY_VALUES = ["published", "hidden", "offline", "draft"]
|
||||
VISIBILITY_DESCRIPTION = (
|
||||
"published (default): listed and open to everyone. hidden: not listed, searched or indexed, "
|
||||
"anyone with the link can open it. offline: only the owner and administrators can open it. "
|
||||
"draft: only the owner, not yet published."
|
||||
)
|
||||
|
||||
|
||||
VISIBILITY_OMITTED = {
|
||||
"create": " Optional and backwards compatible: when omitted the item is published.",
|
||||
"edit": " Optional and backwards compatible: when omitted the current value is kept.",
|
||||
"value": "",
|
||||
}
|
||||
|
||||
|
||||
def visibility_field(mode="create"):
|
||||
return field(
|
||||
"value" if mode == "value" else "visibility",
|
||||
"form",
|
||||
"enum",
|
||||
mode == "value",
|
||||
"published",
|
||||
VISIBILITY_DESCRIPTION + VISIBILITY_OMITTED[mode],
|
||||
VISIBILITY_VALUES,
|
||||
)
|
||||
|
||||
|
||||
def visibility_endpoint(id, kind, path, slug_name, example, auth="user"):
|
||||
area = kind if kind == "news" else f"{kind}s"
|
||||
owner_text = (
|
||||
"Administrators only." if auth == "admin" else "Only the owner can change it."
|
||||
)
|
||||
return endpoint(
|
||||
id=id,
|
||||
method="POST",
|
||||
path=path,
|
||||
title=f"Set {kind} visibility",
|
||||
summary=(
|
||||
f"Set the visibility of a {kind} to published, hidden, offline or draft. "
|
||||
f"{owner_text} Hidden items disappear from every listing, search and sitemap but "
|
||||
"open by direct link; offline and draft items are visible only to the owner and "
|
||||
"administrators (everyone else gets a 404)."
|
||||
),
|
||||
auth=auth,
|
||||
encoding="form",
|
||||
params=[
|
||||
field(
|
||||
slug_name,
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
example,
|
||||
f"Slug or UID of the {kind}.",
|
||||
),
|
||||
visibility_field("value"),
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": f"/{area}/example-slug",
|
||||
"data": {
|
||||
"uid": "0190c0de-0000-7000-8000-000000000000",
|
||||
"visibility": "hidden",
|
||||
"url": f"/{area}/example-slug",
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .._shared import endpoint, field
|
||||
from .._shared import endpoint, field, visibility_endpoint
|
||||
|
||||
GROUP = {
|
||||
"slug": "admin",
|
||||
@@ -482,6 +482,14 @@ four ways to sign requests.
|
||||
field("uid", "path", "string", True, "NEWS_UID", "Article UID.")
|
||||
],
|
||||
),
|
||||
visibility_endpoint(
|
||||
"admin-news-visibility",
|
||||
"news",
|
||||
"/admin/news/{uid}/visibility",
|
||||
"uid",
|
||||
"NEWS_UID",
|
||||
auth="admin",
|
||||
),
|
||||
endpoint(
|
||||
id="admin-news-landing",
|
||||
method="POST",
|
||||
@@ -718,6 +726,7 @@ four ways to sign requests.
|
||||
field("target_model", "json", "string", True, "x-ai/grok-4.3", "Model actually sent upstream."),
|
||||
field("provider", "json", "string", False, "openrouter", "Provider name to route through. Blank = the default provider."),
|
||||
field("fallback_model", "json", "string", False, "molodetz-pro", "Another already-configured source_model of the same kind, tried once automatically when this route fails after its own retries are exhausted. Blank = no fallback."),
|
||||
field("tools_mode", "json", "string", False, "emulated", "'native' (default) forwards tools untouched. 'emulated' declares that this model does not support tool calling: tools are rewritten into the system prompt before sending and the reply is converted back into OpenAI tool_calls."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .._shared import COMMENT_TARGETS, GIST_LANGUAGES, PROJECT_TYPES, endpoint, field
|
||||
from .._shared import (
|
||||
COMMENT_TARGETS,
|
||||
GIST_LANGUAGES,
|
||||
PROJECT_TYPES,
|
||||
endpoint,
|
||||
field,
|
||||
visibility_field,
|
||||
)
|
||||
from devplacepy.constants import TOPICS
|
||||
|
||||
GROUP = {
|
||||
@@ -143,6 +150,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field(),
|
||||
field(
|
||||
"content",
|
||||
"form",
|
||||
@@ -276,6 +284,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field("edit"),
|
||||
field(
|
||||
"post_slug",
|
||||
"path",
|
||||
@@ -548,6 +557,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field(),
|
||||
field(
|
||||
"title",
|
||||
"form",
|
||||
@@ -649,6 +659,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field("edit"),
|
||||
field(
|
||||
"project_slug",
|
||||
"path",
|
||||
@@ -771,7 +782,7 @@ four ways to sign requests.
|
||||
id="projects-private",
|
||||
method="POST",
|
||||
path="/projects/{project_slug}/private",
|
||||
title="Set project visibility",
|
||||
title="Make a project private or public",
|
||||
summary="Mark a project you own private or public. Send value=1 for private, value=0 for public. A project you hide as a member stays visible to administrators; a project you hide as an administrator is visible only to you, not to other administrators (this also hides its files and any attached containers).",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
@@ -1023,6 +1034,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field(),
|
||||
field(
|
||||
"title",
|
||||
"form",
|
||||
@@ -1068,6 +1080,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field("edit"),
|
||||
field(
|
||||
"gist_slug",
|
||||
"path",
|
||||
|
||||
@@ -79,6 +79,31 @@ four ways to sign requests.
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="messages-delete",
|
||||
method="POST",
|
||||
path="/messages/{message_uid}/delete",
|
||||
title="Delete a sent message",
|
||||
summary="Soft-delete a direct message. Administrators only, and only messages they sent themselves; the bubble is removed live for both participants.",
|
||||
auth="admin",
|
||||
encoding="none",
|
||||
destructive=True,
|
||||
params=[
|
||||
field(
|
||||
"message_uid",
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
"MESSAGE_UID",
|
||||
"Uid of the message to delete.",
|
||||
),
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/messages?with_uid=RECEIVER_UID",
|
||||
"data": {"deleted_uid": "MESSAGE_UID"},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="messages-conversations",
|
||||
method="GET",
|
||||
|
||||
@@ -11,7 +11,7 @@ GROUP = {
|
||||
Attachment storage. Upload a file - or hand the server a public URL to fetch - to receive an
|
||||
attachment record, then reference its `uid` in an `attachment_uids` field when creating a post,
|
||||
comment, project, gist, message, or issue - see
|
||||
[Posts, Comments, Projects, Gists & News](/docs/content.html). Images and videos embed and
|
||||
[Posts, Comments, Projects, Gists & News](/docs/content.html). GIFs from GIPHY are attached the same way, through `/giphy/attach`. Images and videos embed and
|
||||
play inline once posted; other types render as download links. The record's `is_image` and
|
||||
`is_video` flags indicate how the file is displayed.
|
||||
|
||||
@@ -112,6 +112,82 @@ four ways to sign requests.
|
||||
"mime_type": "image/png",
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-status",
|
||||
method="GET",
|
||||
path="/giphy/status",
|
||||
title="GIF picker availability",
|
||||
summary="Tell whether an administrator has configured GIPHY, so a client knows whether to offer a GIF picker.",
|
||||
auth="user",
|
||||
sample_response={"configured": True},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-search",
|
||||
method="GET",
|
||||
path="/giphy/search",
|
||||
title="Search GIFs",
|
||||
summary="Search GIPHY by keyword through the server, so the API key never reaches a browser.",
|
||||
auth="user",
|
||||
params=[
|
||||
field("q", "query", "string", True, "happy dance", "Search keywords, 1-200 characters."),
|
||||
field("limit", "query", "integer", False, "24", "Number of results, 1-50 (default 24)."),
|
||||
],
|
||||
notes=[
|
||||
"Results are capped to the content rating an administrator configured (default `pg`) and cached for five minutes.",
|
||||
"Each member may make 30 GIF searches or trending requests per minute; beyond that the answer is `429` with `Retry-After`. `503` means GIPHY is not configured, `400` a missing `q`, `502` an upstream failure.",
|
||||
],
|
||||
sample_response={"query": "happy dance", "results": [{
|
||||
"id": "xT9IgDEI1iZyb2wqo8",
|
||||
"title": "happy dance GIF",
|
||||
"url": "https://giphy.com/gifs/xT9IgDEI1iZyb2wqo8",
|
||||
"preview": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/200w.gif", "width": 200, "height": 150, "size": 90210},
|
||||
"original": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/giphy.gif", "width": 480, "height": 360, "size": 1048576},
|
||||
}]},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-trending",
|
||||
method="GET",
|
||||
path="/giphy/trending",
|
||||
title="Trending GIFs",
|
||||
summary="List the GIFs currently trending on GIPHY, in the same shape as Search GIFs.",
|
||||
auth="user",
|
||||
params=[
|
||||
field("limit", "query", "integer", False, "24", "Number of results, 1-50 (default 24)."),
|
||||
],
|
||||
notes=["Shares the per-member rate limit, the cache and the error codes of Search GIFs."],
|
||||
sample_response={"results": [{
|
||||
"id": "xT9IgDEI1iZyb2wqo8",
|
||||
"title": "happy dance GIF",
|
||||
"url": "https://giphy.com/gifs/xT9IgDEI1iZyb2wqo8",
|
||||
"preview": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/200w.gif", "width": 200, "height": 150, "size": 90210},
|
||||
"original": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/giphy.gif", "width": 480, "height": 360, "size": 1048576},
|
||||
}]},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-attach",
|
||||
method="POST",
|
||||
path="/giphy/attach",
|
||||
title="Attach a GIF",
|
||||
summary="Download a GIPHY GIF on the server and store it as an attachment, exactly like an upload.",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field("gif_id", "json", "string", True, "xT9IgDEI1iZyb2wqo8", "GIPHY id from Search GIFs or Trending GIFs."),
|
||||
field("title", "json", "string", False, "happy dance", "Optional title used as the attachment's filename."),
|
||||
],
|
||||
notes=[
|
||||
"Accepts a JSON body or a form. Returns `201` with the same record as Upload a file; reference its `uid` in `attachment_uids`. `413` if larger than the upload limit, `415` when GIF uploads are disabled, `404` for an unknown id.",
|
||||
],
|
||||
sample_response={
|
||||
"uid": "ATTACHMENT_UID",
|
||||
"original_filename": "happy dance.gif",
|
||||
"url": "/static/uploads/attachments/ab/cd/ATTACHMENT_UID.gif",
|
||||
"file_size": 1048576,
|
||||
"mime_type": "image/gif",
|
||||
"is_image": True,
|
||||
"is_video": False,
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="uploads-list",
|
||||
method="GET",
|
||||
|
||||
@@ -72,7 +72,7 @@ def _user_facts(user: dict) -> dict:
|
||||
"following": _count("follows", follower_uid=uid),
|
||||
"bookmarks": _count("bookmarks", user_uid=uid),
|
||||
"unread_notifications": _count("notifications", user_uid=uid, read=False),
|
||||
"unread_messages": _count("messages", receiver_uid=uid, read=False),
|
||||
"unread_messages": _count("messages", receiver_uid=uid, read=False, deleted_at=None),
|
||||
},
|
||||
"recent_posts": [
|
||||
{
|
||||
|
||||
@@ -31,9 +31,11 @@ def _post_pool() -> list[str]:
|
||||
if cached is not None:
|
||||
return cached
|
||||
slugs: list[str] = []
|
||||
from devplacepy.database import listed_sql
|
||||
|
||||
if "posts" in db.tables:
|
||||
rows = db.query(
|
||||
"SELECT slug, uid FROM posts WHERE deleted_at IS NULL ORDER BY RANDOM() LIMIT :limit",
|
||||
f"SELECT slug, uid FROM posts WHERE deleted_at IS NULL AND {listed_sql('posts')} ORDER BY RANDOM() LIMIT :limit",
|
||||
limit=POOL_SIZE,
|
||||
)
|
||||
slugs = [row["slug"] or row["uid"] for row in rows]
|
||||
|
||||
@@ -24,6 +24,7 @@ from devplacepy.services.containers.service import ContainerService
|
||||
from devplacepy.services.containers.workspace_service import WorkspaceService
|
||||
from devplacepy.services.dbapi.service import DbApiJobService
|
||||
from devplacepy.services.devii import DeviiService
|
||||
from devplacepy.services.giphy.service import GiphyService
|
||||
from devplacepy.services.gitea.service import IssueTrackerService
|
||||
from devplacepy.services.jobs.award_service import AwardService
|
||||
from devplacepy.services.jobs.deepsearch.service import DeepsearchService
|
||||
@@ -73,6 +74,7 @@ SERVICE_CLASSES = [
|
||||
IssueCreateService,
|
||||
PlanningReportService,
|
||||
IssueTrackerService,
|
||||
GiphyService,
|
||||
ContainerService,
|
||||
WorkspaceService,
|
||||
XmlrpcService,
|
||||
|
||||
@@ -38,6 +38,7 @@ from devplacepy.routers import (
|
||||
gists,
|
||||
quizzes,
|
||||
uploads,
|
||||
giphy,
|
||||
media,
|
||||
push,
|
||||
leaderboard,
|
||||
@@ -72,6 +73,7 @@ logging.basicConfig(
|
||||
level=getattr(logging, LOG_LEVEL, logging.INFO),
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
)
|
||||
logging.getLogger("alembic.runtime.migration").setLevel(logging.WARNING)
|
||||
|
||||
app = FastAPI(
|
||||
title="DevPlace",
|
||||
@@ -125,6 +127,7 @@ app.include_router(issues.router, prefix="/issues")
|
||||
app.include_router(gists.router, prefix="/gists")
|
||||
app.include_router(news.router, prefix="/news")
|
||||
app.include_router(uploads.router, prefix="/uploads")
|
||||
app.include_router(giphy.router, prefix="/giphy")
|
||||
app.include_router(media.router, prefix="/media")
|
||||
app.include_router(zips.router, prefix="/zips")
|
||||
app.include_router(forks.router, prefix="/forks")
|
||||
|
||||
@@ -6,6 +6,9 @@ from devplacepy.models.shared import (
|
||||
normalize_poll_options,
|
||||
_declared_age,
|
||||
normalize_index_list,
|
||||
VisibilityForm,
|
||||
VisibilityCreateMixin,
|
||||
VisibilityEditMixin,
|
||||
)
|
||||
from devplacepy.models.auth import (
|
||||
SignupForm,
|
||||
@@ -78,6 +81,7 @@ from devplacepy.models.uploads import (
|
||||
UploadUrlForm,
|
||||
GenerateImageForm,
|
||||
AttachmentRenameForm,
|
||||
GiphyAttachForm,
|
||||
)
|
||||
from devplacepy.models.tools import (
|
||||
PlanningForm,
|
||||
@@ -161,6 +165,9 @@ __all__ = [
|
||||
"normalize_poll_options",
|
||||
"_declared_age",
|
||||
"normalize_index_list",
|
||||
"VisibilityForm",
|
||||
"VisibilityCreateMixin",
|
||||
"VisibilityEditMixin",
|
||||
"SignupForm",
|
||||
"LoginForm",
|
||||
"ForgotPasswordForm",
|
||||
@@ -217,6 +224,7 @@ __all__ = [
|
||||
"IssueAttachmentForm",
|
||||
"IssueStatusForm",
|
||||
"UploadUrlForm",
|
||||
"GiphyAttachForm",
|
||||
"GenerateImageForm",
|
||||
"AttachmentRenameForm",
|
||||
"PlanningForm",
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from typing import Literal, Optional
|
||||
from pydantic import BaseModel, Field
|
||||
from devplacepy.models.shared import VisibilityCreateMixin, VisibilityEditMixin
|
||||
|
||||
|
||||
class SkillInstallForm(BaseModel):
|
||||
@@ -23,7 +24,7 @@ class MessageForm(BaseModel):
|
||||
client_id: Optional[str] = Field(default=None, max_length=64)
|
||||
|
||||
|
||||
class GistForm(BaseModel):
|
||||
class GistForm(VisibilityCreateMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(default="", max_length=5000)
|
||||
source_code: str = Field(min_length=1, max_length=400000)
|
||||
@@ -31,7 +32,7 @@ class GistForm(BaseModel):
|
||||
attachment_uids: list[str] = []
|
||||
|
||||
|
||||
class GistEditForm(BaseModel):
|
||||
class GistEditForm(VisibilityEditMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(default="", max_length=5000)
|
||||
source_code: str = Field(min_length=1, max_length=400000)
|
||||
|
||||
@@ -4,10 +4,14 @@ from typing import Literal
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
from devplacepy.constants import TOPICS
|
||||
from devplacepy.rendering import is_single_emoji
|
||||
from devplacepy.models.shared import normalize_poll_options
|
||||
from devplacepy.models.shared import (
|
||||
normalize_poll_options,
|
||||
VisibilityCreateMixin,
|
||||
VisibilityEditMixin,
|
||||
)
|
||||
|
||||
|
||||
class PostForm(BaseModel):
|
||||
class PostForm(VisibilityCreateMixin):
|
||||
content: str = Field(min_length=10, max_length=125000)
|
||||
title: str = Field(default="", max_length=500)
|
||||
topic: str = "random"
|
||||
@@ -50,7 +54,7 @@ class WarJoinForm(BaseModel):
|
||||
return value
|
||||
|
||||
|
||||
class PostEditForm(BaseModel):
|
||||
class PostEditForm(VisibilityEditMixin):
|
||||
content: str = Field(min_length=10, max_length=125000)
|
||||
title: str = Field(default="", max_length=500)
|
||||
topic: str = "random"
|
||||
|
||||
@@ -2,10 +2,15 @@
|
||||
|
||||
from typing import Literal
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from devplacepy.models.shared import normalize_european_date, normalize_website_url
|
||||
from devplacepy.models.shared import (
|
||||
normalize_european_date,
|
||||
normalize_website_url,
|
||||
VisibilityCreateMixin,
|
||||
VisibilityEditMixin,
|
||||
)
|
||||
|
||||
|
||||
class ProjectForm(BaseModel):
|
||||
class ProjectForm(VisibilityCreateMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(min_length=1, max_length=5000)
|
||||
release_date: str = ""
|
||||
@@ -33,7 +38,7 @@ class ProjectForm(BaseModel):
|
||||
return normalize_website_url(value)
|
||||
|
||||
|
||||
class ProjectEditForm(BaseModel):
|
||||
class ProjectEditForm(VisibilityEditMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(min_length=1, max_length=5000)
|
||||
release_date: str = ""
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
from urllib.parse import urlparse
|
||||
from pydantic import BaseModel, field_validator
|
||||
from devplacepy.database.visibility import VISIBILITY_VALUES, DEFAULT_VISIBILITY
|
||||
|
||||
|
||||
def normalize_european_date(value):
|
||||
@@ -73,3 +76,41 @@ def normalize_index_list(value):
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return indexes
|
||||
|
||||
|
||||
def parse_visibility(value, default):
|
||||
text = str(value if value is not None else "").strip().lower()
|
||||
if not text:
|
||||
return default
|
||||
if text not in VISIBILITY_VALUES:
|
||||
raise ValueError(
|
||||
"Visibility must be one of: " + ", ".join(VISIBILITY_VALUES)
|
||||
)
|
||||
return text
|
||||
|
||||
|
||||
class VisibilityCreateMixin(BaseModel):
|
||||
visibility: str = DEFAULT_VISIBILITY
|
||||
|
||||
@field_validator("visibility", mode="before")
|
||||
@classmethod
|
||||
def valid_visibility(cls, value):
|
||||
return parse_visibility(value, DEFAULT_VISIBILITY)
|
||||
|
||||
|
||||
class VisibilityEditMixin(BaseModel):
|
||||
visibility: Optional[str] = None
|
||||
|
||||
@field_validator("visibility", mode="before")
|
||||
@classmethod
|
||||
def valid_visibility(cls, value):
|
||||
return parse_visibility(value, None)
|
||||
|
||||
|
||||
class VisibilityForm(BaseModel):
|
||||
value: str
|
||||
|
||||
@field_validator("value", mode="before")
|
||||
@classmethod
|
||||
def valid_value(cls, value):
|
||||
return parse_visibility(value, None) or DEFAULT_VISIBILITY
|
||||
|
||||
@@ -16,3 +16,8 @@ class GenerateImageForm(BaseModel):
|
||||
|
||||
class AttachmentRenameForm(BaseModel):
|
||||
filename: str = Field(min_length=1, max_length=255)
|
||||
|
||||
|
||||
class GiphyAttachForm(BaseModel):
|
||||
gif_id: str = Field(min_length=1, max_length=64)
|
||||
title: Optional[str] = Field(default=None, max_length=255)
|
||||
|
||||
@@ -34,6 +34,7 @@ Prefixes are wired in `main.py`:
|
||||
| `/issues` | issues/ package - issue tracker backed by Gitea (no local issue store): `index.py` (list `?state=`/`?page=`, detail `/{number}` with comments), `create.py` (async AI-enhanced filing `/create` enqueues a `issue_create` job, status at `/jobs/{uid}`), `comment.py` (synchronous, pushes to Gitea + notifies admins), `status.py` (admin open/closed), `attachments.py` (file attachments on open issues + comments, mirrored to Gitea native assets; add/list/delete with owner-or-admin + open-state guards) |
|
||||
| `/gists` | gists.py |
|
||||
| `/news` | news.py |
|
||||
| `/giphy` | giphy.py - GIF picker backend: `GET /giphy/status`, `GET /giphy/search`, `GET /giphy/trending` (cached, 30 calls/min/user, pure JSON) and `POST /giphy/attach` (stores the GIF as an attachment, audit `attachment.giphy`); see `services/giphy/CLAUDE.md` |
|
||||
| `/uploads` | uploads.py - attachment management CRUD for the signed-in user over the ONE `attachments` table (the same rows that appear on posts/comments/projects/gists/issues). Create: `POST /upload` (multipart), `POST /upload-url` (server-side fetch). Read: `GET ""` (own attachments, paginated 24/page newest-first, `?page=`, `?linked=true|false` via `database.get_user_attachments`), `GET /{attachment_uid}` (one, via `database.get_user_attachment`). Update: `PATCH /{attachment_uid}` (rename display filename via `attachments.rename_attachment`; the original file extension is ALWAYS preserved - renaming can never change the file type, the upload-time security control - audit `attachment.rename`). Delete: `DELETE /delete/{attachment_uid}` (soft delete). All `require_user_api` (401 for guests); read/rename/delete of another user's row is owner-or-admin. JSON-only router (no HTML/`respond`); list uses `UploadsListOut`, single/rename return `UploadItemOut`. Devii tools mirror every face: `upload_file`/`attach_url`/`list_attachments`/`get_attachment`/`rename_attachment`/`delete_attachment` |
|
||||
| `/media` | media.py - profile media gallery item soft delete/restore: `POST /media/{uid}/delete` and `POST /media/{uid}/restore` (owner or admin) |
|
||||
| `/openai` | openai_gateway.py |
|
||||
@@ -377,3 +378,7 @@ The profile **Activity** tab (`/profile/{username}?tab=activity`, public) interl
|
||||
- **Comment items** get `"url": f"{resolve_object_url(target_type, target_uid)}#comment-{c['uid']}"`, the SEO-friendly parent detail URL plus the `#comment-{uid}` anchor. This is **byte-identical to a comment notification's `target_url`** (`content.create_comment_record`), so clicking an activity comment reproduces the notification click exactly: it lands on the parent post and `NotificationManager.js` scrolls to / highlights the `id="comment-{uid}"` element (`_comment.html`). Reuse `resolve_object_url` (`database.py`) - never rebuild this URL by hand, and do not call `resolve_object_url("comment", uid)` here (it re-fetches the row the loop already has).
|
||||
- **Frontend:** the card is wrapped in the shared full-card overlay link (`card-link-host` + `_card_link.html`), the same pattern `_post_card.html` uses; inner content links (mentions, URLs rendered by `render_content`) stay clickable via the existing `card-link-host a:not(.card-link)` z-index rule (`base.css`). No new CSS.
|
||||
- **API:** `ProfileOut.activities` is `list[Any]`, so the added `url` (and, on comments, the existing `target_type` + `uid` = parent target uid) flow into the JSON response with no schema change - this is the parent-post reference exposed on each comment item.
|
||||
|
||||
## Content visibility routes
|
||||
|
||||
Posts, gists, projects and news share one four-value status (published, hidden, draft, offline); the specification, truth table and choke points are in `devplacepy/database/CLAUDE.md` ("Content visibility"). It is an ordinary field of the create and edit forms of posts, gists and projects (`visibility`, via the `_visibility_field.html` partial), handled by `create_content_item` and `edit_content_item` like every other field; there are no dedicated toggle routes. The only dedicated route is the administrator's `POST /admin/news/{uid}/visibility` (`VisibilityForm{value}`), because news has no edit form. Hidden items are `noindex,nofollow`. Comments, votes, reactions, bookmarks and notes refuse (404) a target the viewer cannot open (`content.can_view_target`), and the saved/notes lists skip such targets. The devRant API resolves rants through `post_by_id(rant_id, viewer)` and lists listed rants only. Audit key: `<kind>.visibility.set`. Devii: the `visibility` parameter on the create and edit tools, plus `admin_set_news_visibility`.
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
import logging
|
||||
from typing import Optional
|
||||
from urllib.parse import quote
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
|
||||
@@ -17,7 +18,12 @@ from devplacepy.schemas import (
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.services.openai_gateway import model_stats_query, quota, routing
|
||||
from devplacepy.services.openai_gateway import (
|
||||
health,
|
||||
model_stats_query,
|
||||
quota,
|
||||
routing,
|
||||
)
|
||||
from devplacepy.templating import templates
|
||||
from devplacepy.utils import not_found, require_admin
|
||||
|
||||
@@ -149,10 +155,14 @@ def _error_response(
|
||||
|
||||
|
||||
@router.get("/gateway", response_class=HTMLResponse)
|
||||
async def gateway_config_page(request: Request, tab: str = "models"):
|
||||
async def gateway_config_page(
|
||||
request: Request, tab: str = "models", tested: str = "", test_image: str = ""
|
||||
):
|
||||
admin = require_admin(request)
|
||||
if tab not in TABS:
|
||||
tab = "models"
|
||||
if not test_image.startswith("http"):
|
||||
test_image = ""
|
||||
quota_rules = quota.quota_rule_store.list()
|
||||
for rule in quota_rules:
|
||||
rule["spent_24h_usd"] = round(
|
||||
@@ -172,6 +182,8 @@ async def gateway_config_page(request: Request, tab: str = "models"):
|
||||
"default_provider": _default_provider_summary(),
|
||||
"quota_defaults": _quota_defaults_summary(),
|
||||
"stats_ranges": list(model_stats_query.RANGE_SECONDS.keys()),
|
||||
"tested": tested,
|
||||
"test_image": test_image,
|
||||
},
|
||||
model=AdminGatewayOut,
|
||||
)
|
||||
@@ -472,6 +484,7 @@ MODEL_FORM_DEFAULTS = {
|
||||
"off_peak_end": "",
|
||||
"off_peak_discount_pct": "0",
|
||||
"fallback_model": "",
|
||||
"tools_mode": "native",
|
||||
"is_active": "1",
|
||||
}
|
||||
|
||||
@@ -505,6 +518,7 @@ def _model_form_values_from_route(route: dict) -> dict:
|
||||
"off_peak_end": _minutes_to_time(route.get("off_peak_end_minute")),
|
||||
"off_peak_discount_pct": _num("off_peak_discount_pct"),
|
||||
"fallback_model": route.get("fallback_model", ""),
|
||||
"tools_mode": route.get("tools_mode", "native"),
|
||||
"is_active": _bool_str(route.get("is_active", True)),
|
||||
}
|
||||
|
||||
@@ -548,6 +562,7 @@ def _model_payload_kwargs(data: dict) -> dict:
|
||||
"off_peak_end_minute": _time_to_minutes(data.get("off_peak_end", "")),
|
||||
"off_peak_discount_pct": _num("off_peak_discount_pct"),
|
||||
"fallback_model": data.get("fallback_model", ""),
|
||||
"tools_mode": data.get("tools_mode", "native"),
|
||||
"is_active": data.get("is_active", "1"),
|
||||
}
|
||||
|
||||
@@ -675,6 +690,58 @@ async def delete_model_page(request: Request, source_model: str):
|
||||
return RedirectResponse(url=f"{INDEX_URL}?tab=models", status_code=302)
|
||||
|
||||
|
||||
@router.post("/gateway/models/test-all")
|
||||
async def test_all_models_page(request: Request):
|
||||
admin = require_admin(request)
|
||||
tested, failed = await health.test_all()
|
||||
audit.record(
|
||||
request,
|
||||
"gateway.models.test_all",
|
||||
user=admin,
|
||||
target_type="gateway_models",
|
||||
target_uid="all",
|
||||
target_label=f"{tested} routes",
|
||||
summary=(
|
||||
f"admin {admin['username']} tested all gateway model routes: "
|
||||
f"{tested - failed} ok, {failed} failed"
|
||||
),
|
||||
)
|
||||
return RedirectResponse(
|
||||
url=f"{INDEX_URL}?tab=models&tested=all", status_code=302
|
||||
)
|
||||
|
||||
|
||||
@router.post("/gateway/models/{source_model:path}/test")
|
||||
async def test_model_page(request: Request, source_model: str):
|
||||
admin = require_admin(request)
|
||||
route = routing.model_store.get(source_model)
|
||||
if route is None:
|
||||
raise not_found("Model route not found")
|
||||
result = await health.probe_route(source_model)
|
||||
health.record_health(source_model, result)
|
||||
audit.record(
|
||||
request,
|
||||
"gateway.model.test",
|
||||
user=admin,
|
||||
target_type="gateway_model",
|
||||
target_uid=source_model,
|
||||
target_label=source_model,
|
||||
summary=(
|
||||
f"admin {admin['username']} tested gateway model route "
|
||||
f"{source_model}: {'ok' if result.ok else result.code}"
|
||||
),
|
||||
)
|
||||
url = f"{INDEX_URL}?tab=models&tested={quote(source_model, safe='')}"
|
||||
if (
|
||||
result.ok
|
||||
and route.kind == "image"
|
||||
and result.image_url.startswith("http")
|
||||
and len(result.image_url) <= 2000
|
||||
):
|
||||
url += f"&test_image={quote(result.image_url, safe='')}"
|
||||
return RedirectResponse(url=url, status_code=302)
|
||||
|
||||
|
||||
# --- Quota rules: JSON API (Devii + programmatic clients) -------------------
|
||||
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from fastapi import APIRouter, Request
|
||||
from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
@@ -15,6 +16,9 @@ from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.responses import respond, action_result
|
||||
from devplacepy.schemas import AdminNewsOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.models import VisibilityForm
|
||||
from devplacepy.content import set_visibility_item
|
||||
from devplacepy.dependencies import json_or_form
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@@ -182,3 +186,21 @@ async def admin_news_delete(request: Request, uid: str):
|
||||
links=[audit.target("news", uid, article.get("title"))],
|
||||
)
|
||||
return action_result(request, "/admin/news")
|
||||
|
||||
|
||||
@router.post("/news/{uid}/visibility")
|
||||
async def admin_news_visibility(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[VisibilityForm, Depends(json_or_form(VisibilityForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
return set_visibility_item(
|
||||
request,
|
||||
"news",
|
||||
"news",
|
||||
admin,
|
||||
uid,
|
||||
data.value,
|
||||
"/admin/news",
|
||||
)
|
||||
|
||||
@@ -6,6 +6,7 @@ from fastapi.responses import RedirectResponse, JSONResponse, HTMLResponse
|
||||
from devplacepy.database import get_table, db, paginate, resolve_object_url, _now_iso
|
||||
from devplacepy.utils import generate_uid, require_user, time_ago, redirect_back
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.content import can_view_item, can_view_target
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import SavedOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -50,7 +51,8 @@ async def saved_page(request: Request, before: str = None):
|
||||
if table.has_column("deleted_at"):
|
||||
clauses.append(table.table.columns.deleted_at.is_(None))
|
||||
for obj in table.find(*clauses):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
if can_view_item(table_name, obj, user):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
|
||||
items = []
|
||||
for row in rows:
|
||||
@@ -95,6 +97,8 @@ async def toggle_bookmark(request: Request, target_type: str, target_uid: str):
|
||||
user = require_user(request)
|
||||
if target_type not in BOOKMARKABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
|
||||
bookmarks = get_table("bookmarks")
|
||||
existing = bookmarks.find_one(
|
||||
|
||||
@@ -8,11 +8,12 @@ from fastapi.responses import JSONResponse
|
||||
from devplacepy.database import get_table, resolve_object_url
|
||||
from devplacepy.content import (
|
||||
is_owner,
|
||||
can_view_target,
|
||||
create_comment_record,
|
||||
delete_comment_record,
|
||||
edit_comment_record,
|
||||
)
|
||||
from devplacepy.utils import require_user, is_admin
|
||||
from devplacepy.utils import require_user, is_admin, not_found
|
||||
from devplacepy.models import CommentForm, CommentEditForm
|
||||
from devplacepy.responses import action_result, wants_json, json_error
|
||||
from devplacepy.schemas import CommentEditOut
|
||||
@@ -26,6 +27,8 @@ router = APIRouter()
|
||||
async def create_comment(request: Request, data: Annotated[CommentForm, Depends(json_or_form(CommentForm))]):
|
||||
user = require_user(request)
|
||||
target_uid = data.target_uid or data.post_uid
|
||||
if not can_view_target(data.target_type, target_uid, user):
|
||||
raise not_found("Not found")
|
||||
comment_uid, comment_url = create_comment_record(
|
||||
request,
|
||||
user,
|
||||
|
||||
@@ -114,7 +114,7 @@ async def create_rant(request: Request):
|
||||
async def get_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
viewer = resolve_actor(request, params)
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, viewer)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
detail = load_rant_detail(post, viewer)
|
||||
@@ -127,7 +127,7 @@ async def edit_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.", fail_reason="not_found")
|
||||
if not is_owner(post, user):
|
||||
@@ -166,7 +166,7 @@ async def delete_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
if not (is_owner(post, user) or is_admin(user)):
|
||||
@@ -181,7 +181,7 @@ async def vote_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
value = as_int(params.get("vote"))
|
||||
@@ -208,7 +208,7 @@ async def _set_favorite(request: Request, rant_id: str, saved: bool):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
set_bookmark(request, user, "post", post["uid"], saved)
|
||||
@@ -221,7 +221,7 @@ async def comment_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
text = (params.get("comment") or "").strip()
|
||||
|
||||
@@ -69,6 +69,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "content-visibility",
|
||||
"title": "Content visibility",
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "code-farm",
|
||||
"title": "Code Farm",
|
||||
@@ -292,6 +298,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_COMPONENTS,
|
||||
},
|
||||
{
|
||||
"slug": "component-dp-gif-picker",
|
||||
"title": "dp-gif-picker",
|
||||
"kind": "prose",
|
||||
"section": SECTION_COMPONENTS,
|
||||
},
|
||||
{
|
||||
"slug": "component-dp-upload",
|
||||
"title": "dp-upload",
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from devplacepy.attachments import (
|
||||
RemoteFetchError,
|
||||
is_extension_allowed,
|
||||
store_attachment_from_url,
|
||||
)
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import GiphyAttachForm
|
||||
from devplacepy.responses import json_error
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.giphy import runtime
|
||||
from devplacepy.services.giphy.client import MAX_LIMIT, GiphyError
|
||||
from devplacepy.services.giphy.config import giphy_config
|
||||
from devplacepy.services.giphy.limits import UserRateLimiter
|
||||
from devplacepy.utils import require_user_api, track_action
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
limiter = UserRateLimiter()
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def giphy_status(request: Request):
|
||||
require_user_api(request)
|
||||
return JSONResponse({"configured": giphy_config().is_configured})
|
||||
|
||||
|
||||
def _limit_value(limit: int) -> int:
|
||||
return max(1, min(int(limit), MAX_LIMIT))
|
||||
|
||||
|
||||
def _error(exc: GiphyError) -> JSONResponse:
|
||||
status = exc.status if 400 <= exc.status < 600 else 502
|
||||
return json_error(status, exc.message)
|
||||
|
||||
|
||||
def _throttled(user: dict) -> JSONResponse | None:
|
||||
if limiter.allow(user["uid"]):
|
||||
return None
|
||||
response = json_error(429, "Too many GIF searches, slow down")
|
||||
response.headers["Retry-After"] = str(int(limiter.window_seconds))
|
||||
return response
|
||||
|
||||
|
||||
@router.get("/search")
|
||||
async def search_gifs(
|
||||
request: Request, q: str = Query(default="", max_length=200), limit: int = 24
|
||||
):
|
||||
user = require_user_api(request)
|
||||
query = q.strip()
|
||||
if not query:
|
||||
return json_error(400, "q is required")
|
||||
if (refused := _throttled(user)) is not None:
|
||||
return refused
|
||||
try:
|
||||
gifs = await runtime.get_client().search(query, limit=_limit_value(limit))
|
||||
except GiphyError as exc:
|
||||
return _error(exc)
|
||||
return JSONResponse({"query": query, "results": gifs})
|
||||
|
||||
|
||||
@router.get("/trending")
|
||||
async def trending_gifs(request: Request, limit: int = 24):
|
||||
user = require_user_api(request)
|
||||
if (refused := _throttled(user)) is not None:
|
||||
return refused
|
||||
try:
|
||||
gifs = await runtime.get_client().trending(limit=_limit_value(limit))
|
||||
except GiphyError as exc:
|
||||
return _error(exc)
|
||||
return JSONResponse({"results": gifs})
|
||||
|
||||
|
||||
@router.post("/attach")
|
||||
async def attach_gif(
|
||||
request: Request,
|
||||
data: Annotated[GiphyAttachForm, Depends(json_or_form(GiphyAttachForm))],
|
||||
):
|
||||
user = require_user_api(request)
|
||||
client = runtime.get_client()
|
||||
try:
|
||||
gif = await client.get_gif(data.gif_id)
|
||||
except GiphyError as exc:
|
||||
return _error(exc)
|
||||
|
||||
url = (gif.get("original") or {}).get("url") or (gif.get("preview") or {}).get("url")
|
||||
if not url:
|
||||
return json_error(502, "GIPHY returned no downloadable file")
|
||||
|
||||
stem = (data.title or gif.get("title") or gif.get("id") or "giphy").strip()
|
||||
filename = f"{stem[:200]}.gif"
|
||||
if not is_extension_allowed(".gif"):
|
||||
return json_error(415, "GIF uploads are disabled")
|
||||
|
||||
try:
|
||||
result = await store_attachment_from_url(url, user["uid"], filename)
|
||||
except RemoteFetchError as exc:
|
||||
logger.warning("Giphy attach failed for %s: %s", gif.get("id"), exc.message)
|
||||
return json_error(exc.status, exc.message)
|
||||
|
||||
audit.record(
|
||||
request,
|
||||
"attachment.giphy",
|
||||
user=user,
|
||||
target_type="attachment",
|
||||
target_uid=result.get("uid"),
|
||||
target_label=filename,
|
||||
metadata={
|
||||
"gif_id": gif.get("id"),
|
||||
"source_host": urlparse(url).hostname,
|
||||
"size": result.get("file_size"),
|
||||
"mime": result.get("mime_type"),
|
||||
},
|
||||
summary=f"{user['username']} attached a GIF from Giphy",
|
||||
links=[audit.attachment_link(result.get("uid"), filename)],
|
||||
)
|
||||
track_action(user["uid"], "upload")
|
||||
return JSONResponse(result, status_code=201)
|
||||
@@ -11,6 +11,8 @@ from devplacepy.database import (
|
||||
get_gist_languages,
|
||||
get_recent_comments_by_target_uids,
|
||||
paginate,
|
||||
listed_clause,
|
||||
is_listed,
|
||||
text_search_clause,
|
||||
resolve_object_url,
|
||||
mark_notifications_read_by_target,
|
||||
@@ -84,13 +86,17 @@ def get_gists_list(user_uid=None, language=None, search="", before=None, viewer=
|
||||
gists_table, search, ("title", "description"), author_field="user_uid"
|
||||
)
|
||||
clauses = [search_match] if search_match is not None else []
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == user_uid else None
|
||||
listed = listed_clause(gists_table, own_uid=own_uid)
|
||||
count_clauses = clauses + ([listed] if listed is not None else [])
|
||||
|
||||
total = gists_table.count(*clauses, deleted_at=None, **filters)
|
||||
total = gists_table.count(*count_clauses, deleted_at=None, **filters)
|
||||
gists, next_cursor = paginate(
|
||||
gists_table,
|
||||
*clauses,
|
||||
before=before,
|
||||
viewer_uid=viewer["uid"] if viewer else None,
|
||||
include_unlisted_of=own_uid,
|
||||
**filters,
|
||||
)
|
||||
|
||||
@@ -167,6 +173,7 @@ async def gist_detail(request: Request, gist_slug: str):
|
||||
title=gist.get("title", "Gist"),
|
||||
description=gist.get("description") or "",
|
||||
seo_target=("gist", gist["uid"]),
|
||||
robots="index,follow" if is_listed("gists", gist) else "noindex,nofollow",
|
||||
og_type="article",
|
||||
og_image=first_image_url(gist, detail["attachments"]),
|
||||
breadcrumbs=[
|
||||
@@ -216,6 +223,7 @@ async def create_gist(request: Request, data: Annotated[GistForm, Depends(json_o
|
||||
"description": description or None,
|
||||
"source_code": source_code,
|
||||
"language": language,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
title,
|
||||
XP_GIST,
|
||||
@@ -245,6 +253,7 @@ async def edit_gist(
|
||||
"description": data.description.strip() or None,
|
||||
"source_code": data.source_code.strip(),
|
||||
"language": language,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
"/gists",
|
||||
target_type="gist",
|
||||
@@ -254,3 +263,4 @@ async def edit_gist(
|
||||
async def delete_gist(request: Request, gist_slug: str):
|
||||
user = require_user(request)
|
||||
return delete_content_item(request, "gists", "gist", user, gist_slug, "/gists")
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ from datetime import datetime
|
||||
from typing import Annotated, Optional
|
||||
from fastapi import Depends, APIRouter, Request, WebSocket, WebSocketDisconnect
|
||||
from devplacepy.models import MessageForm
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
db,
|
||||
@@ -19,12 +19,14 @@ from devplacepy.attachments import get_attachments_batch
|
||||
from devplacepy.templating import clear_messages_cache
|
||||
from devplacepy.utils import (
|
||||
require_user,
|
||||
is_admin,
|
||||
not_found,
|
||||
time_ago,
|
||||
_user_from_session,
|
||||
_user_from_api_key,
|
||||
)
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.responses import respond, action_result
|
||||
from devplacepy.responses import respond, action_result, json_error, wants_json
|
||||
from devplacepy.schemas import ConversationOut, MessagesOut
|
||||
from devplacepy.services import presence
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -36,6 +38,8 @@ from devplacepy.services.messaging import (
|
||||
message_frame,
|
||||
message_hub,
|
||||
message_relay,
|
||||
broadcast_deleted,
|
||||
delete_message_record,
|
||||
persist_message,
|
||||
redeem_ticket,
|
||||
stamp_content_revision,
|
||||
@@ -66,7 +70,7 @@ def mark_conversation_read(user_uid: str, other_uid: str) -> None:
|
||||
return
|
||||
with db:
|
||||
db.query(
|
||||
"UPDATE messages SET read = 1 WHERE receiver_uid = :me AND sender_uid = :other AND read = 0",
|
||||
"UPDATE messages SET read = 1 WHERE receiver_uid = :me AND sender_uid = :other AND read = 0 AND deleted_at IS NULL",
|
||||
me=user_uid,
|
||||
other=other_uid,
|
||||
)
|
||||
@@ -85,7 +89,7 @@ def get_conversations(user_uid: str):
|
||||
" ORDER BY created_at DESC, id DESC"
|
||||
" ) AS rn"
|
||||
" FROM messages"
|
||||
" WHERE sender_uid = :me OR receiver_uid = :me"
|
||||
" WHERE (sender_uid = :me OR receiver_uid = :me) AND deleted_at IS NULL"
|
||||
") WHERE rn = 1 ORDER BY created_at DESC",
|
||||
me=user_uid,
|
||||
)
|
||||
@@ -127,7 +131,7 @@ def get_conversation_messages(user_uid: str, other_uid: str, before: str = ""):
|
||||
"SELECT * FROM messages"
|
||||
" WHERE ((sender_uid = :me AND receiver_uid = :other)"
|
||||
" OR (sender_uid = :other AND receiver_uid = :me))"
|
||||
" AND created_at < :before"
|
||||
" AND created_at < :before AND deleted_at IS NULL"
|
||||
" ORDER BY created_at DESC, id DESC LIMIT :lim",
|
||||
me=user_uid,
|
||||
other=other_uid,
|
||||
@@ -139,8 +143,9 @@ def get_conversation_messages(user_uid: str, other_uid: str, before: str = ""):
|
||||
msgs = list(
|
||||
db.query(
|
||||
"SELECT * FROM messages"
|
||||
" WHERE (sender_uid = :me AND receiver_uid = :other)"
|
||||
" OR (sender_uid = :other AND receiver_uid = :me)"
|
||||
" WHERE ((sender_uid = :me AND receiver_uid = :other)"
|
||||
" OR (sender_uid = :other AND receiver_uid = :me))"
|
||||
" AND deleted_at IS NULL"
|
||||
" ORDER BY created_at DESC, id DESC LIMIT :lim",
|
||||
me=user_uid,
|
||||
other=other_uid,
|
||||
@@ -293,6 +298,33 @@ async def send_message(request: Request, data: Annotated[MessageForm, Depends(js
|
||||
request, f"/messages?with_uid={receiver_uid}", data=frame
|
||||
)
|
||||
|
||||
@router.post("/{message_uid}/delete")
|
||||
async def delete_message(request: Request, message_uid: str):
|
||||
user = require_user(request)
|
||||
message = get_table("messages").find_one(uid=message_uid, deleted_at=None)
|
||||
if not is_admin(user) or (message and message["sender_uid"] != user["uid"]):
|
||||
audit.record(
|
||||
request,
|
||||
"message.delete",
|
||||
user=user,
|
||||
result="denied",
|
||||
target_type="message",
|
||||
target_uid=message_uid,
|
||||
summary=f"{user['username']} denied deleting message {message_uid}",
|
||||
)
|
||||
if wants_json(request):
|
||||
return json_error(403, "Not allowed")
|
||||
return RedirectResponse(url="/messages", status_code=302)
|
||||
if not message:
|
||||
raise not_found("Message not found")
|
||||
delete_message_record(request, user, message)
|
||||
await broadcast_deleted(message)
|
||||
return action_result(
|
||||
request,
|
||||
f"/messages?with_uid={message['receiver_uid']}",
|
||||
data={"deleted_uid": message_uid},
|
||||
)
|
||||
|
||||
async def broadcast_message(
|
||||
sender: dict, message: dict, client_id: Optional[str] = None,
|
||||
ai_processed: bool = False, ai_pending: bool = False,
|
||||
@@ -312,7 +344,7 @@ async def _await_ai_and_push(
|
||||
) -> bool:
|
||||
await asyncio.gather(*pending, return_exceptions=True)
|
||||
pending.clear()
|
||||
row = get_table("messages").find_one(uid=message["uid"])
|
||||
row = get_table("messages").find_one(uid=message["uid"], deleted_at=None)
|
||||
if not row:
|
||||
return False
|
||||
changed = row.get("content") != message.get("content")
|
||||
@@ -358,6 +390,7 @@ async def _sync_missed(user_uid: str, data: dict, websocket: WebSocket) -> None:
|
||||
" OR (sender_uid = :other AND receiver_uid = :me))"
|
||||
" AND (created_at > :since"
|
||||
" OR (updated_at IS NOT NULL AND updated_at > :since))"
|
||||
" AND deleted_at IS NULL"
|
||||
" ORDER BY created_at ASC, id ASC LIMIT :lim",
|
||||
me=user_uid,
|
||||
other=with_uid,
|
||||
@@ -372,6 +405,7 @@ async def _sync_missed(user_uid: str, data: dict, websocket: WebSocket) -> None:
|
||||
" WHERE (sender_uid = :me OR receiver_uid = :me)"
|
||||
" AND (created_at > :since"
|
||||
" OR (updated_at IS NOT NULL AND updated_at > :since))"
|
||||
" AND deleted_at IS NULL"
|
||||
" ORDER BY created_at ASC, id ASC LIMIT :lim",
|
||||
me=user_uid,
|
||||
since=since,
|
||||
|
||||
@@ -15,11 +15,12 @@ from devplacepy.database import (
|
||||
get_user_bookmarks,
|
||||
get_user_notes,
|
||||
paginate,
|
||||
is_listed,
|
||||
resolve_object_url,
|
||||
mark_notifications_read_by_target,
|
||||
)
|
||||
from devplacepy.utils import get_current_user, time_ago, not_found
|
||||
from devplacepy.content import canonical_redirect
|
||||
from devplacepy.content import canonical_redirect, can_view_item
|
||||
from devplacepy.seo import (
|
||||
base_seo_context,
|
||||
website_schema,
|
||||
@@ -105,6 +106,8 @@ async def news_detail_page(request: Request, news_slug: str):
|
||||
article = resolve_by_slug(news_table, news_slug)
|
||||
if not article:
|
||||
raise not_found("News article not found")
|
||||
if not can_view_item("news", article, user):
|
||||
raise not_found("News article not found")
|
||||
redirect = canonical_redirect("news", article, news_slug)
|
||||
if redirect:
|
||||
return redirect
|
||||
@@ -139,6 +142,7 @@ async def news_detail_page(request: Request, news_slug: str):
|
||||
title=article.get("title", "News Article"),
|
||||
description=article.get("description", "") or "",
|
||||
seo_target=("news", article["uid"]),
|
||||
robots="index,follow" if is_listed("news", article) else "noindex,nofollow",
|
||||
og_type="article",
|
||||
og_image=image_url or None,
|
||||
breadcrumbs=[
|
||||
|
||||
@@ -8,6 +8,7 @@ from devplacepy.database import get_table, db, paginate, resolve_object_url, _no
|
||||
from devplacepy.models import NoteForm
|
||||
from devplacepy.utils import generate_uid, require_user, time_ago, redirect_back
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.content import can_view_item, can_view_target
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import NotesOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -52,7 +53,8 @@ async def notes_page(request: Request, before: str = None):
|
||||
if table.has_column("deleted_at"):
|
||||
clauses.append(table.table.columns.deleted_at.is_(None))
|
||||
for obj in table.find(*clauses):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
if can_view_item(table_name, obj, user):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
|
||||
items = []
|
||||
for row in rows:
|
||||
@@ -104,6 +106,8 @@ async def set_note(
|
||||
user = require_user(request)
|
||||
if target_type not in NOTABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
|
||||
notes = get_table("notes")
|
||||
existing = notes.find_one(
|
||||
|
||||
@@ -14,6 +14,8 @@ from devplacepy.database import (
|
||||
mark_notifications_read_by_target,
|
||||
get_featured_topics,
|
||||
get_blocked_uids,
|
||||
listed_sql,
|
||||
is_listed,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
@@ -83,6 +85,7 @@ async def create_post(request: Request, data: Annotated[PostForm, Depends(json_o
|
||||
"topic": topic,
|
||||
"project_uid": project_uid or None,
|
||||
"image": image_filename,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
slug_text,
|
||||
XP_POST,
|
||||
@@ -147,7 +150,7 @@ def _next_post_url(post: dict, viewer: dict | None) -> str | None:
|
||||
blocked = get_blocked_uids(viewer["uid"]) if viewer else frozenset()
|
||||
rows = db.query(
|
||||
"SELECT slug, uid, user_uid FROM posts WHERE created_at < :created_at "
|
||||
"AND deleted_at IS NULL ORDER BY created_at DESC LIMIT :limit",
|
||||
f"AND deleted_at IS NULL AND {listed_sql('posts')} ORDER BY created_at DESC LIMIT :limit",
|
||||
created_at=post["created_at"],
|
||||
limit=10,
|
||||
)
|
||||
@@ -177,7 +180,8 @@ def post_page_context(
|
||||
request,
|
||||
title=post.get("title") or "Post",
|
||||
description=post.get("content", ""),
|
||||
robots=robots or "index,follow",
|
||||
robots=robots
|
||||
or ("index,follow" if is_listed("posts", post) else "noindex,nofollow"),
|
||||
seo_target=("post", post["uid"]),
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
@@ -207,7 +211,7 @@ def post_page_context(
|
||||
if "posts" in db.tables:
|
||||
rows = list(
|
||||
db.query(
|
||||
"SELECT p.*, u.username FROM posts p JOIN users u ON p.user_uid = u.uid WHERE p.topic = :t AND p.uid != :uid AND p.deleted_at IS NULL ORDER BY p.created_at DESC LIMIT 5",
|
||||
f"SELECT p.*, u.username FROM posts p JOIN users u ON p.user_uid = u.uid WHERE p.topic = :t AND p.uid != :uid AND p.deleted_at IS NULL AND {listed_sql('posts', 'p')} ORDER BY p.created_at DESC LIMIT 5",
|
||||
t=post.get("topic", ""),
|
||||
uid=post["uid"],
|
||||
)
|
||||
@@ -275,6 +279,7 @@ async def edit_post(
|
||||
"content": data.content.strip(),
|
||||
"title": data.title.strip() or None,
|
||||
"topic": data.topic,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
"/feed",
|
||||
target_type="post",
|
||||
@@ -297,3 +302,4 @@ async def delete_post(request: Request, post_slug: str):
|
||||
return delete_content_item(
|
||||
request, "posts", "post", user, post_slug, "/feed", inline_image_field="image"
|
||||
)
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@ from devplacepy.models import ProfileForm
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.database import (
|
||||
get_setting,
|
||||
listed_clause,
|
||||
is_listed,
|
||||
get_table,
|
||||
get_customization_prefs,
|
||||
get_notification_prefs,
|
||||
@@ -138,6 +140,12 @@ async def my_profile_page(request: Request, tab: str = "posts", page: int = 1):
|
||||
user = require_user(request)
|
||||
return await profile_page(request, user["username"], tab, page)
|
||||
|
||||
def _visible_clauses(table, viewer, profile_user):
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == profile_user["uid"] else None
|
||||
listed = listed_clause(table, own_uid=own_uid)
|
||||
return [listed] if listed is not None else []
|
||||
|
||||
|
||||
@router.get("/{username}", response_class=HTMLResponse)
|
||||
async def profile_page(
|
||||
request: Request, username: str, tab: str = "posts", page: int = 1
|
||||
@@ -147,6 +155,7 @@ async def profile_page(
|
||||
profile_user = users.find_one(username=username)
|
||||
if not profile_user:
|
||||
raise not_found("Profile not found")
|
||||
viewing_self = bool(current_user and current_user["uid"] == profile_user["uid"])
|
||||
if current_user:
|
||||
mark_notifications_read_by_target(
|
||||
current_user["uid"], f"/profile/{profile_user['username']}"
|
||||
@@ -188,6 +197,7 @@ async def profile_page(
|
||||
posts_table = get_table("posts")
|
||||
raw_posts = list(
|
||||
posts_table.find(
|
||||
*_visible_clauses(posts_table, current_user, profile_user),
|
||||
user_uid=profile_user["uid"],
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
@@ -229,13 +239,20 @@ async def profile_page(
|
||||
projects = list(
|
||||
get_table("projects").find(user_uid=profile_user["uid"], deleted_at=None)
|
||||
)
|
||||
projects = [p for p in projects if can_view_project(p, current_user)]
|
||||
projects = [
|
||||
p
|
||||
for p in projects
|
||||
if (viewing_self or is_listed("projects", p))
|
||||
and can_view_project(p, current_user)
|
||||
]
|
||||
gists = []
|
||||
if tab == "gists" or include_collections:
|
||||
gists_raw = list(
|
||||
get_table("gists").find(user_uid=profile_user["uid"], deleted_at=None)
|
||||
)
|
||||
for g in gists_raw:
|
||||
if not (viewing_self or is_listed("gists", g)):
|
||||
continue
|
||||
gists.append({"gist": g, "time_ago": time_ago(g["created_at"])})
|
||||
posts_count = get_user_post_count(profile_user["uid"])
|
||||
|
||||
@@ -243,6 +260,7 @@ async def profile_page(
|
||||
if tab == "activity":
|
||||
posts_table = get_table("posts")
|
||||
for p in posts_table.find(
|
||||
*_visible_clauses(posts_table, current_user, profile_user),
|
||||
user_uid=profile_user["uid"],
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
|
||||
@@ -11,7 +11,7 @@ Each project card links to `/projects/{project_uid}` showing full project detail
|
||||
|
||||
**Project overview page.** The detail page is a dedicated project showcase: one encompassing dark card (`.project-shell`, the site `--bg-card` surface with clipped corners) wraps the hero, the section tab bar and the two-column body, and every inner panel (tab bar, sidebar cards, devlog post cards, empty state, comments section) sits one elevation lighter on `--bg-secondary`. The hero's cover banner is the attachment referenced by `projects.cover_attachment_uid`, falling back to the first image attachment (brand-gradient band when neither exists); the title block, type/platform chips and author row render OVERLAID on the banner behind a bottom scrim (dark text-shadow for readability) beside the optional `projects.logo_attachment_uid` tile, with an owner-set **Visit Website** CTA (`projects.website_url`). Cover and logo ride the ONE existing upload pipeline: `dp-upload` widgets (`name="cover_attachment_uid"`/`"logo_attachment_uid"`, `max-files="1"`) in the create/edit modals upload to `/uploads/upload`, the route validates each uid via `database.get_user_attachment` (must exist, belong to the actor, be an image - `_hero_attachment_uid`) and links it to the project through `attachments.link_attachments`; an empty value on edit keeps the current image (no removal control). `website_url`/`repo_url` are normalized by `models.normalize_website_url` (scheme-less input gets `https://`, non-http(s) rejected) and render with `rel="noopener nofollow"`. Below the hero an anchor **section tab bar** (`.project-tabs`, underline style, Overview `.active`) links `#about` / `#devlog` / `#screenshots` (only when gallery images exist) / `#comments` / the Files page - server-rendered anchors, no JS tab state. The main column holds **About** (description + non-image attachments), the **Devlog** (every post whose `project_uid` points at the project via `_post_card.html` - the template loads `feed.css` for the card styles alongside `post.css`, the same rule as `news.html`) with `devlog_count` (`content.count_project_devlog`) and an owner **Post update** button (`.project-devlog-post-btn`) opening the shared composer preset to `topic=devlog` + this project (the form lives ONCE in `templates/_post_composer_form.html`, locals `_composer_topic`/`_composer_project`, included by `feed.html` and `project_detail.html` - never fork a second copy), a **Screenshots** gallery (image attachments minus the cover/logo, thumbnails, `data-lightbox`, capped at 12 rendered), and the comment thread; the sidebar holds Links (website/repository/files/fork source), the Stats card (5 `.project-stat` entries + a last-update line) and the Author card. Owners add gallery images via the More-menu **Add screenshots** modal: `_attachment_form.html` uploads, then `POST /projects/{slug}/screenshots` (`ProjectScreenshotsForm`, owner-only, audit `project.screenshots.add`) links the uids through the same `link_attachments` choke point; Devii action `project_add_screenshots`, docs id `projects-screenshots`. `comment_count`/`devlog_count` ride `ProjectDetailOut`; the new project fields ride `ProjectOut`; the page og:image prefers the cover attachment. **Locator discipline:** the page has several `Files` anchors (action row, tab bar, sidebar) and, for owners, a second hidden `textarea[name='content']`/Post button inside the composer modal - tests MUST scope (`.project-detail-actions a:has-text('Files')`, `.comment-form textarea[name='content']`).
|
||||
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest into a `dp-action-menu` component. The trigger is `<button data-action-menu-trigger>` labelled **More**; the secondary actions (Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) are real elements inside `<div data-action-menu-source hidden>`, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/components/AppActionMenu.js` (auto-defined via `components/index.js`) builds the menu through `app.contextMenu` and each item's `onSelect` activates the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), the `data-modal` Edit trigger, and the `data-confirm`/`data-confirm-danger` forms, which the component confirms itself before re-clicking (the element sits in a hidden container, so delegated handlers would never see it). Reuse `<dp-action-menu>` for any future action row that overflows; do not write a bespoke controller.
|
||||
|
||||
**Owner editing.** Mirrors post editing exactly: an owner-only **Edit** menu item (`data-modal="edit-project-modal"`) opens the `modal()` macro's `edit-project-modal`, a plain `POST` form to `/projects/edit/{slug}` (route `edit_project` in `routers/projects/index.py`, body `ProjectEditForm`, owner-gated through the shared `content.edit_content_item` which returns 403 JSON / redirect for non-owners and stamps `updated_at`). The modal is the create modal pre-filled from the `project` row (title, description, type/status radios pre-checked, dates via `format_date()` back to DD/MM/YYYY). `is_private`/`read_only` are NOT edited here - they stay on their dedicated toggles. The platforms tag widget reuses the create modal's `platforms-input`/`platforms`/`platforms-tags` ids; `ProfileEditor.initPlatformTags` now **seeds existing tags** from the hidden `#platforms` value on load, so both the empty create form and the pre-filled edit form work from the same code. Devii tool `edit_project`; documented in `docs_api.py` (`projects-edit`).
|
||||
|
||||
|
||||
@@ -35,6 +35,8 @@ from devplacepy.database import (
|
||||
count_forks,
|
||||
get_top_authors,
|
||||
get_user_attachment,
|
||||
listed_clause,
|
||||
is_listed,
|
||||
)
|
||||
from devplacepy.project_files import count_files
|
||||
from devplacepy.services.jobs import queue
|
||||
@@ -116,13 +118,17 @@ def get_projects_list(
|
||||
clauses.append(visible)
|
||||
|
||||
order = ["-stars", "-created_at"] if tab == "popular" else ["-created_at"]
|
||||
total = projects.count(*clauses, **filters)
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == user_uid else None
|
||||
listed = listed_clause(projects, own_uid=own_uid)
|
||||
count_clauses = clauses + ([listed] if listed is not None else [])
|
||||
total = projects.count(*count_clauses, **filters)
|
||||
page, next_cursor = paginate(
|
||||
projects,
|
||||
*clauses,
|
||||
before=before,
|
||||
order=order,
|
||||
viewer_uid=viewer["uid"] if viewer else None,
|
||||
include_unlisted_of=own_uid,
|
||||
**filters,
|
||||
)
|
||||
|
||||
@@ -234,7 +240,11 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
)
|
||||
|
||||
base = site_url(request)
|
||||
robots = "noindex,nofollow" if project.get("is_private") else "index,follow"
|
||||
robots = (
|
||||
"noindex,nofollow"
|
||||
if project.get("is_private") or not is_listed("projects", project)
|
||||
else "index,follow"
|
||||
)
|
||||
cover_url = next(
|
||||
(
|
||||
a["url"]
|
||||
@@ -465,6 +475,7 @@ async def create_project(request: Request, data: Annotated[ProjectForm, Depends(
|
||||
"cover_attachment_uid": cover_uid,
|
||||
"logo_attachment_uid": logo_uid,
|
||||
"is_private": 1 if data.is_private else 0,
|
||||
"visibility": data.visibility,
|
||||
"read_only": 0,
|
||||
},
|
||||
title,
|
||||
@@ -495,6 +506,7 @@ async def edit_project(
|
||||
"status": data.status,
|
||||
"website_url": data.website_url or None,
|
||||
"repo_url": data.repo_url or None,
|
||||
"visibility": data.visibility,
|
||||
}
|
||||
hero_uids = []
|
||||
for field in ("cover_attachment_uid", "logo_attachment_uid"):
|
||||
@@ -616,3 +628,4 @@ async def set_project_readonly(
|
||||
return _set_project_flag(request, project_slug, "read_only", data.value)
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ from fastapi.responses import RedirectResponse, JSONResponse
|
||||
from devplacepy.database import get_table, db, _now_iso
|
||||
from devplacepy.utils import generate_uid, require_user, track_action, redirect_back
|
||||
from devplacepy.models import ReactionForm
|
||||
from devplacepy.content import can_view_target
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.dependencies import json_or_form
|
||||
|
||||
@@ -25,6 +26,8 @@ async def react(
|
||||
user = require_user(request)
|
||||
if target_type not in REACTABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
|
||||
emoji = data.emoji
|
||||
reactions = get_table("reactions")
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from devplacepy.constants import TOPICS, TOPIC_LABELS
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.database import get_table, listed_clause
|
||||
from devplacepy.routers.feed import get_feed_posts, enrich_post_cards
|
||||
from devplacepy.utils import get_current_user, not_found
|
||||
from devplacepy.seo import list_page_seo, next_page_url
|
||||
@@ -17,11 +17,13 @@ router = APIRouter()
|
||||
async def topics_hub(request: Request):
|
||||
user = get_current_user(request)
|
||||
posts_table = get_table("posts")
|
||||
listed_match = listed_clause(posts_table)
|
||||
listed = [listed_match] if listed_match is not None else []
|
||||
topics = [
|
||||
{
|
||||
"key": topic,
|
||||
"label": TOPIC_LABELS.get(topic, topic.title()),
|
||||
"post_count": posts_table.count(topic=topic, deleted_at=None),
|
||||
"post_count": posts_table.count(*listed, topic=topic, deleted_at=None),
|
||||
}
|
||||
for topic in TOPICS
|
||||
]
|
||||
|
||||
@@ -5,7 +5,7 @@ from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from fastapi.responses import RedirectResponse, JSONResponse
|
||||
from devplacepy.utils import require_user, redirect_back
|
||||
from devplacepy.content import apply_vote
|
||||
from devplacepy.content import apply_vote, can_view_target
|
||||
from devplacepy.models import VoteForm
|
||||
from devplacepy.dependencies import json_or_form
|
||||
|
||||
@@ -25,6 +25,8 @@ async def vote(
|
||||
user = require_user(request)
|
||||
if target_type not in VOTABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
result = apply_vote(request, user, target_type, target_uid, data.value)
|
||||
|
||||
if request.headers.get("x-requested-with") == "fetch":
|
||||
|
||||
@@ -4,11 +4,21 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
from pydantic import ConfigDict, Field
|
||||
from pydantic import ConfigDict, Field, field_validator
|
||||
|
||||
from devplacepy.database.visibility import normalize_visibility
|
||||
from devplacepy.schemas.base import _Out
|
||||
|
||||
|
||||
class VisibleOut(_Out):
|
||||
visibility: str = "published"
|
||||
|
||||
@field_validator("visibility", mode="before")
|
||||
@classmethod
|
||||
def default_visibility(cls, value):
|
||||
return normalize_visibility(value)
|
||||
|
||||
|
||||
class UserOut(_Out):
|
||||
uid: str = ""
|
||||
username: str = ""
|
||||
@@ -80,7 +90,7 @@ class ProjectLinkOut(_Out):
|
||||
url: Optional[str] = None
|
||||
|
||||
|
||||
class PostOut(_Out):
|
||||
class PostOut(VisibleOut):
|
||||
uid: str = ""
|
||||
slug: Optional[str] = None
|
||||
user_uid: Optional[str] = None
|
||||
@@ -94,7 +104,7 @@ class PostOut(_Out):
|
||||
updated_at: Optional[str] = None
|
||||
|
||||
|
||||
class GistOut(_Out):
|
||||
class GistOut(VisibleOut):
|
||||
uid: str = ""
|
||||
slug: Optional[str] = None
|
||||
user_uid: Optional[str] = None
|
||||
@@ -107,7 +117,7 @@ class GistOut(_Out):
|
||||
updated_at: Optional[str] = None
|
||||
|
||||
|
||||
class ProjectOut(_Out):
|
||||
class ProjectOut(VisibleOut):
|
||||
uid: str = ""
|
||||
slug: Optional[str] = None
|
||||
user_uid: Optional[str] = None
|
||||
|
||||
@@ -51,6 +51,8 @@ class AdminGatewayOut(_Out):
|
||||
quota_defaults: dict = {}
|
||||
stats_ranges: list = []
|
||||
admin_section: Optional[str] = None
|
||||
tested: str = ""
|
||||
test_image: str = ""
|
||||
|
||||
|
||||
class AdminGatewayProviderFormOut(_Out):
|
||||
|
||||
+8
-2
@@ -421,9 +421,13 @@ def make_sitemap(base_url):
|
||||
|
||||
|
||||
def _collect(table, limit, label, **query):
|
||||
from devplacepy.database import listed_clause
|
||||
|
||||
if table.has_column("deleted_at") and "deleted_at" not in query:
|
||||
query["deleted_at"] = None
|
||||
rows = list(table.find(_limit=limit, **query))
|
||||
listed = listed_clause(table)
|
||||
clauses = [listed] if listed is not None else []
|
||||
rows = list(table.find(*clauses, _limit=limit, **query))
|
||||
if len(rows) >= limit:
|
||||
logger.warning("sitemap: %s truncated at %d entries", label, limit)
|
||||
return rows
|
||||
@@ -574,11 +578,13 @@ def _build_sitemap(base_url):
|
||||
)
|
||||
)
|
||||
|
||||
from devplacepy.database import listed_sql
|
||||
|
||||
if "users" in db.tables:
|
||||
post_counts = {}
|
||||
if "posts" in db.tables:
|
||||
for row in db.query(
|
||||
"SELECT user_uid, COUNT(*) AS c FROM posts WHERE deleted_at IS NULL GROUP BY user_uid"
|
||||
f"SELECT user_uid, COUNT(*) AS c FROM posts WHERE deleted_at IS NULL AND {listed_sql('posts')} GROUP BY user_uid"
|
||||
):
|
||||
post_counts[row["user_uid"]] = row["c"]
|
||||
users = _collect(
|
||||
|
||||
@@ -27,7 +27,7 @@ Admin-only, enterprise-grade backups built on the **same async-job pattern as zi
|
||||
|
||||
**`prune_orphans` (`devplace backups prune`) is offload-aware for the same reason:** a `done` row with a confirmed `remote_uploaded_at` is skipped even when its `local_path` is empty/missing - that is the normal steady state after `enforce_local_retention` purges the local copy, not an orphan. Only a `done` row with no confirmed remote copy AND a missing local file counts as truly orphaned and gets hard-deleted. Without this check, running the CLI prune after offload has done its job would delete the DB record for every successfully offloaded backup, discarding the only pointer to its `remote_path`.
|
||||
|
||||
**Operational prerequisite (production, not automatic):** the `rclone` binary is installed in the shipped Docker image, but a working WebDAV remote still needs to exist at `config.RCLONE_CONFIG_FILE` (default `$HOME/.config/rclone/rclone.conf` inside the app container, overridable via `DEVPLACE_RCLONE_CONFIG`) with a remote named to match `config.BACKUP_OFFLOAD_REMOTE`'s prefix (default `storagebox`) pointing at the Hetzner Storage Box's WebDAV endpoint and credentials - `rclone config` (interactive) or a hand-written `rclone.conf` generates it. Until that file exists, every `upload_pending` attempt fails fast (`rclone` errors "didn't find section") and is logged and retried next cycle; local retention and rotation both stay disabled the whole time (see above), so backups simply accumulate locally with no data loss, they just never leave the box. **In Docker, `HOME=/app` (the bind-mounted repo root, `docker-compose.yml`), so the default config path resolves to `<repo>/.config/rclone/rclone.conf` on the host - `.gitignore` excludes `/.config/` precisely because this file holds live remote-storage credentials; never force-add it.**
|
||||
**Operational prerequisite (production, not automatic):** the `rclone` binary is installed in the shipped Docker image, but a working WebDAV remote still needs to exist at `config.RCLONE_CONFIG_FILE` (default `$HOME/.config/rclone/rclone.conf` inside the app container, overridable via `DEVPLACE_RCLONE_CONFIG`) with a remote named to match `config.BACKUP_OFFLOAD_REMOTE`'s prefix (default `storagebox`) pointing at the Hetzner Storage Box's WebDAV endpoint and credentials - `rclone config` (interactive) or a hand-written `rclone.conf` generates it. Until that file exists, every `upload_pending` attempt fails fast (`rclone` errors "didn't find section") and is logged and retried next cycle; local retention and rotation both stay disabled the whole time (see above), so backups simply accumulate locally with no data loss, they just never leave the box. **In Docker, `HOME=/app` (the bind-mounted repo root, `docker-compose.yml`), so the default config path resolves to `<repo>/.config/rclone/rclone.conf` on the host - `.gitignore` excludes `/.config/` precisely because this file holds live remote-storage credentials; never force-add it.** When the `rclone` binary itself is absent (`shutil.which(config.RCLONE_BIN)` finds nothing, e.g. bare-metal hosts without rclone installed), `run_cycle` skips the whole cycle with one `Offload skipped: rclone binary not found ...` log line naming the `backup_offload_enabled` kill-switch, and retries next cycle - installing rclone needs no restart.
|
||||
|
||||
## Disk usage warning
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from devplacepy import config
|
||||
@@ -118,6 +119,12 @@ async def enforce_remote_retention(keep_remote: int, log=lambda message: None) -
|
||||
|
||||
|
||||
async def run_cycle(*, keep_local: int, keep_remote: int, log=lambda message: None) -> dict:
|
||||
if shutil.which(config.RCLONE_BIN) is None:
|
||||
log(
|
||||
f"Offload skipped: rclone binary not found ({config.RCLONE_BIN}); "
|
||||
"install rclone or disable backup_offload_enabled"
|
||||
)
|
||||
return {"uploaded": 0, "purged_local": 0, "removed_remote": 0}
|
||||
uploaded = await upload_pending(log=log)
|
||||
purged_local = await enforce_local_retention(keep_local, log=log)
|
||||
removed_remote = await enforce_remote_retention(keep_remote, log=log)
|
||||
|
||||
@@ -122,32 +122,36 @@ class DbApiJobService(JobService):
|
||||
}
|
||||
|
||||
async def _stream(self, uid: str, sql: str, max_rows: int):
|
||||
connection = sqlite3.connect(
|
||||
f"file:{_database_path()}?mode=ro", uri=True, timeout=30
|
||||
)
|
||||
db_path = _database_path()
|
||||
loop = asyncio.get_running_loop()
|
||||
rows: list[dict] = []
|
||||
truncated = False
|
||||
try:
|
||||
connection.row_factory = sqlite3.Row
|
||||
connection.execute("PRAGMA query_only=ON")
|
||||
cursor = connection.execute(sql)
|
||||
while True:
|
||||
batch = cursor.fetchmany(BATCH_SIZE)
|
||||
if not batch:
|
||||
break
|
||||
for raw in batch:
|
||||
if len(rows) >= max_rows:
|
||||
truncated = True
|
||||
|
||||
def _run() -> tuple[list[dict], bool]:
|
||||
out: list[dict] = []
|
||||
stop = False
|
||||
conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=30)
|
||||
try:
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA query_only=ON")
|
||||
cursor = conn.execute(sql)
|
||||
while True:
|
||||
batch = cursor.fetchmany(BATCH_SIZE)
|
||||
if not batch:
|
||||
break
|
||||
rows.append(dict(raw))
|
||||
hub.publish(
|
||||
uid, {"type": "progress", "row_count": len(rows)}
|
||||
)
|
||||
if truncated:
|
||||
break
|
||||
await asyncio.sleep(0)
|
||||
finally:
|
||||
connection.close()
|
||||
for raw in batch:
|
||||
if len(out) >= max_rows:
|
||||
stop = True
|
||||
break
|
||||
out.append(dict(raw))
|
||||
loop.call_soon_threadsafe(hub.publish, uid, {"type": "progress", "row_count": len(out)})
|
||||
if stop:
|
||||
break
|
||||
finally:
|
||||
conn.close()
|
||||
return out, stop
|
||||
|
||||
rows, truncated = await asyncio.to_thread(_run)
|
||||
return rows, truncated
|
||||
|
||||
def cleanup(self, job: dict) -> None:
|
||||
|
||||
@@ -47,3 +47,16 @@ def confirm() -> Param:
|
||||
|
||||
ATTACHMENTS = "Comma separated attachment uids returned by upload_file."
|
||||
TARGET_TYPE = "Target type, e.g. post, comment, project, gist."
|
||||
|
||||
VISIBILITY = (
|
||||
"Visibility: published (default, listed and open to everyone), hidden (not listed or "
|
||||
"searchable, anyone with the link can open it), offline (taken offline, only you can see "
|
||||
"it, publish again at any time) or draft (only you, not yet published). "
|
||||
)
|
||||
VISIBILITY_CREATE = VISIBILITY + "Optional: omit it and the item is published."
|
||||
VISIBILITY_EDIT = VISIBILITY + "Optional: omit it to keep the current value."
|
||||
VISIBILITY_VALUE = (
|
||||
"New visibility: published, hidden, offline or draft. Only the owner can change it. "
|
||||
"Offline and draft hide the item from everyone but the owner and administrators."
|
||||
)
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from ..spec import Action
|
||||
from ._shared import body, confirm, path, query
|
||||
from ._shared import VISIBILITY_VALUE, body, confirm, path, query
|
||||
|
||||
|
||||
ADMIN_ACTIONS: tuple[Action, ...] = (
|
||||
@@ -218,6 +218,17 @@ ADMIN_ACTIONS: tuple[Action, ...] = (
|
||||
params=(path("uid", "News uid."),),
|
||||
requires_admin=True,
|
||||
),
|
||||
Action(
|
||||
name="admin_set_news_visibility",
|
||||
method="POST",
|
||||
path="/admin/news/{uid}/visibility",
|
||||
summary="Set the visibility of a news article (published, hidden, offline or draft)",
|
||||
params=(
|
||||
path("uid", "News uid."),
|
||||
body("value", VISIBILITY_VALUE, required=True),
|
||||
),
|
||||
requires_admin=True,
|
||||
),
|
||||
Action(
|
||||
name="admin_landing_news",
|
||||
method="POST",
|
||||
|
||||
@@ -127,6 +127,7 @@ GATEWAY_ACTIONS: tuple[Action, ...] = (
|
||||
Param(name="off_peak_end_minute", location="body", description="Off-peak window end, UTC minutes since midnight (0-1439). A value less than the start wraps past midnight.", required=False, type="integer"),
|
||||
Param(name="off_peak_discount_pct", location="body", description="Percentage discount (0-100) applied to the active tier's rates during the off-peak window.", required=False, type="number"),
|
||||
body("fallback_model", "Another already-configured source_model of the same kind, tried once automatically when this route fails after its own retries are exhausted. Blank disables fallback."),
|
||||
body("tools_mode", "'native' (default) forwards tools untouched; 'emulated' declares the model does not support tool calling, so the gateway rewrites tools into the prompt and converts the reply back into OpenAI tool_calls."),
|
||||
Param(name="is_active", location="body", description="Whether the route is active ('1' or '0').", required=False, type="boolean"),
|
||||
),
|
||||
),
|
||||
|
||||
@@ -3,7 +3,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from ..spec import Action
|
||||
from ._shared import ATTACHMENTS, body, confirm, path, query
|
||||
from ._shared import (
|
||||
ATTACHMENTS,
|
||||
VISIBILITY_CREATE,
|
||||
VISIBILITY_EDIT,
|
||||
body,
|
||||
confirm,
|
||||
path,
|
||||
query,
|
||||
)
|
||||
|
||||
|
||||
GIST_ACTIONS: tuple[Action, ...] = (
|
||||
@@ -44,6 +52,7 @@ GIST_ACTIONS: tuple[Action, ...] = (
|
||||
body("description", "Gist description."),
|
||||
body("language", "Programming language."),
|
||||
body("attachment_uids", ATTACHMENTS),
|
||||
body("visibility", VISIBILITY_CREATE),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
@@ -60,6 +69,7 @@ GIST_ACTIONS: tuple[Action, ...] = (
|
||||
body("source_code", "Gist source code.", required=True),
|
||||
body("description", "Gist description."),
|
||||
body("language", "Programming language."),
|
||||
body("visibility", VISIBILITY_EDIT),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from ..spec import Action
|
||||
from ._shared import ATTACHMENTS, body, query
|
||||
from ._shared import ATTACHMENTS, body, confirm, path, query
|
||||
|
||||
|
||||
MESSAGE_ACTIONS: tuple[Action, ...] = (
|
||||
@@ -34,5 +34,12 @@ MESSAGE_ACTIONS: tuple[Action, ...] = (
|
||||
body("receiver_uid", "Recipient user uid.", required=True),
|
||||
body("attachment_uids", ATTACHMENTS),
|
||||
),
|
||||
), Action(
|
||||
name="delete_message",
|
||||
method="POST",
|
||||
path="/messages/{message_uid}/delete",
|
||||
summary="Delete a direct message you sent (administrators only, own messages only). Soft delete, confirmation required",
|
||||
params=(path("message_uid", "Uid of the message."), confirm()),
|
||||
requires_admin=True,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -3,7 +3,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from ..spec import Action
|
||||
from ._shared import ATTACHMENTS, body, confirm, path, query
|
||||
from ._shared import (
|
||||
ATTACHMENTS,
|
||||
VISIBILITY_CREATE,
|
||||
VISIBILITY_EDIT,
|
||||
body,
|
||||
confirm,
|
||||
path,
|
||||
query,
|
||||
)
|
||||
|
||||
|
||||
POSTS_ACTIONS: tuple[Action, ...] = (
|
||||
@@ -63,6 +71,7 @@ POSTS_ACTIONS: tuple[Action, ...] = (
|
||||
"war_faction_b",
|
||||
"Optional Opinion War faction B name (max 30 chars). Must differ from faction A.",
|
||||
),
|
||||
body("visibility", VISIBILITY_CREATE),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
@@ -99,6 +108,7 @@ POSTS_ACTIONS: tuple[Action, ...] = (
|
||||
"poll_options",
|
||||
"Poll options as a JSON array of strings, or one option per line, or comma separated. At least two are required for the poll to be created.",
|
||||
),
|
||||
body("visibility", VISIBILITY_EDIT),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
|
||||
@@ -3,7 +3,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from ..spec import Action
|
||||
from ._shared import ATTACHMENTS, body, confirm, path, query
|
||||
from ._shared import (
|
||||
ATTACHMENTS,
|
||||
VISIBILITY_CREATE,
|
||||
VISIBILITY_EDIT,
|
||||
body,
|
||||
confirm,
|
||||
path,
|
||||
query,
|
||||
)
|
||||
|
||||
|
||||
PROJECTS_ACTIONS: tuple[Action, ...] = (
|
||||
@@ -52,6 +60,7 @@ PROJECTS_ACTIONS: tuple[Action, ...] = (
|
||||
body("cover_attachment_uid", "Attachment uid of an uploaded cover image (upload_file/attach_url first)."),
|
||||
body("logo_attachment_uid", "Attachment uid of an uploaded project logo (upload_file/attach_url first)."),
|
||||
body("attachment_uids", ATTACHMENTS),
|
||||
body("visibility", VISIBILITY_CREATE),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
@@ -75,6 +84,7 @@ PROJECTS_ACTIONS: tuple[Action, ...] = (
|
||||
body("repo_url", "Updated source repository URL (http/https)."),
|
||||
body("cover_attachment_uid", "Attachment uid of a new cover image; empty keeps the current one."),
|
||||
body("logo_attachment_uid", "Attachment uid of a new project logo; empty keeps the current one."),
|
||||
body("visibility", VISIBILITY_EDIT),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
|
||||
@@ -73,6 +73,46 @@ UPLOAD_ACTIONS: tuple[Action, ...] = (
|
||||
),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="search_gifs",
|
||||
method="GET",
|
||||
path="/giphy/search",
|
||||
summary="Search GIPHY for GIFs by keyword",
|
||||
description=(
|
||||
"Searches GIPHY and returns {query, results}. Each result carries id, title, preview (a small "
|
||||
"still-animated rendition) and original (the full GIF). Pass a result's id to attach_gif to "
|
||||
"attach it. Answers 503 when GIPHY is not configured by an administrator and 429 when you search "
|
||||
"too fast."
|
||||
),
|
||||
params=(
|
||||
query("q", "Search keywords, 1-200 characters.", required=True),
|
||||
query("limit", "Number of results, 1-50 (default 24)."),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="trending_gifs",
|
||||
method="GET",
|
||||
path="/giphy/trending",
|
||||
summary="List the GIFs currently trending on GIPHY",
|
||||
description="Returns {results} in the same shape as search_gifs. Pass a result's id to attach_gif.",
|
||||
params=(query("limit", "Number of results, 1-50 (default 24)."),),
|
||||
),
|
||||
Action(
|
||||
name="attach_gif",
|
||||
method="POST",
|
||||
path="/giphy/attach",
|
||||
summary="Store a GIPHY GIF as an attachment, returning its uid",
|
||||
description=(
|
||||
"Downloads the GIF with the given GIPHY id on the server and stores it as an attachment exactly "
|
||||
"like an upload, returning {uid, url, mime_type, ...}. Pass the returned uid in attachment_uids "
|
||||
"when you create_post, create_comment or send_message to attach it. Find ids with search_gifs or "
|
||||
"trending_gifs."
|
||||
),
|
||||
params=(
|
||||
body("gif_id", "GIPHY id of the GIF, from search_gifs or trending_gifs.", required=True),
|
||||
body("title", "Optional title used as the attachment's filename."),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="generate_image",
|
||||
method="POST",
|
||||
|
||||
@@ -63,6 +63,7 @@ CONFIRM_REQUIRED = {
|
||||
"regenerate_avatar",
|
||||
"delete_post",
|
||||
"delete_comment",
|
||||
"delete_message",
|
||||
"delete_gist",
|
||||
"delete_attachment",
|
||||
"delete_issue_attachment",
|
||||
@@ -248,6 +249,12 @@ def confirmation_error(name: str, arguments: dict[str, Any]) -> ToolInputError |
|
||||
f"Deleting comment '{uid}' is permanent and cannot be undone. "
|
||||
"Ask the user to confirm explicitly, then call again with confirm=true."
|
||||
)
|
||||
if name == "delete_message":
|
||||
uid = str(arguments.get("message_uid", "")).strip() or "(unspecified)"
|
||||
return ToolInputError(
|
||||
f"Deleting message '{uid}' removes it from the conversation for both participants. "
|
||||
"Show the user the exact message, get explicit confirmation, then call again with confirm=true."
|
||||
)
|
||||
if name == "community_delete":
|
||||
slug = str(arguments.get("slug", "")).strip() or "(unspecified)"
|
||||
return ToolInputError(
|
||||
|
||||
@@ -5,6 +5,7 @@ from typing import Optional
|
||||
from devplacepy.attachments import get_attachments_batch
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
listed_clause,
|
||||
get_users_by_uids,
|
||||
get_user_votes,
|
||||
get_vote_counts,
|
||||
@@ -49,9 +50,15 @@ def build_rant_list(posts: list, viewer: Optional[dict]) -> list:
|
||||
|
||||
def list_rants(sort: str, limit: int, skip: int, viewer: Optional[dict]) -> list:
|
||||
order = SORT_ORDER.get(sort, SORT_ORDER["recent"])
|
||||
posts_table = get_table("posts")
|
||||
listed = listed_clause(posts_table)
|
||||
posts = list(
|
||||
get_table("posts").find(
|
||||
deleted_at=None, order_by=order, _limit=limit, _offset=skip
|
||||
posts_table.find(
|
||||
*([listed] if listed is not None else []),
|
||||
deleted_at=None,
|
||||
order_by=order,
|
||||
_limit=limit,
|
||||
_offset=skip,
|
||||
)
|
||||
)
|
||||
return build_rant_list(posts, viewer)
|
||||
@@ -64,9 +71,14 @@ def search_rants(term: str, viewer: Optional[dict], limit: int = 50) -> list:
|
||||
)
|
||||
if clause is None:
|
||||
return []
|
||||
listed = listed_clause(posts_table)
|
||||
posts = list(
|
||||
posts_table.find(
|
||||
clause, deleted_at=None, order_by=["-created_at"], _limit=limit
|
||||
clause,
|
||||
*([listed] if listed is not None else []),
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
_limit=limit,
|
||||
)
|
||||
)
|
||||
return build_rant_list(posts, viewer)
|
||||
|
||||
@@ -29,11 +29,16 @@ def now_unix() -> int:
|
||||
return int(datetime.now(timezone.utc).timestamp())
|
||||
|
||||
|
||||
def post_by_id(rant_id: object) -> Optional[dict]:
|
||||
def post_by_id(rant_id: object, viewer: Optional[dict] = None) -> Optional[dict]:
|
||||
from devplacepy.content import can_view_item
|
||||
|
||||
identifier = as_int(rant_id)
|
||||
if not identifier:
|
||||
return None
|
||||
return get_table("posts").find_one(id=identifier, deleted_at=None)
|
||||
post = get_table("posts").find_one(id=identifier, deleted_at=None)
|
||||
if post and not can_view_item("posts", post, viewer):
|
||||
return None
|
||||
return post
|
||||
|
||||
|
||||
def comment_by_id(comment_id: object) -> Optional[dict]:
|
||||
|
||||
@@ -4,6 +4,7 @@ from typing import Optional
|
||||
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
listed_clause,
|
||||
get_users_by_uids,
|
||||
get_user_votes,
|
||||
get_vote_counts,
|
||||
@@ -28,8 +29,12 @@ def skills_from_bio(bio: Optional[str]) -> str:
|
||||
|
||||
|
||||
def _user_rants(user: dict, viewer: Optional[dict]) -> list:
|
||||
posts_table = get_table("posts")
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == user["uid"] else None
|
||||
listed = listed_clause(posts_table, own_uid=own_uid)
|
||||
posts = list(
|
||||
get_table("posts").find(
|
||||
posts_table.find(
|
||||
*([listed] if listed is not None else []),
|
||||
user_uid=user["uid"],
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# Giphy (GIF picker)
|
||||
|
||||
Members search GIPHY from a "GIF" button beside the attachment button in the post composer, every comment form and the direct-message input. A picked GIF is stored as a normal attachment. There is no local GIF store, no table and no client-side key.
|
||||
|
||||
## Layout
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `config.py` | `giphy_api_key` (secret), `giphy_base_url`, `giphy_rating` as `CONFIG_FIELDS` (the admin-editable settings), `giphy_config()` reads them through `get_setting` |
|
||||
| `client.py` | `GiphyClient` (`search`, `trending`, `get_gif`), `normalize_gif`, `GiphyError(message, status)`, the module-level 5 minute `TtlCache` |
|
||||
| `limits.py` | `TtlCache` (256 entries, 300 s) and `UserRateLimiter` (30 calls per 60 s per user, 2048 users tracked) |
|
||||
| `runtime.py` | `get_client()` / `set_client()`, the test seam (the gitea `runtime` pattern) |
|
||||
| `fake.py` | `FakeGiphyClient` for tests |
|
||||
| `service.py` | `GiphyService` (`BaseService`, off by default): a daily health probe against `trending` and the home of the admin config fields |
|
||||
| `routers/giphy.py` | `/giphy/status`, `/search`, `/trending`, `/attach` |
|
||||
|
||||
## Rules
|
||||
|
||||
- **A GIF is an attachment, never markup.** `POST /giphy/attach` downloads the GIF through `attachments.store_attachment_from_url` (SSRF guarded, size capped, the same pipeline as an upload) and returns the upload record. The picker fires `dp-gif:attached` and the form's `dp-upload` (attachment mode) adds the uid to `attachment_uids`. Nothing is inserted into the textarea; the GIF renders in the attachment gallery like any file. Do not add a markdown insertion path: it would render the GIF twice.
|
||||
- **The key never leaves the server.** The browser talks only to `/giphy/*`. The picker hides itself when `/giphy/status` says `configured: false`. The key is edited in `/admin/services` (the Giphy service), stored in `site_settings`, never hardcoded.
|
||||
- **Hard caps.** `limit` is clamped to `MAX_LIMIT` (50), `q` to 200 characters, the cache to 256 entries / 300 s, the per-user limiter to 30 calls per 60 s (`429` with `Retry-After`). The cache is keyed without the API key and never stores errors. These are static ceilings on purpose: a free GIPHY key allows roughly 100 calls an hour, so repeated and rapid queries must not reach the upstream.
|
||||
- **Errors use `responses.json_error`** (`{"error": {"status", "message"}}`) so `Http` shows the message. `503` not configured, `400` missing `q`, `404` unknown id, `429` throttled, `502` upstream failure.
|
||||
- **Outbound HTTP is the stealth client only** (`stealth.stealth_async_client`); attachments go through `guarded_async_client`.
|
||||
- **Audit:** `attachment.giphy` (`events.md`), recorded with the gif id and the source host. The GIF is a normal attachment afterwards, so it is reportable and soft-deletable like every other.
|
||||
- **Privacy:** GIPHY receives the search words and a picked id from the server, never the member's IP or account. Disclosed in `/docs/privacy.html`.
|
||||
- **Attribution:** the panel shows "Powered by GIPHY"; do not remove it (GIPHY terms).
|
||||
- **Devii:** `search_gifs`, `trending_gifs`, `attach_gif` (`catalog/uploads.py`), docs in `docs_api/groups/uploads.py`. Not load tested: the routes call a third party with a rate-limited key, so `locustfile.py` deliberately has no task for them.
|
||||
|
||||
## Testing
|
||||
|
||||
`runtime.set_client(FakeGiphyClient())` swaps the client in-process (api and unit tiers). The e2e tier runs the server in a subprocess, so it points `giphy_base_url` at a local fake GIPHY HTTP server through the settings table instead, and stubs only `/giphy/attach` in the browser (the SSRF guard rightly refuses a loopback GIF URL).
|
||||
@@ -0,0 +1,5 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.services.giphy.service import GiphyService
|
||||
|
||||
__all__ = ["GiphyService"]
|
||||
@@ -0,0 +1,134 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from devplacepy import stealth
|
||||
from devplacepy.services.giphy.config import (
|
||||
HTTP_TIMEOUT_SECONDS,
|
||||
RESULT_LIMIT_DEFAULT,
|
||||
GiphyConfig,
|
||||
giphy_config,
|
||||
)
|
||||
from devplacepy.services.giphy.limits import TtlCache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
MAX_LIMIT = 50
|
||||
|
||||
_cache = TtlCache()
|
||||
|
||||
|
||||
def clear_cache() -> None:
|
||||
_cache.clear()
|
||||
|
||||
|
||||
class GiphyError(Exception):
|
||||
def __init__(self, message: str, status: int = 0):
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
self.message = message
|
||||
|
||||
|
||||
def _pick_image(images: dict, keys: tuple[str, ...]) -> dict:
|
||||
for key in keys:
|
||||
image = images.get(key)
|
||||
if not isinstance(image, dict):
|
||||
continue
|
||||
url = image.get("url") or image.get("webp") or ""
|
||||
if not url:
|
||||
continue
|
||||
return {
|
||||
"url": url,
|
||||
"width": int(image.get("width") or 0),
|
||||
"height": int(image.get("height") or 0),
|
||||
"size": int(image.get("size") or 0),
|
||||
}
|
||||
return {}
|
||||
|
||||
|
||||
def normalize_gif(item: dict) -> dict:
|
||||
images = item.get("images") or {}
|
||||
return {
|
||||
"id": str(item.get("id") or ""),
|
||||
"title": str(item.get("title") or "").strip(),
|
||||
"url": str(item.get("url") or ""),
|
||||
"preview": _pick_image(images, ("fixed_width", "downsized", "original")),
|
||||
"original": _pick_image(images, ("original", "downsized")),
|
||||
}
|
||||
|
||||
|
||||
class GiphyClient:
|
||||
def __init__(self, config: GiphyConfig | None = None):
|
||||
self.config = config or giphy_config()
|
||||
|
||||
def _require_config(self) -> None:
|
||||
if not self.config.is_configured:
|
||||
raise GiphyError("Giphy integration is not configured", status=503)
|
||||
|
||||
def _params(self, **extra) -> dict:
|
||||
return {"api_key": self.config.api_key, "rating": self.config.rating, **extra}
|
||||
|
||||
def _cache_key(self, path: str, params: dict) -> tuple:
|
||||
shared = {k: v for k, v in params.items() if k != "api_key"}
|
||||
return (self.config.base_url, path, tuple(sorted(shared.items())))
|
||||
|
||||
async def _get(self, path: str, params: dict) -> dict:
|
||||
self._require_config()
|
||||
key = self._cache_key(path, params)
|
||||
cached = _cache.get(key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
data = await self._fetch(path, params)
|
||||
_cache.put(key, data)
|
||||
return data
|
||||
|
||||
async def _fetch(self, path: str, params: dict) -> dict:
|
||||
url = f"{self.config.base_url.rstrip('/')}/{path}"
|
||||
logger.debug("Giphy GET %s", path)
|
||||
try:
|
||||
async with stealth.stealth_async_client(timeout=HTTP_TIMEOUT_SECONDS) as client:
|
||||
response = await client.get(url, params=params)
|
||||
except httpx.HTTPError as exc:
|
||||
logger.warning("Giphy request failed: %s", exc)
|
||||
raise GiphyError(f"Giphy request failed: {exc}") from exc
|
||||
if response.status_code >= 400:
|
||||
logger.warning(
|
||||
"Giphy %s returned %s: %s",
|
||||
path,
|
||||
response.status_code,
|
||||
response.text[:200],
|
||||
)
|
||||
raise GiphyError(
|
||||
f"Giphy returned {response.status_code}", status=response.status_code
|
||||
)
|
||||
return response.json()
|
||||
|
||||
async def search(self, query: str, limit: int = RESULT_LIMIT_DEFAULT) -> list[dict]:
|
||||
data = await self._get(
|
||||
"search",
|
||||
self._params(
|
||||
q=query,
|
||||
limit=max(1, min(int(limit), MAX_LIMIT)),
|
||||
offset=0,
|
||||
lang="en",
|
||||
),
|
||||
)
|
||||
return [normalize_gif(item) for item in data.get("data", [])]
|
||||
|
||||
async def trending(self, limit: int = RESULT_LIMIT_DEFAULT) -> list[dict]:
|
||||
data = await self._get(
|
||||
"trending",
|
||||
self._params(limit=max(1, min(int(limit), MAX_LIMIT)), offset=0),
|
||||
)
|
||||
return [normalize_gif(item) for item in data.get("data", [])]
|
||||
|
||||
async def get_gif(self, gif_id: str) -> dict:
|
||||
if not gif_id:
|
||||
raise GiphyError("missing GIF id", status=400)
|
||||
data = await self._get(gif_id, self._params())
|
||||
item = data.get("data") or {}
|
||||
if not item:
|
||||
raise GiphyError("GIF not found", status=404)
|
||||
return normalize_gif(item)
|
||||
@@ -0,0 +1,59 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from devplacepy.database import get_setting
|
||||
from devplacepy.services.base import ConfigField
|
||||
|
||||
API_BASE_DEFAULT = "https://api.giphy.com/v1/gifs"
|
||||
HTTP_TIMEOUT_SECONDS = 15.0
|
||||
RESULT_LIMIT_DEFAULT = 24
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GiphyConfig:
|
||||
api_key: str
|
||||
base_url: str
|
||||
rating: str
|
||||
|
||||
@property
|
||||
def is_configured(self) -> bool:
|
||||
return bool(self.api_key)
|
||||
|
||||
|
||||
CONFIG_FIELDS: list[ConfigField] = [
|
||||
ConfigField(
|
||||
"giphy_api_key",
|
||||
"Giphy API key",
|
||||
type="password",
|
||||
default="",
|
||||
secret=True,
|
||||
help="API key from the Giphy developer dashboard. Without it the GIF picker stays hidden.",
|
||||
group="Giphy",
|
||||
),
|
||||
ConfigField(
|
||||
"giphy_base_url",
|
||||
"Giphy API base URL",
|
||||
type="url",
|
||||
default=API_BASE_DEFAULT,
|
||||
help="Endpoint root of the Giphy API, without the trailing search or trending path.",
|
||||
group="Giphy",
|
||||
),
|
||||
ConfigField(
|
||||
"giphy_rating",
|
||||
"Content rating",
|
||||
type="choice",
|
||||
default="pg",
|
||||
options=["g", "pg", "pg-13", "r"],
|
||||
help="Highest content rating returned by search and trending.",
|
||||
group="Giphy",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def giphy_config() -> GiphyConfig:
|
||||
return GiphyConfig(
|
||||
api_key=get_setting("giphy_api_key", ""),
|
||||
base_url=get_setting("giphy_base_url", API_BASE_DEFAULT) or API_BASE_DEFAULT,
|
||||
rating=get_setting("giphy_rating", "pg") or "pg",
|
||||
)
|
||||
@@ -0,0 +1,65 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.services.giphy.client import MAX_LIMIT, GiphyError, normalize_gif
|
||||
|
||||
|
||||
def _gif(gif_id: str, title: str, rating: str = "pg") -> dict:
|
||||
base = f"https://media.giphy.test/media/{gif_id}/giphy"
|
||||
return {
|
||||
"id": gif_id,
|
||||
"title": title,
|
||||
"url": f"https://giphy.test/gifs/{gif_id}",
|
||||
"images": {
|
||||
"fixed_width": {
|
||||
"url": f"{base}.gif",
|
||||
"width": "200",
|
||||
"height": "150",
|
||||
"size": "1024",
|
||||
},
|
||||
"original": {
|
||||
"url": f"{base}.gif",
|
||||
"width": "480",
|
||||
"height": "360",
|
||||
"size": "4096",
|
||||
},
|
||||
},
|
||||
"rating": rating,
|
||||
}
|
||||
|
||||
|
||||
class FakeGiphyClient:
|
||||
def __init__(self, gifs: list[dict] | None = None, configured: bool = True):
|
||||
self.gifs = (
|
||||
list(gifs)
|
||||
if gifs is not None
|
||||
else [_gif("fake1", "Fake happy"), _gif("fake2", "Fake dance")]
|
||||
)
|
||||
self.configured = configured
|
||||
self.queries: list[str] = []
|
||||
|
||||
def _require_config(self) -> None:
|
||||
if not self.configured:
|
||||
raise GiphyError("Giphy integration is not configured", status=503)
|
||||
|
||||
def _limit(self, limit: int) -> int:
|
||||
return max(1, min(int(limit), MAX_LIMIT))
|
||||
|
||||
async def search(self, query: str, limit: int = 24) -> list[dict]:
|
||||
self._require_config()
|
||||
self.queries.append(query)
|
||||
needle = (query or "").strip().lower()
|
||||
matches = [
|
||||
item for item in self.gifs if needle in item["title"].lower()
|
||||
] or (list(self.gifs) if not needle else [])
|
||||
return [normalize_gif(item) for item in matches[: self._limit(limit)]]
|
||||
|
||||
async def trending(self, limit: int = 24) -> list[dict]:
|
||||
self._require_config()
|
||||
return [normalize_gif(item) for item in self.gifs[: self._limit(limit)]]
|
||||
|
||||
async def get_gif(self, gif_id: str) -> dict:
|
||||
self._require_config()
|
||||
for item in self.gifs:
|
||||
if item["id"] == gif_id:
|
||||
return normalize_gif(item)
|
||||
raise GiphyError("GIF not found", status=404)
|
||||
@@ -0,0 +1,76 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import time
|
||||
from collections import OrderedDict, deque
|
||||
|
||||
CACHE_MAX_ENTRIES = 256
|
||||
CACHE_TTL_SECONDS = 300.0
|
||||
USER_CALLS_PER_WINDOW = 30
|
||||
USER_WINDOW_SECONDS = 60.0
|
||||
USER_MAX_TRACKED = 2048
|
||||
|
||||
|
||||
class TtlCache:
|
||||
def __init__(
|
||||
self,
|
||||
max_entries: int = CACHE_MAX_ENTRIES,
|
||||
ttl_seconds: float = CACHE_TTL_SECONDS,
|
||||
):
|
||||
self.max_entries = max_entries
|
||||
self.ttl_seconds = ttl_seconds
|
||||
self._items: OrderedDict = OrderedDict()
|
||||
|
||||
def get(self, key):
|
||||
entry = self._items.get(key)
|
||||
if entry is None:
|
||||
return None
|
||||
stored_at, value = entry
|
||||
if time.monotonic() - stored_at > self.ttl_seconds:
|
||||
del self._items[key]
|
||||
return None
|
||||
self._items.move_to_end(key)
|
||||
return value
|
||||
|
||||
def put(self, key, value) -> None:
|
||||
self._items[key] = (time.monotonic(), value)
|
||||
self._items.move_to_end(key)
|
||||
while len(self._items) > self.max_entries:
|
||||
self._items.popitem(last=False)
|
||||
|
||||
def clear(self) -> None:
|
||||
self._items.clear()
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._items)
|
||||
|
||||
|
||||
class UserRateLimiter:
|
||||
def __init__(
|
||||
self,
|
||||
max_calls: int = USER_CALLS_PER_WINDOW,
|
||||
window_seconds: float = USER_WINDOW_SECONDS,
|
||||
max_tracked: int = USER_MAX_TRACKED,
|
||||
):
|
||||
self.max_calls = max_calls
|
||||
self.window_seconds = window_seconds
|
||||
self.max_tracked = max_tracked
|
||||
self._calls: OrderedDict[str, deque] = OrderedDict()
|
||||
|
||||
def allow(self, user_uid: str) -> bool:
|
||||
now = time.monotonic()
|
||||
calls = self._calls.get(user_uid)
|
||||
if calls is None:
|
||||
calls = deque()
|
||||
self._calls[user_uid] = calls
|
||||
self._calls.move_to_end(user_uid)
|
||||
while calls and now - calls[0] > self.window_seconds:
|
||||
calls.popleft()
|
||||
while len(self._calls) > self.max_tracked:
|
||||
self._calls.popitem(last=False)
|
||||
if len(calls) >= self.max_calls:
|
||||
return False
|
||||
calls.append(now)
|
||||
return True
|
||||
|
||||
def reset(self) -> None:
|
||||
self._calls.clear()
|
||||
@@ -0,0 +1,16 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
_client = None
|
||||
|
||||
|
||||
def get_client():
|
||||
if _client is not None:
|
||||
return _client
|
||||
from devplacepy.services.giphy.client import GiphyClient
|
||||
|
||||
return GiphyClient()
|
||||
|
||||
|
||||
def set_client(client) -> None:
|
||||
global _client
|
||||
_client = client
|
||||
@@ -0,0 +1,38 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
|
||||
from devplacepy.services.base import BaseService
|
||||
from devplacepy.services.giphy.client import GiphyError
|
||||
from devplacepy.services.giphy.config import CONFIG_FIELDS, giphy_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GiphyService(BaseService):
|
||||
interval_key = "giphy_interval"
|
||||
default_enabled = False
|
||||
title = "Giphy"
|
||||
description = (
|
||||
"Holds the Giphy API key used by the GIF picker in comments and posts. The key is "
|
||||
"entered by an administrator; members never see it. While the interval runs the "
|
||||
"service only checks that the key still answers."
|
||||
)
|
||||
config_fields = CONFIG_FIELDS
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(name="giphy", interval_seconds=86400)
|
||||
|
||||
async def run_once(self) -> None:
|
||||
config = giphy_config()
|
||||
if not config.is_configured:
|
||||
self.log("Giphy API key not configured; GIF picker stays hidden")
|
||||
return
|
||||
from devplacepy.services.giphy import runtime
|
||||
|
||||
try:
|
||||
gifs = await runtime.get_client().trending(limit=1)
|
||||
except GiphyError as exc:
|
||||
self.log(f"Giphy check failed: {exc}")
|
||||
return
|
||||
self.log(f"Giphy reachable, {len(gifs)} result(s) on trending")
|
||||
@@ -10,7 +10,7 @@ The `/messages` feature is a live WebSocket chat layered over the SAME `messages
|
||||
|
||||
## Bounded page queries (load-bearing performance rule)
|
||||
|
||||
`get_conversations` is ONE window-function query (`ROW_NUMBER() OVER (PARTITION BY other_uid ORDER BY created_at DESC, id DESC)` over `sender_uid = :me OR receiver_uid = :me`, the `get_recent_comments_by_target_uids` precedent) that returns only each partner's latest row - never load the user's full message history into Python to build the sidebar. It backs both the server-rendered `GET /messages` page and the JSON `GET /messages/conversations` route (same helper, no duplicated query). `get_conversation_messages` loads at most `CONVERSATION_MESSAGE_LIMIT` (500) most-recent rows (`ORDER BY created_at DESC, id DESC LIMIT`, reversed to ascending in Python); older history stays in the DB and is simply not rendered. Both are covered by `idx_messages_conversation`/`_rev` (verified `EXPLAIN QUERY PLAN`, MULTI-INDEX OR, no table scan). `messages` is NOT in `SOFT_DELETE_TABLES`, so neither query filters `deleted_at`. Any new message listing must stay bounded and indexed the same way.
|
||||
`get_conversations` is ONE window-function query (`ROW_NUMBER() OVER (PARTITION BY other_uid ORDER BY created_at DESC, id DESC)` over `sender_uid = :me OR receiver_uid = :me`, the `get_recent_comments_by_target_uids` precedent) that returns only each partner's latest row - never load the user's full message history into Python to build the sidebar. It backs both the server-rendered `GET /messages` page and the JSON `GET /messages/conversations` route (same helper, no duplicated query). `get_conversation_messages` loads at most `CONVERSATION_MESSAGE_LIMIT` (500) most-recent rows (`ORDER BY created_at DESC, id DESC LIMIT`, reversed to ascending in Python); older history stays in the DB and is simply not rendered. Both are covered by `idx_messages_conversation`/`_rev` (verified `EXPLAIN QUERY PLAN`, MULTI-INDEX OR, no table scan). `messages` IS in `SOFT_DELETE_TABLES`, so every read (conversation window, history, `_sync_missed`, relay ticks, unread counts in `templating.jinja_unread_messages`/`notification_relay`/`docs_live`, `_find_recent_duplicate`, `stamp_content_revision`) filters `deleted_at IS NULL`, and `persist_message` inserts `deleted_at: None, deleted_by: None`. Any new message listing must stay bounded and indexed the same way.
|
||||
|
||||
## `GET /messages/conversations`
|
||||
|
||||
@@ -58,9 +58,13 @@ Because the WS accepts on every worker, a message persisted on worker A must sti
|
||||
|
||||
`"messages": ("content",)` is in the `CORRECTABLE_FIELDS` registry and `persist_message` invokes `schedule_correction`/`schedule_modification` (sender = the user), so typing `@ai <instruction>` in a DM runs the AI modifier (default on + sync) and an enabled correction rewrites the content. Both send paths funnel through `routers/messages._finalize_and_broadcast`: it broadcasts the persisted row immediately (`ai_pending=true` when sync futures were stashed on `request.scope[PENDING_SCOPE_KEY]`), then applies any pending SYNC correction/modifier. The HTTP `POST /send` path awaits that apply so the JSON body is the final text; the WS send path schedules it with `asyncio.create_task` so the receive loop never blocks on the gateway. After an in-place rewrite, `_run_correction`/`_run_modification` call `push_content_revision` which stamps `messages.updated_at` and pushes a second `message` frame (`ai_processed=true`) to local sockets. Cross-worker peers pick the revision up from `message_relay._tick_updates` (`SELECT ... WHERE updated_at > watermark`, independent of the new-row `id` watermark and of `was_delivered`). The client matches the second frame on `data-msg-uid` and replaces the bubble body. The WS loop also accepts `{type:"sync", since, with_uid?}` to replay rows created or revised after `since` (reconnect catch-up) and `{type:"ping"}` (ignored). `ContentRefused` on a WS send returns `{type:"error"}` and keeps the socket open.
|
||||
|
||||
## Administrators delete their own sent messages
|
||||
|
||||
`POST /messages/{message_uid}/delete` (`require_user`) is allowed ONLY when `is_admin(user)` AND `message.sender_uid == user.uid`; members and admins deleting someone else's message get a `denied` audit row (`message.delete`) and a 403 (JSON) or redirect. It is a soft delete (`removal.delete_message_record`): the message and its attachments share one stamp, the receiver's unread cache is cleared, and the row is restorable from `/admin/trash`. Live removal: `removal.broadcast_deleted` sends `{type:"message_deleted", uid, sender_uid, receiver_uid}` to both participants through `message_hub` (marking `deleted:{uid}` as delivered), and `message_relay._tick_deleted` (a `deleted_at` watermark) fills the cross-worker gap, skipping uids the hub already delivered. `AppChat.js` renders a `.message-delete-btn` on the admin's own bubbles (`self-admin` attribute on `<dp-chat>`, server-rendered in `messages.html` and built by `_buildBubble`/`_applyFrameContent`), confirms through `app.dialog.confirm`, and `_removeBubble` handles both the HTTP result and the `message_deleted` frame. A restore does not push a frame; the message reappears on reload. Devii: `delete_message` (`requires_admin`, `CONFIRM_REQUIRED`, declared `confirm`).
|
||||
|
||||
## WS protocol frames
|
||||
|
||||
Client -> server: `{type:"send", receiver_uid, content, attachment_uids?, client_id}`, `{type:"typing", receiver_uid}` (throttled client-side), `{type:"read", with_uid}`, `{type:"active", with_uid}` (marks the sender as currently viewing that conversation - see "Active-conversation notification suppression" below), `{type:"sync", since, with_uid?}` (replay created-or-revised rows after `since`), `{type:"ping"}` (keepalive, ignored). Server -> client: `{type:"ready", user_uid}` (sent first on connect), `{type:"message", uid, sender_uid, sender_username, sender_role, receiver_uid, content, created_at, time_ago, client_id, attachments, ai_processed, ai_pending}` (broadcast to BOTH sender and receiver sockets so multi-tab and the sender's own optimistic bubble reconcile via the echoed `client_id`; a first frame may set `ai_pending` while a sync job runs, and a later frame for the same `uid` sets `ai_processed` with the rewritten body), `{type:"typing", from_uid}` (to the receiver only), `{type:"read", by_uid}` (read-receipt to the other user), `{type:"error", client_id, text}` (sender only, on a dropped send, e.g. the recipient blocked the sender, or a screened body). There is no `presence` frame on this socket - see below. `active` has no server -> client echo; it only writes the DB-backed marker consumed server-side by `persist_message`.
|
||||
Client -> server: `{type:"send", receiver_uid, content, attachment_uids?, client_id}`, `{type:"typing", receiver_uid}` (throttled client-side), `{type:"read", with_uid}`, `{type:"active", with_uid}` (marks the sender as currently viewing that conversation - see "Active-conversation notification suppression" below), `{type:"sync", since, with_uid?}` (replay created-or-revised rows after `since`), `{type:"ping"}` (keepalive, ignored). Server -> client: `{type:"ready", user_uid}` (sent first on connect), `{type:"message", uid, sender_uid, sender_username, sender_role, receiver_uid, content, created_at, time_ago, client_id, attachments, ai_processed, ai_pending}` (broadcast to BOTH sender and receiver sockets so multi-tab and the sender's own optimistic bubble reconcile via the echoed `client_id`; a first frame may set `ai_pending` while a sync job runs, and a later frame for the same `uid` sets `ai_processed` with the rewritten body), `{type:"typing", from_uid}` (to the receiver only), `{type:"read", by_uid}` (read-receipt to the other user), `{type:"message_deleted", uid, sender_uid, receiver_uid}` (both participants, see "Administrators delete their own sent messages"), `{type:"error", client_id, text}` (sender only, on a dropped send, e.g. the recipient blocked the sender, or a screened body). There is no `presence` frame on this socket - see below. `active` has no server -> client echo; it only writes the DB-backed marker consumed server-side by `persist_message`.
|
||||
|
||||
## Active-conversation notification suppression
|
||||
|
||||
|
||||
@@ -12,9 +12,13 @@ from devplacepy.services.messaging.persist import (
|
||||
stamp_content_revision,
|
||||
)
|
||||
from devplacepy.services.messaging.relay import message_relay
|
||||
from devplacepy.services.messaging.removal import broadcast_deleted, delete_message_record, deleted_frame
|
||||
from devplacepy.services.messaging.tickets import issue_ticket, redeem_ticket
|
||||
|
||||
__all__ = [
|
||||
"broadcast_deleted",
|
||||
"delete_message_record",
|
||||
"deleted_frame",
|
||||
"is_actively_viewing",
|
||||
"issue_ticket",
|
||||
"message_frame",
|
||||
|
||||
@@ -73,6 +73,7 @@ def _find_recent_duplicate(sender_uid: str, client_id: str) -> Optional[dict[str
|
||||
row = get_table("messages").find_one(
|
||||
sender_uid=sender_uid,
|
||||
client_id=client_id,
|
||||
deleted_at=None,
|
||||
created_at={">=": cutoff},
|
||||
)
|
||||
return dict(row) if row else None
|
||||
@@ -124,6 +125,8 @@ def persist_message(
|
||||
"created_at": created_at,
|
||||
"updated_at": None,
|
||||
"client_id": client_id,
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
|
||||
@@ -213,7 +216,7 @@ def stamp_content_revision(message_uid: str) -> Optional[dict[str, Any]]:
|
||||
if not message_uid:
|
||||
return None
|
||||
table = get_table("messages")
|
||||
row = table.find_one(uid=message_uid)
|
||||
row = table.find_one(uid=message_uid, deleted_at=None)
|
||||
if not row:
|
||||
return None
|
||||
updated_at = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
@@ -8,6 +8,7 @@ from typing import Optional
|
||||
from devplacepy.database import db, get_users_by_uids
|
||||
from devplacepy.services.messaging.hub import message_hub
|
||||
from devplacepy.services.messaging.persist import message_frame
|
||||
from devplacepy.services.messaging.removal import deleted_frame, deleted_key
|
||||
|
||||
logger = logging.getLogger("messaging.relay")
|
||||
|
||||
@@ -20,6 +21,7 @@ class MessageRelay:
|
||||
self._task: Optional[asyncio.Task] = None
|
||||
self._watermark: int = 0
|
||||
self._update_watermark: str = ""
|
||||
self._deleted_watermark: str = ""
|
||||
self._primed: bool = False
|
||||
|
||||
def start(self) -> None:
|
||||
@@ -40,12 +42,14 @@ class MessageRelay:
|
||||
if not self._primed:
|
||||
self._watermark = self._max_id()
|
||||
self._update_watermark = datetime.now(timezone.utc).isoformat()
|
||||
self._deleted_watermark = self._update_watermark
|
||||
self._primed = True
|
||||
logger.debug("message relay primed at watermark %d", self._watermark)
|
||||
while message_hub.has_connections():
|
||||
try:
|
||||
await self._tick()
|
||||
await self._tick_updates()
|
||||
await self._tick_deleted()
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception("message relay tick failed")
|
||||
await asyncio.sleep(POLL_INTERVAL_SECONDS)
|
||||
@@ -58,7 +62,7 @@ class MessageRelay:
|
||||
return
|
||||
rows = list(
|
||||
db.query(
|
||||
"SELECT * FROM messages WHERE id > :wm ORDER BY id ASC LIMIT :lim",
|
||||
"SELECT * FROM messages WHERE id > :wm AND deleted_at IS NULL ORDER BY id ASC LIMIT :lim",
|
||||
wm=self._watermark,
|
||||
lim=BATCH_LIMIT,
|
||||
)
|
||||
@@ -95,7 +99,7 @@ class MessageRelay:
|
||||
rows = list(
|
||||
db.query(
|
||||
"SELECT * FROM messages"
|
||||
" WHERE updated_at IS NOT NULL AND updated_at > :uwm"
|
||||
" WHERE updated_at IS NOT NULL AND updated_at > :uwm AND deleted_at IS NULL"
|
||||
" ORDER BY updated_at ASC LIMIT :lim",
|
||||
uwm=self._update_watermark,
|
||||
lim=BATCH_LIMIT,
|
||||
@@ -131,5 +135,31 @@ class MessageRelay:
|
||||
len(rows),
|
||||
)
|
||||
|
||||
async def _tick_deleted(self) -> None:
|
||||
if "messages" not in db.tables or not self._deleted_watermark:
|
||||
return
|
||||
rows = list(
|
||||
db.query(
|
||||
"SELECT * FROM messages"
|
||||
" WHERE deleted_at IS NOT NULL AND deleted_at > :dwm"
|
||||
" ORDER BY deleted_at ASC LIMIT :lim",
|
||||
dwm=self._deleted_watermark,
|
||||
lim=BATCH_LIMIT,
|
||||
)
|
||||
)
|
||||
if not rows:
|
||||
return
|
||||
connected = message_hub.connected_user_uids()
|
||||
for row in rows:
|
||||
key = deleted_key(row["uid"])
|
||||
if message_hub.was_delivered(key):
|
||||
continue
|
||||
if row["sender_uid"] in connected or row["receiver_uid"] in connected:
|
||||
message_hub.mark_delivered(key)
|
||||
await message_hub.send_to_users(
|
||||
[row["sender_uid"], row["receiver_uid"]], deleted_frame(dict(row))
|
||||
)
|
||||
self._deleted_watermark = max(row["deleted_at"] for row in rows)
|
||||
|
||||
|
||||
message_relay = MessageRelay()
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from typing import Any
|
||||
|
||||
from devplacepy.database import _now_iso, soft_delete, soft_delete_in
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.messaging.hub import message_hub
|
||||
from devplacepy.templating import clear_messages_cache
|
||||
|
||||
|
||||
def deleted_frame(message: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"type": "message_deleted",
|
||||
"uid": message["uid"],
|
||||
"sender_uid": message["sender_uid"],
|
||||
"receiver_uid": message["receiver_uid"],
|
||||
}
|
||||
|
||||
|
||||
def delete_message_record(request: Any, user: dict[str, Any], message: dict[str, Any]) -> None:
|
||||
actor = user["uid"]
|
||||
stamp = _now_iso()
|
||||
soft_delete_in("attachments", "target_uid", [message["uid"]], actor, stamp=stamp, target_type="message")
|
||||
soft_delete("messages", actor, stamp=stamp, uid=message["uid"])
|
||||
clear_messages_cache(message["receiver_uid"])
|
||||
audit.record(
|
||||
request,
|
||||
"message.delete",
|
||||
user=user,
|
||||
target_type="message",
|
||||
target_uid=message["uid"],
|
||||
summary=f"{user['username']} deleted a message they sent",
|
||||
links=[
|
||||
audit.target("message", message["uid"]),
|
||||
audit.recipient(message["receiver_uid"]),
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
async def broadcast_deleted(message: dict[str, Any]) -> None:
|
||||
message_hub.mark_delivered(deleted_key(message["uid"]))
|
||||
await message_hub.send_to_users(
|
||||
[message["sender_uid"], message["receiver_uid"]], deleted_frame(message)
|
||||
)
|
||||
|
||||
|
||||
def deleted_key(message_uid: str) -> str:
|
||||
return f"deleted:{message_uid}"
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import csv
|
||||
import gzip
|
||||
import ipaddress
|
||||
@@ -232,12 +233,15 @@ async def _download(url: str, destination: Path) -> bool:
|
||||
if response.status_code != 200:
|
||||
raise GeoError(f"{url} answered {response.status_code}")
|
||||
total = 0
|
||||
with open(destination, "wb") as handle:
|
||||
handle = await asyncio.to_thread(open, destination, "wb")
|
||||
try:
|
||||
async for chunk in response.aiter_bytes():
|
||||
total += len(chunk)
|
||||
if total > SPEC.max_download_bytes:
|
||||
raise GeoError(f"{url} exceeds {SPEC.max_download_bytes} bytes")
|
||||
handle.write(chunk)
|
||||
await asyncio.to_thread(handle.write, chunk)
|
||||
finally:
|
||||
await asyncio.to_thread(handle.close)
|
||||
return True
|
||||
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ class NotificationRelayService(BaseService):
|
||||
rows = list(
|
||||
db.query(
|
||||
"SELECT COUNT(*) AS c FROM messages "
|
||||
"WHERE receiver_uid = :u AND read = 0",
|
||||
"WHERE receiver_uid = :u AND read = 0 AND deleted_at IS NULL",
|
||||
u=user_uid,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -36,6 +36,17 @@ Callers may send an optional `X-App-Reference` header to tag gateway calls by ap
|
||||
|
||||
`apps.py` is the single specification of every internal gateway app: the `GATEWAY_APPS` registry (`GatewayApp(reference, name)`, title `DevPlace <name>`), the `APP_REFERENCE_*` constants callers import, `validate_app_reference` and `attribution_headers(app_reference)`. Every upstream request (chat, embeddings, images, vision sub-call, fallback retries) carries, for ALL providers and not only OpenRouter: `X-OpenRouter-Title` (OpenRouter's current app-title header, https://openrouter.ai/docs/app-attribution), the legacy `X-Title` with the same value, and `HTTP-Referer` = `DEVPLACE_SITE_URL` when set. The title is per app (`DevPlace Devii`, `DevPlace News`, ...), so a provider dashboard splits spend by sub-app; unknown or `default` references fall back to `DevPlace`. OpenRouter documents no sub-app field beyond the title, so the title is the carrier. Apps: Devii, News, Bots, AI Correction, AI Modifier, Quiz Grading, SEO Metadata, DeepSearch, AI Usage Analyzer, Issue Enhance, Issue Planning, DB API, Awards, Image Generation, Community Discovery, Community Sync. A new internal caller adds one registry row and sends `X-App-Reference`. `gateway_complete(..., app_reference=)` carries the label for correction-style callers.
|
||||
|
||||
## Tool-call emulation (`tool_emulation.py`, per-model `tools_mode`)
|
||||
|
||||
A model route (`gateway_models.tools_mode`, `native` default or `emulated`) declares whether the model supports tool calling. **The admin's word is the truth: there is no probing and no auto-detection**, and an `emulated` flag is never second-guessed. The flag rides the route overlay as `gateway_tools_mode`, so it is decided per attempt and a fallback chain may freely mix native and emulated models. The client keeps speaking plain OpenAI: it sends `tools` and gets `tool_calls` back, whatever the model underneath.
|
||||
|
||||
- **Request seam.** `gateway._chat_upstream_body` calls `emulate_request(payload)` first, BEFORE the Responses/Meta transpile, so the Meta path (`chat_to_responses`, Muse aliasing) simply receives a tool-less Chat payload. As a side effect the Meta reserved tool-name gate never fires, because `tools[]` is gone; `_response_tool_aliases` returns `{}` for emulated routes. `emulate_request` strips `tools`/`tool_choice`/`parallel_tool_calls`/`functions`/`function_call`, appends a Hermes-format catalog (`<tools>` JSON lines plus the `<tool_call>{"name","arguments"}</tool_call>` contract) to the system message (creating one when absent), maps `tool_choice` (`none` drops the catalog, `required`/named add a MUST directive, `parallel_tool_calls: false` adds a one-call rule), rewrites history (assistant `tool_calls` become `<tool_call>` blocks, `role: tool`/`function` messages become one merged user message of `<tool_response>` blocks matched back to names by call id), and adds `<tool_response>` as a stop sequence so the model cannot invent a result. It never mutates the caller's payload.
|
||||
- **Response seams.** Non-streaming: `convert_completion` parses blocks out of `message.content` into `tool_calls` (generated `call_` ids, `finish_reason: "tool_calls"`, leftover text kept, `content: null` when empty), applied on both the chat and the Responses non-stream branches. Streaming: `_emulate_tool_stream` wraps the finished SSE iterator, so ledger/TTFT accounting stays inside the inner generator. `ToolCallStream` relays text live with a partial-`<tool_call>` holdback, buffers once a call opens, and emits each call whole in ONE `tool_calls` delta (never incremental arguments: that is where vLLM's Hermes parser kept breaking). Usage chunks and non-content deltas pass through.
|
||||
- **Parsing is tolerant, validation strict.** Accepts code fences, an array of calls in one block, `parameters` for `arguments`, string-encoded arguments, and an unterminated final block. A call is valid only if the name is declared and the schema's `required` keys are present. An invalid block is NEVER dropped silently: it stays in `content` as text and is logged (`emulated tools: ...`). Known gap: there is no repair retry; an invalid call is returned as text.
|
||||
- **Hard caps (400, no upstream call, no ledger row).** `MAX_TOOLS` 64, `MAX_CATALOG_CHARS` 60000, only `function` tools, `MAX_CALLS_PER_TURN` 8 (extra calls dropped with a warning). The catalog costs prompt tokens on every call, billed like any input.
|
||||
- **Surfaces.** `ModelRouteIn.tools_mode`, the admin model form (**Tool calling** select), the `tools: emulated` badge, `POST /admin/gateway/models` (`tools_mode`, documented in `docs_api`) and the Devii `gateway_model_set` tool.
|
||||
- **Not covered:** constrained/guided decoding (provider-specific), and the format is Hermes JSON only. Tests: `tests/unit/services/openai_gateway/tool_emulation.py` (pure) and the `test_emulated_tools_*` tests in `gateway.py` (through the real Meta Responses path, streaming and non-streaming).
|
||||
|
||||
## Per-worker runtime
|
||||
|
||||
**Serves in every worker.** Config/enabled come from `site_settings` (via the service's `get_config()`/`is_enabled()`), so any uvicorn worker answers - not just the supervisor worker. Per-worker runtime (`GatewayRuntime`: `httpx.AsyncClient` pool + `asyncio.Semaphore` sized to `gateway_instances`, plus counters and a `VisionCache`) is created lazily on first request and rebuilt when `instances`/`timeout`/cache size change. `gateway_instances` is the **scaling knob** (concurrency per worker); process scaling is uvicorn workers.
|
||||
|
||||
@@ -30,6 +30,15 @@ from devplacepy.services.openai_gateway.responses import (
|
||||
responses_object_to_chat_completion,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.system_message import apply_system_directives
|
||||
from devplacepy.services.openai_gateway.tool_emulation import (
|
||||
ToolCallStream,
|
||||
ToolEmulationError,
|
||||
convert_completion,
|
||||
declared_tools,
|
||||
emulate_request,
|
||||
emulation_enabled,
|
||||
parallel_allowed,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.thinking import (
|
||||
apply_thinking,
|
||||
client_thinking_enabled,
|
||||
@@ -151,6 +160,8 @@ def _chat_upstream_body(
|
||||
) -> dict:
|
||||
# The handler reasons about a Chat payload throughout (routing, vision,
|
||||
# precheck, fallback); only the bytes on the wire change per protocol.
|
||||
if emulation_enabled(cfg):
|
||||
payload = emulate_request(payload)
|
||||
if protocol != "responses":
|
||||
return payload
|
||||
if muse_code.is_meta_upstream(cfg.get("gateway_upstream_url")):
|
||||
@@ -183,13 +194,69 @@ def _response_tool_aliases(payload: dict, protocol: str, cfg: dict) -> dict:
|
||||
# Rebuilds the exact map _chat_upstream_body applied on the wire for
|
||||
# this attempt (same payload, same deterministic rule), so response
|
||||
# names restore without threading state through the fallback path.
|
||||
if protocol != "responses":
|
||||
if protocol != "responses" or emulation_enabled(cfg):
|
||||
return {}
|
||||
if not muse_code.is_meta_upstream(cfg.get("gateway_upstream_url")):
|
||||
return {}
|
||||
return muse_code.tool_aliases_for(payload)
|
||||
|
||||
|
||||
def _tool_emulation_rejection(error: ToolEmulationError) -> JSONResponse:
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": {"message": str(error), "type": "invalid_request_error"}},
|
||||
)
|
||||
|
||||
|
||||
def _emulated_tools(payload: dict, cfg: dict):
|
||||
# Same rebuild-from-the-attempt's-own-payload rule as _response_tool_aliases:
|
||||
# the declared tools come from the Chat payload of the attempt that was
|
||||
# actually sent, so a fallback crossing into or out of an emulated model
|
||||
# needs no threaded state.
|
||||
if not emulation_enabled(cfg):
|
||||
return None
|
||||
declared = declared_tools(payload)
|
||||
if not declared:
|
||||
return None
|
||||
return declared, parallel_allowed(payload)
|
||||
|
||||
|
||||
def _emulate_completion(data: dict, emulated, log) -> dict:
|
||||
if emulated is None:
|
||||
return data
|
||||
declared, allow_parallel = emulated
|
||||
data, warnings = convert_completion(data, declared, allow_parallel)
|
||||
for warning in warnings[:3]:
|
||||
log(f"emulated tools: {warning}")
|
||||
return data
|
||||
|
||||
|
||||
async def _emulate_tool_stream(inner, emulated, log):
|
||||
declared, allow_parallel = emulated
|
||||
translator = ToolCallStream(declared, allow_parallel)
|
||||
try:
|
||||
async for line in inner:
|
||||
body = line[len("data:") :].strip() if line.startswith("data:") else ""
|
||||
if body == "[DONE]":
|
||||
for out in translator.finish():
|
||||
yield f"data: {json.dumps(out)}\n\n"
|
||||
yield line
|
||||
continue
|
||||
try:
|
||||
chunk = json.loads(body)
|
||||
except ValueError:
|
||||
yield line
|
||||
continue
|
||||
if not isinstance(chunk, dict) or chunk.get("error"):
|
||||
yield line
|
||||
continue
|
||||
for out in translator.feed(chunk):
|
||||
yield f"data: {json.dumps(out)}\n\n"
|
||||
finally:
|
||||
for warning in translator.warnings[:3]:
|
||||
log(f"emulated tools: {warning}")
|
||||
|
||||
|
||||
def _apply_served_model(headers: dict, resp) -> dict:
|
||||
# Everything else in `headers` (cost, tokens, latency, context) is our
|
||||
# own measurement and must never be overwritten by an upstream that
|
||||
@@ -633,7 +700,10 @@ class GatewayRuntime:
|
||||
break
|
||||
|
||||
send_start = time.monotonic()
|
||||
send_body = _chat_upstream_body(payload, body, cfg, protocol, log)
|
||||
try:
|
||||
send_body = _chat_upstream_body(payload, body, cfg, protocol, log)
|
||||
except ToolEmulationError as tool_error:
|
||||
return _tool_emulation_rejection(tool_error)
|
||||
resp, exc, timing = await self._send(
|
||||
client,
|
||||
sem,
|
||||
@@ -683,9 +753,12 @@ class GatewayRuntime:
|
||||
payload["model"] = model
|
||||
headers = _fallback_headers(cfg, "gateway_api_key", app_reference)
|
||||
send_start = time.monotonic()
|
||||
send_body = _chat_upstream_body(
|
||||
payload, body, cfg, protocol, log
|
||||
)
|
||||
try:
|
||||
send_body = _chat_upstream_body(
|
||||
payload, body, cfg, protocol, log
|
||||
)
|
||||
except ToolEmulationError as tool_error:
|
||||
return _tool_emulation_rejection(tool_error)
|
||||
resp, exc, timing = await self._send(
|
||||
client,
|
||||
sem,
|
||||
@@ -843,6 +916,9 @@ class GatewayRuntime:
|
||||
send_start,
|
||||
log,
|
||||
)
|
||||
emulated = _emulated_tools(payload, cfg)
|
||||
if emulated is not None:
|
||||
body_iter = _emulate_tool_stream(body_iter, emulated, log)
|
||||
return StreamingResponse(
|
||||
body_iter,
|
||||
media_type="text/event-stream",
|
||||
@@ -878,6 +954,9 @@ class GatewayRuntime:
|
||||
converted = muse_code.restore_completion_names(
|
||||
converted, _response_tool_aliases(payload, protocol, cfg)
|
||||
)
|
||||
converted = _emulate_completion(
|
||||
converted, _emulated_tools(payload, cfg), log
|
||||
)
|
||||
if convert_warnings:
|
||||
log(
|
||||
"responses->chat conversion warnings: "
|
||||
@@ -892,6 +971,13 @@ class GatewayRuntime:
|
||||
)
|
||||
resp_headers = finalize(200, True, None, data.get("usage"))
|
||||
log(f"chat POST -> 200 ({timing['upstream_latency_ms']:.0f}ms)")
|
||||
emulated = _emulated_tools(payload, cfg)
|
||||
if emulated is not None:
|
||||
return Response(
|
||||
content=json.dumps(_emulate_completion(data, emulated, log)).encode("utf-8"),
|
||||
media_type="application/json",
|
||||
headers=resp_headers,
|
||||
)
|
||||
return Response(
|
||||
content=resp.content,
|
||||
media_type="application/json",
|
||||
|
||||
@@ -16,6 +16,10 @@ from devplacepy.database import (
|
||||
sync_local_cache,
|
||||
)
|
||||
from devplacepy.services.openai_gateway import muse_code, opencode_zen
|
||||
from devplacepy.services.openai_gateway.tool_emulation import (
|
||||
TOOLS_MODE_NATIVE,
|
||||
TOOLS_MODES,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -205,6 +209,8 @@ MODEL_HEALTH_COLUMNS = (
|
||||
"health_checked_at",
|
||||
)
|
||||
|
||||
MODEL_TOOLS_COLUMN = "tools_mode"
|
||||
|
||||
HEALTH_OK = "ok"
|
||||
HEALTH_BAD = "bad"
|
||||
|
||||
@@ -241,6 +247,7 @@ def ensure_tables() -> None:
|
||||
"is_active INTEGER DEFAULT 1, health_status TEXT DEFAULT '', "
|
||||
"health_code TEXT DEFAULT '', health_message TEXT DEFAULT '', "
|
||||
"health_checked_at TEXT DEFAULT '', "
|
||||
"tools_mode TEXT DEFAULT 'native', "
|
||||
"created_at TEXT, updated_at TEXT)"
|
||||
)
|
||||
models_table = get_table(MODELS_TABLE)
|
||||
@@ -252,6 +259,8 @@ def ensure_tables() -> None:
|
||||
for column in MODEL_HEALTH_COLUMNS:
|
||||
if not models_table.has_column(column):
|
||||
models_table.create_column_by_example(column, "")
|
||||
if not models_table.has_column(MODEL_TOOLS_COLUMN):
|
||||
models_table.create_column_by_example(MODEL_TOOLS_COLUMN, TOOLS_MODE_NATIVE)
|
||||
try:
|
||||
db.query(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS idx_gateway_providers_name ON "
|
||||
@@ -340,8 +349,17 @@ class ModelRouteIn(BaseModel):
|
||||
off_peak_end_minute: Optional[int] = Field(default=None, ge=0, le=1439)
|
||||
off_peak_discount_pct: float = Field(default=0.0, ge=0, le=100)
|
||||
fallback_model: str = Field(default="", max_length=128)
|
||||
tools_mode: str = Field(default=TOOLS_MODE_NATIVE, max_length=16)
|
||||
is_active: bool = True
|
||||
|
||||
@field_validator("tools_mode")
|
||||
@classmethod
|
||||
def _clean_tools_mode(cls, value: str) -> str:
|
||||
value = (value or TOOLS_MODE_NATIVE).strip().lower()
|
||||
if value not in TOOLS_MODES:
|
||||
raise ValueError(f"tools_mode must be one of: {', '.join(TOOLS_MODES)}")
|
||||
return value
|
||||
|
||||
@field_validator("source_model", "target_model")
|
||||
@classmethod
|
||||
def _clean_model(cls, value: str) -> str:
|
||||
@@ -407,6 +425,7 @@ class ModelRoute:
|
||||
health_code: str
|
||||
health_message: str
|
||||
health_checked_at: str
|
||||
tools_mode: str
|
||||
|
||||
|
||||
def _opt_float(row: dict, key: str) -> Optional[float]:
|
||||
@@ -448,6 +467,7 @@ def _route_from_row(row: dict) -> ModelRoute:
|
||||
health_code=str(row.get("health_code") or ""),
|
||||
health_message=str(row.get("health_message") or ""),
|
||||
health_checked_at=str(row.get("health_checked_at") or ""),
|
||||
tools_mode=str(row.get("tools_mode") or TOOLS_MODE_NATIVE),
|
||||
)
|
||||
|
||||
|
||||
@@ -579,6 +599,7 @@ class ModelStore:
|
||||
"off_peak_end_minute": payload.off_peak_end_minute,
|
||||
"off_peak_discount_pct": payload.off_peak_discount_pct,
|
||||
"fallback_model": payload.fallback_model,
|
||||
"tools_mode": payload.tools_mode,
|
||||
"is_active": 1 if payload.is_active else 0,
|
||||
"updated_at": _now(),
|
||||
}
|
||||
@@ -789,6 +810,7 @@ def chat_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[dic
|
||||
"gateway_off_peak_end_minute": route.off_peak_end_minute,
|
||||
"gateway_off_peak_discount_pct": route.off_peak_discount_pct,
|
||||
"gateway_provider_name": route.provider or "default",
|
||||
"gateway_tools_mode": route.tools_mode,
|
||||
}
|
||||
if route.provider:
|
||||
_provider_overlay(
|
||||
|
||||
@@ -0,0 +1,444 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import copy
|
||||
import json
|
||||
import re
|
||||
import uuid
|
||||
from typing import Optional
|
||||
|
||||
TOOLS_MODE_NATIVE = "native"
|
||||
TOOLS_MODE_EMULATED = "emulated"
|
||||
TOOLS_MODES = (TOOLS_MODE_NATIVE, TOOLS_MODE_EMULATED)
|
||||
|
||||
MAX_TOOLS = 64
|
||||
MAX_CATALOG_CHARS = 60_000
|
||||
MAX_CALLS_PER_TURN = 8
|
||||
MAX_STOP_SEQUENCES = 4
|
||||
|
||||
CALL_OPEN = "<tool_call>"
|
||||
CALL_CLOSE = "</tool_call>"
|
||||
RESPONSE_OPEN = "<tool_response>"
|
||||
RESPONSE_CLOSE = "</tool_response>"
|
||||
|
||||
_CALL_BLOCK = re.compile(
|
||||
re.escape(CALL_OPEN) + r"(.*?)(?:" + re.escape(CALL_CLOSE) + r"|\Z)", re.DOTALL
|
||||
)
|
||||
_FENCE = re.compile(r"^```[a-zA-Z0-9_-]*\s*\n?(.*?)\n?```$", re.DOTALL)
|
||||
|
||||
REMOVED_PAYLOAD_KEYS = (
|
||||
"tools",
|
||||
"tool_choice",
|
||||
"parallel_tool_calls",
|
||||
"functions",
|
||||
"function_call",
|
||||
)
|
||||
|
||||
|
||||
class ToolEmulationError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def emulation_enabled(cfg: dict) -> bool:
|
||||
return (cfg.get("gateway_tools_mode") or TOOLS_MODE_NATIVE) == TOOLS_MODE_EMULATED
|
||||
|
||||
|
||||
def _function_specs(payload: dict) -> list[dict]:
|
||||
tools = payload.get("tools")
|
||||
if not tools:
|
||||
return []
|
||||
if not isinstance(tools, list):
|
||||
raise ToolEmulationError("tools must be a list")
|
||||
if len(tools) > MAX_TOOLS:
|
||||
raise ToolEmulationError(
|
||||
f"too many tools for an emulated-tools model: {len(tools)} > {MAX_TOOLS}"
|
||||
)
|
||||
specs = []
|
||||
for tool in tools:
|
||||
if not isinstance(tool, dict) or tool.get("type") != "function":
|
||||
raise ToolEmulationError(
|
||||
"an emulated-tools model supports only tools of type 'function'"
|
||||
)
|
||||
function = tool.get("function")
|
||||
if not isinstance(function, dict) or not str(function.get("name") or "").strip():
|
||||
raise ToolEmulationError("every tool needs function.name")
|
||||
specs.append(function)
|
||||
return specs
|
||||
|
||||
|
||||
def declared_tools(payload: dict) -> dict[str, dict]:
|
||||
return {str(spec["name"]): spec for spec in _function_specs(payload)}
|
||||
|
||||
|
||||
def parallel_allowed(payload: dict) -> bool:
|
||||
return payload.get("parallel_tool_calls") is not False
|
||||
|
||||
|
||||
def _choice_directive(payload: dict) -> tuple[bool, str]:
|
||||
choice = payload.get("tool_choice")
|
||||
if choice == "none":
|
||||
return False, ""
|
||||
if choice == "required":
|
||||
return True, "You MUST call at least one tool in this reply."
|
||||
if isinstance(choice, dict):
|
||||
name = str((choice.get("function") or {}).get("name") or "").strip()
|
||||
if name:
|
||||
return True, f'You MUST call the tool "{name}" in this reply.'
|
||||
return True, ""
|
||||
|
||||
|
||||
def _catalog_text(specs: list[dict]) -> str:
|
||||
lines = [json.dumps({"type": "function", "function": spec}, ensure_ascii=False) for spec in specs]
|
||||
catalog = "\n".join(lines)
|
||||
if len(catalog) > MAX_CATALOG_CHARS:
|
||||
raise ToolEmulationError(
|
||||
f"tool catalog too large for an emulated-tools model: "
|
||||
f"{len(catalog)} > {MAX_CATALOG_CHARS} characters"
|
||||
)
|
||||
return catalog
|
||||
|
||||
|
||||
def build_instruction(payload: dict, specs: list[dict]) -> str:
|
||||
include, directive = _choice_directive(payload)
|
||||
if not include or not specs:
|
||||
return ""
|
||||
parts = [
|
||||
"You can call tools. The available tools are listed below inside "
|
||||
"<tools></tools>, one JSON object per line.",
|
||||
"<tools>\n" + _catalog_text(specs) + "\n</tools>",
|
||||
"To call a tool, reply with one block per call, in exactly this format:\n"
|
||||
f"{CALL_OPEN}\n"
|
||||
'{"name": "<tool name>", "arguments": {<arguments object>}}\n'
|
||||
f"{CALL_CLOSE}",
|
||||
"Rules: use only the tool names listed above. The arguments must be a "
|
||||
"valid JSON object that matches the tool's parameter schema. After the "
|
||||
"call blocks, stop: the results arrive in the next message as "
|
||||
f"{RESPONSE_OPEN} blocks. Never write {RESPONSE_OPEN} yourself. When no "
|
||||
"tool is needed, answer normally without any call block.",
|
||||
]
|
||||
if not parallel_allowed(payload):
|
||||
parts.append("Emit at most one tool call per reply.")
|
||||
if directive:
|
||||
parts.append(directive)
|
||||
return "\n\n".join(parts)
|
||||
|
||||
|
||||
def _text_of(content) -> str:
|
||||
if content is None:
|
||||
return ""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
return "\n".join(
|
||||
str(part.get("text", ""))
|
||||
for part in content
|
||||
if isinstance(part, dict) and part.get("type") == "text"
|
||||
)
|
||||
return str(content)
|
||||
|
||||
|
||||
def _arguments_object(raw):
|
||||
if isinstance(raw, dict):
|
||||
return raw
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
parsed = json.loads(raw) if raw.strip() else {}
|
||||
except ValueError:
|
||||
return raw
|
||||
return parsed
|
||||
return {}
|
||||
|
||||
|
||||
def _call_block(name: str, arguments) -> str:
|
||||
body = json.dumps({"name": name, "arguments": arguments}, ensure_ascii=False)
|
||||
return f"{CALL_OPEN}\n{body}\n{CALL_CLOSE}"
|
||||
|
||||
|
||||
def _response_block(call_id: str, name: str, content) -> str:
|
||||
body = json.dumps(
|
||||
{"tool_call_id": call_id, "name": name, "content": _text_of(content)},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
return f"{RESPONSE_OPEN}\n{body}\n{RESPONSE_CLOSE}"
|
||||
|
||||
|
||||
def _rewrite_history(messages: list) -> list:
|
||||
names: dict[str, str] = {}
|
||||
rewritten: list = []
|
||||
for message in messages:
|
||||
if not isinstance(message, dict):
|
||||
rewritten.append(message)
|
||||
continue
|
||||
role = message.get("role")
|
||||
if role == "assistant" and message.get("tool_calls"):
|
||||
blocks = []
|
||||
for call in message["tool_calls"]:
|
||||
function = (call or {}).get("function") or {}
|
||||
name = str(function.get("name") or "")
|
||||
call_id = str((call or {}).get("id") or "")
|
||||
if call_id:
|
||||
names[call_id] = name
|
||||
blocks.append(_call_block(name, _arguments_object(function.get("arguments"))))
|
||||
text = _text_of(message.get("content")).strip()
|
||||
converted = {k: v for k, v in message.items() if k != "tool_calls"}
|
||||
converted["content"] = "\n".join(([text] if text else []) + blocks)
|
||||
rewritten.append(converted)
|
||||
elif role in ("tool", "function"):
|
||||
call_id = str(message.get("tool_call_id") or "")
|
||||
name = str(message.get("name") or names.get(call_id) or "")
|
||||
block = _response_block(call_id, name, message.get("content"))
|
||||
previous = rewritten[-1] if rewritten else None
|
||||
if (
|
||||
isinstance(previous, dict)
|
||||
and previous.get("_emulated_tool_results")
|
||||
):
|
||||
previous["content"] += "\n" + block
|
||||
else:
|
||||
rewritten.append(
|
||||
{"role": "user", "content": block, "_emulated_tool_results": True}
|
||||
)
|
||||
else:
|
||||
rewritten.append(message)
|
||||
for message in rewritten:
|
||||
if isinstance(message, dict):
|
||||
message.pop("_emulated_tool_results", None)
|
||||
return rewritten
|
||||
|
||||
|
||||
def _with_system_instruction(messages: list, instruction: str) -> list:
|
||||
if not instruction:
|
||||
return messages
|
||||
if messages and isinstance(messages[0], dict) and messages[0].get("role") == "system":
|
||||
first = dict(messages[0])
|
||||
content = first.get("content")
|
||||
if isinstance(content, list):
|
||||
first["content"] = list(content) + [{"type": "text", "text": instruction}]
|
||||
else:
|
||||
existing = _text_of(content)
|
||||
first["content"] = (existing + "\n\n" if existing else "") + instruction
|
||||
return [first] + messages[1:]
|
||||
return [{"role": "system", "content": instruction}] + messages
|
||||
|
||||
|
||||
def _with_stop(payload: dict) -> None:
|
||||
stop = payload.get("stop")
|
||||
sequences = [stop] if isinstance(stop, str) else list(stop or [])
|
||||
if RESPONSE_OPEN in sequences or len(sequences) >= MAX_STOP_SEQUENCES:
|
||||
return
|
||||
payload["stop"] = sequences + [RESPONSE_OPEN]
|
||||
|
||||
|
||||
def emulate_request(payload: dict) -> dict:
|
||||
specs = _function_specs(payload)
|
||||
history = payload.get("messages") or []
|
||||
has_tool_history = any(
|
||||
isinstance(m, dict) and (m.get("role") in ("tool", "function") or m.get("tool_calls"))
|
||||
for m in history
|
||||
)
|
||||
if not specs and not has_tool_history:
|
||||
return payload
|
||||
converted = {k: copy.copy(v) for k, v in payload.items() if k not in REMOVED_PAYLOAD_KEYS}
|
||||
messages = _rewrite_history(list(history))
|
||||
converted["messages"] = _with_system_instruction(messages, build_instruction(payload, specs))
|
||||
if specs:
|
||||
_with_stop(converted)
|
||||
return converted
|
||||
|
||||
|
||||
def _strip_fence(text: str) -> str:
|
||||
stripped = text.strip()
|
||||
match = _FENCE.match(stripped)
|
||||
return match.group(1).strip() if match else stripped
|
||||
|
||||
|
||||
def _call_objects(raw: str) -> list:
|
||||
try:
|
||||
parsed = json.loads(_strip_fence(raw))
|
||||
except ValueError:
|
||||
return []
|
||||
items = parsed if isinstance(parsed, list) else [parsed]
|
||||
return [item for item in items if isinstance(item, dict)]
|
||||
|
||||
|
||||
def _normalize_call(item: dict, declared: dict[str, dict]) -> tuple[Optional[dict], Optional[str]]:
|
||||
function = item.get("function") if isinstance(item.get("function"), dict) else item
|
||||
name = str(function.get("name") or "").strip()
|
||||
if name not in declared:
|
||||
return None, f"unknown tool {name!r}"
|
||||
arguments = function.get("arguments", function.get("parameters", {}))
|
||||
if isinstance(arguments, str):
|
||||
try:
|
||||
arguments = json.loads(arguments) if arguments.strip() else {}
|
||||
except ValueError:
|
||||
return None, f"arguments of {name!r} are not valid JSON"
|
||||
if not isinstance(arguments, dict):
|
||||
return None, f"arguments of {name!r} are not an object"
|
||||
schema = declared[name].get("parameters")
|
||||
required = schema.get("required") if isinstance(schema, dict) else None
|
||||
missing = [key for key in (required or []) if key not in arguments]
|
||||
if missing:
|
||||
return None, f"call to {name!r} misses required arguments: {', '.join(missing)}"
|
||||
return (
|
||||
{
|
||||
"id": "call_" + uuid.uuid4().hex[:24],
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": name,
|
||||
"arguments": json.dumps(arguments, ensure_ascii=False),
|
||||
},
|
||||
},
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def parse_tool_calls(
|
||||
text: str, declared: dict[str, dict], allow_parallel: bool = True
|
||||
) -> tuple[str, list[dict], list[str]]:
|
||||
if CALL_OPEN not in (text or ""):
|
||||
return text or "", [], []
|
||||
calls: list[dict] = []
|
||||
warnings: list[str] = []
|
||||
kept: list[str] = []
|
||||
cursor = 0
|
||||
for match in _CALL_BLOCK.finditer(text):
|
||||
kept.append(text[cursor : match.start()])
|
||||
cursor = match.end()
|
||||
items = _call_objects(match.group(1))
|
||||
if not items:
|
||||
warnings.append("a tool_call block did not contain valid JSON")
|
||||
kept.append(match.group(0))
|
||||
continue
|
||||
for item in items:
|
||||
call, problem = _normalize_call(item, declared)
|
||||
if call is None:
|
||||
warnings.append(problem or "invalid tool call")
|
||||
kept.append(match.group(0))
|
||||
continue
|
||||
calls.append(call)
|
||||
kept.append(text[cursor:])
|
||||
if not allow_parallel and len(calls) > 1:
|
||||
warnings.append(f"parallel calls disabled: dropped {len(calls) - 1} extra call(s)")
|
||||
calls = calls[:1]
|
||||
if len(calls) > MAX_CALLS_PER_TURN:
|
||||
warnings.append(f"dropped {len(calls) - MAX_CALLS_PER_TURN} call(s) over the per-turn cap")
|
||||
calls = calls[:MAX_CALLS_PER_TURN]
|
||||
return "".join(kept).strip(), calls, warnings
|
||||
|
||||
|
||||
def convert_completion(
|
||||
completion: dict, declared: dict[str, dict], allow_parallel: bool = True
|
||||
) -> tuple[dict, list[str]]:
|
||||
warnings: list[str] = []
|
||||
for choice in completion.get("choices") or []:
|
||||
message = choice.get("message") if isinstance(choice, dict) else None
|
||||
if not isinstance(message, dict) or message.get("tool_calls"):
|
||||
continue
|
||||
text = message.get("content")
|
||||
if not isinstance(text, str):
|
||||
continue
|
||||
remaining, calls, found = parse_tool_calls(text, declared, allow_parallel)
|
||||
warnings.extend(found)
|
||||
if calls:
|
||||
message["tool_calls"] = calls
|
||||
message["content"] = remaining or None
|
||||
choice["finish_reason"] = "tool_calls"
|
||||
return completion, warnings
|
||||
|
||||
|
||||
def _holdback(text: str) -> int:
|
||||
longest = min(len(CALL_OPEN) - 1, len(text))
|
||||
for size in range(longest, 0, -1):
|
||||
if CALL_OPEN.startswith(text[-size:]):
|
||||
return size
|
||||
return 0
|
||||
|
||||
|
||||
class ToolCallStream:
|
||||
def __init__(self, declared: dict[str, dict], allow_parallel: bool = True):
|
||||
self.declared = declared
|
||||
self.allow_parallel = allow_parallel
|
||||
self.warnings: list[str] = []
|
||||
self._base: dict = {}
|
||||
self._pending = ""
|
||||
self._call_text = ""
|
||||
self._in_call = False
|
||||
self._flushed = False
|
||||
self._saw_calls = False
|
||||
|
||||
def _chunk(self, delta: dict, finish_reason=None) -> dict:
|
||||
chunk = {k: v for k, v in self._base.items() if k not in ("choices", "usage")}
|
||||
chunk["choices"] = [{"index": 0, "delta": delta, "finish_reason": finish_reason}]
|
||||
return chunk
|
||||
|
||||
def _content_chunk(self, text: str) -> list[dict]:
|
||||
return [self._chunk({"content": text})] if text else []
|
||||
|
||||
def _consume(self, text: str) -> list[dict]:
|
||||
if self._in_call:
|
||||
self._call_text += text
|
||||
return []
|
||||
self._pending += text
|
||||
index = self._pending.find(CALL_OPEN)
|
||||
if index >= 0:
|
||||
out = self._content_chunk(self._pending[:index])
|
||||
self._call_text = self._pending[index:]
|
||||
self._pending = ""
|
||||
self._in_call = True
|
||||
return out
|
||||
keep = _holdback(self._pending)
|
||||
emit = self._pending[: len(self._pending) - keep]
|
||||
self._pending = self._pending[len(self._pending) - keep :]
|
||||
return self._content_chunk(emit)
|
||||
|
||||
def flush(self) -> list[dict]:
|
||||
if self._flushed:
|
||||
return []
|
||||
self._flushed = True
|
||||
if not self._in_call:
|
||||
text, self._pending = self._pending, ""
|
||||
return self._content_chunk(text)
|
||||
remaining, calls, found = parse_tool_calls(
|
||||
self._call_text, self.declared, self.allow_parallel
|
||||
)
|
||||
self.warnings.extend(found)
|
||||
out = self._content_chunk(remaining)
|
||||
if calls:
|
||||
self._saw_calls = True
|
||||
out.append(
|
||||
self._chunk(
|
||||
{
|
||||
"tool_calls": [
|
||||
{"index": i, **call} for i, call in enumerate(calls)
|
||||
]
|
||||
}
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
def feed(self, chunk: dict) -> list[dict]:
|
||||
choices = chunk.get("choices") or []
|
||||
if not choices:
|
||||
return [chunk]
|
||||
self._base = chunk
|
||||
choice = choices[0]
|
||||
delta = choice.get("delta") or {}
|
||||
out: list[dict] = []
|
||||
text = delta.get("content")
|
||||
passthrough = {k: v for k, v in delta.items() if k != "content"}
|
||||
if isinstance(text, str) and text:
|
||||
out.extend(self._consume(text))
|
||||
finish = choice.get("finish_reason")
|
||||
if passthrough:
|
||||
out.append(self._chunk(passthrough))
|
||||
if finish is not None:
|
||||
out.extend(self.flush())
|
||||
out.append(
|
||||
self._chunk(
|
||||
{}, "tool_calls" if self._saw_calls and finish == "stop" else finish
|
||||
)
|
||||
)
|
||||
if chunk.get("usage"):
|
||||
out.append({**{k: v for k, v in chunk.items() if k != "choices"}, "choices": []})
|
||||
return out
|
||||
|
||||
def finish(self) -> list[dict]:
|
||||
return self.flush()
|
||||
@@ -13,6 +13,7 @@ def _instantiate_all():
|
||||
from devplacepy.services.bot.service import BotsService
|
||||
from devplacepy.services.containers.service import ContainerService
|
||||
from devplacepy.services.devii import DeviiService
|
||||
from devplacepy.services.giphy.service import GiphyService
|
||||
from devplacepy.services.gitea.service import IssueTrackerService
|
||||
from devplacepy.services.news import NewsService
|
||||
from devplacepy.services.openai_gateway import GatewayService
|
||||
@@ -28,6 +29,7 @@ def _instantiate_all():
|
||||
ContainerService(),
|
||||
DeviiService(),
|
||||
IssueTrackerService(),
|
||||
GiphyService(),
|
||||
NewsService(),
|
||||
GatewayService(),
|
||||
PubSubService(),
|
||||
|
||||
@@ -6,7 +6,7 @@ import base64
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from devplacepy.database import db
|
||||
from devplacepy.database import db, listed_sql
|
||||
from devplacepy.database.core import _in_clause
|
||||
from devplacepy.database.sync import EXPORT_CHANGED
|
||||
from devplacepy.services.sync.normalize import instant_iso
|
||||
@@ -61,6 +61,7 @@ class Exporter:
|
||||
kind = ""
|
||||
table = ""
|
||||
extra_where = ""
|
||||
listed = False
|
||||
|
||||
def changed(self) -> str:
|
||||
return EXPORT_CHANGED[self.table]
|
||||
@@ -85,6 +86,8 @@ class Exporter:
|
||||
params: dict = {"kind": self.kind, "n": limit + 1}
|
||||
if self.extra_where:
|
||||
clauses.append(self.extra_where)
|
||||
if self.listed:
|
||||
clauses.append(listed_sql(self.table, "t"))
|
||||
if since:
|
||||
clauses.append(f"{changed} > :since")
|
||||
params["since"] = since
|
||||
@@ -133,6 +136,7 @@ class ProjectExporter(Exporter):
|
||||
kind = KIND_PROJECTS
|
||||
table = "projects"
|
||||
extra_where = "COALESCE(is_private, 0) = 0"
|
||||
listed = True
|
||||
|
||||
def record(self, row: dict, context: dict) -> dict:
|
||||
return {
|
||||
@@ -154,6 +158,7 @@ class ProjectExporter(Exporter):
|
||||
class PostExporter(Exporter):
|
||||
kind = KIND_POSTS
|
||||
table = "posts"
|
||||
listed = True
|
||||
|
||||
def prepare(self, rows: list[dict]) -> dict:
|
||||
project_uids = [row["project_uid"] for row in rows if row.get("project_uid")]
|
||||
|
||||
@@ -516,6 +516,23 @@ dp-chat[mode="embed"] {
|
||||
margin-bottom: var(--space-sm);
|
||||
}
|
||||
|
||||
.messages-input-area dp-gif-picker {
|
||||
flex-shrink: 0;
|
||||
align-items: flex-end;
|
||||
}
|
||||
|
||||
.messages-input-area .gif-picker-wrapper {
|
||||
left: auto;
|
||||
right: 0;
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.messages-input-area .gif-picker-wrapper {
|
||||
left: var(--space-sm);
|
||||
right: var(--space-sm);
|
||||
}
|
||||
}
|
||||
|
||||
.messages-send-btn {
|
||||
flex-shrink: 0;
|
||||
width: 36px;
|
||||
@@ -752,3 +769,24 @@ dp-chat[mode="embed"] {
|
||||
.message-report-btn:focus {
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
.message-delete-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--space-xs);
|
||||
margin-top: var(--space-xs);
|
||||
margin-left: var(--space-sm);
|
||||
padding: 0;
|
||||
background: none;
|
||||
border: none;
|
||||
color: inherit;
|
||||
font-size: 0.7rem;
|
||||
opacity: 0.8;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.message-delete-btn:hover,
|
||||
.message-delete-btn:focus {
|
||||
opacity: 1;
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
@@ -143,6 +143,58 @@
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
/* Overflow action menu (dp-action-menu): a popover panel holding the
|
||||
secondary actions as real markup, so they work with and without JS. */
|
||||
.action-menu-panel {
|
||||
position: fixed;
|
||||
inset: auto;
|
||||
z-index: var(--z-chrome-menu);
|
||||
margin: 0;
|
||||
min-width: 180px;
|
||||
max-width: 280px;
|
||||
padding: 0.25rem;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
background: var(--bg-card);
|
||||
color: var(--text-primary);
|
||||
box-shadow: var(--shadow-lg);
|
||||
overflow: auto;
|
||||
}
|
||||
|
||||
.action-menu-panel:not(:popover-open) {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.action-menu-panel > form,
|
||||
.action-menu-panel > a,
|
||||
.action-menu-panel > button {
|
||||
display: block;
|
||||
width: 100%;
|
||||
margin: 0 0 0.125rem;
|
||||
padding: 0.5rem 0.625rem;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
background: none;
|
||||
color: var(--text-primary);
|
||||
font-family: inherit;
|
||||
font-size: 0.875rem;
|
||||
text-align: left;
|
||||
text-decoration: none;
|
||||
cursor: pointer;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.action-menu-panel > form:hover,
|
||||
.action-menu-panel > a:hover,
|
||||
.action-menu-panel > button:hover,
|
||||
.action-menu-panel > form:focus-within {
|
||||
background: var(--bg-card-hover);
|
||||
}
|
||||
|
||||
.action-menu-panel button[data-confirm-danger] {
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
dp-content {
|
||||
display: block;
|
||||
}
|
||||
@@ -159,6 +211,144 @@ dp-upload[hidden] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
dp-gif-picker {
|
||||
position: relative;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
dp-gif-picker[hidden] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.gif-toggle-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
min-width: 2.25rem;
|
||||
padding: var(--space-xs) var(--space-sm);
|
||||
font-size: 0.75rem;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.04em;
|
||||
color: var(--text-muted);
|
||||
background: none;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.gif-toggle-btn:hover,
|
||||
.gif-toggle-btn[aria-expanded="true"] {
|
||||
color: var(--text-primary);
|
||||
border-color: var(--border-light);
|
||||
}
|
||||
|
||||
.gif-picker-wrapper {
|
||||
position: absolute;
|
||||
bottom: calc(100% + var(--space-xs));
|
||||
left: 0;
|
||||
z-index: var(--z-popover);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--space-sm);
|
||||
width: min(22rem, 92vw);
|
||||
max-height: 26rem;
|
||||
padding: var(--space-sm);
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
box-shadow: var(--shadow-lg);
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.gif-picker-wrapper[hidden] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.gif-picker-search {
|
||||
width: 100%;
|
||||
padding: var(--space-xs) var(--space-sm);
|
||||
font-size: 0.875rem;
|
||||
color: var(--text-primary);
|
||||
background: var(--bg-primary);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
}
|
||||
|
||||
.gif-picker-status {
|
||||
margin: 0;
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
.gif-picker-status[hidden] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.gif-picker-status.error {
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
.gif-picker-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: var(--space-xs);
|
||||
min-height: 0;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.gif-picker-item {
|
||||
min-width: 0;
|
||||
padding: 0;
|
||||
background: none;
|
||||
border: 1px solid transparent;
|
||||
border-radius: var(--radius);
|
||||
cursor: pointer;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.gif-picker-item:hover,
|
||||
.gif-picker-item:focus-visible {
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
.gif-picker-item.busy {
|
||||
opacity: 0.5;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.gif-picker-item img {
|
||||
display: block;
|
||||
width: 100%;
|
||||
height: 6rem;
|
||||
object-fit: cover;
|
||||
}
|
||||
|
||||
.gif-picker-credit {
|
||||
margin: 0;
|
||||
font-size: 0.6875rem;
|
||||
color: var(--text-muted);
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.composer-attachments-actions {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
gap: var(--space-sm);
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.gif-picker-wrapper {
|
||||
position: fixed;
|
||||
left: var(--space-sm);
|
||||
right: var(--space-sm);
|
||||
bottom: var(--space-sm);
|
||||
width: auto;
|
||||
max-height: 70dvh;
|
||||
}
|
||||
}
|
||||
|
||||
.dp-upload-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
|
||||
@@ -512,3 +512,4 @@
|
||||
pointer-events: none;
|
||||
animation: engagement-busy-pulse 0.9s ease-in-out infinite;
|
||||
}
|
||||
|
||||
|
||||
@@ -69,6 +69,31 @@
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.gw-badge-ok {
|
||||
color: var(--success);
|
||||
border-color: var(--success);
|
||||
}
|
||||
|
||||
.gw-badge-bad {
|
||||
color: var(--danger);
|
||||
border-color: var(--danger);
|
||||
}
|
||||
|
||||
.gw-row-bad {
|
||||
background: rgba(var(--danger-rgb), 0.08);
|
||||
}
|
||||
|
||||
.gw-health-error {
|
||||
margin-top: 0.25rem;
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.gw-test-image {
|
||||
max-width: 100%;
|
||||
border-radius: var(--radius);
|
||||
}
|
||||
|
||||
.gw-actions {
|
||||
text-align: right;
|
||||
white-space: nowrap;
|
||||
|
||||
@@ -594,9 +594,6 @@
|
||||
padding-top: 1rem;
|
||||
border-top: 1px solid var(--border);
|
||||
}
|
||||
.project-actions-overflow {
|
||||
display: none;
|
||||
}
|
||||
.project-star-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
|
||||
@@ -5,6 +5,7 @@ import { Http } from "./Http.js";
|
||||
class AwardGiver {
|
||||
constructor() {
|
||||
this.btn = document.querySelector("[data-award-give]");
|
||||
this.username = this.btn ? this.btn.dataset.username : "";
|
||||
this.modal = document.getElementById("award-give-modal");
|
||||
if (!this.btn || !this.modal) return;
|
||||
this.form = this.modal.querySelector("form");
|
||||
@@ -37,7 +38,7 @@ class AwardGiver {
|
||||
|
||||
async submit(event) {
|
||||
event.preventDefault();
|
||||
const username = this.btn.dataset.username;
|
||||
const username = this.username;
|
||||
const description = (this.textarea?.value || "").trim();
|
||||
if (!description) return;
|
||||
const submitBtn = this.form.querySelector('button[type="submit"]');
|
||||
|
||||
@@ -14,7 +14,7 @@ This file documents JS module organization, custom web components, and shared fr
|
||||
## Custom web components (`static/js/components/`)
|
||||
|
||||
Self-contained, presentational UI is built as custom elements with the `dp-` prefix:
|
||||
`dp-avatar`, `dp-code`, `dp-content`, `dp-title`, `dp-toast`, `dp-dialog`, `dp-context-menu`, `dp-upload`, `dp-lightbox`, `dp-chat`, `dp-quiz-player`, `dp-quiz-builder`.
|
||||
`dp-avatar`, `dp-code`, `dp-content`, `dp-title`, `dp-toast`, `dp-dialog`, `dp-context-menu`, `dp-upload`, `dp-gif-picker`, `dp-lightbox`, `dp-chat`, `dp-quiz-player`, `dp-quiz-builder`.
|
||||
|
||||
`dp-quiz-player` (`AppQuizPlayer.js`) and `dp-quiz-builder` (`AppQuizBuilder.js`) both **adopt** server-rendered markup rather than building it (the `dp-chat` `mode="page"` pattern): the player intercepts each question's real `<form method="post">` through `Http.sendForm`, renders the graded result in place, shows a `Checking…` state while a `free_text` grade is in flight, ticks the remaining time from `Format.duration`, and advances to the next slide; the builder toggles the kind-specific field groups from the question-kind picker and submits the add-question form through `Http.sendForm`. Neither re-renders a prompt (the server already rendered it through `render_content`), neither opens a WebSocket, and neither implements a fetch wrapper, a poller or a dialog. Removing the JS leaves a fully working no-JS quiz.
|
||||
|
||||
@@ -101,6 +101,10 @@ File validation: max 5MB, allowed extensions: `.png`, `.jpg`, `.jpeg`, `.gif`, `
|
||||
|
||||
Media is served **inline** (not forced-download) for known-safe types only. The set `INLINE_MEDIA_EXTENSIONS` in `main.py` (`UploadStaticFiles`) and the matching `map $uri $upload_disposition` in `nginx/nginx.conf.template` must stay in sync: images/video/audio -> `inline` (so `<video>` plays and seeks via Range), everything else -> `attachment`. SVG is deliberately excluded from both (stored-XSS defense). `ContentRenderer.js` embeds direct video URLs typed into content via `videoExtRe`, mirroring its image handling.
|
||||
|
||||
### The GIF picker (`dp-gif-picker`)
|
||||
|
||||
`AppGifPicker.js` is the one "GIF" button: a toggle plus a searchable GIPHY panel (Escape / outside click close it, a bottom sheet at 480px and below). It uses `Http.getJson`/`Http.postJson` (never raw `fetch`), asks `/giphy/status` once per page (`AppGifPicker.availability`, shared promise) and hides itself when GIPHY is not configured. Picking a GIF POSTs `/giphy/attach` and dispatches a bubbling `dp-gif:attached` carrying the attachment record; a `dp-upload` in `attachment` mode in the same `<form>` listens on the form (`addAttached`) and adds the uid to its hidden `attachment_uids`, so a GIF is an attachment exactly like an uploaded file and is never written into the textarea. Place `<dp-gif-picker>` beside the form's `dp-upload` in the markup (composer, `_comment_form.html`, `messages.html`, the `dp-chat` embed skeleton); there is no JS injection. Server rules: `devplacepy/services/giphy/CLAUDE.md`.
|
||||
|
||||
### The upload button (`dp-upload`)
|
||||
|
||||
Every file-upload UI is the one custom element `dp-upload` (`static/js/components/AppUpload.js`); it replaced `AttachmentUploader.js` (deleted). It shows a clean button with a count badge + removable filename chips (no thumbnail grid), and runs in one of three `mode`s:
|
||||
|
||||
@@ -1,32 +0,0 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
export class ProjectActionsMenu {
|
||||
constructor(root) {
|
||||
this.root = root;
|
||||
this.trigger = root.querySelector(".project-actions-more");
|
||||
this.overflow = root.querySelector(".project-actions-overflow");
|
||||
if (!this.trigger || !this.overflow) {
|
||||
return;
|
||||
}
|
||||
if (!this.overflow.querySelector("[data-menu-action]")) {
|
||||
this.trigger.hidden = true;
|
||||
return;
|
||||
}
|
||||
this.trigger.addEventListener("click", (event) => this.open(event));
|
||||
}
|
||||
|
||||
open(event) {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
const items = Array.from(
|
||||
this.overflow.querySelectorAll("[data-menu-action]")
|
||||
).map((element) => ({
|
||||
icon: element.dataset.menuIcon || "",
|
||||
label: element.dataset.menuLabel || element.textContent.trim(),
|
||||
danger: element.hasAttribute("data-confirm-danger"),
|
||||
onSelect: () => element.click(),
|
||||
}));
|
||||
const rect = this.trigger.getBoundingClientRect();
|
||||
window.app.contextMenu.open(rect.left, rect.bottom + 4, items);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
import { Component } from "./Component.js";
|
||||
|
||||
// Generic overflow menu: a visible trigger plus the actions that do not fit
|
||||
// next to the primary ones. The menu markup lives in the light DOM (a popover)
|
||||
// instead of a hidden container, so the actions stay real buttons, links and
|
||||
// forms: no JavaScript still means working markup, and every handler that
|
||||
// listens on the document (confirm, modal, form submit) keeps seeing them.
|
||||
export class AppActionMenu extends Component {
|
||||
connectedCallback() {
|
||||
if (this._built) return;
|
||||
this._built = true;
|
||||
this.panel = this.querySelector("[data-action-menu-panel]");
|
||||
this.trigger = this.querySelector("[data-action-menu-trigger]");
|
||||
if (!this.panel || !this.trigger) return;
|
||||
if (!this.trigger.id) this.trigger.id = `action-menu-${Math.random().toString(36).slice(2, 9)}`;
|
||||
this.panel.id = this.panel.id || `${this.trigger.id}-panel`;
|
||||
// A [popover] panel is display:none until it is shown, so the hidden
|
||||
// attribute is only the no-JS fallback and has to go once we own it.
|
||||
this.panel.hidden = false;
|
||||
this.trigger.setAttribute("aria-controls", this.panel.id);
|
||||
this.trigger.hidden = !this.actions().length;
|
||||
this.trigger.addEventListener("click", (event) => this.toggle(event));
|
||||
this.panel.addEventListener("keydown", (event) => {
|
||||
if (event.key === "Escape") this.close();
|
||||
});
|
||||
// Keep the panel from lingering over whatever the action opened (the
|
||||
// confirm dialog): click and submit both count, and the panel closes
|
||||
// before the dialog is shown.
|
||||
// Capture phase: the document confirm handler calls
|
||||
// stopImmediatePropagation when it intercepts the click, so a bubbling
|
||||
// listener never runs and the panel would stay open over the dialog
|
||||
// backdrop.
|
||||
this.panel.addEventListener(
|
||||
"click",
|
||||
(event) => {
|
||||
if (event.target.closest("[data-menu-action]")) this.close();
|
||||
},
|
||||
true
|
||||
);
|
||||
this.panel.addEventListener("submit", () => this.close(), true);
|
||||
this.panel.addEventListener("toggle", (event) => {
|
||||
this.trigger.setAttribute("aria-expanded", String(event.newState === "open"));
|
||||
});
|
||||
// The popover API reports closing through toggle; without it nothing
|
||||
// closes the panel except our own handlers, so mark it as a plain block.
|
||||
if (typeof this.panel.showPopover !== "function") {
|
||||
this.panel.removeAttribute("popover");
|
||||
this.panel.hidden = true;
|
||||
}
|
||||
}
|
||||
|
||||
actions() {
|
||||
return Array.from(this.panel.querySelectorAll("[data-menu-action]")).filter(
|
||||
(element) => !element.hidden
|
||||
);
|
||||
}
|
||||
|
||||
toggle(event) {
|
||||
if (event) {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
}
|
||||
if (this.trigger.getAttribute("aria-expanded") === "true") this.close();
|
||||
else this.open();
|
||||
}
|
||||
|
||||
open(event) {
|
||||
if (event) {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
}
|
||||
if (!this.actions().length) return;
|
||||
if (typeof this.panel.showPopover === "function") {
|
||||
try {
|
||||
this.panel.showPopover();
|
||||
} catch (error) {
|
||||
this.panel.removeAttribute("popover");
|
||||
this.panel.hidden = false;
|
||||
}
|
||||
} else {
|
||||
this.panel.hidden = false;
|
||||
}
|
||||
this.trigger.setAttribute("aria-expanded", "true");
|
||||
}
|
||||
|
||||
close() {
|
||||
if (typeof this.panel.hidePopover === "function" && this.panel.matches(":popover-open")) {
|
||||
this.panel.hidePopover();
|
||||
} else if (!this.panel.hasAttribute("popover")) {
|
||||
this.panel.hidden = true;
|
||||
}
|
||||
this.trigger.setAttribute("aria-expanded", "false");
|
||||
}
|
||||
}
|
||||
|
||||
customElements.define("dp-action-menu", AppActionMenu);
|
||||
@@ -99,6 +99,7 @@ export class AppChat extends Component {
|
||||
this._startScrollWatcher();
|
||||
this._ensureJumpButton();
|
||||
this._bindActiveConversation();
|
||||
this._bindDelete();
|
||||
this._connect();
|
||||
this._initPresence();
|
||||
this._scrollThreadToEnd();
|
||||
@@ -132,6 +133,7 @@ export class AppChat extends Component {
|
||||
_readConfig() {
|
||||
this.mode = this.attr("mode", "page");
|
||||
this.selfUid = this.attr("self-uid");
|
||||
this.selfIsAdmin = this.attr("self-admin", "false") === "true";
|
||||
this.withUid = this.attr("with-uid") || null;
|
||||
this.conversationsUrl = this.attr("conversations-url", "/messages/conversations");
|
||||
this.searchUrl = this.attr("search-url", "/messages/search");
|
||||
@@ -239,7 +241,7 @@ export class AppChat extends Component {
|
||||
sendSpinner.setAttribute("aria-hidden", "true");
|
||||
this.sendBtn.append(sendIcon, sendSpinner);
|
||||
|
||||
this.form.append(this.input, this.upload, this.sendBtn);
|
||||
this.form.append(this.input, this.upload, document.createElement("dp-gif-picker"), this.sendBtn);
|
||||
this.main.append(header, this.thread, this.form);
|
||||
this.appendChild(this.main);
|
||||
}
|
||||
@@ -501,6 +503,7 @@ export class AppChat extends Component {
|
||||
receipt.hidden = true;
|
||||
receipt.innerHTML = "✓✓";
|
||||
bubble.appendChild(receipt);
|
||||
if (this.selfIsAdmin && uid) bubble.appendChild(this._deleteButton(uid));
|
||||
} else if (uid) {
|
||||
bubble.appendChild(this._reportButton(uid));
|
||||
}
|
||||
@@ -527,6 +530,61 @@ export class AppChat extends Component {
|
||||
return btn;
|
||||
}
|
||||
|
||||
_deleteButton(uid) {
|
||||
const btn = document.createElement("button");
|
||||
btn.type = "button";
|
||||
btn.className = "message-delete-btn";
|
||||
btn.dataset.deleteUid = uid;
|
||||
btn.setAttribute("aria-label", "Delete this message");
|
||||
btn.title = "Delete";
|
||||
const icon = document.createElement("span");
|
||||
icon.className = "icon";
|
||||
icon.setAttribute("aria-hidden", "true");
|
||||
icon.textContent = "\u{1F5D1}";
|
||||
const label = document.createElement("span");
|
||||
label.className = "label";
|
||||
label.textContent = " Delete";
|
||||
btn.append(icon, label);
|
||||
return btn;
|
||||
}
|
||||
|
||||
_bindDelete() {
|
||||
this.addEventListener("click", (event) => {
|
||||
const btn = event.target.closest(".message-delete-btn");
|
||||
if (!btn || !this.contains(btn)) return;
|
||||
event.preventDefault();
|
||||
this._confirmDelete(btn.dataset.deleteUid);
|
||||
});
|
||||
}
|
||||
|
||||
async _confirmDelete(uid) {
|
||||
const dialog = window.app && window.app.dialog;
|
||||
const message = "Delete this message for everyone in the conversation?";
|
||||
const ok = dialog
|
||||
? await dialog.confirm({ title: "Delete message", message, confirmLabel: "Delete", danger: true })
|
||||
: window.confirm(message);
|
||||
if (!ok) return;
|
||||
try {
|
||||
await Http.send(`/messages/${encodeURIComponent(uid)}/delete`);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
this._removeBubble(uid);
|
||||
}
|
||||
|
||||
_removeBubble(uid) {
|
||||
if (!uid || !this.thread) return;
|
||||
const bubble = this.thread.querySelector(`.message-bubble[data-msg-uid="${uid}"]`);
|
||||
if (!bubble) return;
|
||||
const next = bubble.nextElementSibling;
|
||||
bubble.remove();
|
||||
this._normalizeDateSeparators();
|
||||
if (next && next.classList.contains("message-bubble")) {
|
||||
next.classList.toggle("grouped", shouldGroup(this._previousMetaBefore(next), this._bubbleMeta(next)));
|
||||
}
|
||||
this._refreshConversations();
|
||||
}
|
||||
|
||||
_contentSource(el) {
|
||||
if (!el) return "";
|
||||
if (el.dataset && Object.prototype.hasOwnProperty.call(el.dataset, "source")) {
|
||||
@@ -744,6 +802,9 @@ export class AppChat extends Component {
|
||||
case "read":
|
||||
if (frame.by_uid === this.withUid) this._markReceipts();
|
||||
break;
|
||||
case "message_deleted":
|
||||
this._removeBubble(frame.uid);
|
||||
break;
|
||||
case "error":
|
||||
this._handleError(frame);
|
||||
break;
|
||||
@@ -828,6 +889,9 @@ export class AppChat extends Component {
|
||||
if (frame.uid && !bubble.classList.contains("mine") && !bubble.querySelector(".message-report-btn")) {
|
||||
bubble.appendChild(this._reportButton(frame.uid));
|
||||
}
|
||||
if (frame.uid && this.selfIsAdmin && bubble.classList.contains("mine") && !bubble.querySelector(".message-delete-btn")) {
|
||||
bubble.appendChild(this._deleteButton(frame.uid));
|
||||
}
|
||||
|
||||
if (frame.ai_pending && !frame.ai_processed) {
|
||||
this._showAiPending(bubble);
|
||||
|
||||
@@ -51,6 +51,13 @@ export class AppDialog extends Component {
|
||||
overlay.addEventListener("click", (e) => {
|
||||
if (e.target === overlay) this.dismiss();
|
||||
});
|
||||
// The tiny backdrop line: anything else opened in the top layer while
|
||||
// the dialog runs (the overflow action menu) is light-dismissed first,
|
||||
// so the same click never reaches the dialog backdrop underneath.
|
||||
overlay.addEventListener("toggle", (event) => {
|
||||
if (event.newState !== "open") return;
|
||||
this.closeOtherPopovers();
|
||||
});
|
||||
overlay.querySelector(".dialog-close").addEventListener("click", () => this.dismiss());
|
||||
this.cancelBtn.addEventListener("click", () => this.dismiss());
|
||||
this.confirmBtn.addEventListener("click", () => this.accept());
|
||||
@@ -113,7 +120,18 @@ export class AppDialog extends Component {
|
||||
}
|
||||
|
||||
this.lastFocus = document.activeElement;
|
||||
this.closeOtherPopovers();
|
||||
this.overlay.classList.add("visible");
|
||||
// Top layer: a popover element paints above every z-index, so a menu
|
||||
// left open above cannot cover the dialog and its backdrop.
|
||||
if (typeof this.overlay.showPopover === "function" && !this.overlay.matches(":popover-open")) {
|
||||
try {
|
||||
this.overlay.showPopover();
|
||||
} catch (err) {
|
||||
// Already open elsewhere or unsupported context: the CSS
|
||||
// overlay still works, simply below the top layer.
|
||||
}
|
||||
}
|
||||
const focusTarget = mode === "prompt" ? this.input : this.confirmBtn;
|
||||
setTimeout(() => {
|
||||
focusTarget.focus();
|
||||
@@ -148,8 +166,22 @@ export class AppDialog extends Component {
|
||||
this.close(this.mode === "prompt" ? null : this.mode === "alert" ? undefined : false);
|
||||
}
|
||||
|
||||
closeOtherPopovers() {
|
||||
for (const el of document.querySelectorAll(":popover-open")) {
|
||||
if (el === this.overlay || !el.matches("[data-action-menu-panel]")) continue;
|
||||
if (typeof el.hidePopover === "function") el.hidePopover();
|
||||
}
|
||||
}
|
||||
|
||||
close(value) {
|
||||
this.overlay.classList.remove("visible");
|
||||
if (this.overlay.matches(":popover-open")) {
|
||||
try {
|
||||
this.overlay.hidePopover();
|
||||
} catch (err) {
|
||||
// Nothing to undo when the dialog never made it to the top layer.
|
||||
}
|
||||
}
|
||||
if (this.lastFocus && this.lastFocus.focus) this.lastFocus.focus();
|
||||
const resolve = this.resolver;
|
||||
this.resolver = null;
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
import { Component } from "./Component.js";
|
||||
import { Http } from "../Http.js";
|
||||
|
||||
const DEBOUNCE_MS = 350;
|
||||
const TOO_MANY_REQUESTS = 429;
|
||||
|
||||
export class AppGifPicker extends Component {
|
||||
static availability(endpoint) {
|
||||
if (!AppGifPicker._availability) {
|
||||
AppGifPicker._availability = Http.getJson(`${endpoint}/status`)
|
||||
.then((data) => Boolean(data && data.configured))
|
||||
.catch(() => false);
|
||||
}
|
||||
return AppGifPicker._availability;
|
||||
}
|
||||
|
||||
connectedCallback() {
|
||||
if (this._built) {
|
||||
return;
|
||||
}
|
||||
this._built = true;
|
||||
this.endpoint = this.attr("endpoint", "/giphy");
|
||||
this.limit = this.intAttr("limit", 24);
|
||||
this.loaded = false;
|
||||
this.timer = null;
|
||||
this.requestId = 0;
|
||||
this.hidden = true;
|
||||
this.build();
|
||||
AppGifPicker.availability(this.endpoint).then((available) => {
|
||||
this.hidden = !available;
|
||||
});
|
||||
}
|
||||
|
||||
disconnectedCallback() {
|
||||
this.unbindDismiss();
|
||||
clearTimeout(this.timer);
|
||||
}
|
||||
|
||||
build() {
|
||||
this.button = document.createElement("button");
|
||||
this.button.type = "button";
|
||||
this.button.className = "gif-toggle-btn";
|
||||
this.button.title = "Add GIF";
|
||||
this.button.setAttribute("aria-label", "Add GIF");
|
||||
this.button.setAttribute("aria-haspopup", "dialog");
|
||||
this.button.setAttribute("aria-expanded", "false");
|
||||
this.button.textContent = "GIF";
|
||||
this.button.addEventListener("click", (event) => {
|
||||
event.preventDefault();
|
||||
this.toggle();
|
||||
});
|
||||
|
||||
this.panel = document.createElement("div");
|
||||
this.panel.className = "gif-picker-wrapper";
|
||||
this.panel.setAttribute("role", "dialog");
|
||||
this.panel.setAttribute("aria-label", "Pick a GIF");
|
||||
this.panel.hidden = true;
|
||||
|
||||
this.searchEl = document.createElement("input");
|
||||
this.searchEl.type = "search";
|
||||
this.searchEl.className = "gif-picker-search";
|
||||
this.searchEl.placeholder = "Search GIPHY";
|
||||
this.searchEl.setAttribute("aria-label", "Search GIPHY");
|
||||
this.searchEl.maxLength = 200;
|
||||
this.searchEl.addEventListener("input", () => this.debounceSearch());
|
||||
|
||||
this.statusEl = document.createElement("p");
|
||||
this.statusEl.className = "gif-picker-status";
|
||||
this.statusEl.setAttribute("role", "status");
|
||||
this.statusEl.hidden = true;
|
||||
|
||||
this.gridEl = document.createElement("div");
|
||||
this.gridEl.className = "gif-picker-grid";
|
||||
|
||||
const credit = document.createElement("p");
|
||||
credit.className = "gif-picker-credit";
|
||||
credit.textContent = "Powered by GIPHY";
|
||||
|
||||
this.panel.append(this.searchEl, this.statusEl, this.gridEl, credit);
|
||||
this.append(this.button, this.panel);
|
||||
|
||||
this.onDocumentClick = (event) => {
|
||||
if (!this.contains(event.target)) this.hide();
|
||||
};
|
||||
this.onKeydown = (event) => {
|
||||
if (event.key !== "Escape") return;
|
||||
this.hide();
|
||||
this.button.focus();
|
||||
};
|
||||
}
|
||||
|
||||
bindDismiss() {
|
||||
document.addEventListener("click", this.onDocumentClick);
|
||||
document.addEventListener("keydown", this.onKeydown);
|
||||
}
|
||||
|
||||
unbindDismiss() {
|
||||
document.removeEventListener("click", this.onDocumentClick);
|
||||
document.removeEventListener("keydown", this.onKeydown);
|
||||
}
|
||||
|
||||
toggle() {
|
||||
if (this.panel.hidden) this.show();
|
||||
else this.hide();
|
||||
}
|
||||
|
||||
show() {
|
||||
this.panel.hidden = false;
|
||||
this.button.setAttribute("aria-expanded", "true");
|
||||
this.bindDismiss();
|
||||
if (!this.loaded) {
|
||||
this.loaded = true;
|
||||
this.loadTrending();
|
||||
}
|
||||
this.searchEl.focus();
|
||||
}
|
||||
|
||||
hide() {
|
||||
this.panel.hidden = true;
|
||||
this.button.setAttribute("aria-expanded", "false");
|
||||
this.unbindDismiss();
|
||||
}
|
||||
|
||||
debounceSearch() {
|
||||
clearTimeout(this.timer);
|
||||
this.timer = setTimeout(() => {
|
||||
const value = this.searchEl.value.trim();
|
||||
if (value) this.loadSearch(value);
|
||||
else this.loadTrending();
|
||||
}, DEBOUNCE_MS);
|
||||
}
|
||||
|
||||
loadTrending() {
|
||||
return this.load(`${this.endpoint}/trending?limit=${this.limit}`, "No trending GIFs");
|
||||
}
|
||||
|
||||
loadSearch(query) {
|
||||
const url = `${this.endpoint}/search?q=${encodeURIComponent(query)}&limit=${this.limit}`;
|
||||
return this.load(url, "No GIFs found");
|
||||
}
|
||||
|
||||
async load(url, emptyMessage) {
|
||||
const requestId = ++this.requestId;
|
||||
this.setStatus("Loading GIFs...");
|
||||
let data;
|
||||
try {
|
||||
data = await Http.getJson(url);
|
||||
} catch (error) {
|
||||
if (requestId !== this.requestId) return;
|
||||
this.gridEl.replaceChildren();
|
||||
const slow = error.status === TOO_MANY_REQUESTS;
|
||||
this.setStatus(slow ? "Too many searches, wait a moment" : "Could not load GIFs", true);
|
||||
return;
|
||||
}
|
||||
if (requestId !== this.requestId) return;
|
||||
this.render(data.results || [], emptyMessage);
|
||||
}
|
||||
|
||||
render(gifs, emptyMessage) {
|
||||
this.gridEl.replaceChildren();
|
||||
if (!gifs.length) {
|
||||
this.setStatus(emptyMessage);
|
||||
return;
|
||||
}
|
||||
this.clearStatus();
|
||||
for (const gif of gifs) {
|
||||
this.gridEl.appendChild(this.renderItem(gif));
|
||||
}
|
||||
}
|
||||
|
||||
renderItem(gif) {
|
||||
const item = document.createElement("button");
|
||||
item.type = "button";
|
||||
item.className = "gif-picker-item";
|
||||
const label = gif.title || "GIF";
|
||||
item.setAttribute("aria-label", `Attach ${label}`);
|
||||
const img = document.createElement("img");
|
||||
img.src = (gif.preview && gif.preview.url) || "";
|
||||
img.alt = "";
|
||||
img.loading = "lazy";
|
||||
item.appendChild(img);
|
||||
item.addEventListener("click", (event) => {
|
||||
event.preventDefault();
|
||||
this.select(gif, item);
|
||||
});
|
||||
return item;
|
||||
}
|
||||
|
||||
async select(gif, item) {
|
||||
if (!gif || !gif.id || item.classList.contains("busy")) return;
|
||||
item.classList.add("busy");
|
||||
this.setStatus("Attaching GIF...");
|
||||
try {
|
||||
const result = await Http.postJson(
|
||||
`${this.endpoint}/attach`,
|
||||
{ gif_id: gif.id, title: gif.title || "" },
|
||||
{ silent: true },
|
||||
);
|
||||
if (!result || !result.uid) {
|
||||
this.setStatus("Could not attach GIF", true);
|
||||
return;
|
||||
}
|
||||
this.dispatchEvent(new CustomEvent("dp-gif:attached", { detail: result, bubbles: true }));
|
||||
this.clearStatus();
|
||||
this.hide();
|
||||
} catch (error) {
|
||||
this.setStatus(error.message || "Could not attach GIF", true);
|
||||
} finally {
|
||||
item.classList.remove("busy");
|
||||
}
|
||||
}
|
||||
|
||||
setStatus(message, error = false) {
|
||||
this.statusEl.textContent = message;
|
||||
this.statusEl.hidden = false;
|
||||
this.statusEl.classList.toggle("error", error);
|
||||
}
|
||||
|
||||
clearStatus() {
|
||||
this.statusEl.textContent = "";
|
||||
this.statusEl.hidden = true;
|
||||
this.statusEl.classList.remove("error");
|
||||
}
|
||||
}
|
||||
|
||||
if (!customElements.get("dp-gif-picker")) {
|
||||
customElements.define("dp-gif-picker", AppGifPicker);
|
||||
}
|
||||
@@ -94,6 +94,9 @@ export class AppUpload extends Component {
|
||||
if (this.boolAttr("paste")) {
|
||||
form.addEventListener("paste", (event) => this.handlePaste(event));
|
||||
}
|
||||
if (this.mode === "attachment") {
|
||||
form.addEventListener("dp-gif:attached", (event) => this.addAttached(event.detail));
|
||||
}
|
||||
form.addEventListener("submit", (event) => {
|
||||
if (this.busyCount > 0) {
|
||||
event.preventDefault();
|
||||
@@ -107,6 +110,21 @@ export class AppUpload extends Component {
|
||||
}
|
||||
}
|
||||
|
||||
addAttached(result) {
|
||||
if (!result || !result.uid || this.items.some((item) => item.uid === result.uid)) {
|
||||
return;
|
||||
}
|
||||
if (this.maxFiles > 0 && this.items.length >= this.maxFiles) {
|
||||
this.showError(`You can attach at most ${this.maxFiles} files.`);
|
||||
return;
|
||||
}
|
||||
this.clearError();
|
||||
this.items.push({ uid: result.uid, name: result.original_filename || result.filename || "GIF" });
|
||||
this.syncUids();
|
||||
this.renderChips();
|
||||
this.updateCount();
|
||||
}
|
||||
|
||||
open() {
|
||||
this.input.click();
|
||||
}
|
||||
|
||||
@@ -5,9 +5,11 @@ import "./AppCode.js";
|
||||
import "./AppContent.js";
|
||||
import "./AppTitle.js";
|
||||
import "./AppUpload.js";
|
||||
import "./AppGifPicker.js";
|
||||
import "./AppToast.js";
|
||||
import "./AppDialog.js";
|
||||
import "./AppContextMenu.js";
|
||||
import "./AppActionMenu.js";
|
||||
import "./AppLightbox.js";
|
||||
import "./AppDocsChat.js";
|
||||
import "./AppIsslop.js";
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user