Update
This commit is contained in:
parent
818568c609
commit
48bb6c2ec2
@ -2,6 +2,8 @@
|
||||
|
||||
This file provides guidance to Claude Code when working with code in this repository. It holds only what applies regardless of which part of the codebase is being touched. Deep, subsystem-specific detail lives in nested `CLAUDE.md` files placed inside the relevant directory - Claude Code auto-loads a nested file only when a file under that directory is read or edited, so the always-loaded cost of this repository stays proportional to this file alone. See "Subsystem map" below for the full list.
|
||||
|
||||
It is a big project, whatever you are implementing, it is probably done before. You should look it up and match the implementation structurely and visually. For inconsistency there is zero tolerance policy. Develop dry, kiss, re-usable code, consistent with existing implementation. Literally always try to find relatable examples before making a modification. If no-example exists, explain to user what is the case and let user decide what to do and how to continue.
|
||||
|
||||
## Project
|
||||
|
||||
DevPlace is a server-rendered social network for developers. FastAPI backend serves Jinja2 templates with pure ES6 module JavaScript on the frontend. SQLite via the `dataset` library (auto-syncs schema). No JS framework, no NPM, no JWT.
|
||||
@ -99,6 +101,7 @@ Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA
|
||||
| `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Max avatars in the feed's live "Online now" panel. |
|
||||
| `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS` | `20` | Grace margin (hysteresis) before an online user is dropped, kills dot/roster flicker at the boundary. |
|
||||
| `DEVPLACE_DATA_DIR` | `<repo>/data` | Single root for ALL runtime/user-generated data OUTSIDE the package and OUTSIDE `/static`. Point at a volume in prod. |
|
||||
| `DEVPLACE_OUTBOUND_PROXY_URL` | unset | Fallback for the `outbound_proxy_url` site setting (below) when the DB/settings row is unavailable (early CLI contexts). Prefer configuring the setting via `/admin/settings` - it applies live with no restart. |
|
||||
|
||||
## Subsystem map
|
||||
|
||||
@ -235,7 +238,7 @@ Devii is also reachable over **Telegram** (one supervised long-poller subprocess
|
||||
|
||||
- **No comments, no docstrings in source.** Code is self-documenting.
|
||||
- **Forbidden variable name patterns:** `_new`, `_old`, `_temp`, `_v2`, `better_`, `my_`, `the_`.
|
||||
- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: `routers/proxy.py` relays the user's own headers verbatim. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter).
|
||||
- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: `routers/proxy.py` relays the user's own headers verbatim. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
|
||||
- **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
|
||||
- **`RedirectResponse(url=..., status_code=302)`** - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
|
||||
- **Ownership checks:** compare `resource["user_uid"] == user["uid"]` before edit/delete (use `content.is_owner`). **Deletes additionally allow any admin:** `is_owner(...) or is_admin(user)` on every content delete endpoint, so an administrator may delete any member's content while members are limited to their own. Edits stay owner-only. Deletes are **soft** and cascade under one shared stamp - see `devplacepy/database/CLAUDE.md`.
|
||||
|
||||
0
devplacepy/awards/__init__.py
Normal file
0
devplacepy/awards/__init__.py
Normal file
33
devplacepy/awards/images.py
Normal file
33
devplacepy/awards/images.py
Normal file
@ -0,0 +1,33 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from io import BytesIO
|
||||
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def enforce_rgba_png(file_bytes: bytes) -> bytes:
|
||||
img = Image.open(BytesIO(file_bytes)).convert("RGBA")
|
||||
width, height = img.size
|
||||
if width > 1 and height > 1:
|
||||
corner = img.getpixel((0, 0))
|
||||
if len(corner) == 4 and corner[3] == 255:
|
||||
bg = corner[:3]
|
||||
data = img.getdata()
|
||||
cleaned = []
|
||||
for pixel in data:
|
||||
if pixel[:3] == bg:
|
||||
cleaned.append((pixel[0], pixel[1], pixel[2], 0))
|
||||
else:
|
||||
cleaned.append(pixel)
|
||||
img.putdata(cleaned)
|
||||
buf = BytesIO()
|
||||
img.save(buf, format="PNG")
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def resize_award_png(source: bytes, size: int) -> bytes:
|
||||
img = Image.open(BytesIO(source)).convert("RGBA")
|
||||
img = img.resize((size, size), Image.LANCZOS)
|
||||
buf = BytesIO()
|
||||
img.save(buf, format="PNG")
|
||||
return buf.getvalue()
|
||||
@ -68,6 +68,21 @@ INTERNAL_GATEWAY_URL = f"{INTERNAL_BASE_URL}/openai/v1/chat/completions"
|
||||
INTERNAL_EMBED_URL = f"{INTERNAL_BASE_URL}/openai/v1/embeddings"
|
||||
INTERNAL_MODEL = "molodetz"
|
||||
INTERNAL_EMBED_MODEL = "molodetz~embed"
|
||||
INTERNAL_IMAGE_MODEL = "molodetz-img-small"
|
||||
|
||||
AWARD_GIVE_COOLDOWN_HOURS_DEFAULT = 24
|
||||
AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT = 24
|
||||
AWARD_DISPLAY_HOURS_DEFAULT = 24
|
||||
AWARD_DESCRIPTION_MAX = 125
|
||||
AWARD_IMAGE_MODEL_DEFAULT = "molodetz-img-small"
|
||||
AWARD_IMAGE_SIZE_DEFAULT = "512x512"
|
||||
AWARD_GENERATION_TIMEOUT_SECONDS = 120.0
|
||||
AWARD_IMAGE_PROMPT_DEFAULT = (
|
||||
"Generate a single decorative developer award emblem/badge as a PNG with a fully "
|
||||
"transparent background (alpha channel). No rectangular backdrop, no drop shadow "
|
||||
"plate, no text labels rendered in the image. Center one stylized trophy/medal "
|
||||
"icon that visually matches this message:"
|
||||
)
|
||||
DEFAULT_CORRECTION_PROMPT = "Leave literary as is, only do punctuation and casing"
|
||||
DEFAULT_MODIFIER_PROMPT = (
|
||||
"Execute what is behind `@ai` (the prompt) and replace that part including `@ai`"
|
||||
|
||||
@ -73,10 +73,17 @@ class CurlResponseStream(httpx.AsyncByteStream):
|
||||
|
||||
|
||||
class CurlTransport(httpx.AsyncBaseTransport):
|
||||
def __init__(self, *, impersonate: str = IMPERSONATE_TARGET, verify: bool = True) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
impersonate: str = IMPERSONATE_TARGET,
|
||||
verify: bool = True,
|
||||
proxy: str | None = None,
|
||||
) -> None:
|
||||
self._session = AsyncSession()
|
||||
self._impersonate = impersonate
|
||||
self._verify = verify
|
||||
self._proxy = proxy
|
||||
|
||||
async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
|
||||
headers = {
|
||||
@ -97,6 +104,7 @@ class CurlTransport(httpx.AsyncBaseTransport):
|
||||
data=body or None,
|
||||
impersonate=self._impersonate,
|
||||
verify=self._verify,
|
||||
proxy=self._proxy,
|
||||
stream=True,
|
||||
allow_redirects=False,
|
||||
timeout=resolve_timeout(request),
|
||||
|
||||
@ -180,7 +180,7 @@ Site settings are seeded on startup (`site_settings` table):
|
||||
| `customization_js_enabled` | `"1"` | When `"0"`, custom CSS still serves but custom JS is suppressed |
|
||||
| `extra_head` | `""` | Raw HTML emitted verbatim into every page `<head>` by `templating.extra_head_tag()`; site-wide trusted-admin input, not sanitized |
|
||||
|
||||
Besides the site/news/upload keys above, the **Operational** group is admin-editable at `/admin/settings`: `site_url` (public origin for absolute links incl. container ingress; resolved by `seo.public_base_url()` = setting -> `DEVPLACE_SITE_URL` env -> request origin), `rate_limit_per_minute`, `rate_limit_window_seconds`, `news_service_interval`, `session_max_age_days`, `session_remember_days`, `registration_open`, `maintenance_mode`, `maintenance_message`, `docs_search_mode` (`agent`|`bm25`, default `agent` - picks the `/docs/search.html` surface: the in-page Devii chat or the classic BM25 list; a viewer over their daily AI limit, or a guest when Devii is disabled, auto-falls-back to BM25 via `routers/docs/views.py` `_agent_search_state`). The **Custom Code** key `extra_head` is the sole key in the settings handler's `CLEARABLE_SETTINGS` set, so saving an empty textarea removes it (the default loop skips empty values).
|
||||
Besides the site/news/upload keys above, the **Operational** group is admin-editable at `/admin/settings`: `site_url` (public origin for absolute links incl. container ingress; resolved by `seo.public_base_url()` = setting -> `DEVPLACE_SITE_URL` env -> request origin), `rate_limit_per_minute`, `rate_limit_window_seconds`, `news_service_interval`, `session_max_age_days`, `session_remember_days`, `registration_open`, `maintenance_mode`, `maintenance_message`, `docs_search_mode` (`agent`|`bm25`, default `agent` - picks the `/docs/search.html` surface: the in-page Devii chat or the classic BM25 list; a viewer over their daily AI limit, or a guest when Devii is disabled, auto-falls-back to BM25 via `routers/docs/views.py` `_agent_search_state`), `outbound_proxy_url` (empty by default - when set, every `stealth.stealth_async_client`/`stealth_sync_client` call across the whole app routes through it via `stealth.configured_proxy_url()`; validated as `http(s)://`/`socks5(h)://` with a host in `AdminSettingsForm`; falls back to `DEVPLACE_OUTBOUND_PROXY_URL` when unset - see the "Outbound HTTP" note in the root `CLAUDE.md`). The **Custom Code** key `extra_head` is the sole key in the settings handler's `CLEARABLE_SETTINGS` set, so saving an empty textarea removes it (the default loop skips empty values).
|
||||
|
||||
The seed block in `database.py` is guarded by `if "site_settings" in tables:` - on a brand-new DB the table does not exist yet (dataset creates tables lazily on first insert), so none of these rows are written until the table exists. Correct runtime behavior therefore relies on every consumer passing the production default to `get_setting`/`get_int_setting`, not on the seed.
|
||||
|
||||
|
||||
@ -8,7 +8,24 @@ from .relations import _relations_cache, get_user_relations, get_blocked_uids, g
|
||||
from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, build_pagination
|
||||
from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns, soft_delete, soft_delete_in, restore, purge, list_deleted, count_deleted, restore_event, purge_event
|
||||
from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_polls_by_post_uids, get_poll_for_post
|
||||
from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage
|
||||
from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage, AWARD_USAGE_KEY, add_award_usage, get_award_usage
|
||||
from .awards import (
|
||||
AWARDS_PER_PAGE,
|
||||
award_display_hours,
|
||||
award_give_cooldown_hours,
|
||||
award_receive_cooldown_hours,
|
||||
award_is_prominent,
|
||||
can_give_award,
|
||||
can_receive_award,
|
||||
count_published_awards,
|
||||
enrich_award,
|
||||
get_prominent_award,
|
||||
get_user_awards,
|
||||
has_giver_cooldown,
|
||||
has_receiver_cooldown,
|
||||
recompute_user_award_stats,
|
||||
revoke_award,
|
||||
)
|
||||
from .seo_meta import SEO_META_TYPES, get_seo_metadata, get_seo_metadata_batch, has_fresh_seo_metadata, upsert_seo_metadata, mark_seo_metadata_stale
|
||||
from .activity import record_activity, record_unique_activity, get_user_activity, _activity_cache, _ACTIVITY_TABLES, get_activity_calendar, _activity_level, get_first_activity_date, HEATMAP_WEEKS, get_activity_heatmap, get_activity_months, get_streaks
|
||||
from .customization import CUSTOMIZATION_GLOBAL_SCOPE, CUSTOMIZATION_LANGS, _customizations_cache, _customization_key, CUSTOMIZATION_PREF_COLUMNS, get_customization_prefs, set_customization_pref, get_custom_overrides, get_custom_override, list_custom_overrides, set_custom_override, delete_custom_override
|
||||
|
||||
193
devplacepy/database/awards.py
Normal file
193
devplacepy/database/awards.py
Normal file
@ -0,0 +1,193 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from devplacepy.config import (
|
||||
AWARD_DISPLAY_HOURS_DEFAULT,
|
||||
AWARD_GIVE_COOLDOWN_HOURS_DEFAULT,
|
||||
AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT,
|
||||
)
|
||||
from .core import db
|
||||
from .pagination import build_pagination
|
||||
from .settings import get_int_setting
|
||||
from .core import get_table, _now_iso
|
||||
from .users import get_users_by_uids
|
||||
from .content import resolve_by_slug
|
||||
from .soft_delete import soft_delete, soft_delete_in
|
||||
|
||||
AWARDS_PER_PAGE = 12
|
||||
|
||||
|
||||
def _awards_table():
|
||||
return get_table("awards")
|
||||
|
||||
|
||||
def award_give_cooldown_hours() -> int:
|
||||
return max(1, get_int_setting("award_give_cooldown_hours", AWARD_GIVE_COOLDOWN_HOURS_DEFAULT))
|
||||
|
||||
|
||||
def award_receive_cooldown_hours() -> int:
|
||||
return max(
|
||||
1, get_int_setting("award_receive_cooldown_hours", AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT)
|
||||
)
|
||||
|
||||
|
||||
def award_display_hours() -> int:
|
||||
return max(1, get_int_setting("award_display_hours", AWARD_DISPLAY_HOURS_DEFAULT))
|
||||
|
||||
|
||||
def _cooldown_cutoff(hours: int) -> str:
|
||||
return (datetime.now(timezone.utc) - timedelta(hours=hours)).isoformat()
|
||||
|
||||
|
||||
def has_giver_cooldown(giver_uid: str) -> bool:
|
||||
if not giver_uid or "awards" not in db.tables:
|
||||
return False
|
||||
cutoff = _cooldown_cutoff(award_give_cooldown_hours())
|
||||
row = _awards_table().find_one(
|
||||
giver_uid=giver_uid, deleted_at=None, created_at={">=": cutoff}
|
||||
)
|
||||
return row is not None
|
||||
|
||||
|
||||
def has_receiver_cooldown(receiver_uid: str) -> bool:
|
||||
if not receiver_uid or "awards" not in db.tables:
|
||||
return False
|
||||
cutoff = _cooldown_cutoff(award_receive_cooldown_hours())
|
||||
row = _awards_table().find_one(
|
||||
receiver_uid=receiver_uid, deleted_at=None, created_at={">=": cutoff}
|
||||
)
|
||||
return row is not None
|
||||
|
||||
|
||||
def can_receive_award(receiver_uid: str) -> bool:
|
||||
return not has_receiver_cooldown(receiver_uid)
|
||||
|
||||
|
||||
def can_give_award(giver_uid: str, receiver_uid: str) -> bool:
|
||||
if not giver_uid or not receiver_uid or giver_uid == receiver_uid:
|
||||
return False
|
||||
return not has_giver_cooldown(giver_uid) and not has_receiver_cooldown(receiver_uid)
|
||||
|
||||
|
||||
def _published_filter():
|
||||
return {"deleted_at": None, "generated_at": {">": ""}}
|
||||
|
||||
|
||||
def count_published_awards(receiver_uid: str) -> int:
|
||||
if not receiver_uid or "awards" not in db.tables:
|
||||
return 0
|
||||
return _awards_table().count(receiver_uid=receiver_uid, **_published_filter())
|
||||
|
||||
|
||||
def _latest_published(receiver_uid: str):
|
||||
if not receiver_uid or "awards" not in db.tables:
|
||||
return None
|
||||
rows = list(
|
||||
_awards_table().find(
|
||||
receiver_uid=receiver_uid,
|
||||
deleted_at=None,
|
||||
generated_at={">": ""},
|
||||
order_by=["-generated_at"],
|
||||
_limit=1,
|
||||
)
|
||||
)
|
||||
return rows[0] if rows else None
|
||||
|
||||
|
||||
def recompute_user_award_stats(receiver_uid: str) -> None:
|
||||
if not receiver_uid or "users" not in db.tables:
|
||||
return
|
||||
count = count_published_awards(receiver_uid)
|
||||
latest = _latest_published(receiver_uid)
|
||||
users = get_table("users")
|
||||
payload = {
|
||||
"uid": receiver_uid,
|
||||
"award_count": count,
|
||||
"last_award_at": latest.get("generated_at") if latest else None,
|
||||
"last_award_slug": latest.get("slug") if latest else None,
|
||||
"last_award_uid": latest.get("uid") if latest else None,
|
||||
}
|
||||
users.update(payload, ["uid"])
|
||||
|
||||
|
||||
def award_is_prominent(user: dict | None) -> bool:
|
||||
if not user or not user.get("last_award_at") or not user.get("last_award_uid"):
|
||||
return False
|
||||
award = resolve_by_slug(_awards_table(), user["last_award_uid"])
|
||||
if not award or not award.get("generated_at"):
|
||||
return False
|
||||
try:
|
||||
published = datetime.fromisoformat(award["generated_at"])
|
||||
if published.tzinfo is None:
|
||||
published = published.replace(tzinfo=timezone.utc)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
window = timedelta(hours=award_display_hours())
|
||||
return datetime.now(timezone.utc) - published <= window
|
||||
|
||||
|
||||
def enrich_award(row: dict, givers: dict | None = None) -> dict:
|
||||
item = dict(row)
|
||||
giver_uid = row.get("giver_uid", "")
|
||||
giver = (givers or {}).get(giver_uid) or get_users_by_uids([giver_uid]).get(giver_uid)
|
||||
item["giver"] = giver
|
||||
item["image_url"] = f"/awards/{row.get('slug', '')}/256"
|
||||
item["thumb_url"] = f"/awards/{row.get('slug', '')}/64"
|
||||
return item
|
||||
|
||||
|
||||
def get_user_awards(receiver_uid: str, page: int = 1, per_page: int = AWARDS_PER_PAGE):
|
||||
if not receiver_uid or "awards" not in db.tables:
|
||||
return [], build_pagination(page, 0, per_page)
|
||||
table = _awards_table()
|
||||
total = table.count(receiver_uid=receiver_uid, **_published_filter())
|
||||
offset = max(0, (page - 1) * per_page)
|
||||
rows = list(
|
||||
table.find(
|
||||
receiver_uid=receiver_uid,
|
||||
deleted_at=None,
|
||||
generated_at={">": ""},
|
||||
order_by=["-generated_at"],
|
||||
_limit=per_page,
|
||||
_offset=offset,
|
||||
)
|
||||
)
|
||||
giver_uids = [row.get("giver_uid") for row in rows if row.get("giver_uid")]
|
||||
givers = get_users_by_uids(giver_uids)
|
||||
items = [enrich_award(row, givers) for row in rows]
|
||||
return items, build_pagination(page, total, per_page)
|
||||
|
||||
|
||||
def get_prominent_award(profile_user: dict) -> dict | None:
|
||||
if not award_is_prominent(profile_user):
|
||||
return None
|
||||
award = resolve_by_slug(_awards_table(), profile_user.get("last_award_uid", ""))
|
||||
if not award:
|
||||
return None
|
||||
return enrich_award(award)
|
||||
|
||||
|
||||
def revoke_award(award_uid: str, admin_uid: str) -> dict | None:
|
||||
table = _awards_table()
|
||||
row = table.find_one(uid=award_uid)
|
||||
if not row or row.get("deleted_at"):
|
||||
return None
|
||||
stamp = _now_iso()
|
||||
attachment_uids = [
|
||||
uid
|
||||
for uid in (
|
||||
row.get("attachment_uid_512"),
|
||||
row.get("attachment_uid_256"),
|
||||
row.get("attachment_uid_64"),
|
||||
)
|
||||
if uid
|
||||
]
|
||||
soft_delete("awards", admin_uid, stamp=stamp, uid=award_uid)
|
||||
from devplacepy.attachments import soft_delete_attachments_for
|
||||
|
||||
soft_delete_attachments_for("award", [award_uid], admin_uid)
|
||||
if attachment_uids:
|
||||
soft_delete_in("attachments", "uid", attachment_uids, admin_uid, stamp=stamp)
|
||||
recompute_user_award_stats(row.get("receiver_uid", ""))
|
||||
return row
|
||||
@ -40,6 +40,12 @@ def resolve_object_url(target_type: str, target_uid: str) -> str:
|
||||
comment.get("target_uid") or comment.get("post_uid", ""),
|
||||
)
|
||||
return f"{parent_url}#comment-{target_uid}"
|
||||
if target_type == "award":
|
||||
award = resolve_by_slug(get_table("awards"), target_uid)
|
||||
if award:
|
||||
receiver = get_table("users").find_one(uid=award.get("receiver_uid", ""))
|
||||
if receiver:
|
||||
return f"/profile/{receiver['username']}?tab=awards#award-{award.get('slug', '')}"
|
||||
return "/feed"
|
||||
|
||||
|
||||
|
||||
@ -61,10 +61,11 @@ def _ensure_cache_state() -> None:
|
||||
global _cache_state_ready
|
||||
if _cache_state_ready:
|
||||
return
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS cache_state "
|
||||
"(name TEXT PRIMARY KEY, version INTEGER NOT NULL DEFAULT 0)"
|
||||
)
|
||||
with db:
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS cache_state "
|
||||
"(name TEXT PRIMARY KEY, version INTEGER NOT NULL DEFAULT 0)"
|
||||
)
|
||||
_cache_state_ready = True
|
||||
|
||||
|
||||
@ -74,14 +75,15 @@ def get_cache_version(name: str) -> int:
|
||||
return cached
|
||||
try:
|
||||
_ensure_cache_state()
|
||||
row = next(
|
||||
iter(
|
||||
db.query(
|
||||
"SELECT version FROM cache_state WHERE name = :name", name=name
|
||||
)
|
||||
),
|
||||
None,
|
||||
)
|
||||
with db:
|
||||
row = next(
|
||||
iter(
|
||||
db.query(
|
||||
"SELECT version FROM cache_state WHERE name = :name", name=name
|
||||
)
|
||||
),
|
||||
None,
|
||||
)
|
||||
version = int(row["version"]) if row else 0
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not read cache version {name}: {e}")
|
||||
@ -93,16 +95,15 @@ def get_cache_version(name: str) -> int:
|
||||
def bump_cache_version(name: str) -> None:
|
||||
try:
|
||||
_ensure_cache_state()
|
||||
db.query(
|
||||
"INSERT OR IGNORE INTO cache_state (name, version) VALUES (:name, 0)",
|
||||
name=name,
|
||||
)
|
||||
db.query(
|
||||
"UPDATE cache_state SET version = version + 1 WHERE name = :name", name=name
|
||||
)
|
||||
connection = db.executable
|
||||
if connection.in_transaction():
|
||||
connection.commit()
|
||||
with db:
|
||||
db.query(
|
||||
"INSERT OR IGNORE INTO cache_state (name, version) VALUES (:name, 0)",
|
||||
name=name,
|
||||
)
|
||||
db.query(
|
||||
"UPDATE cache_state SET version = version + 1 WHERE name = :name",
|
||||
name=name,
|
||||
)
|
||||
_cache_version_cache.pop(name)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not bump cache version {name}: {e}")
|
||||
|
||||
@ -17,6 +17,7 @@ NOTIFICATION_TYPES = [
|
||||
{"key": "issue", "label": "Issue tracker", "description": "Updates on issue reports you filed"},
|
||||
{"key": "reminder", "label": "Reminders", "description": "A reminder or scheduled task you asked Devii to run fires"},
|
||||
{"key": "harvest_stolen", "label": "Farm raids", "description": "Someone steals a ready build from your Code Farm"},
|
||||
{"key": "award", "label": "Awards", "description": "Someone gives you an award on your profile"},
|
||||
]
|
||||
|
||||
|
||||
|
||||
@ -658,6 +658,67 @@ def init_db():
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not create unique index on seo_usage: {e}")
|
||||
|
||||
award_usage = get_table("award_usage")
|
||||
for column, example in (
|
||||
("user_uid", ""),
|
||||
("calls", 0),
|
||||
("prompt_tokens", 0),
|
||||
("completion_tokens", 0),
|
||||
("total_tokens", 0),
|
||||
("cost_usd", 0.0),
|
||||
("upstream_latency_ms", 0.0),
|
||||
("total_latency_ms", 0.0),
|
||||
("updated_at", ""),
|
||||
):
|
||||
if not award_usage.has_column(column):
|
||||
award_usage.create_column_by_example(column, example)
|
||||
try:
|
||||
if "award_usage" in db.tables:
|
||||
db.query(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS idx_award_usage_user "
|
||||
"ON award_usage (user_uid)"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not create unique index on award_usage: {e}")
|
||||
|
||||
awards = get_table("awards")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("slug", ""),
|
||||
("description", ""),
|
||||
("giver_uid", ""),
|
||||
("receiver_uid", ""),
|
||||
("attachment_uid_512", ""),
|
||||
("attachment_uid_256", ""),
|
||||
("attachment_uid_64", ""),
|
||||
("generated_at", ""),
|
||||
("created_at", ""),
|
||||
("job_uid", ""),
|
||||
("deleted_at", ""),
|
||||
("deleted_by", ""),
|
||||
):
|
||||
if not awards.has_column(column):
|
||||
awards.create_column_by_example(column, example)
|
||||
try:
|
||||
if "awards" in db.tables:
|
||||
db.query(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS idx_awards_slug ON awards (slug)"
|
||||
)
|
||||
db.query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_awards_receiver_created "
|
||||
"ON awards (receiver_uid, created_at)"
|
||||
)
|
||||
db.query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_awards_giver_created "
|
||||
"ON awards (giver_uid, created_at)"
|
||||
)
|
||||
db.query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_awards_generated "
|
||||
"ON awards (receiver_uid, generated_at)"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not create awards indexes: {e}")
|
||||
|
||||
seo_metadata = get_table("seo_metadata")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
@ -1155,7 +1216,9 @@ def init_db():
|
||||
"customization_enabled": "1",
|
||||
"customization_js_enabled": "1",
|
||||
"audit_log_retention_days": "90",
|
||||
"statistics_tracking_enabled": "1",
|
||||
"docs_search_mode": "agent",
|
||||
"outbound_proxy_url": "",
|
||||
}
|
||||
for key, value in operational_defaults.items():
|
||||
existing = db["site_settings"].find_one(key=key)
|
||||
@ -1164,6 +1227,72 @@ def init_db():
|
||||
{"uid": f"default_{key}", "key": key, "value": value}
|
||||
)
|
||||
|
||||
with db:
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS visit_stats_hourly ("
|
||||
"bucket_start TEXT NOT NULL, "
|
||||
"page_group TEXT NOT NULL, "
|
||||
"referrer_group TEXT NOT NULL, "
|
||||
"views INTEGER NOT NULL DEFAULT 0, "
|
||||
"member_views INTEGER NOT NULL DEFAULT 0, "
|
||||
"guest_views INTEGER NOT NULL DEFAULT 0)"
|
||||
)
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS visit_unique_slots ("
|
||||
"bucket_start TEXT NOT NULL, "
|
||||
"visitor_hash TEXT NOT NULL, "
|
||||
"page_group TEXT NOT NULL, "
|
||||
"user_uid TEXT)"
|
||||
)
|
||||
_index(db, "visit_stats_hourly", "idx_visit_hourly_bucket", ["bucket_start"])
|
||||
_index(
|
||||
db,
|
||||
"visit_stats_hourly",
|
||||
"idx_visit_hourly_page_time",
|
||||
["page_group", "bucket_start"],
|
||||
)
|
||||
_index(
|
||||
db,
|
||||
"visit_stats_hourly",
|
||||
"idx_visit_hourly_ref_time",
|
||||
["referrer_group", "bucket_start"],
|
||||
)
|
||||
_index(
|
||||
db,
|
||||
"visit_stats_hourly",
|
||||
"idx_visit_hourly_unique_row",
|
||||
["bucket_start", "page_group", "referrer_group"],
|
||||
unique=True,
|
||||
)
|
||||
_index(db, "visit_unique_slots", "idx_visit_unique_bucket", ["bucket_start"])
|
||||
_index(
|
||||
db,
|
||||
"visit_unique_slots",
|
||||
"idx_visit_unique_hash",
|
||||
["bucket_start", "visitor_hash"],
|
||||
)
|
||||
_index(
|
||||
db,
|
||||
"visit_unique_slots",
|
||||
"idx_visit_unique_row",
|
||||
["bucket_start", "visitor_hash", "page_group"],
|
||||
unique=True,
|
||||
)
|
||||
_index(db, "issue_tickets", "idx_issue_tickets_created", ["created_at"])
|
||||
_index(db, "devii_turns", "idx_devii_turns_started", ["started_at"])
|
||||
_index(db, "instance_events", "idx_instance_events_created", ["created_at"])
|
||||
_index(db, "game_steals", "idx_game_steals_stolen_at", ["stolen_at"])
|
||||
_index(db, "messages", "idx_messages_created_at", ["created_at"])
|
||||
_index(db, "notifications", "idx_notifications_created", ["created_at"])
|
||||
_index(db, "follows", "idx_follows_created_at", ["created_at"])
|
||||
_index(db, "reactions", "idx_reactions_created_at", ["created_at"])
|
||||
_index(db, "votes", "idx_votes_created_at", ["created_at"])
|
||||
_index(db, "bookmarks", "idx_bookmarks_created_at", ["created_at"])
|
||||
_index(db, "badges", "idx_badges_created_at", ["created_at"])
|
||||
_index(db, "audit_log", "idx_audit_result_created", ["result", "created_at"])
|
||||
_index(db, "jobs", "idx_jobs_created_at", ["created_at"])
|
||||
_index(db, "attachments", "idx_attachments_created_at", ["created_at"])
|
||||
|
||||
_backfill_gamification()
|
||||
backfill_api_keys()
|
||||
migrate_ai_gateway_settings()
|
||||
@ -1210,6 +1339,15 @@ def migrate_ai_gateway_settings() -> None:
|
||||
if get_setting("bot_model", "") == "deepseek-chat":
|
||||
set_setting("bot_model", "molodetz")
|
||||
logger.info("Migrated bot_model to molodetz")
|
||||
from devplacepy.services.openai_gateway.routing import (
|
||||
migrate_retired_image_gateway,
|
||||
seed_default_deepseek_routes,
|
||||
seed_default_image_routes,
|
||||
)
|
||||
|
||||
seed_default_deepseek_routes()
|
||||
seed_default_image_routes()
|
||||
migrate_retired_image_gateway()
|
||||
|
||||
|
||||
def backfill_api_keys() -> int:
|
||||
@ -1240,6 +1378,14 @@ def backfill_api_keys() -> int:
|
||||
users.create_column_by_example("avatar_seed", "")
|
||||
if not users.has_column("last_seen"):
|
||||
users.create_column_by_example("last_seen", "")
|
||||
if not users.has_column("award_count"):
|
||||
users.create_column_by_example("award_count", 0)
|
||||
if not users.has_column("last_award_at"):
|
||||
users.create_column_by_example("last_award_at", "")
|
||||
if not users.has_column("last_award_slug"):
|
||||
users.create_column_by_example("last_award_slug", "")
|
||||
if not users.has_column("last_award_uid"):
|
||||
users.create_column_by_example("last_award_uid", "")
|
||||
with db:
|
||||
db.query(
|
||||
"UPDATE users SET ai_modifier_enabled = 1 WHERE ai_modifier_enabled IS NULL"
|
||||
|
||||
@ -41,6 +41,7 @@ SOFT_DELETE_TABLES = [
|
||||
"email_accounts",
|
||||
"user_relations",
|
||||
"seo_metadata",
|
||||
"awards",
|
||||
]
|
||||
|
||||
|
||||
|
||||
@ -126,3 +126,14 @@ def add_seo_usage(totals: dict) -> None:
|
||||
|
||||
def get_seo_usage() -> dict:
|
||||
return _get_usage("seo_usage", SEO_USAGE_KEY)
|
||||
|
||||
|
||||
AWARD_USAGE_KEY = "award"
|
||||
|
||||
|
||||
def add_award_usage(totals: dict) -> None:
|
||||
_add_usage("award_usage", AWARD_USAGE_KEY, totals)
|
||||
|
||||
|
||||
def get_award_usage() -> dict:
|
||||
return _get_usage("award_usage", AWARD_USAGE_KEY)
|
||||
|
||||
@ -63,6 +63,22 @@ four ways to sign requests.
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/admin/media"},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-revoke-award",
|
||||
method="POST",
|
||||
path="/admin/awards/{uid}/revoke",
|
||||
title="Revoke award",
|
||||
summary=(
|
||||
"Soft-delete a published award and its linked attachments, then recompute "
|
||||
"receiver stats. Restorable from admin trash."
|
||||
),
|
||||
auth="admin",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("uid", "path", "string", True, "", "Award uid to revoke."),
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/profile/receiver?tab=awards"},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-media-purge",
|
||||
method="POST",
|
||||
@ -171,6 +187,84 @@ four ways to sign requests.
|
||||
"This is the endpoint the Devii assistant calls as `site_analytics`; see [Devii internals](/docs/devii-internals.html).",
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="admin-statistics",
|
||||
method="GET",
|
||||
path="/admin/statistics/data",
|
||||
title="Platform statistics",
|
||||
summary=(
|
||||
"Tabbed platform statistics with KPI cards, period-over-period deltas, "
|
||||
"time-series data for charts, and breakdown tables. Covers visitors, members, "
|
||||
"content, engagement, social, AI, Devii, services, containers, game, awards, "
|
||||
"moderation, tools, and storage."
|
||||
),
|
||||
auth="admin",
|
||||
interactive=True,
|
||||
params=[
|
||||
field(
|
||||
"tab",
|
||||
"query",
|
||||
"string",
|
||||
False,
|
||||
"overview",
|
||||
"Tab key (overview, visitors, members, content, ...).",
|
||||
),
|
||||
field(
|
||||
"hours",
|
||||
"query",
|
||||
"int",
|
||||
False,
|
||||
"168",
|
||||
"Lookback window in hours (24, 168, 720, 2160, or 0 for all time).",
|
||||
),
|
||||
field(
|
||||
"compare",
|
||||
"query",
|
||||
"int",
|
||||
False,
|
||||
"1",
|
||||
"Include previous-period comparison (1 or 0).",
|
||||
),
|
||||
field(
|
||||
"top_n",
|
||||
"query",
|
||||
"int",
|
||||
False,
|
||||
"10",
|
||||
"Rows in breakdown tables (1-50).",
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"The HTML dashboard lives at `/admin/statistics`. Visitor metrics require the statistics tracking middleware (hourly aggregation, 90-day retention).",
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="admin-statistics-page",
|
||||
method="GET",
|
||||
path="/admin/statistics",
|
||||
title="Statistics dashboard",
|
||||
summary="Admin HTML dashboard for platform statistics with charts and tabs.",
|
||||
auth="admin",
|
||||
interactive=False,
|
||||
params=[
|
||||
field(
|
||||
"tab",
|
||||
"query",
|
||||
"string",
|
||||
False,
|
||||
"overview",
|
||||
"Initial tab to render.",
|
||||
),
|
||||
field(
|
||||
"hours",
|
||||
"query",
|
||||
"int",
|
||||
False,
|
||||
"168",
|
||||
"Initial time window in hours.",
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="admin-ai-usage",
|
||||
method="GET",
|
||||
|
||||
@ -32,6 +32,11 @@ The gateway additionally serves **text embeddings** at `/openai/v1/embeddings`.
|
||||
generic model `molodetz~embed`, which the gateway maps to the configured embedding model (OpenRouter's
|
||||
Qwen3 8B embedding model by default). Usage and cost are tracked per call exactly like chat and vision.
|
||||
|
||||
The gateway also serves **image generation** at `/openai/v1/images/generations`. Clients request the
|
||||
generic model `molodetz-img-small`, which the gateway maps to the configured image model (OpenRouter's
|
||||
Flux 1.1 Pro by default). Cost is tracked per call with a flat per-image price when the upstream
|
||||
returns no native cost.
|
||||
|
||||
## Model routing and providers
|
||||
|
||||
On top of the single default upstream above, an administrator can register additional named
|
||||
@ -60,7 +65,7 @@ and dollar cost directly from the response with no extra request:
|
||||
| Header | Meaning |
|
||||
|--------|---------|
|
||||
| `X-Gateway-Model` | Upstream model actually used for the call |
|
||||
| `X-Gateway-Backend` | Backend that served it: `chat`, `embed`, or passthrough |
|
||||
| `X-Gateway-Backend` | Backend that served it: `chat`, `embed`, `image`, or passthrough |
|
||||
| `X-Gateway-Prompt-Tokens` | Input (prompt) tokens |
|
||||
| `X-Gateway-Completion-Tokens` | Output (completion) tokens |
|
||||
| `X-Gateway-Total-Tokens` | Total tokens (prompt + completion) |
|
||||
@ -83,7 +88,7 @@ and dollar cost directly from the response with no extra request:
|
||||
Dollar costs use the upstream's native `cost` field when it returns one
|
||||
(`X-Gateway-Cost-Native: 1`); otherwise they are computed from the per-million prices of the matched
|
||||
model route, falling back to the prices configured on the `openai` service when no route matches. The
|
||||
one denied path that makes no upstream call (embeddings disabled) returns no usage headers.
|
||||
denied paths that make no upstream call (embeddings or image generation disabled) return no usage headers.
|
||||
|
||||
Administrators enable and configure this gateway under [Background Services](/docs/services.html)
|
||||
(the `openai` service).
|
||||
@ -172,6 +177,53 @@ for signing DevPlace's own requests.
|
||||
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="gateway-images",
|
||||
method="POST",
|
||||
path="/openai/v1/images/generations",
|
||||
title="Image generation",
|
||||
summary="OpenAI-compatible image generation. Request model molodetz-img-small.",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field(
|
||||
"model",
|
||||
"json",
|
||||
"string",
|
||||
False,
|
||||
"molodetz-img-small",
|
||||
"Image model id; the gateway maps molodetz-img-small to the configured model, or to a matching image model route's provider and target model.",
|
||||
),
|
||||
field(
|
||||
"prompt",
|
||||
"json",
|
||||
"string",
|
||||
True,
|
||||
'"a decorative developer award emblem"',
|
||||
"Text prompt describing the image to generate.",
|
||||
),
|
||||
field(
|
||||
"size",
|
||||
"json",
|
||||
"string",
|
||||
False,
|
||||
"512x512",
|
||||
"Output dimensions (provider-dependent).",
|
||||
),
|
||||
field(
|
||||
"response_format",
|
||||
"json",
|
||||
"string",
|
||||
False,
|
||||
"b64_json",
|
||||
"Return format: url or b64_json.",
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"Returns `503` when the gateway service is not running or image generation is disabled.",
|
||||
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="gateway-passthrough",
|
||||
method="POST",
|
||||
|
||||
@ -32,7 +32,7 @@ four ways to sign requests.
|
||||
False,
|
||||
"posts",
|
||||
"Profile tab.",
|
||||
["posts", "activity", "followers", "following", "media"],
|
||||
["posts", "activity", "followers", "following", "media", "awards"],
|
||||
),
|
||||
],
|
||||
),
|
||||
@ -60,7 +60,7 @@ four ways to sign requests.
|
||||
False,
|
||||
"posts",
|
||||
"Profile tab.",
|
||||
["posts", "activity", "followers", "following", "media"],
|
||||
["posts", "activity", "followers", "following", "media", "awards"],
|
||||
),
|
||||
],
|
||||
),
|
||||
@ -254,6 +254,40 @@ four ways to sign requests.
|
||||
],
|
||||
sample_response={"api_key": "NEW_UUID"},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-give-award",
|
||||
method="POST",
|
||||
path="/profile/{username}/award",
|
||||
title="Give a member an award",
|
||||
summary="Create a pending award on another member's profile and enqueue image generation.",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field(
|
||||
"username",
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
"{{ username }}",
|
||||
"Receiver username.",
|
||||
),
|
||||
field(
|
||||
"description",
|
||||
"body",
|
||||
"string",
|
||||
True,
|
||||
"Great work on the release!",
|
||||
"Award message (1-125 characters).",
|
||||
),
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"data": {
|
||||
"award_uid": "AWARD_UID",
|
||||
"award_slug": "abc123-great-work",
|
||||
},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-regenerate-avatar",
|
||||
method="POST",
|
||||
@ -650,6 +684,37 @@ four ways to sign requests.
|
||||
auth="public",
|
||||
interactive=True,
|
||||
),
|
||||
endpoint(
|
||||
id="award-image",
|
||||
method="GET",
|
||||
path="/awards/{slug_or_uid}/{size}",
|
||||
title="Award image redirect",
|
||||
summary="Redirect to the stored PNG attachment for a published award.",
|
||||
auth="public",
|
||||
params=[
|
||||
field(
|
||||
"slug_or_uid",
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
"abc123-great-work",
|
||||
"Award slug or bare uid.",
|
||||
),
|
||||
field(
|
||||
"size",
|
||||
"path",
|
||||
"enum",
|
||||
True,
|
||||
"256",
|
||||
"Image size.",
|
||||
["512", "256", "64"],
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"> Pending or revoked awards return 404.",
|
||||
"> Response includes long-lived cache headers.",
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="avatar",
|
||||
method="GET",
|
||||
|
||||
@ -56,6 +56,7 @@ from devplacepy.routers import (
|
||||
notifications,
|
||||
votes,
|
||||
avatar,
|
||||
awards,
|
||||
follow,
|
||||
relations,
|
||||
admin,
|
||||
@ -95,6 +96,7 @@ from devplacepy.services.jobs.issue_create_service import IssueCreateService
|
||||
from devplacepy.services.jobs.planning_service import PlanningReportService
|
||||
from devplacepy.services.jobs.seo.service import SeoService
|
||||
from devplacepy.services.jobs.seo_meta_service import SeoMetaService
|
||||
from devplacepy.services.jobs.award_service import AwardService
|
||||
from devplacepy.services.backup import BackupService
|
||||
from devplacepy.services.dbapi.service import DbApiJobService
|
||||
from devplacepy.services.pubsub import PubSubService
|
||||
@ -255,6 +257,7 @@ async def lifespan(app: FastAPI):
|
||||
service_manager.register(ForkService())
|
||||
service_manager.register(SeoService())
|
||||
service_manager.register(SeoMetaService())
|
||||
service_manager.register(AwardService())
|
||||
service_manager.register(BackupService())
|
||||
service_manager.register(DbApiJobService())
|
||||
service_manager.register(PubSubService())
|
||||
@ -281,9 +284,15 @@ async def lifespan(app: FastAPI):
|
||||
logger.info(
|
||||
f"Worker pid {os.getpid()} declined service lock; another worker owns background services"
|
||||
)
|
||||
from devplacepy.services.statistics.tracking import start_visit_flusher
|
||||
|
||||
start_visit_flusher()
|
||||
logger.info(f"DevPlace started on port {PORT}")
|
||||
yield
|
||||
logger.info("Shutting down services...")
|
||||
from devplacepy.services.statistics.tracking import flush_visits
|
||||
|
||||
flush_visits()
|
||||
await service_manager.shutdown_all()
|
||||
await background.stop()
|
||||
|
||||
@ -428,6 +437,7 @@ app.include_router(reactions.router, prefix="/reactions")
|
||||
app.include_router(bookmarks.router, prefix="/bookmarks")
|
||||
app.include_router(polls.router, prefix="/polls")
|
||||
app.include_router(avatar.router, prefix="/avatar")
|
||||
app.include_router(awards.router, prefix="/awards")
|
||||
app.include_router(follow.router, prefix="/follow")
|
||||
app.include_router(relations.router)
|
||||
app.include_router(leaderboard.router, prefix="/leaderboard")
|
||||
@ -580,6 +590,15 @@ async def track_presence(request: Request, call_next):
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def visit_statistics(request: Request, call_next):
|
||||
from devplacepy.services.statistics.tracking import track_visit
|
||||
|
||||
response = await call_next(request)
|
||||
track_visit(request, response.status_code)
|
||||
return response
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def response_timing(request: Request, call_next):
|
||||
start = time.perf_counter()
|
||||
|
||||
@ -4,6 +4,7 @@ import re
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal, Optional
|
||||
from urllib.parse import urlsplit
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
from devplacepy.constants import TOPICS, REACTION_EMOJI
|
||||
from devplacepy.config import DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT
|
||||
@ -238,6 +239,10 @@ class CustomizationToggleForm(BaseModel):
|
||||
value: bool = False
|
||||
|
||||
|
||||
class AwardGiveForm(BaseModel):
|
||||
description: str = Field(min_length=1, max_length=125)
|
||||
|
||||
|
||||
class NotificationPrefForm(BaseModel):
|
||||
notification_type: str = Field(min_length=1, max_length=40)
|
||||
channel: Literal["in_app", "push", "telegram"]
|
||||
@ -551,8 +556,20 @@ class AdminSettingsForm(BaseModel):
|
||||
maintenance_mode: str = Field(default="", max_length=1)
|
||||
maintenance_message: str = Field(default="", max_length=300)
|
||||
docs_search_mode: str = Field(default="", max_length=20)
|
||||
outbound_proxy_url: str = Field(default="", max_length=500)
|
||||
extra_head: str = Field(default="", max_length=50000)
|
||||
|
||||
@field_validator("outbound_proxy_url")
|
||||
@classmethod
|
||||
def validate_outbound_proxy_url(cls, value):
|
||||
text = value.strip()
|
||||
if not text:
|
||||
return text
|
||||
parsed = urlsplit(text)
|
||||
if parsed.scheme not in ("http", "https", "socks5", "socks5h") or not parsed.hostname:
|
||||
raise ValueError("Proxy URL must be http(s):// or socks5(h):// with a host, e.g. http://user:pass@host:port")
|
||||
return text
|
||||
|
||||
|
||||
class GamePlantForm(BaseModel):
|
||||
slot: int = Field(ge=0, le=64)
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.routers.admin import (
|
||||
awards,
|
||||
aiquota,
|
||||
aiusage,
|
||||
auditlog,
|
||||
@ -14,13 +15,16 @@ from devplacepy.routers.admin import (
|
||||
notifications,
|
||||
services,
|
||||
settings,
|
||||
statistics,
|
||||
trash,
|
||||
users,
|
||||
)
|
||||
from devplacepy.routers.admin.index import router
|
||||
|
||||
router.include_router(awards.router)
|
||||
router.include_router(users.router)
|
||||
router.include_router(aiusage.router)
|
||||
router.include_router(statistics.router)
|
||||
router.include_router(aiquota.router)
|
||||
router.include_router(media.router)
|
||||
router.include_router(trash.router)
|
||||
|
||||
49
devplacepy/routers/admin/awards.py
Normal file
49
devplacepy/routers/admin/awards.py
Normal file
@ -0,0 +1,49 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.database.awards import revoke_award
|
||||
from devplacepy.responses import action_result, json_error, wants_json
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.utils import not_found, require_admin, safe_next
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _redirect_back(request: Request, award: dict) -> str:
|
||||
referer = request.headers.get("referer", "")
|
||||
if referer and safe_next(referer, "") == referer:
|
||||
return referer
|
||||
receiver = get_table("users").find_one(uid=award.get("receiver_uid", ""))
|
||||
if receiver:
|
||||
return f"/profile/{receiver['username']}?tab=awards"
|
||||
return "/admin"
|
||||
|
||||
|
||||
@router.post("/awards/{uid}/revoke")
|
||||
async def admin_revoke_award(request: Request, uid: str):
|
||||
admin = require_admin(request)
|
||||
row = revoke_award(uid, admin["uid"])
|
||||
if not row:
|
||||
if wants_json(request):
|
||||
return json_error(404, "Award not found")
|
||||
raise not_found("Award not found")
|
||||
receiver = get_table("users").find_one(uid=row.get("receiver_uid", ""))
|
||||
logger.info("Admin %s revoked award %s", admin["username"], uid)
|
||||
audit.record(
|
||||
request,
|
||||
"award.revoke",
|
||||
user=admin,
|
||||
target_type="award",
|
||||
target_uid=uid,
|
||||
target_label=row.get("slug", uid),
|
||||
summary=f"admin {admin['username']} revoked award {row.get('slug', uid)}",
|
||||
links=[
|
||||
audit.target("award", uid, row.get("slug")),
|
||||
audit.target("user", row.get("receiver_uid"), receiver.get("username") if receiver else None),
|
||||
],
|
||||
)
|
||||
return action_result(request, _redirect_back(request, row))
|
||||
@ -25,6 +25,8 @@ def _default_provider_summary() -> dict:
|
||||
"model": cfg.get("gateway_model", ""),
|
||||
"embed_url": cfg.get("gateway_embed_url", ""),
|
||||
"embed_model": cfg.get("gateway_embed_model", ""),
|
||||
"image_url": cfg.get("gateway_image_url", ""),
|
||||
"image_model": cfg.get("gateway_image_model", ""),
|
||||
"vision_url": cfg.get("gateway_vision_url", ""),
|
||||
"vision_model": cfg.get("gateway_vision_model", ""),
|
||||
}
|
||||
|
||||
90
devplacepy/routers/admin/statistics.py
Normal file
90
devplacepy/routers/admin/statistics.py
Normal file
@ -0,0 +1,90 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas.statistics import StatisticsOut
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.services.statistics.build import build_statistics_tab
|
||||
from devplacepy.services.statistics.common import VALID_TABS
|
||||
from devplacepy.utils import require_admin
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
TAB_LABELS = (
|
||||
("overview", "Overview", "\U0001f4ca"),
|
||||
("visitors", "Visitors", "\U0001f441\ufe0f"),
|
||||
("members", "Members", "\U0001f465"),
|
||||
("content", "Content", "\U0001f4dd"),
|
||||
("engagement", "Engagement", "\U0001f525"),
|
||||
("social", "Social", "\U0001f91d"),
|
||||
("ai", "AI", "\U0001f916"),
|
||||
("devii", "Devii", "\u2728"),
|
||||
("services", "Services", "\u2699\ufe0f"),
|
||||
("containers", "Containers", "\U0001f4e6"),
|
||||
("game", "Game", "\U0001f3ae"),
|
||||
("awards", "Awards", "\U0001f3c6"),
|
||||
("moderation", "Moderation", "\U0001f6e1\ufe0f"),
|
||||
("tools", "Tools", "\U0001f527"),
|
||||
("storage", "Storage", "\U0001f4be"),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/statistics", response_class=HTMLResponse)
|
||||
async def admin_statistics(request: Request, tab: str = "overview", hours: int = 168):
|
||||
admin = require_admin(request)
|
||||
active = tab if tab in VALID_TABS else "overview"
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Statistics - Admin",
|
||||
description="Platform statistics with trends, visitors, content, engagement, and operations.",
|
||||
robots="noindex,nofollow",
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Admin", "url": "/admin"},
|
||||
{"name": "Statistics", "url": "/admin/statistics"},
|
||||
],
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
initial = build_statistics_tab(active, hours, compare=True, top_n=10)
|
||||
tabs = [
|
||||
{"key": key, "label": label, "icon": icon, "active": key == active}
|
||||
for key, label, icon in TAB_LABELS
|
||||
]
|
||||
return respond(
|
||||
request,
|
||||
"admin_statistics.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"admin_section": "statistics",
|
||||
"tabs": tabs,
|
||||
"active_tab": active,
|
||||
"window_hours": hours,
|
||||
"initial": initial,
|
||||
},
|
||||
model=StatisticsOut,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/statistics/data")
|
||||
async def admin_statistics_data(
|
||||
request: Request,
|
||||
tab: str = "overview",
|
||||
hours: int = 168,
|
||||
compare: int = 1,
|
||||
top_n: int = 10,
|
||||
):
|
||||
require_admin(request)
|
||||
return JSONResponse(
|
||||
build_statistics_tab(
|
||||
tab,
|
||||
hours,
|
||||
compare=bool(compare),
|
||||
top_n=top_n,
|
||||
)
|
||||
)
|
||||
@ -24,13 +24,14 @@ logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
TRASH_TABLES = [
|
||||
{"key": "posts", "label": "Posts", "type": "post"},
|
||||
{"key": "comments", "label": "Comments", "type": "comment"},
|
||||
{"key": "gists", "label": "Gists", "type": "gist"},
|
||||
{"key": "projects", "label": "Projects", "type": "project"},
|
||||
{"key": "news", "label": "News", "type": "news"},
|
||||
{"key": "project_files", "label": "Project files", "type": None},
|
||||
{"key": "attachments", "label": "Attachments", "type": None},
|
||||
{"key": "posts", "label": "Posts", "icon": "\U0001f4dd", "type": "post"},
|
||||
{"key": "comments", "label": "Comments", "icon": "\U0001f4ac", "type": "comment"},
|
||||
{"key": "gists", "label": "Gists", "icon": "\U0001f4cb", "type": "gist"},
|
||||
{"key": "projects", "label": "Projects", "icon": "\U0001f680", "type": "project"},
|
||||
{"key": "news", "label": "News", "icon": "\U0001f4f0", "type": "news"},
|
||||
{"key": "awards", "label": "Awards", "icon": "\U0001f3c6", "type": "award"},
|
||||
{"key": "project_files", "label": "Project files", "icon": "\U0001f4c1", "type": None},
|
||||
{"key": "attachments", "label": "Attachments", "icon": "\U0001f4ce", "type": None},
|
||||
]
|
||||
_TRASH_KEYS = {entry["key"] for entry in TRASH_TABLES}
|
||||
_TRASH_TYPE = {entry["key"]: entry["type"] for entry in TRASH_TABLES}
|
||||
@ -113,6 +114,10 @@ async def admin_trash_restore(request: Request, table: str, uid: str):
|
||||
row = get_table(table).find_one(uid=uid)
|
||||
if row and row.get("deleted_at"):
|
||||
restored = restore_event(row["deleted_at"])
|
||||
if table == "awards":
|
||||
from devplacepy.database.awards import recompute_user_award_stats
|
||||
|
||||
recompute_user_award_stats(row.get("receiver_uid", ""))
|
||||
logger.info(
|
||||
f"Admin {admin['username']} restored {table} {uid} ({restored} rows)"
|
||||
)
|
||||
|
||||
36
devplacepy/routers/awards.py
Normal file
36
devplacepy/routers/awards.py
Normal file
@ -0,0 +1,36 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import RedirectResponse
|
||||
from devplacepy.attachments import _row_to_attachment
|
||||
from devplacepy.database import get_table, resolve_by_slug
|
||||
from devplacepy.utils import not_found
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_VALID_SIZES = {"512", "256", "64"}
|
||||
_CACHE_CONTROL = "public, max-age=86400, immutable"
|
||||
|
||||
|
||||
@router.get("/{slug_or_uid}/{size}")
|
||||
async def award_image(request: Request, slug_or_uid: str, size: str):
|
||||
if size not in _VALID_SIZES:
|
||||
raise not_found("Award image not found")
|
||||
award = resolve_by_slug(get_table("awards"), slug_or_uid)
|
||||
if not award or not award.get("generated_at"):
|
||||
raise not_found("Award image not found")
|
||||
attachment_uid = award.get(f"attachment_uid_{size}") or ""
|
||||
if not attachment_uid:
|
||||
raise not_found("Award image not found")
|
||||
row = get_table("attachments").find_one(uid=attachment_uid, deleted_at=None)
|
||||
attachment = _row_to_attachment(row) if row else None
|
||||
if not attachment:
|
||||
raise not_found("Award image not found")
|
||||
url = attachment.get("url") or ""
|
||||
if not url:
|
||||
raise not_found("Award image not found")
|
||||
headers = {
|
||||
"Cache-Control": _CACHE_CONTROL,
|
||||
"ETag": f'"{attachment_uid}"',
|
||||
}
|
||||
return RedirectResponse(url=url, status_code=302, headers=headers)
|
||||
@ -123,6 +123,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "awards",
|
||||
"title": "Profile awards",
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "ai-correction",
|
||||
"title": "AI content correction",
|
||||
|
||||
@ -4,6 +4,7 @@ from devplacepy.routers.profile import (
|
||||
ai_correction,
|
||||
ai_modifier,
|
||||
avatar,
|
||||
award,
|
||||
customization,
|
||||
notifications,
|
||||
telegram,
|
||||
@ -11,6 +12,7 @@ from devplacepy.routers.profile import (
|
||||
from devplacepy.routers.profile.index import router
|
||||
from devplacepy.routers.profile.usage import _ai_quota
|
||||
|
||||
router.include_router(award.router)
|
||||
router.include_router(customization.router)
|
||||
router.include_router(notifications.router)
|
||||
router.include_router(ai_correction.router)
|
||||
|
||||
106
devplacepy/routers/profile/award.py
Normal file
106
devplacepy/routers/profile/award.py
Normal file
@ -0,0 +1,106 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from devplacepy.database import get_blocked_uids, get_table
|
||||
from devplacepy.database.awards import can_give_award, has_giver_cooldown, has_receiver_cooldown
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import AwardGiveForm
|
||||
from devplacepy.responses import action_result, json_error, wants_json
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.jobs import queue
|
||||
from devplacepy.utils import generate_uid, make_combined_slug, require_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/{username}/award")
|
||||
async def give_award(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[AwardGiveForm, json_or_form(AwardGiveForm)],
|
||||
):
|
||||
giver = require_user(request)
|
||||
target = get_table("users").find_one(username=username)
|
||||
redirect = f"/profile/{username}"
|
||||
|
||||
def deny(message: str, status: int = 400):
|
||||
if wants_json(request):
|
||||
return json_error(status, message)
|
||||
return action_result(request, redirect, status_code=302)
|
||||
|
||||
if not target:
|
||||
return deny("User not found", 404)
|
||||
if target["uid"] == giver["uid"]:
|
||||
return deny("You cannot give yourself an award")
|
||||
blocked = get_blocked_uids(giver["uid"])
|
||||
if target["uid"] in blocked:
|
||||
return deny("You cannot give an award to a blocked user")
|
||||
reverse_blocked = get_blocked_uids(target["uid"])
|
||||
if giver["uid"] in reverse_blocked:
|
||||
return deny("You cannot give an award to this user")
|
||||
if has_giver_cooldown(giver["uid"]):
|
||||
return deny("You can give another award later")
|
||||
if has_receiver_cooldown(target["uid"]):
|
||||
return deny("This user received an award recently")
|
||||
if not (giver.get("api_key") or "").strip():
|
||||
return deny("Your account has no API key for award generation")
|
||||
|
||||
description = data.description.strip()
|
||||
uid = generate_uid()
|
||||
slug = make_combined_slug(description, uid)
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
get_table("awards").insert(
|
||||
{
|
||||
"uid": uid,
|
||||
"slug": slug,
|
||||
"description": description,
|
||||
"giver_uid": giver["uid"],
|
||||
"receiver_uid": target["uid"],
|
||||
"attachment_uid_512": "",
|
||||
"attachment_uid_256": "",
|
||||
"attachment_uid_64": "",
|
||||
"generated_at": None,
|
||||
"created_at": now,
|
||||
"job_uid": "",
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
job_uid = queue.enqueue(
|
||||
"award",
|
||||
{
|
||||
"award_uid": uid,
|
||||
"giver_uid": giver["uid"],
|
||||
"receiver_uid": target["uid"],
|
||||
"description": description,
|
||||
"api_key": giver.get("api_key", ""),
|
||||
},
|
||||
"user",
|
||||
giver["uid"],
|
||||
)
|
||||
get_table("awards").update({"uid": uid, "job_uid": job_uid}, ["uid"])
|
||||
logger.info("%s gave award %s to %s", giver["username"], uid, username)
|
||||
audit.record(
|
||||
request,
|
||||
"award.give",
|
||||
user=giver,
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=username,
|
||||
summary=f"{giver['username']} gave award to {username}",
|
||||
links=[
|
||||
audit.target("user", target["uid"], username),
|
||||
audit.target("award", uid, slug),
|
||||
audit.job(job_uid),
|
||||
],
|
||||
)
|
||||
return action_result(
|
||||
request,
|
||||
redirect,
|
||||
data={"ok": True, "award_uid": uid, "award_slug": slug},
|
||||
)
|
||||
@ -28,6 +28,11 @@ from devplacepy.database import (
|
||||
mark_notifications_read_by_target,
|
||||
resolve_object_url,
|
||||
)
|
||||
from devplacepy.database.awards import (
|
||||
can_give_award,
|
||||
get_prominent_award,
|
||||
get_user_awards,
|
||||
)
|
||||
from devplacepy.content import can_view_project, enrich_items
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
@ -147,6 +152,17 @@ async def profile_page(
|
||||
if tab == "media":
|
||||
media, media_pagination = get_user_media(profile_user["uid"], page)
|
||||
|
||||
awards, awards_pagination = [], None
|
||||
if tab == "awards":
|
||||
awards, awards_pagination = get_user_awards(profile_user["uid"], page)
|
||||
prominent_award = get_prominent_award(profile_user)
|
||||
awards_count = int(profile_user.get("award_count") or 0)
|
||||
can_give = bool(
|
||||
current_user
|
||||
and current_user["uid"] != profile_user["uid"]
|
||||
and can_give_award(current_user["uid"], profile_user["uid"])
|
||||
)
|
||||
|
||||
posts = []
|
||||
if tab == "posts":
|
||||
posts_table = get_table("posts")
|
||||
@ -404,6 +420,11 @@ async def profile_page(
|
||||
"follow_pagination": follow_pagination,
|
||||
"followers_count": follow_counts["followers"],
|
||||
"following_count": follow_counts["following"],
|
||||
"awards": awards,
|
||||
"awards_pagination": awards_pagination,
|
||||
"awards_count": awards_count,
|
||||
"prominent_award": prominent_award,
|
||||
"can_give_award": can_give,
|
||||
},
|
||||
model=ProfileOut,
|
||||
)
|
||||
|
||||
@ -114,6 +114,7 @@ from devplacepy.schemas.gateway import (
|
||||
GatewayUsageOut,
|
||||
UserAiUsageOut,
|
||||
)
|
||||
from devplacepy.schemas.statistics import StatisticsOut
|
||||
from devplacepy.schemas.auth import (
|
||||
AuthPageOut,
|
||||
DeviiPageOut,
|
||||
|
||||
19
devplacepy/schemas/awards.py
Normal file
19
devplacepy/schemas/awards.py
Normal file
@ -0,0 +1,19 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from typing import Optional
|
||||
|
||||
from devplacepy.schemas.base import _Out
|
||||
from devplacepy.schemas.content import UserOut
|
||||
|
||||
|
||||
class AwardOut(_Out):
|
||||
uid: str = ""
|
||||
slug: str = ""
|
||||
description: str = ""
|
||||
giver_uid: str = ""
|
||||
receiver_uid: str = ""
|
||||
generated_at: Optional[str] = None
|
||||
created_at: Optional[str] = None
|
||||
image_url: Optional[str] = None
|
||||
thumb_url: Optional[str] = None
|
||||
giver: Optional[UserOut] = None
|
||||
@ -6,6 +6,7 @@ from typing import Any, Optional
|
||||
|
||||
from devplacepy.schemas.base import _Out
|
||||
from devplacepy.schemas.content import BadgeOut, ProjectOut, UserOut
|
||||
from devplacepy.schemas.awards import AwardOut
|
||||
from devplacepy.schemas.listings import FeedItemOut, GistItemOut
|
||||
|
||||
|
||||
@ -70,6 +71,11 @@ class ProfileOut(_Out):
|
||||
media: list[MediaItemOut] = []
|
||||
media_pagination: Optional[Any] = None
|
||||
notification_prefs: list[Any] = []
|
||||
awards: list[AwardOut] = []
|
||||
awards_pagination: Optional[Any] = None
|
||||
awards_count: int = 0
|
||||
prominent_award: Optional[AwardOut] = None
|
||||
can_give_award: bool = False
|
||||
|
||||
|
||||
class TelegramPairOut(_Out):
|
||||
|
||||
52
devplacepy/schemas/statistics.py
Normal file
52
devplacepy/schemas/statistics.py
Normal file
@ -0,0 +1,52 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class StatisticsMetricOut(BaseModel):
|
||||
key: str
|
||||
label: str
|
||||
value: Any
|
||||
format: str = "int"
|
||||
delta: Optional[float] = None
|
||||
direction: Optional[str] = None
|
||||
|
||||
|
||||
class StatisticsPointOut(BaseModel):
|
||||
t: str
|
||||
v: float
|
||||
|
||||
|
||||
class StatisticsSeriesOut(BaseModel):
|
||||
key: str
|
||||
label: str
|
||||
points: list[StatisticsPointOut]
|
||||
|
||||
|
||||
class StatisticsTableOut(BaseModel):
|
||||
key: str
|
||||
title: str
|
||||
columns: list[str]
|
||||
rows: list[list[Any]]
|
||||
|
||||
|
||||
class StatisticsHighlightOut(BaseModel):
|
||||
label: str
|
||||
value: Any
|
||||
|
||||
|
||||
class StatisticsOut(BaseModel):
|
||||
tab: str
|
||||
window_hours: int
|
||||
granularity: str
|
||||
generated_at: str
|
||||
compare: bool = True
|
||||
cards: list[StatisticsMetricOut] = []
|
||||
series: list[StatisticsSeriesOut] = []
|
||||
tables: list[StatisticsTableOut] = []
|
||||
highlights: list[StatisticsHighlightOut] = []
|
||||
notes: dict[str, Any] = {}
|
||||
@ -4,6 +4,7 @@ CATEGORY_BY_PREFIX: dict[str, str] = {
|
||||
"auth": "auth",
|
||||
"profile": "account",
|
||||
"follow": "social",
|
||||
"award": "social",
|
||||
"relation": "social",
|
||||
"push": "push",
|
||||
"notification": "notification",
|
||||
|
||||
@ -64,6 +64,16 @@ class BackgroundQueue:
|
||||
if inline:
|
||||
self._inline += 1
|
||||
logger.warning("background task %s failed: %s", getattr(fn, "__name__", fn), exc)
|
||||
finally:
|
||||
self._release_db_lock()
|
||||
|
||||
def _release_db_lock(self) -> None:
|
||||
try:
|
||||
from devplacepy.database import refresh_snapshot
|
||||
|
||||
refresh_snapshot()
|
||||
except Exception as exc:
|
||||
logger.warning("background queue could not release db lock: %s", exc)
|
||||
|
||||
async def start(self) -> None:
|
||||
if self.running:
|
||||
|
||||
@ -31,6 +31,25 @@ ADMIN_ACTIONS: tuple[Action, ...] = (
|
||||
params=(query("top_n", "How many top authors to include (1-50)."),),
|
||||
requires_admin=True,
|
||||
),
|
||||
Action(
|
||||
name="admin_statistics",
|
||||
method="GET",
|
||||
path="/admin/statistics/data",
|
||||
summary="Platform statistics tab data with trends (admin only)",
|
||||
description=(
|
||||
"Returns JSON for one statistics tab: KPI cards with period-over-period deltas, "
|
||||
"time-series points for line charts, breakdown tables, and highlight metrics. "
|
||||
"Tabs: overview, visitors, members, content, engagement, social, ai, devii, "
|
||||
"services, containers, game, awards, moderation, tools, storage."
|
||||
),
|
||||
params=(
|
||||
query("tab", "Tab key (default overview)."),
|
||||
query("hours", "Lookback window in hours (24, 168, 720, 2160, or 0 for all time)."),
|
||||
query("compare", "Include previous-period comparison (1 or 0, default 1)."),
|
||||
query("top_n", "Rows in breakdown tables (default 10)."),
|
||||
),
|
||||
requires_admin=True,
|
||||
),
|
||||
Action(
|
||||
name="ai_usage",
|
||||
method="GET",
|
||||
@ -305,6 +324,18 @@ ADMIN_ACTIONS: tuple[Action, ...] = (
|
||||
params=(path("uid", "Attachment uid to purge."), confirm()),
|
||||
requires_admin=True,
|
||||
),
|
||||
Action(
|
||||
name="admin_revoke_award",
|
||||
method="POST",
|
||||
path="/admin/awards/{uid}/revoke",
|
||||
summary="Revoke a published award (admin only)",
|
||||
description=(
|
||||
"Soft-deletes the award and linked attachments, then recomputes receiver stats. "
|
||||
"Confirmation is required in the UI; Devii should confirm before calling."
|
||||
),
|
||||
params=(path("uid", "Award uid to revoke."), confirm()),
|
||||
requires_admin=True,
|
||||
),
|
||||
Action(
|
||||
name="admin_reset_guest_ai_quota",
|
||||
method="POST",
|
||||
|
||||
@ -27,7 +27,7 @@ GATEWAY_ACTIONS: tuple[Action, ...] = (
|
||||
summary="Create or update a gateway provider (admin only)",
|
||||
description=(
|
||||
"Adds or updates a named upstream provider. base_url is the OpenAI-compatible "
|
||||
"chat-completions endpoint; the embeddings endpoint is derived from it."
|
||||
"chat-completions endpoint; the embeddings and images endpoints are derived from it."
|
||||
),
|
||||
handler="http",
|
||||
requires_admin=True,
|
||||
@ -60,7 +60,8 @@ GATEWAY_ACTIONS: tuple[Action, ...] = (
|
||||
summary="List OpenAI gateway model routes (admin only)",
|
||||
description=(
|
||||
"Returns JSON: every source-model route with its provider, target model, kind "
|
||||
"(chat/embed), optional vision model, context window, and per-model pricing economy."
|
||||
"(chat/embed/image), optional vision model, context window, and per-model pricing economy "
|
||||
"(including any tiered/off-peak pricing configured on it)."
|
||||
),
|
||||
handler="http",
|
||||
requires_admin=True,
|
||||
@ -73,9 +74,14 @@ GATEWAY_ACTIONS: tuple[Action, ...] = (
|
||||
summary="Create or update a gateway model route (admin only)",
|
||||
description=(
|
||||
"Maps a requested source_model onto a provider + target_model, each with its own "
|
||||
"pricing. kind is 'chat' or 'embed'. A vision_model adds image-to-text augmentation "
|
||||
"for chat routes. Prices are USD per 1,000,000 tokens; chat uses cache-hit/cache-miss/"
|
||||
"output, embeddings use input, vision uses input/output."
|
||||
"pricing. kind is 'chat', 'embed', or 'image'. A vision_model adds image-to-text augmentation "
|
||||
"for chat routes. Prices are USD per 1,000,000 tokens for chat/embed/vision; image routes "
|
||||
"use price_input_per_m as a flat USD per generated image. Optionally, a route can also "
|
||||
"charge a different (tier-2) rate once the request's input tokens exceed "
|
||||
"context_tier_threshold_tokens, and/or apply a percentage discount during a fixed "
|
||||
"UTC off-peak window - leave the tier2/off-peak fields unset to keep the flat rates "
|
||||
"above at all times. off_peak_start_minute and off_peak_end_minute must be set "
|
||||
"together (both or neither) or the call is rejected."
|
||||
),
|
||||
handler="http",
|
||||
requires_admin=True,
|
||||
@ -83,14 +89,22 @@ GATEWAY_ACTIONS: tuple[Action, ...] = (
|
||||
body("source_model", "Model name clients request.", required=True),
|
||||
body("provider", "Provider name, or blank for the default upstream."),
|
||||
body("target_model", "Model name sent upstream.", required=True),
|
||||
body("kind", "Route kind: 'chat' or 'embed'."),
|
||||
body("kind", "Route kind: 'chat', 'embed', or 'image'."),
|
||||
body("vision_provider", "Provider for image description, or blank for the route provider."),
|
||||
body("vision_model", "Vision model name (blank disables the merge)."),
|
||||
Param(name="context_window", location="body", description="Max context tokens (0 = unknown).", required=False, type="integer"),
|
||||
Param(name="price_cache_hit_per_m", location="body", description="USD per 1M cache-hit input tokens.", required=False, type="number"),
|
||||
Param(name="price_cache_miss_per_m", location="body", description="USD per 1M cache-miss input tokens.", required=False, type="number"),
|
||||
Param(name="price_output_per_m", location="body", description="USD per 1M output tokens.", required=False, type="number"),
|
||||
Param(name="price_input_per_m", location="body", description="USD per 1M input tokens (embed/vision).", required=False, type="number"),
|
||||
Param(name="price_input_per_m", location="body", description="USD per 1M input tokens (embed/vision), or flat USD per image for image routes.", required=False, type="number"),
|
||||
Param(name="context_tier_threshold_tokens", location="body", description="Input tokens above which tier-2 rates apply (0 disables tiering).", required=False, type="integer"),
|
||||
Param(name="price_cache_hit_per_m_tier2", location="body", description="Tier-2 USD per 1M cache-hit input tokens (unset = keep tier-1 rate above threshold).", required=False, type="number"),
|
||||
Param(name="price_cache_miss_per_m_tier2", location="body", description="Tier-2 USD per 1M cache-miss input tokens (unset = keep tier-1 rate above threshold).", required=False, type="number"),
|
||||
Param(name="price_output_per_m_tier2", location="body", description="Tier-2 USD per 1M output tokens (unset = keep tier-1 rate above threshold).", required=False, type="number"),
|
||||
Param(name="price_input_per_m_tier2", location="body", description="Tier-2 USD per 1M input tokens, embed/vision (unset = keep tier-1 rate above threshold).", required=False, type="number"),
|
||||
Param(name="off_peak_start_minute", location="body", description="Off-peak window start, UTC minutes since midnight (0-1439). Must be set together with off_peak_end_minute.", required=False, type="integer"),
|
||||
Param(name="off_peak_end_minute", location="body", description="Off-peak window end, UTC minutes since midnight (0-1439). A value less than the start wraps past midnight.", required=False, type="integer"),
|
||||
Param(name="off_peak_discount_pct", location="body", description="Percentage discount (0-100) applied to the active tier's rates during the off-peak window.", required=False, type="number"),
|
||||
Param(name="is_active", location="body", description="Whether the route is active ('1' or '0').", required=False, type="boolean"),
|
||||
),
|
||||
),
|
||||
|
||||
@ -61,6 +61,21 @@ PROFILE_ACTIONS: tuple[Action, ...] = (
|
||||
),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="give_award",
|
||||
method="POST",
|
||||
path="/profile/{username}/award",
|
||||
summary="Give another member an award on their profile",
|
||||
description=(
|
||||
"Creates a pending award row and enqueues image generation billed to the giver's "
|
||||
"API key. The receiver is notified when generation completes. Cannot award yourself; "
|
||||
"blocked pairs are rejected; cooldowns apply."
|
||||
),
|
||||
params=(
|
||||
path("username", "Receiver username."),
|
||||
body("description", "Short award message (1-125 characters).", required=True),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="regenerate_avatar",
|
||||
method="POST",
|
||||
|
||||
@ -18,7 +18,7 @@ def arg(
|
||||
|
||||
|
||||
TYPES = (
|
||||
"One of: comment, reply, mention, vote, follow, message, badge, level, issue."
|
||||
"One of: comment, reply, mention, vote, follow, message, badge, level, issue, award."
|
||||
)
|
||||
|
||||
NOTIFICATION_ACTIONS: tuple[Action, ...] = (
|
||||
|
||||
@ -21,32 +21,6 @@ from ..text import html_to_text
|
||||
|
||||
logger = logging.getLogger("devii.fetch")
|
||||
|
||||
CHROME_VERSION = "131"
|
||||
CHROME_VERSION_FULL = "131.0.0.0"
|
||||
STEALTH_HEADERS = {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
|
||||
f"(KHTML, like Gecko) Chrome/{CHROME_VERSION_FULL} Safari/537.36"
|
||||
),
|
||||
"Accept": (
|
||||
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,"
|
||||
"image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
|
||||
),
|
||||
"Accept-Language": "en-US,en;q=0.9",
|
||||
"sec-ch-ua": (
|
||||
f'"Google Chrome";v="{CHROME_VERSION}", "Chromium";v="{CHROME_VERSION}", '
|
||||
'"Not_A Brand";v="24"'
|
||||
),
|
||||
"sec-ch-ua-mobile": "?0",
|
||||
"sec-ch-ua-platform": '"Linux"',
|
||||
"Sec-Fetch-Dest": "document",
|
||||
"Sec-Fetch-Mode": "navigate",
|
||||
"Sec-Fetch-Site": "none",
|
||||
"Sec-Fetch-User": "?1",
|
||||
"Upgrade-Insecure-Requests": "1",
|
||||
"Cache-Control": "max-age=0",
|
||||
"DNT": "1",
|
||||
}
|
||||
TITLE = re.compile(r"<title[^>]*>(.*?)</title>", re.IGNORECASE | re.DOTALL)
|
||||
MIN_FETCH_CHARS = 1000
|
||||
ALLOWED_METHODS = ("GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS")
|
||||
@ -230,9 +204,7 @@ class FetchController:
|
||||
raise_5xx: bool = False,
|
||||
) -> tuple[str, str, str, int, dict[str, str]]:
|
||||
limit = self._settings.fetch_max_bytes
|
||||
merged_headers = dict(STEALTH_HEADERS)
|
||||
if headers:
|
||||
merged_headers.update({str(k): str(v) for k, v in headers.items()})
|
||||
merged_headers = {str(k): str(v) for k, v in headers.items()} if headers else {}
|
||||
request_kwargs: dict[str, Any] = {}
|
||||
if json_body is not None:
|
||||
request_kwargs["json"] = json_body
|
||||
@ -240,43 +212,48 @@ class FetchController:
|
||||
request_kwargs["data"] = form
|
||||
elif content is not None:
|
||||
request_kwargs["content"] = content
|
||||
|
||||
async def _once(client: httpx.AsyncClient, target: str) -> tuple[str, str, str, int, dict[str, str]]:
|
||||
async with client.stream(method, target, **request_kwargs) as response:
|
||||
if raise_5xx and response.status_code >= 500:
|
||||
raise UpstreamError(
|
||||
f"Server returned {response.status_code}.",
|
||||
status=response.status_code,
|
||||
url=target,
|
||||
)
|
||||
chunks: list[bytes] = []
|
||||
total = 0
|
||||
async for chunk in response.aiter_bytes():
|
||||
chunks.append(chunk)
|
||||
total += len(chunk)
|
||||
if total >= limit:
|
||||
break
|
||||
raw = b"".join(chunks)[:limit]
|
||||
encoding = response.encoding or "utf-8"
|
||||
try:
|
||||
body = raw.decode(encoding, errors="replace")
|
||||
except LookupError:
|
||||
body = raw.decode("utf-8", errors="replace")
|
||||
content_type = response.headers.get("content-type", "").lower()
|
||||
response_headers = {key: value for key, value in response.headers.items()}
|
||||
return body, str(response.url), content_type, response.status_code, response_headers
|
||||
|
||||
try:
|
||||
async with stealth.stealth_async_client(
|
||||
headers=merged_headers,
|
||||
follow_redirects=True,
|
||||
timeout=self._settings.fetch_timeout_seconds,
|
||||
) as client:
|
||||
async with client.stream(method, url, **request_kwargs) as response:
|
||||
if raise_5xx and response.status_code >= 500:
|
||||
raise UpstreamError(
|
||||
f"Server returned {response.status_code}.",
|
||||
status=response.status_code,
|
||||
url=url,
|
||||
)
|
||||
chunks: list[bytes] = []
|
||||
total = 0
|
||||
async for chunk in response.aiter_bytes():
|
||||
chunks.append(chunk)
|
||||
total += len(chunk)
|
||||
if total >= limit:
|
||||
break
|
||||
raw = b"".join(chunks)[:limit]
|
||||
encoding = response.encoding or "utf-8"
|
||||
try:
|
||||
body = raw.decode(encoding, errors="replace")
|
||||
except LookupError:
|
||||
body = raw.decode("utf-8", errors="replace")
|
||||
content_type = response.headers.get("content-type", "").lower()
|
||||
response_headers = {
|
||||
key: value for key, value in response.headers.items()
|
||||
}
|
||||
return (
|
||||
body,
|
||||
str(response.url),
|
||||
content_type,
|
||||
response.status_code,
|
||||
response_headers,
|
||||
)
|
||||
result = await _once(client, url)
|
||||
if method == "GET" and "html" in result[2]:
|
||||
gate_url = stealth.detect_consent_gate(result[0])
|
||||
if gate_url:
|
||||
try:
|
||||
await client.get(gate_url)
|
||||
result = await _once(client, url)
|
||||
except httpx.HTTPError as exc:
|
||||
logger.debug("consent gate follow-up failed for %s: %s", url, exc)
|
||||
return result
|
||||
except httpx.TimeoutException as exc:
|
||||
raise NetworkError(f"Request timed out fetching {url}", url=url) from exc
|
||||
except httpx.HTTPError as exc:
|
||||
|
||||
207
devplacepy/services/jobs/award_service.py
Normal file
207
devplacepy/services/jobs/award_service.py
Normal file
@ -0,0 +1,207 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from io import BytesIO
|
||||
|
||||
from devplacepy import stealth
|
||||
from devplacepy.attachments import link_attachments, store_attachment
|
||||
from devplacepy.awards.images import enforce_rgba_png, resize_award_png
|
||||
from devplacepy.config import (
|
||||
AWARD_GENERATION_TIMEOUT_SECONDS,
|
||||
AWARD_IMAGE_MODEL_DEFAULT,
|
||||
AWARD_IMAGE_PROMPT_DEFAULT,
|
||||
AWARD_IMAGE_SIZE_DEFAULT,
|
||||
INTERNAL_BASE_URL,
|
||||
)
|
||||
from devplacepy.database import add_award_usage, get_award_usage, get_setting, get_table
|
||||
from devplacepy.database.awards import recompute_user_award_stats
|
||||
from devplacepy.services.base import ConfigField
|
||||
from devplacepy.services.correction import new_usage_totals
|
||||
from devplacepy.services.jobs.base import JobService
|
||||
from devplacepy.services.openai_gateway.usage import accumulate_usage, usage_metric_cards
|
||||
from devplacepy.utils import create_notification
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AwardService(JobService):
|
||||
kind = "award"
|
||||
title = "Awards"
|
||||
description = (
|
||||
"Generates award images off the request path, stores PNG attachments, and "
|
||||
"notifies receivers when ready."
|
||||
)
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(name="award", interval_seconds=2)
|
||||
self.config_fields = list(self.config_fields) + [
|
||||
ConfigField(
|
||||
"award_give_cooldown_hours",
|
||||
"Give cooldown (hours)",
|
||||
type="int",
|
||||
default=24,
|
||||
minimum=1,
|
||||
maximum=168,
|
||||
group="Awards",
|
||||
),
|
||||
ConfigField(
|
||||
"award_receive_cooldown_hours",
|
||||
"Receive cooldown (hours)",
|
||||
type="int",
|
||||
default=24,
|
||||
minimum=1,
|
||||
maximum=168,
|
||||
group="Awards",
|
||||
),
|
||||
ConfigField(
|
||||
"award_display_hours",
|
||||
"Prominent display (hours)",
|
||||
type="int",
|
||||
default=24,
|
||||
minimum=1,
|
||||
maximum=168,
|
||||
group="Awards",
|
||||
),
|
||||
ConfigField(
|
||||
"award_image_model",
|
||||
"Image model",
|
||||
type="str",
|
||||
default=AWARD_IMAGE_MODEL_DEFAULT,
|
||||
group="Awards",
|
||||
),
|
||||
ConfigField(
|
||||
"award_image_prompt",
|
||||
"Image prompt",
|
||||
type="text",
|
||||
default=AWARD_IMAGE_PROMPT_DEFAULT,
|
||||
group="Awards",
|
||||
),
|
||||
]
|
||||
|
||||
async def process(self, job: dict) -> dict:
|
||||
totals = new_usage_totals()
|
||||
try:
|
||||
result = await asyncio.to_thread(self._generate, job, totals)
|
||||
except Exception as exc:
|
||||
self._audit_failed(job, str(exc))
|
||||
raise
|
||||
if totals["calls"]:
|
||||
add_award_usage(totals)
|
||||
return result
|
||||
|
||||
def _generate(self, job: dict, totals: dict) -> dict:
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
payload = job.get("payload") or {}
|
||||
award_uid = str(payload.get("award_uid", ""))
|
||||
giver_uid = str(payload.get("giver_uid", ""))
|
||||
receiver_uid = str(payload.get("receiver_uid", ""))
|
||||
description = str(payload.get("description", ""))
|
||||
api_key = str(payload.get("api_key", ""))
|
||||
awards = get_table("awards")
|
||||
row = awards.find_one(uid=award_uid)
|
||||
if not row or row.get("deleted_at") or row.get("generated_at"):
|
||||
return {"award_uid": award_uid, "skipped": True}
|
||||
source_png = self._fetch_image(api_key, description, totals)
|
||||
source_png = enforce_rgba_png(source_png)
|
||||
uid512 = store_attachment(
|
||||
resize_award_png(source_png, 512), "award-512.png", receiver_uid
|
||||
)["uid"]
|
||||
uid256 = store_attachment(
|
||||
resize_award_png(source_png, 256), "award-256.png", giver_uid
|
||||
)["uid"]
|
||||
uid64 = store_attachment(
|
||||
resize_award_png(source_png, 64), "award-64.png", giver_uid
|
||||
)["uid"]
|
||||
link_attachments([uid512, uid256, uid64], "award", award_uid)
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
awards.update(
|
||||
{
|
||||
"uid": award_uid,
|
||||
"attachment_uid_512": uid512,
|
||||
"attachment_uid_256": uid256,
|
||||
"attachment_uid_64": uid64,
|
||||
"generated_at": now,
|
||||
},
|
||||
["uid"],
|
||||
)
|
||||
recompute_user_award_stats(receiver_uid)
|
||||
users = get_table("users")
|
||||
giver = users.find_one(uid=giver_uid) or {}
|
||||
receiver = users.find_one(uid=receiver_uid) or {}
|
||||
slug = row.get("slug", "")
|
||||
create_notification(
|
||||
receiver_uid,
|
||||
"award",
|
||||
f"@{giver.get('username', 'someone')} gave you an award",
|
||||
giver_uid,
|
||||
f"/profile/{receiver.get('username', '')}?tab=awards#award-{slug}",
|
||||
)
|
||||
audit.record_system(
|
||||
"award.complete",
|
||||
actor_kind="service",
|
||||
target_type="award",
|
||||
target_uid=award_uid,
|
||||
target_label=description[:80],
|
||||
summary=f"Award ready for {receiver.get('username', receiver_uid)}",
|
||||
links=[
|
||||
audit.target("award", award_uid, slug),
|
||||
audit.target("user", receiver_uid, receiver.get("username")),
|
||||
audit.target("user", giver_uid, giver.get("username")),
|
||||
audit.job(job.get("uid", "")),
|
||||
],
|
||||
)
|
||||
return {"award_uid": award_uid, "slug": slug}
|
||||
|
||||
def _fetch_image(self, api_key: str, description: str, totals: dict) -> bytes:
|
||||
prompt = get_setting("award_image_prompt", AWARD_IMAGE_PROMPT_DEFAULT)
|
||||
model = get_setting("award_image_model", AWARD_IMAGE_MODEL_DEFAULT)
|
||||
size = AWARD_IMAGE_SIZE_DEFAULT
|
||||
body = {
|
||||
"model": model,
|
||||
"prompt": f"{prompt}\n\n{description}".strip(),
|
||||
"size": size,
|
||||
"background": "transparent",
|
||||
"output_format": "png",
|
||||
}
|
||||
url = f"{INTERNAL_BASE_URL}/openai/v1/images/generations"
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
with stealth.stealth_sync_client(timeout=AWARD_GENERATION_TIMEOUT_SECONDS) as client:
|
||||
response = client.post(url, json=body, headers=headers)
|
||||
accumulate_usage(totals, response)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
items = data.get("data") or []
|
||||
if not items:
|
||||
raise RuntimeError("image API returned no data")
|
||||
encoded = items[0].get("b64_json") or ""
|
||||
if not encoded:
|
||||
raise RuntimeError("image API returned no b64_json")
|
||||
return base64.b64decode(encoded)
|
||||
|
||||
def _audit_failed(self, job: dict, error: str) -> None:
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
payload = job.get("payload") or {}
|
||||
award_uid = str(payload.get("award_uid", ""))
|
||||
audit.record_system(
|
||||
"award.failed",
|
||||
actor_kind="service",
|
||||
target_type="award",
|
||||
target_uid=award_uid,
|
||||
summary=f"Award generation failed: {error[:80]}",
|
||||
metadata={"error": error[:500], "award_uid": award_uid},
|
||||
result="failure",
|
||||
links=[audit.job(job.get("uid", "")), audit.target("award", award_uid)],
|
||||
)
|
||||
|
||||
def collect_metrics(self) -> dict:
|
||||
base = super().collect_metrics()
|
||||
base["stats"] = base.get("stats", []) + usage_metric_cards(get_award_usage())
|
||||
return base
|
||||
@ -52,6 +52,10 @@ The gateway records one row per upstream call (chat, vision, passthrough) and su
|
||||
|
||||
**Financial data is admin-only everywhere.** Any monetary figure (USD cost, pricing, spend, limit) is restricted to administrators; members and guests see only the percentage of quota used - this rule is enforced consistently across the profile card, `ai_correction`/`ai_modifier` usage displays, and the Devii cost tools.
|
||||
|
||||
## Image generation
|
||||
|
||||
`POST /openai/v1/images/generations` exposes an OpenAI-compatible image-generation endpoint. Clients send the generic model `molodetz-img-small` (`config.INTERNAL_IMAGE_MODEL`), which `handle_images` remaps to `gateway_image_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty or `molodetz-img`). It defaults to OpenRouter's `black-forest-labs/flux-1.1-pro` at `https://openrouter.ai/api/v1/images/generations` (`config.IMAGE_*_DEFAULT`, $0.04 per image fallback). `handle_images` mirrors `handle_embeddings`: build the payload, forward via `_send`, and record one ledger row. The config fields are the **Images** group (`gateway_image_enabled` default on, `gateway_image_url`, `gateway_image_model`, `gateway_image_key`) plus the Pricing-group `gateway_image_price_per_call`. `effective_config()` falls the image key back to `gateway_api_key` then `OPENROUTER_API_KEY`. Usage is recorded with **`backend="image"`**; `usage.compute_cost` adds an `image` branch (flat per-call, native OpenRouter `cost` still preferred via `extract_image_usage`). `routing.image_overlay` resolves per-route provider/url/key and uses `price_input_per_m` as the per-image price. `routing.seed_default_image_routes()` (from `migrate_ai_gateway_settings`) idempotently seeds `molodetz-img-small` -> Flux on the `openrouter` provider when `OPENROUTER_API_KEY` is set. When `gateway_image_enabled` is off the endpoint returns 503 with no ledger row.
|
||||
|
||||
## Embeddings
|
||||
|
||||
`POST /openai/v1/embeddings` exposes an OpenAI-compatible text-embeddings model. Clients send the generic model `molodetz~embed` (`config.INTERNAL_EMBED_MODEL`), which `handle_embeddings` remaps to `gateway_embed_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty). It defaults to OpenRouter's `qwen/qwen3-embedding-8b` at `https://openrouter.ai/api/v1/embeddings` (`config.EMBED_*_DEFAULT`, $0.01 per 1M input tokens). `handle_embeddings` mirrors `handle_chat` but is simpler: no vision augmentation and no streaming - build the payload, forward via `_send`, and record one ledger row through the same `finalize(...)` closure. The config fields are the **Embeddings** group (`gateway_embed_enabled` default on, `gateway_embed_url`, `gateway_embed_model`, `gateway_embed_key`) plus the Pricing-group `gateway_embed_price_input_per_m`. `effective_config()` falls the embed key back to `gateway_vision_key` then `OPENROUTER_API_KEY` (NOT `gateway_api_key`: that is the DeepSeek chat upstream key, whereas embeddings target OpenRouter like vision does). Usage is recorded with **`backend="embed"`**; `usage.compute_cost` adds an `embed` branch (input-only, completion always 0, native OpenRouter `cost` still preferred) and `Pricing` gained `embed_input_per_m`. `analytics.py` groups by `backend` generically, so embed rows roll up automatically; `caching_savings` counts only **non-native** chat rows (native-priced rows did not use the configured cache-hit/miss rates, so folding them in would report a fictional saving). When `gateway_embed_enabled` is off the endpoint returns 503 with no ledger row.
|
||||
@ -75,11 +79,11 @@ Layered ON TOP of the single-provider service config above, which stays THE impl
|
||||
**Storage.** Two dataset tables, ensured in `init_db` via `routing.ensure_tables`, cross-worker cache-invalidated under the `"gateway_routing"` cache-version name (module-level `provider_store`/`model_store` over a shared `_ROUTING_CACHE`; writes `bump_cache_version` and clear the cache). They are admin config, NOT in `SOFT_DELETE_TABLES` - hard CRUD, mirroring `site_settings`:
|
||||
|
||||
- `gateway_providers` - named upstreams: `name` + `base_url` (chat-completions URL) + `api_key` + `is_active`; the embeddings URL is derived by swapping `/chat/completions` -> `/embeddings`.
|
||||
- `gateway_models` - source->target routes: `source_model` (unique, what clients request) -> `provider` (blank = default) + `target_model` + `kind` (chat|embed) + optional `vision_provider`/`vision_model` for the **text+vision merge** (when set, image content is described by that vision model before forwarding) + `context_window` + its **own economy** (`price_cache_hit_per_m`/`price_cache_miss_per_m`/`price_output_per_m`/`price_input_per_m`, USD per 1M tokens) + `is_active`.
|
||||
- `gateway_models` - source->target routes: `source_model` (unique, what clients request) -> `provider` (blank = default) + `target_model` + `kind` (chat|embed|image) + optional `vision_provider`/`vision_model` for the **text+vision merge** (when set, image content is described by that vision model before forwarding) + `context_window` + its **own economy** (`price_cache_hit_per_m`/`price_cache_miss_per_m`/`price_output_per_m`/`price_input_per_m`, USD per 1M tokens for chat/embed/vision; for image routes `price_input_per_m` is a flat USD per image, plus the tiered/off-peak fields below) + `is_active`.
|
||||
|
||||
**Resolution is a per-request overlay, not a fork.** At request time `routing.chat_overlay(requested, cfg)` / `routing.embed_overlay(requested, cfg)` resolve an active route by the requested model name and return a per-request OVERLAY dict of `gateway_*` cfg keys (`gateway_force_model`+`gateway_model`=target, `gateway_upstream_url`/`gateway_api_key` from the provider, the price keys, an augmented `gateway_model_context_map`, and vision overlay keys); `handle_chat`/`handle_embeddings` merge it onto the base cfg (`cfg = {**cfg, **overlay}`) BEFORE everything else, so the existing model-selection / `pricing_from_cfg` / `parse_context_map` / vision / url+key paths transparently use the route's provider, target model, pricing, vision model and context window. `_ensure` (the httpx pool / semaphore / breaker / vision cache) reads only the non-overlaid pool keys, so the connection pool is never churned per request.
|
||||
**Resolution is a per-request overlay, not a fork.** At request time `routing.chat_overlay(requested, cfg)` / `routing.embed_overlay(requested, cfg)` / `routing.image_overlay(requested, cfg)` resolve an active route by the requested model name and return a per-request OVERLAY dict of `gateway_*` cfg keys (`gateway_force_model`+`gateway_model`=target, `gateway_upstream_url`/`gateway_api_key` from the provider, the price keys, an augmented `gateway_model_context_map`, and vision overlay keys); `handle_chat`/`handle_embeddings` merge it onto the base cfg (`cfg = {**cfg, **overlay}`) BEFORE everything else, so the existing model-selection / `pricing_from_cfg` / `parse_context_map` / vision / url+key paths transparently use the route's provider, target model, pricing, vision model and context window. `_ensure` (the httpx pool / semaphore / breaker / vision cache) reads only the non-overlaid pool keys, so the connection pool is never churned per request.
|
||||
|
||||
**No matching route = `None` overlay = byte-identical legacy behavior** - this is the "nobody feels the transformation" guarantee: `molodetz`/`molodetz~embed` and every existing caller are byte-identical when no route matches. A route with a blank provider overlays only model+pricing(+vision), keeping the default upstream url/key.
|
||||
**No matching route = `None` overlay = byte-identical legacy behavior** - this is the "nobody feels the transformation" guarantee: `molodetz`/`molodetz~embed`/`molodetz-img-small` and every existing caller are byte-identical when no route matches. A route with a blank provider overlays only model+pricing(+vision), keeping the default upstream url/key.
|
||||
|
||||
**CRUD.** Admin JSON at `/admin/gateway/{providers,models}` (`routers/admin/gateway_configs.py`, `require_admin`, Pydantic `ProviderIn`/`ModelRouteIn` validation, accepts both JSON from `static/js/GatewayAdmin.js` and form from Devii), audited under `gateway.provider.*`/`gateway.model.*` (category `ai`), included in the admin package with the page at `/admin/gateway` (`templates/admin_gateway.html`, sidebar link, `admin_section="gateway"`).
|
||||
|
||||
@ -88,3 +92,13 @@ Layered ON TOP of the single-provider service config above, which stays THE impl
|
||||
When adding a routed value, overlay it as the matching `gateway_*` cfg key so the runtime needs no new branch.
|
||||
|
||||
**Tests:** `tests/unit/services/openai_gateway/routing.py` (overlay/economy/kind isolation), `tests/unit/services/openai_gateway/gateway.py::test_model_route_overrides_upstream` (end-to-end through `handle_chat`), and `tests/api/admin/gateway/` (admin CRUD + validation + role gating).
|
||||
|
||||
## Tiered (context-length) and off-peak pricing (model-agnostic variable pricing)
|
||||
|
||||
Real providers sometimes charge more than a flat per-1M rate for one component: a rate that jumps once a request crosses a context-length threshold, or a fixed time-of-day discount window. This is layered on top of the cache-hit/cache-miss/output shape (which is already exactly DeepSeek's real billing model - see below), on the SAME `gateway_models` route row, so it stays fully provider-agnostic and opt-in per route.
|
||||
|
||||
- **Per-route fields** (each optional/zero by default = feature off, so an unconfigured route is byte-identical to before): `context_tier_threshold_tokens` (0 disables tiering; when the request's input token count exceeds it, `price_*_per_m_tier2` rates apply instead of the tier-1 rates above) and the four nullable `price_cache_hit_per_m_tier2`/`price_cache_miss_per_m_tier2`/`price_output_per_m_tier2`/`price_input_per_m_tier2` (a component left `None` keeps its tier-1 rate even above threshold - so a provider that only re-prices input above a size threshold, keeping output flat, needs just one tier2 field set), plus `off_peak_start_minute`/`off_peak_end_minute` (nullable, UTC minutes-since-midnight, both-or-neither enforced by `ModelRouteIn`'s model validator; a window where start > end wraps past midnight, e.g. DeepSeek's old V3/R1-era 16:30-00:30 UTC window) and `off_peak_discount_pct` (0-100, multiplies whichever tier rate is active). The same fields double for vision (`price_input/output_per_m_tier2`) and embed (`price_input_per_m_tier2`) exactly like their tier-1 counterparts already do.
|
||||
- **Overlay + computation.** `chat_overlay`/`embed_overlay` propagate all of these into the per-request cfg dict under `gateway_*` keys exactly like the existing price fields (section above); `usage.pricing_from_cfg` reads them generically (defaulting to `None`/`0`/disabled when absent), so **Layer A (the single global flat Pricing config fields) never gains these dimensions** - only a `gateway_models` route can enable them, preserving the "no matching route = byte-identical legacy behavior" guarantee. `usage.compute_cost` selects tier1 vs tier2 per rate component (`_tiered_rate`) based on whether `norm["prompt"]` (billable input tokens) exceeds the threshold, then applies the off-peak discount (`_effective_rate`/`_off_peak_active`, UTC wraparound-aware) to whichever rate was selected. **The response header format and `compute_cost`'s return shape (`total, input_cost, output_cost, native`) are unchanged** - this is purely an internal rate-selection step before the existing input/output split math runs; a native upstream `cost` (OpenRouter) still overrides the modeled total exactly as before.
|
||||
- **Migration.** New `gateway_models` columns are added via `has_column`/`create_column_by_example` in `routing.ensure_tables()` (the `CREATE TABLE IF NOT EXISTS` DDL string alone would never reach a pre-existing table - see the `database/CLAUDE.md` column-ensure idiom).
|
||||
- **Admin UI.** `/admin/gateway`'s model-route form has a "Tiered / off-peak pricing (optional)" subsection; off-peak start/end render as `<input type="time">` (converted to/from UTC minutes-of-day by `GatewayAdmin.js`), and the routes table shows `tiered`/`off-peak` badges when a route has either dimension configured.
|
||||
- **DeepSeek's real pricing is already the tier-1 shape, not a new dimension.** DeepSeek's actual API (verified against `api-docs.deepseek.com/quick_start/pricing`) bills three flat per-1M rates - cache-hit input, cache-miss input, output - with no current context-length tier or off-peak window for the V4 models; that shape was already fully modeled by the pre-existing `chat_cache_hit_per_m`/`chat_cache_miss_per_m`/`chat_output_per_m` fields before this section's tier2/off-peak fields existed. `routing.seed_default_deepseek_routes()` (called once from `database.migrate_ai_gateway_settings()` at the end of `init_db()`) idempotently inserts two ready-made routes - `deepseek-v4-flash` (`$0.0028`/`$0.14`/`$0.28` per 1M, 1M context) and `deepseek-v4-pro` (`$0.003625`/`$0.435`/`$0.87` per 1M, 1M context) - only when that `source_model` row does not already exist, so a caller or Devii can request either name explicitly and get correctly-priced, decoupled from whatever the single global `gateway_model` default happens to be set to (switching that global setting between the two real models does NOT retroactively fix the flat Pricing config fields - the seeded routes are the model-agnostic, always-correct way to reference a specific priced model). Neither seeded route sets the tier2/off-peak fields (DeepSeek does not use them today); an admin can add them later on the same row if DeepSeek (or any other provider routed here) introduces such pricing.
|
||||
|
||||
@ -15,6 +15,10 @@ VISION_CACHE_SIZE_DEFAULT = 256
|
||||
EMBED_URL_DEFAULT = "https://openrouter.ai/api/v1/embeddings"
|
||||
EMBED_MODEL_DEFAULT = "qwen/qwen3-embedding-8b"
|
||||
|
||||
IMAGE_URL_DEFAULT = "https://openrouter.ai/api/v1/images"
|
||||
IMAGE_MODEL_DEFAULT = "black-forest-labs/flux.2-pro"
|
||||
IMAGE_PRICE_PER_CALL_DEFAULT = 0.04
|
||||
|
||||
VISION_INSTRUCTION = (
|
||||
"Describe this image in detail. Note objects, people, scene, any visible "
|
||||
"text, layout, colors, and anything else that could be relevant for "
|
||||
|
||||
@ -13,11 +13,16 @@ from fastapi.responses import JSONResponse, Response, StreamingResponse
|
||||
from devplacepy import stealth
|
||||
from devplacepy.services.openai_gateway import config
|
||||
from devplacepy.services.openai_gateway.reliability import CircuitBreaker, retry_send
|
||||
from devplacepy.services.openai_gateway.routing import chat_overlay, embed_overlay
|
||||
from devplacepy.services.openai_gateway.routing import (
|
||||
chat_overlay,
|
||||
embed_overlay,
|
||||
image_overlay,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.system_message import apply_system_directives
|
||||
from devplacepy.services.openai_gateway.usage import (
|
||||
GatewayUsageLedger,
|
||||
classify_error,
|
||||
extract_image_usage,
|
||||
extract_params,
|
||||
parse_context_map,
|
||||
pricing_from_cfg,
|
||||
@ -106,6 +111,7 @@ class GatewayRuntime:
|
||||
self.peak_in_flight = 0
|
||||
self.vision_calls = 0
|
||||
self.embed_calls = 0
|
||||
self.image_calls = 0
|
||||
self.last_status = 0
|
||||
self.last_latency_ms = 0
|
||||
|
||||
@ -537,6 +543,171 @@ class GatewayRuntime:
|
||||
log(f"embed POST -> 200 ({timing['upstream_latency_ms']:.0f}ms)")
|
||||
return JSONResponse(content=data, headers=resp_headers)
|
||||
|
||||
async def handle_images(
|
||||
self, body: dict, cfg: dict, owner: tuple, user_agent: str, log=None
|
||||
):
|
||||
log = log or (lambda message: None)
|
||||
overlay = image_overlay(body.get("model"), cfg)
|
||||
if overlay:
|
||||
cfg = {**cfg, **overlay}
|
||||
log(
|
||||
f"routed image model {body.get('model')!r} -> {cfg['gateway_image_model']!r}"
|
||||
)
|
||||
if not cfg["gateway_image_enabled"]:
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.openai_gateway.usage import audit_actor_for
|
||||
|
||||
actor_kind, actor_uid, actor_role = audit_actor_for(owner[0], owner[1])
|
||||
audit.record_system(
|
||||
"ai.gateway.call",
|
||||
actor_kind=actor_kind,
|
||||
actor_uid=actor_uid,
|
||||
actor_role=actor_role,
|
||||
origin="api",
|
||||
result="denied",
|
||||
summary="image generation disabled",
|
||||
metadata={
|
||||
"backend": "image",
|
||||
"endpoint": "images/generations",
|
||||
"owner_kind": owner[0],
|
||||
"owner_id": owner[1],
|
||||
},
|
||||
)
|
||||
return JSONResponse(
|
||||
status_code=503,
|
||||
content={
|
||||
"error": {
|
||||
"message": "Image generation is disabled",
|
||||
"type": "images_disabled",
|
||||
}
|
||||
},
|
||||
)
|
||||
client, sem = self._ensure(cfg)
|
||||
pricing = pricing_from_cfg(cfg)
|
||||
context_map = parse_context_map(cfg.get("gateway_model_context_map"))
|
||||
params = extract_params(body)
|
||||
handle_start = time.monotonic()
|
||||
|
||||
requested = body.get("model")
|
||||
if (
|
||||
cfg["gateway_force_model"]
|
||||
or not requested
|
||||
or requested in ("molodetz-img-small", "molodetz-img")
|
||||
):
|
||||
model = cfg["gateway_image_model"]
|
||||
else:
|
||||
model = requested
|
||||
|
||||
payload = dict(body)
|
||||
payload["model"] = model
|
||||
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if cfg["gateway_image_key"]:
|
||||
headers["Authorization"] = f"Bearer {cfg['gateway_image_key']}"
|
||||
else:
|
||||
log(
|
||||
"No upstream image API key configured (gateway_image_key / gateway_api_key / OPENROUTER_API_KEY); upstream will likely reject the request"
|
||||
)
|
||||
|
||||
resp, exc, timing = await self._send(
|
||||
client,
|
||||
sem,
|
||||
"POST",
|
||||
cfg["gateway_image_url"],
|
||||
headers,
|
||||
cfg,
|
||||
log,
|
||||
json_body=payload,
|
||||
)
|
||||
|
||||
base = {
|
||||
"owner_kind": owner[0],
|
||||
"owner_id": owner[1],
|
||||
"backend": "image",
|
||||
"endpoint": "images/generations",
|
||||
"model": model,
|
||||
"user_agent": user_agent,
|
||||
**params,
|
||||
**timing,
|
||||
}
|
||||
|
||||
def finalize(status_code, success, category, usage=None):
|
||||
base["total_latency_ms"] = round(
|
||||
(time.monotonic() - handle_start) * 1000, 3
|
||||
)
|
||||
base["gateway_overhead_ms"] = round(
|
||||
max(
|
||||
base["total_latency_ms"]
|
||||
- timing["upstream_latency_ms"]
|
||||
- timing["queue_wait_ms"],
|
||||
0.0,
|
||||
),
|
||||
3,
|
||||
)
|
||||
base["status_code"] = status_code
|
||||
base["success"] = success
|
||||
base["error_category"] = category
|
||||
base["usage"] = usage
|
||||
row = self._ledger.record(base, pricing, context_map)
|
||||
return usage_response_headers(row)
|
||||
|
||||
if timing["circuit_open"]:
|
||||
resp_headers = finalize(503, False, "circuit_open")
|
||||
return JSONResponse(
|
||||
status_code=503,
|
||||
content={
|
||||
"error": {
|
||||
"message": "Upstream temporarily unavailable",
|
||||
"type": "circuit_open",
|
||||
}
|
||||
},
|
||||
headers=resp_headers,
|
||||
)
|
||||
if exc is not None:
|
||||
resp_headers = finalize(502, False, classify_error(0, exc))
|
||||
return JSONResponse(
|
||||
status_code=502,
|
||||
content={
|
||||
"error": {
|
||||
"message": f"Upstream connection failed: {exc}",
|
||||
"type": "upstream_error",
|
||||
}
|
||||
},
|
||||
headers=resp_headers,
|
||||
)
|
||||
if resp.status_code != 200:
|
||||
resp_headers = finalize(
|
||||
resp.status_code,
|
||||
False,
|
||||
classify_error(resp.status_code, None, resp.text),
|
||||
)
|
||||
log(f"image upstream POST -> {resp.status_code}: {resp.text[:300]}")
|
||||
return JSONResponse(
|
||||
status_code=resp.status_code,
|
||||
content={"error": {"message": resp.text, "type": "upstream_error"}},
|
||||
headers=resp_headers,
|
||||
)
|
||||
try:
|
||||
data = resp.json()
|
||||
except ValueError:
|
||||
self.errors += 1
|
||||
resp_headers = finalize(502, False, "gateway")
|
||||
log("image upstream returned 200 but body was not valid JSON")
|
||||
return JSONResponse(
|
||||
status_code=502,
|
||||
content={
|
||||
"error": {
|
||||
"message": "invalid upstream response",
|
||||
"type": "upstream_error",
|
||||
}
|
||||
},
|
||||
headers=resp_headers,
|
||||
)
|
||||
self.image_calls += 1
|
||||
resp_headers = finalize(200, True, None, extract_image_usage(data))
|
||||
log(f"image POST -> 200 ({timing['upstream_latency_ms']:.0f}ms)")
|
||||
return JSONResponse(content=data, headers=resp_headers)
|
||||
|
||||
async def handle_passthrough(
|
||||
self,
|
||||
method: str,
|
||||
@ -661,6 +832,7 @@ class GatewayRuntime:
|
||||
"peak_in_flight": self.peak_in_flight,
|
||||
"vision_calls": self.vision_calls,
|
||||
"embed_calls": self.embed_calls,
|
||||
"image_calls": self.image_calls,
|
||||
"last_status": self.last_status,
|
||||
"last_latency_ms": self.last_latency_ms,
|
||||
"pool": self._instances,
|
||||
|
||||
@ -7,7 +7,7 @@ from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
|
||||
from devplacepy.database import (
|
||||
bump_cache_version,
|
||||
@ -21,7 +21,7 @@ logger = logging.getLogger(__name__)
|
||||
PROVIDERS_TABLE = "gateway_providers"
|
||||
MODELS_TABLE = "gateway_models"
|
||||
CACHE_NAME = "gateway_routing"
|
||||
KINDS = ("chat", "embed")
|
||||
KINDS = ("chat", "embed", "image")
|
||||
|
||||
_ROUTING_CACHE: dict = {}
|
||||
|
||||
@ -47,6 +47,27 @@ def _embed_url_from_base(base_url: str) -> str:
|
||||
return base_url
|
||||
|
||||
|
||||
def _image_url_from_base(base_url: str) -> str:
|
||||
base_url = (base_url or "").strip()
|
||||
if not base_url:
|
||||
return ""
|
||||
if base_url.endswith("/chat/completions"):
|
||||
return base_url[: -len("/chat/completions")] + "/images"
|
||||
return base_url
|
||||
|
||||
|
||||
MODEL_TIER2_COLUMNS = (
|
||||
"context_tier_threshold_tokens",
|
||||
"price_cache_hit_per_m_tier2",
|
||||
"price_cache_miss_per_m_tier2",
|
||||
"price_output_per_m_tier2",
|
||||
"price_input_per_m_tier2",
|
||||
"off_peak_start_minute",
|
||||
"off_peak_end_minute",
|
||||
"off_peak_discount_pct",
|
||||
)
|
||||
|
||||
|
||||
def ensure_tables() -> None:
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS "
|
||||
@ -62,8 +83,17 @@ def ensure_tables() -> None:
|
||||
"vision_model TEXT, context_window INTEGER DEFAULT 0, "
|
||||
"price_cache_hit_per_m REAL DEFAULT 0, price_cache_miss_per_m REAL DEFAULT 0, "
|
||||
"price_output_per_m REAL DEFAULT 0, price_input_per_m REAL DEFAULT 0, "
|
||||
"context_tier_threshold_tokens INTEGER DEFAULT 0, "
|
||||
"price_cache_hit_per_m_tier2 REAL, price_cache_miss_per_m_tier2 REAL, "
|
||||
"price_output_per_m_tier2 REAL, price_input_per_m_tier2 REAL, "
|
||||
"off_peak_start_minute INTEGER, off_peak_end_minute INTEGER, "
|
||||
"off_peak_discount_pct REAL DEFAULT 0, "
|
||||
"is_active INTEGER DEFAULT 1, created_at TEXT, updated_at TEXT)"
|
||||
)
|
||||
models_table = get_table(MODELS_TABLE)
|
||||
for column in MODEL_TIER2_COLUMNS:
|
||||
if not models_table.has_column(column):
|
||||
models_table.create_column_by_example(column, 0.0)
|
||||
try:
|
||||
db.query(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS idx_gateway_providers_name ON "
|
||||
@ -121,6 +151,14 @@ class ModelRouteIn(BaseModel):
|
||||
price_cache_miss_per_m: float = Field(default=0.0, ge=0)
|
||||
price_output_per_m: float = Field(default=0.0, ge=0)
|
||||
price_input_per_m: float = Field(default=0.0, ge=0)
|
||||
context_tier_threshold_tokens: int = Field(default=0, ge=0, le=100_000_000)
|
||||
price_cache_hit_per_m_tier2: Optional[float] = Field(default=None, ge=0)
|
||||
price_cache_miss_per_m_tier2: Optional[float] = Field(default=None, ge=0)
|
||||
price_output_per_m_tier2: Optional[float] = Field(default=None, ge=0)
|
||||
price_input_per_m_tier2: Optional[float] = Field(default=None, ge=0)
|
||||
off_peak_start_minute: Optional[int] = Field(default=None, ge=0, le=1439)
|
||||
off_peak_end_minute: Optional[int] = Field(default=None, ge=0, le=1439)
|
||||
off_peak_discount_pct: float = Field(default=0.0, ge=0, le=100)
|
||||
is_active: bool = True
|
||||
|
||||
@field_validator("source_model", "target_model")
|
||||
@ -141,9 +179,19 @@ class ModelRouteIn(BaseModel):
|
||||
def _clean_kind(cls, value: str) -> str:
|
||||
value = (value or "chat").strip().lower()
|
||||
if value not in KINDS:
|
||||
raise ValueError("Kind must be 'chat' or 'embed'")
|
||||
raise ValueError("Kind must be 'chat', 'embed', or 'image'")
|
||||
return value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _check_off_peak_window(self) -> "ModelRouteIn":
|
||||
has_start = self.off_peak_start_minute is not None
|
||||
has_end = self.off_peak_end_minute is not None
|
||||
if has_start != has_end:
|
||||
raise ValueError(
|
||||
"Off-peak start and end minute must both be set, or both left blank"
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelRoute:
|
||||
@ -158,9 +206,27 @@ class ModelRoute:
|
||||
price_cache_miss_per_m: float
|
||||
price_output_per_m: float
|
||||
price_input_per_m: float
|
||||
context_tier_threshold_tokens: int
|
||||
price_cache_hit_per_m_tier2: Optional[float]
|
||||
price_cache_miss_per_m_tier2: Optional[float]
|
||||
price_output_per_m_tier2: Optional[float]
|
||||
price_input_per_m_tier2: Optional[float]
|
||||
off_peak_start_minute: Optional[int]
|
||||
off_peak_end_minute: Optional[int]
|
||||
off_peak_discount_pct: float
|
||||
is_active: bool
|
||||
|
||||
|
||||
def _opt_float(row: dict, key: str) -> Optional[float]:
|
||||
value = row.get(key)
|
||||
return float(value) if value is not None else None
|
||||
|
||||
|
||||
def _opt_int(row: dict, key: str) -> Optional[int]:
|
||||
value = row.get(key)
|
||||
return int(value) if value is not None else None
|
||||
|
||||
|
||||
def _route_from_row(row: dict) -> ModelRoute:
|
||||
return ModelRoute(
|
||||
source_model=str(row.get("source_model") or ""),
|
||||
@ -174,6 +240,16 @@ def _route_from_row(row: dict) -> ModelRoute:
|
||||
price_cache_miss_per_m=float(row.get("price_cache_miss_per_m") or 0.0),
|
||||
price_output_per_m=float(row.get("price_output_per_m") or 0.0),
|
||||
price_input_per_m=float(row.get("price_input_per_m") or 0.0),
|
||||
context_tier_threshold_tokens=int(
|
||||
row.get("context_tier_threshold_tokens") or 0
|
||||
),
|
||||
price_cache_hit_per_m_tier2=_opt_float(row, "price_cache_hit_per_m_tier2"),
|
||||
price_cache_miss_per_m_tier2=_opt_float(row, "price_cache_miss_per_m_tier2"),
|
||||
price_output_per_m_tier2=_opt_float(row, "price_output_per_m_tier2"),
|
||||
price_input_per_m_tier2=_opt_float(row, "price_input_per_m_tier2"),
|
||||
off_peak_start_minute=_opt_int(row, "off_peak_start_minute"),
|
||||
off_peak_end_minute=_opt_int(row, "off_peak_end_minute"),
|
||||
off_peak_discount_pct=float(row.get("off_peak_discount_pct") or 0.0),
|
||||
is_active=_as_bool(row.get("is_active", 1)),
|
||||
)
|
||||
|
||||
@ -291,6 +367,14 @@ class ModelStore:
|
||||
"price_cache_miss_per_m": payload.price_cache_miss_per_m,
|
||||
"price_output_per_m": payload.price_output_per_m,
|
||||
"price_input_per_m": payload.price_input_per_m,
|
||||
"context_tier_threshold_tokens": payload.context_tier_threshold_tokens,
|
||||
"price_cache_hit_per_m_tier2": payload.price_cache_hit_per_m_tier2,
|
||||
"price_cache_miss_per_m_tier2": payload.price_cache_miss_per_m_tier2,
|
||||
"price_output_per_m_tier2": payload.price_output_per_m_tier2,
|
||||
"price_input_per_m_tier2": payload.price_input_per_m_tier2,
|
||||
"off_peak_start_minute": payload.off_peak_start_minute,
|
||||
"off_peak_end_minute": payload.off_peak_end_minute,
|
||||
"off_peak_discount_pct": payload.off_peak_discount_pct,
|
||||
"is_active": 1 if payload.is_active else 0,
|
||||
"updated_at": _now(),
|
||||
}
|
||||
@ -318,6 +402,63 @@ class ModelStore:
|
||||
provider_store = ProviderStore()
|
||||
model_store = ModelStore()
|
||||
|
||||
DEEPSEEK_DEFAULT_ROUTES = (
|
||||
{
|
||||
"source_model": "deepseek-v4-flash",
|
||||
"target_model": "deepseek-v4-flash",
|
||||
"context_window": 1_048_576,
|
||||
"price_cache_hit_per_m": 0.0028,
|
||||
"price_cache_miss_per_m": 0.14,
|
||||
"price_output_per_m": 0.28,
|
||||
},
|
||||
{
|
||||
"source_model": "deepseek-v4-pro",
|
||||
"target_model": "deepseek-v4-pro",
|
||||
"context_window": 1_048_576,
|
||||
"price_cache_hit_per_m": 0.003625,
|
||||
"price_cache_miss_per_m": 0.435,
|
||||
"price_output_per_m": 0.87,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def seed_default_deepseek_routes() -> None:
|
||||
ensure_tables()
|
||||
table = get_table(MODELS_TABLE)
|
||||
seeded = False
|
||||
for defaults in DEEPSEEK_DEFAULT_ROUTES:
|
||||
if table.find_one(source_model=defaults["source_model"]):
|
||||
continue
|
||||
record = {
|
||||
"source_model": defaults["source_model"],
|
||||
"provider": "",
|
||||
"target_model": defaults["target_model"],
|
||||
"kind": "chat",
|
||||
"vision_provider": "",
|
||||
"vision_model": "",
|
||||
"context_window": defaults["context_window"],
|
||||
"price_cache_hit_per_m": defaults["price_cache_hit_per_m"],
|
||||
"price_cache_miss_per_m": defaults["price_cache_miss_per_m"],
|
||||
"price_output_per_m": defaults["price_output_per_m"],
|
||||
"price_input_per_m": 0.0,
|
||||
"context_tier_threshold_tokens": 0,
|
||||
"price_cache_hit_per_m_tier2": None,
|
||||
"price_cache_miss_per_m_tier2": None,
|
||||
"price_output_per_m_tier2": None,
|
||||
"price_input_per_m_tier2": None,
|
||||
"off_peak_start_minute": None,
|
||||
"off_peak_end_minute": None,
|
||||
"off_peak_discount_pct": 0.0,
|
||||
"is_active": 1,
|
||||
"created_at": _now(),
|
||||
"updated_at": _now(),
|
||||
}
|
||||
table.insert(record)
|
||||
seeded = True
|
||||
if seeded:
|
||||
bump_cache_version(CACHE_NAME)
|
||||
_ROUTING_CACHE.clear()
|
||||
|
||||
|
||||
def _provider_overlay(name: str, base_key: str, url_key: str, overlay: dict) -> None:
|
||||
provider = provider_store.get(name)
|
||||
@ -339,6 +480,13 @@ def chat_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[dic
|
||||
"gateway_price_cache_hit_per_m": route.price_cache_hit_per_m,
|
||||
"gateway_price_cache_miss_per_m": route.price_cache_miss_per_m,
|
||||
"gateway_price_output_per_m": route.price_output_per_m,
|
||||
"gateway_price_cache_hit_per_m_tier2": route.price_cache_hit_per_m_tier2,
|
||||
"gateway_price_cache_miss_per_m_tier2": route.price_cache_miss_per_m_tier2,
|
||||
"gateway_price_output_per_m_tier2": route.price_output_per_m_tier2,
|
||||
"gateway_context_tier_threshold_tokens": route.context_tier_threshold_tokens,
|
||||
"gateway_off_peak_start_minute": route.off_peak_start_minute,
|
||||
"gateway_off_peak_end_minute": route.off_peak_end_minute,
|
||||
"gateway_off_peak_discount_pct": route.off_peak_discount_pct,
|
||||
}
|
||||
if route.provider:
|
||||
_provider_overlay(
|
||||
@ -355,6 +503,12 @@ def chat_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[dic
|
||||
overlay["gateway_vision_model"] = route.vision_model
|
||||
overlay["gateway_vision_price_input_per_m"] = route.price_input_per_m
|
||||
overlay["gateway_vision_price_output_per_m"] = route.price_output_per_m
|
||||
overlay["gateway_vision_price_input_per_m_tier2"] = (
|
||||
route.price_input_per_m_tier2
|
||||
)
|
||||
overlay["gateway_vision_price_output_per_m_tier2"] = (
|
||||
route.price_output_per_m_tier2
|
||||
)
|
||||
vision_provider = route.vision_provider or route.provider
|
||||
if vision_provider:
|
||||
_provider_overlay(
|
||||
@ -371,6 +525,11 @@ def embed_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[di
|
||||
"gateway_force_model": True,
|
||||
"gateway_embed_model": route.target_model,
|
||||
"gateway_embed_price_input_per_m": route.price_input_per_m,
|
||||
"gateway_embed_price_input_per_m_tier2": route.price_input_per_m_tier2,
|
||||
"gateway_context_tier_threshold_tokens": route.context_tier_threshold_tokens,
|
||||
"gateway_off_peak_start_minute": route.off_peak_start_minute,
|
||||
"gateway_off_peak_end_minute": route.off_peak_end_minute,
|
||||
"gateway_off_peak_discount_pct": route.off_peak_discount_pct,
|
||||
}
|
||||
provider = provider_store.get(route.provider) if route.provider else None
|
||||
if provider:
|
||||
@ -379,3 +538,129 @@ def embed_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[di
|
||||
if provider.get("api_key"):
|
||||
overlay["gateway_embed_key"] = provider["api_key"]
|
||||
return overlay
|
||||
|
||||
|
||||
def image_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[dict]:
|
||||
route = model_store.resolve(requested_model, "image")
|
||||
if route is None:
|
||||
return None
|
||||
overlay: dict = {
|
||||
"gateway_force_model": True,
|
||||
"gateway_image_model": route.target_model,
|
||||
"gateway_image_price_per_call": route.price_input_per_m,
|
||||
"gateway_off_peak_start_minute": route.off_peak_start_minute,
|
||||
"gateway_off_peak_end_minute": route.off_peak_end_minute,
|
||||
"gateway_off_peak_discount_pct": route.off_peak_discount_pct,
|
||||
}
|
||||
provider = provider_store.get(route.provider) if route.provider else None
|
||||
if provider:
|
||||
if provider.get("base_url"):
|
||||
overlay["gateway_image_url"] = _image_url_from_base(provider["base_url"])
|
||||
if provider.get("api_key"):
|
||||
overlay["gateway_image_key"] = provider["api_key"]
|
||||
return overlay
|
||||
|
||||
|
||||
OPENROUTER_PROVIDER_NAME = "openrouter"
|
||||
FLUX_TARGET_MODEL = "black-forest-labs/flux.2-pro"
|
||||
IMAGE_SOURCE_MODEL = "molodetz-img-small"
|
||||
RETIRED_IMAGE_MODELS = {
|
||||
"black-forest-labs/flux-1.1-pro": FLUX_TARGET_MODEL,
|
||||
"black-forest-labs/flux-schnell": "black-forest-labs/flux.2-klein-4b",
|
||||
}
|
||||
LEGACY_IMAGE_URL = "https://openrouter.ai/api/v1/images/generations"
|
||||
|
||||
|
||||
def seed_default_image_routes() -> None:
|
||||
import os
|
||||
|
||||
from devplacepy.services.openai_gateway import config as gw_config
|
||||
|
||||
ensure_tables()
|
||||
openrouter_key = os.environ.get("OPENROUTER_API_KEY", "")
|
||||
if openrouter_key:
|
||||
existing_provider = provider_store.get(OPENROUTER_PROVIDER_NAME)
|
||||
if existing_provider is None:
|
||||
provider_store.set(
|
||||
ProviderIn(
|
||||
name=OPENROUTER_PROVIDER_NAME,
|
||||
base_url="https://openrouter.ai/api/v1/chat/completions",
|
||||
api_key=openrouter_key,
|
||||
is_active=True,
|
||||
)
|
||||
)
|
||||
models_table = get_table(MODELS_TABLE)
|
||||
if models_table.find_one(source_model=IMAGE_SOURCE_MODEL):
|
||||
return
|
||||
record = {
|
||||
"source_model": IMAGE_SOURCE_MODEL,
|
||||
"provider": OPENROUTER_PROVIDER_NAME if openrouter_key else "",
|
||||
"target_model": FLUX_TARGET_MODEL,
|
||||
"kind": "image",
|
||||
"vision_provider": "",
|
||||
"vision_model": "",
|
||||
"context_window": 0,
|
||||
"price_cache_hit_per_m": 0.0,
|
||||
"price_cache_miss_per_m": 0.0,
|
||||
"price_output_per_m": 0.0,
|
||||
"price_input_per_m": gw_config.IMAGE_PRICE_PER_CALL_DEFAULT,
|
||||
"context_tier_threshold_tokens": 0,
|
||||
"price_cache_hit_per_m_tier2": None,
|
||||
"price_cache_miss_per_m_tier2": None,
|
||||
"price_output_per_m_tier2": None,
|
||||
"price_input_per_m_tier2": None,
|
||||
"off_peak_start_minute": None,
|
||||
"off_peak_end_minute": None,
|
||||
"off_peak_discount_pct": 0.0,
|
||||
"is_active": 1,
|
||||
"created_at": _now(),
|
||||
"updated_at": _now(),
|
||||
}
|
||||
models_table.insert(record)
|
||||
bump_cache_version(CACHE_NAME)
|
||||
_ROUTING_CACHE.clear()
|
||||
|
||||
|
||||
def migrate_retired_image_gateway() -> None:
|
||||
from devplacepy.database import get_setting, set_setting
|
||||
from devplacepy.services.openai_gateway import config as gw_config
|
||||
|
||||
ensure_tables()
|
||||
models_table = get_table(MODELS_TABLE)
|
||||
now = _now()
|
||||
changed = False
|
||||
for row in models_table.find(kind="image"):
|
||||
target = str(row.get("target_model") or "")
|
||||
replacement = RETIRED_IMAGE_MODELS.get(target)
|
||||
if not replacement:
|
||||
continue
|
||||
models_table.update(
|
||||
{
|
||||
"source_model": row["source_model"],
|
||||
"target_model": replacement,
|
||||
"updated_at": now,
|
||||
},
|
||||
["source_model"],
|
||||
)
|
||||
changed = True
|
||||
image_url = get_setting("gateway_image_url", "")
|
||||
if image_url in (LEGACY_IMAGE_URL, ""):
|
||||
set_setting("gateway_image_url", gw_config.IMAGE_URL_DEFAULT)
|
||||
changed = True
|
||||
elif image_url.endswith("/images/generations"):
|
||||
set_setting(
|
||||
"gateway_image_url",
|
||||
image_url[: -len("/generations")],
|
||||
)
|
||||
changed = True
|
||||
image_model = get_setting("gateway_image_model", "")
|
||||
replacement = RETIRED_IMAGE_MODELS.get(image_model)
|
||||
if replacement:
|
||||
set_setting("gateway_image_model", replacement)
|
||||
changed = True
|
||||
elif not image_model:
|
||||
set_setting("gateway_image_model", gw_config.IMAGE_MODEL_DEFAULT)
|
||||
changed = True
|
||||
if changed:
|
||||
bump_cache_version(CACHE_NAME)
|
||||
_ROUTING_CACHE.clear()
|
||||
|
||||
@ -172,6 +172,38 @@ class GatewayService(BaseService):
|
||||
help="The key currently in use; falls back to the vision/OPENROUTER key on boot (embeddings default to OpenRouter). Editable.",
|
||||
group="Embeddings",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_image_enabled",
|
||||
"Image generation",
|
||||
type="bool",
|
||||
default=True,
|
||||
help="Expose image generation at /openai/v1/images/generations.",
|
||||
group="Images",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_image_url",
|
||||
"Images URL",
|
||||
type="url",
|
||||
default=config.IMAGE_URL_DEFAULT,
|
||||
help="OpenAI-compatible images/generations endpoint requests are forwarded to.",
|
||||
group="Images",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_image_model",
|
||||
"Images model",
|
||||
type="str",
|
||||
default=config.IMAGE_MODEL_DEFAULT,
|
||||
help="Image model sent upstream. Clients request it as molodetz-img-small.",
|
||||
group="Images",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_image_key",
|
||||
"Images API key",
|
||||
type="str",
|
||||
default="",
|
||||
help="The key currently in use; falls back to gateway_api_key / OPENROUTER_API_KEY on boot. Editable.",
|
||||
group="Images",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_require_auth",
|
||||
"Require authentication",
|
||||
@ -267,6 +299,15 @@ class GatewayService(BaseService):
|
||||
help="Fallback only; used when the embeddings upstream returns no native cost.",
|
||||
group="Pricing",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_image_price_per_call",
|
||||
"Images price / call ($)",
|
||||
type="float",
|
||||
default=config.IMAGE_PRICE_PER_CALL_DEFAULT,
|
||||
minimum=0,
|
||||
help="Fallback only; used when the image upstream returns no native cost.",
|
||||
group="Pricing",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_rsearch_cost_per_call",
|
||||
"rsearch cost / call ($)",
|
||||
@ -355,6 +396,17 @@ class GatewayService(BaseService):
|
||||
or cfg["gateway_vision_key"]
|
||||
or os.environ.get("OPENROUTER_API_KEY", "")
|
||||
)
|
||||
cfg["gateway_image_key"] = (
|
||||
cfg["gateway_image_key"]
|
||||
or cfg["gateway_api_key"]
|
||||
or os.environ.get("OPENROUTER_API_KEY", "")
|
||||
)
|
||||
if not cfg.get("gateway_image_url"):
|
||||
upstream = cfg.get("gateway_upstream_url", "")
|
||||
if upstream.endswith("/chat/completions"):
|
||||
cfg["gateway_image_url"] = (
|
||||
upstream[: -len("/chat/completions")] + "/images"
|
||||
)
|
||||
return cfg
|
||||
|
||||
def authorize(self, request: Request) -> bool:
|
||||
@ -429,6 +481,18 @@ class GatewayService(BaseService):
|
||||
return await runtime.handle_embeddings(
|
||||
body, cfg, owner, user_agent, self.log
|
||||
)
|
||||
if subpath == "images/generations" and request.method == "POST":
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
self.log("Rejected images request: invalid JSON body")
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON body")
|
||||
if not isinstance(body, dict):
|
||||
self.log("Rejected images request: JSON body was not an object")
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON body")
|
||||
return await runtime.handle_images(
|
||||
body, cfg, owner, user_agent, self.log
|
||||
)
|
||||
body = await request.body()
|
||||
content_type = request.headers.get("content-type", "")
|
||||
return await runtime.handle_passthrough(
|
||||
@ -473,6 +537,7 @@ class GatewayService(BaseService):
|
||||
"peak_in_flight": 0,
|
||||
"vision_calls": 0,
|
||||
"embed_calls": 0,
|
||||
"image_calls": 0,
|
||||
"last_status": 0,
|
||||
"last_latency_ms": 0,
|
||||
"pool": 0,
|
||||
@ -485,12 +550,14 @@ class GatewayService(BaseService):
|
||||
{"label": "In flight", "value": m["in_flight"]},
|
||||
{"label": "Vision calls", "value": m["vision_calls"]},
|
||||
{"label": "Embed calls", "value": m["embed_calls"]},
|
||||
{"label": "Image calls", "value": m["image_calls"]},
|
||||
{"label": "Last status", "value": m["last_status"] or "-"},
|
||||
{"label": "Last latency", "value": f"{m['last_latency_ms']} ms"},
|
||||
{"label": "Pool size", "value": m["pool"]},
|
||||
{"label": "Circuit", "value": "open" if m["circuit_open"] else "closed"},
|
||||
{"label": "Model", "value": cfg["gateway_model"]},
|
||||
{"label": "Embed model", "value": cfg["gateway_embed_model"]},
|
||||
{"label": "Image model", "value": cfg["gateway_image_model"]},
|
||||
{"label": "Requests 24h", "value": s["requests"]},
|
||||
{"label": "Success 24h", "value": f"{s['success_pct']}%"},
|
||||
{"label": "Error rate 24h", "value": f"{s['error_pct']}%"},
|
||||
|
||||
@ -36,6 +36,31 @@ class Pricing:
|
||||
vision_input_per_m: float
|
||||
vision_output_per_m: float
|
||||
embed_input_per_m: float
|
||||
image_per_call: float = 0.0
|
||||
chat_cache_hit_per_m_tier2: Optional[float] = None
|
||||
chat_cache_miss_per_m_tier2: Optional[float] = None
|
||||
chat_output_per_m_tier2: Optional[float] = None
|
||||
vision_input_per_m_tier2: Optional[float] = None
|
||||
vision_output_per_m_tier2: Optional[float] = None
|
||||
embed_input_per_m_tier2: Optional[float] = None
|
||||
context_tier_threshold_tokens: int = 0
|
||||
off_peak_start_minute: Optional[int] = None
|
||||
off_peak_end_minute: Optional[int] = None
|
||||
off_peak_discount_pct: float = 0.0
|
||||
|
||||
|
||||
def _cfg_float(cfg: dict, key: str) -> Optional[float]:
|
||||
value = cfg.get(key)
|
||||
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
||||
return float(value)
|
||||
return None
|
||||
|
||||
|
||||
def _cfg_int(cfg: dict, key: str) -> Optional[int]:
|
||||
value = cfg.get(key)
|
||||
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
||||
return int(value)
|
||||
return None
|
||||
|
||||
|
||||
def pricing_from_cfg(cfg: dict) -> Pricing:
|
||||
@ -71,6 +96,34 @@ def pricing_from_cfg(cfg: dict) -> Pricing:
|
||||
config.EMBED_PRICE_INPUT_PER_M_DEFAULT,
|
||||
)
|
||||
),
|
||||
image_per_call=float(
|
||||
cfg.get(
|
||||
"gateway_image_price_per_call",
|
||||
config.IMAGE_PRICE_PER_CALL_DEFAULT,
|
||||
)
|
||||
),
|
||||
chat_cache_hit_per_m_tier2=_cfg_float(
|
||||
cfg, "gateway_price_cache_hit_per_m_tier2"
|
||||
),
|
||||
chat_cache_miss_per_m_tier2=_cfg_float(
|
||||
cfg, "gateway_price_cache_miss_per_m_tier2"
|
||||
),
|
||||
chat_output_per_m_tier2=_cfg_float(cfg, "gateway_price_output_per_m_tier2"),
|
||||
vision_input_per_m_tier2=_cfg_float(
|
||||
cfg, "gateway_vision_price_input_per_m_tier2"
|
||||
),
|
||||
vision_output_per_m_tier2=_cfg_float(
|
||||
cfg, "gateway_vision_price_output_per_m_tier2"
|
||||
),
|
||||
embed_input_per_m_tier2=_cfg_float(
|
||||
cfg, "gateway_embed_price_input_per_m_tier2"
|
||||
),
|
||||
context_tier_threshold_tokens=int(
|
||||
cfg.get("gateway_context_tier_threshold_tokens", 0) or 0
|
||||
),
|
||||
off_peak_start_minute=_cfg_int(cfg, "gateway_off_peak_start_minute"),
|
||||
off_peak_end_minute=_cfg_int(cfg, "gateway_off_peak_end_minute"),
|
||||
off_peak_discount_pct=float(cfg.get("gateway_off_peak_discount_pct", 0.0) or 0.0),
|
||||
)
|
||||
|
||||
|
||||
@ -88,6 +141,18 @@ def parse_context_map(raw: Any) -> dict[str, int]:
|
||||
return dict(config.MODEL_CONTEXT_MAP_DEFAULT)
|
||||
|
||||
|
||||
def extract_image_usage(data: Optional[dict]) -> dict:
|
||||
data = data or {}
|
||||
usage = data.get("usage") if isinstance(data.get("usage"), dict) else {}
|
||||
cost = usage.get("cost")
|
||||
if cost is None:
|
||||
cost = data.get("cost")
|
||||
result: dict = {}
|
||||
if isinstance(cost, (int, float)) and not isinstance(cost, bool):
|
||||
result["cost"] = float(cost)
|
||||
return result
|
||||
|
||||
|
||||
def normalize_usage(usage: Optional[dict]) -> dict:
|
||||
usage = usage or {}
|
||||
prompt = int(usage.get("prompt_tokens", 0) or 0)
|
||||
@ -117,21 +182,108 @@ def normalize_usage(usage: Optional[dict]) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _off_peak_active(pricing: Pricing, now: Optional[datetime] = None) -> bool:
|
||||
if pricing.off_peak_start_minute is None or pricing.off_peak_end_minute is None:
|
||||
return False
|
||||
moment = now or _now()
|
||||
minute_of_day = moment.hour * 60 + moment.minute
|
||||
start, end = pricing.off_peak_start_minute, pricing.off_peak_end_minute
|
||||
if start == end:
|
||||
return True
|
||||
if start < end:
|
||||
return start <= minute_of_day < end
|
||||
return minute_of_day >= start or minute_of_day < end
|
||||
|
||||
|
||||
def _tiered_rate(base: float, tier2: Optional[float], use_tier2: bool) -> float:
|
||||
if use_tier2 and tier2 is not None:
|
||||
return tier2
|
||||
return base
|
||||
|
||||
|
||||
def _effective_rate(
|
||||
base: float,
|
||||
tier2: Optional[float],
|
||||
use_tier2: bool,
|
||||
pricing: Pricing,
|
||||
off_peak: bool,
|
||||
) -> float:
|
||||
rate = _tiered_rate(base, tier2, use_tier2)
|
||||
if off_peak and pricing.off_peak_discount_pct > 0:
|
||||
rate = rate * (1 - min(pricing.off_peak_discount_pct, 100.0) / 100.0)
|
||||
return rate
|
||||
|
||||
|
||||
def compute_cost(
|
||||
usage: dict, norm: dict, pricing: Pricing, backend: str
|
||||
usage: dict,
|
||||
norm: dict,
|
||||
pricing: Pricing,
|
||||
backend: str,
|
||||
now: Optional[datetime] = None,
|
||||
) -> tuple[float, float, float, bool]:
|
||||
threshold = pricing.context_tier_threshold_tokens
|
||||
use_tier2 = bool(threshold > 0 and norm["prompt"] > threshold)
|
||||
off_peak = _off_peak_active(pricing, now)
|
||||
if backend == "vision":
|
||||
input_cost = norm["prompt"] / PER_MILLION * pricing.vision_input_per_m
|
||||
output_cost = norm["completion"] / PER_MILLION * pricing.vision_output_per_m
|
||||
input_rate = _effective_rate(
|
||||
pricing.vision_input_per_m,
|
||||
pricing.vision_input_per_m_tier2,
|
||||
use_tier2,
|
||||
pricing,
|
||||
off_peak,
|
||||
)
|
||||
output_rate = _effective_rate(
|
||||
pricing.vision_output_per_m,
|
||||
pricing.vision_output_per_m_tier2,
|
||||
use_tier2,
|
||||
pricing,
|
||||
off_peak,
|
||||
)
|
||||
input_cost = norm["prompt"] / PER_MILLION * input_rate
|
||||
output_cost = norm["completion"] / PER_MILLION * output_rate
|
||||
elif backend == "embed":
|
||||
input_cost = norm["prompt"] / PER_MILLION * pricing.embed_input_per_m
|
||||
input_rate = _effective_rate(
|
||||
pricing.embed_input_per_m,
|
||||
pricing.embed_input_per_m_tier2,
|
||||
use_tier2,
|
||||
pricing,
|
||||
off_peak,
|
||||
)
|
||||
input_cost = norm["prompt"] / PER_MILLION * input_rate
|
||||
output_cost = 0.0
|
||||
elif backend == "image":
|
||||
rate = pricing.image_per_call
|
||||
if off_peak and pricing.off_peak_discount_pct > 0:
|
||||
rate = rate * (1 - min(pricing.off_peak_discount_pct, 100.0) / 100.0)
|
||||
input_cost = rate
|
||||
output_cost = 0.0
|
||||
else:
|
||||
input_cost = (
|
||||
norm["cache_hit"] / PER_MILLION * pricing.chat_cache_hit_per_m
|
||||
+ norm["cache_miss"] / PER_MILLION * pricing.chat_cache_miss_per_m
|
||||
hit_rate = _effective_rate(
|
||||
pricing.chat_cache_hit_per_m,
|
||||
pricing.chat_cache_hit_per_m_tier2,
|
||||
use_tier2,
|
||||
pricing,
|
||||
off_peak,
|
||||
)
|
||||
output_cost = norm["completion"] / PER_MILLION * pricing.chat_output_per_m
|
||||
miss_rate = _effective_rate(
|
||||
pricing.chat_cache_miss_per_m,
|
||||
pricing.chat_cache_miss_per_m_tier2,
|
||||
use_tier2,
|
||||
pricing,
|
||||
off_peak,
|
||||
)
|
||||
output_rate = _effective_rate(
|
||||
pricing.chat_output_per_m,
|
||||
pricing.chat_output_per_m_tier2,
|
||||
use_tier2,
|
||||
pricing,
|
||||
off_peak,
|
||||
)
|
||||
input_cost = (
|
||||
norm["cache_hit"] / PER_MILLION * hit_rate
|
||||
+ norm["cache_miss"] / PER_MILLION * miss_rate
|
||||
)
|
||||
output_cost = norm["completion"] / PER_MILLION * output_rate
|
||||
native = usage.get("cost") if isinstance(usage, dict) else None
|
||||
if isinstance(native, (int, float)) and not isinstance(native, bool):
|
||||
total = max(0.0, float(native))
|
||||
|
||||
@ -7,6 +7,7 @@ from typing import Optional
|
||||
|
||||
from devplacepy.config import PRESENCE_ONLINE_LIMIT
|
||||
from devplacepy.database import db, get_users_by_uids
|
||||
from devplacepy.database.awards import award_is_prominent
|
||||
from devplacepy.services import presence
|
||||
from devplacepy.services.base import BaseService
|
||||
from devplacepy.services.pubsub import publish as pubsub_publish
|
||||
@ -104,6 +105,8 @@ class PresenceRelayService(BaseService):
|
||||
"uid": row["uid"],
|
||||
"username": row["username"],
|
||||
"avatar_seed": row.get("avatar_seed") or row["username"],
|
||||
"last_award_slug": row.get("last_award_slug") or "",
|
||||
"award_prominent": award_is_prominent(row),
|
||||
}
|
||||
for row in users
|
||||
],
|
||||
|
||||
1
devplacepy/services/statistics/__init__.py
Normal file
1
devplacepy/services/statistics/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
77
devplacepy/services/statistics/build.py
Normal file
77
devplacepy/services/statistics/build.py
Normal file
@ -0,0 +1,77 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from devplacepy.services.statistics.common import (
|
||||
VALID_TABS,
|
||||
cache_get,
|
||||
cache_set,
|
||||
iso,
|
||||
now_utc,
|
||||
window_config,
|
||||
)
|
||||
from devplacepy.services.statistics.tabs_data import (
|
||||
build_ai,
|
||||
build_awards,
|
||||
build_containers,
|
||||
build_content,
|
||||
build_devii,
|
||||
build_engagement,
|
||||
build_game,
|
||||
build_members,
|
||||
build_moderation,
|
||||
build_overview,
|
||||
build_services,
|
||||
build_social,
|
||||
build_storage,
|
||||
build_tools,
|
||||
build_visitors,
|
||||
)
|
||||
|
||||
TAB_BUILDERS = {
|
||||
"overview": build_overview,
|
||||
"visitors": build_visitors,
|
||||
"members": build_members,
|
||||
"content": build_content,
|
||||
"engagement": build_engagement,
|
||||
"social": build_social,
|
||||
"ai": build_ai,
|
||||
"devii": build_devii,
|
||||
"services": build_services,
|
||||
"containers": build_containers,
|
||||
"game": build_game,
|
||||
"awards": build_awards,
|
||||
"moderation": build_moderation,
|
||||
"tools": build_tools,
|
||||
"storage": build_storage,
|
||||
}
|
||||
|
||||
|
||||
def build_statistics_tab(
|
||||
tab: str,
|
||||
hours: int = 168,
|
||||
*,
|
||||
compare: bool = True,
|
||||
top_n: int = 10,
|
||||
) -> dict:
|
||||
key = (tab or "overview").strip().lower()
|
||||
if key not in VALID_TABS:
|
||||
key = "overview"
|
||||
bounded_top = max(1, min(int(top_n or 10), 50))
|
||||
window = window_config(hours)
|
||||
cache_key = f"stats:{key}:{window['hours']}:{int(compare)}:{bounded_top}"
|
||||
cached = cache_get(cache_key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
builder = TAB_BUILDERS[key]
|
||||
body = builder(window["hours"], compare, bounded_top)
|
||||
payload = {
|
||||
"tab": key,
|
||||
"window_hours": window["hours"],
|
||||
"granularity": window["granularity"],
|
||||
"generated_at": iso(now_utc()),
|
||||
"compare": bool(compare),
|
||||
**body,
|
||||
}
|
||||
cache_set(cache_key, payload)
|
||||
return payload
|
||||
169
devplacepy/services/statistics/common.py
Normal file
169
devplacepy/services/statistics/common.py
Normal file
@ -0,0 +1,169 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional
|
||||
|
||||
from devplacepy.cache import TTLCache
|
||||
|
||||
MAX_WINDOW_HOURS = 2160
|
||||
RETENTION_DAYS = 90
|
||||
|
||||
_stats_cache = TTLCache(ttl=60, max_size=200)
|
||||
|
||||
VALID_TABS = (
|
||||
"overview",
|
||||
"visitors",
|
||||
"members",
|
||||
"content",
|
||||
"engagement",
|
||||
"social",
|
||||
"ai",
|
||||
"devii",
|
||||
"services",
|
||||
"containers",
|
||||
"game",
|
||||
"awards",
|
||||
"moderation",
|
||||
"tools",
|
||||
"storage",
|
||||
)
|
||||
|
||||
|
||||
def now_utc() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def iso(moment: datetime) -> str:
|
||||
return moment.isoformat(timespec="seconds")
|
||||
|
||||
|
||||
def hour_bucket(moment: datetime) -> str:
|
||||
return moment.replace(minute=0, second=0, microsecond=0).isoformat(timespec="seconds")
|
||||
|
||||
|
||||
def window_config(hours: int) -> dict:
|
||||
now = now_utc()
|
||||
if hours <= 0:
|
||||
return {
|
||||
"hours": 0,
|
||||
"start": None,
|
||||
"compare_start": None,
|
||||
"end": iso(now),
|
||||
"granularity": "weekly",
|
||||
}
|
||||
bounded = max(1, min(int(hours), MAX_WINDOW_HOURS))
|
||||
start = now - timedelta(hours=bounded)
|
||||
compare_start = start - timedelta(hours=bounded)
|
||||
if bounded <= 48:
|
||||
granularity = "hourly"
|
||||
elif bounded <= 720:
|
||||
granularity = "daily"
|
||||
else:
|
||||
granularity = "daily"
|
||||
return {
|
||||
"hours": bounded,
|
||||
"start": iso(start),
|
||||
"compare_start": iso(compare_start),
|
||||
"end": iso(now),
|
||||
"granularity": granularity,
|
||||
}
|
||||
|
||||
|
||||
def bucket_expr(granularity: str, column: str = "created_at") -> str:
|
||||
if granularity == "hourly":
|
||||
return f"strftime('%Y-%m-%dT%H:00:00', {column})"
|
||||
if granularity == "weekly":
|
||||
return f"strftime('%Y-W%W', {column})"
|
||||
return f"date({column})"
|
||||
|
||||
|
||||
def delta_pair(current: float, previous: float) -> dict:
|
||||
if previous == 0:
|
||||
pct = 100.0 if current > 0 else 0.0
|
||||
else:
|
||||
pct = round((current - previous) / previous * 100, 1)
|
||||
if pct > 0:
|
||||
direction = "up"
|
||||
elif pct < 0:
|
||||
direction = "down"
|
||||
else:
|
||||
direction = "flat"
|
||||
return {"delta": pct, "direction": direction}
|
||||
|
||||
|
||||
def metric_card(
|
||||
key: str,
|
||||
label: str,
|
||||
value,
|
||||
*,
|
||||
current: Optional[float] = None,
|
||||
previous: Optional[float] = None,
|
||||
format_kind: str = "int",
|
||||
) -> dict:
|
||||
card = {"key": key, "label": label, "value": value, "format": format_kind}
|
||||
if current is not None and previous is not None:
|
||||
card.update(delta_pair(current, previous))
|
||||
return card
|
||||
|
||||
|
||||
def fill_series(
|
||||
points: dict[str, float],
|
||||
start: Optional[str],
|
||||
end: str,
|
||||
granularity: str,
|
||||
) -> list[dict]:
|
||||
if not start:
|
||||
ordered = sorted(points.items())
|
||||
return [{"t": bucket, "v": points[bucket]} for bucket, _ in ordered]
|
||||
try:
|
||||
cursor = datetime.fromisoformat(start)
|
||||
end_dt = datetime.fromisoformat(end)
|
||||
except (TypeError, ValueError):
|
||||
ordered = sorted(points.items())
|
||||
return [{"t": bucket, "v": points[bucket]} for bucket, _ in ordered]
|
||||
if cursor.tzinfo is None:
|
||||
cursor = cursor.replace(tzinfo=timezone.utc)
|
||||
if end_dt.tzinfo is None:
|
||||
end_dt = end_dt.replace(tzinfo=timezone.utc)
|
||||
out: list[dict] = []
|
||||
if granularity == "hourly":
|
||||
step = timedelta(hours=1)
|
||||
fmt = lambda moment: hour_bucket(moment)
|
||||
elif granularity == "weekly":
|
||||
step = timedelta(days=7)
|
||||
fmt = lambda moment: moment.strftime("%Y-W%W")
|
||||
else:
|
||||
step = timedelta(days=1)
|
||||
fmt = lambda moment: moment.date().isoformat()
|
||||
while cursor <= end_dt:
|
||||
key = fmt(cursor)
|
||||
out.append({"t": key, "v": points.get(key, 0)})
|
||||
cursor += step
|
||||
return out
|
||||
|
||||
|
||||
def cache_get(key: str):
|
||||
return _stats_cache.get(key)
|
||||
|
||||
|
||||
def cache_set(key: str, payload: dict) -> None:
|
||||
_stats_cache.set(key, payload)
|
||||
|
||||
|
||||
def series_payload(
|
||||
key: str,
|
||||
label: str,
|
||||
points: dict[str, float],
|
||||
window: dict,
|
||||
) -> dict:
|
||||
return {
|
||||
"key": key,
|
||||
"label": label,
|
||||
"points": fill_series(points, window["start"], window["end"], window["granularity"]),
|
||||
}
|
||||
|
||||
|
||||
def table_payload(key: str, title: str, columns: list[str], rows: list[list]) -> dict:
|
||||
return {"key": key, "title": title, "columns": columns, "rows": rows}
|
||||
1288
devplacepy/services/statistics/tabs_data.py
Normal file
1288
devplacepy/services/statistics/tabs_data.py
Normal file
File diff suppressed because it is too large
Load Diff
207
devplacepy/services/statistics/tracking.py
Normal file
207
devplacepy/services/statistics/tracking.py
Normal file
@ -0,0 +1,207 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import logging
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from devplacepy.config import SECRET_KEY
|
||||
from devplacepy.database import db, get_setting
|
||||
from devplacepy.utils import client_ip, get_current_user
|
||||
|
||||
from .common import RETENTION_DAYS, hour_bucket, now_utc
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
EXCLUDED_PREFIXES = (
|
||||
"/static",
|
||||
"/avatar",
|
||||
"/openai",
|
||||
"/devii/ws",
|
||||
"/xmlrpc",
|
||||
"/dbapi",
|
||||
"/admin/statistics/data",
|
||||
"/admin/ai-usage/data",
|
||||
"/admin/services/data",
|
||||
)
|
||||
|
||||
FLUSH_SECONDS = 30
|
||||
|
||||
_view_buffer: dict[tuple[str, str, str], dict[str, int]] = defaultdict(
|
||||
lambda: {"views": 0, "member_views": 0, "guest_views": 0}
|
||||
)
|
||||
_unique_buffer: list[tuple[str, str, str, Optional[str]]] = []
|
||||
_cached_daily_salt = ""
|
||||
_flush_task: Optional[asyncio.Task] = None
|
||||
|
||||
|
||||
def _tracking_enabled() -> bool:
|
||||
return get_setting("statistics_tracking_enabled", "1") == "1"
|
||||
|
||||
|
||||
def _daily_salt() -> str:
|
||||
global _cached_daily_salt
|
||||
day = now_utc().date().isoformat()
|
||||
if not _cached_daily_salt or not _cached_daily_salt.startswith(day):
|
||||
_cached_daily_salt = f"{day}:{SECRET_KEY}"
|
||||
return _cached_daily_salt
|
||||
|
||||
|
||||
def _visitor_hash(request) -> str:
|
||||
ip = client_ip(request, default="") or ""
|
||||
ua = (request.headers.get("user-agent") or "")[:120]
|
||||
raw = f"{_daily_salt()}:{ip}:{ua}"
|
||||
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _page_group(request) -> str:
|
||||
route = request.scope.get("route")
|
||||
path = getattr(route, "path", None)
|
||||
if path:
|
||||
return path
|
||||
segments = [segment for segment in request.url.path.split("/") if segment]
|
||||
if not segments:
|
||||
return "/"
|
||||
return f"/{segments[0]}"
|
||||
|
||||
|
||||
def _referrer_group(request) -> str:
|
||||
raw = request.headers.get("referer") or request.headers.get("referrer") or ""
|
||||
if not raw:
|
||||
return "direct"
|
||||
try:
|
||||
host = (urlparse(raw).hostname or "").lower()
|
||||
except ValueError:
|
||||
return "other"
|
||||
if not host:
|
||||
return "direct"
|
||||
request_host = (request.url.hostname or "").lower()
|
||||
if host == request_host:
|
||||
return "internal"
|
||||
return host[:120]
|
||||
|
||||
|
||||
def should_track(request, status_code: int) -> bool:
|
||||
if not _tracking_enabled():
|
||||
return False
|
||||
if request.method != "GET":
|
||||
return False
|
||||
if status_code >= 400:
|
||||
return False
|
||||
path = request.url.path
|
||||
for prefix in EXCLUDED_PREFIXES:
|
||||
if path.startswith(prefix):
|
||||
return False
|
||||
accept = (request.headers.get("accept") or "").lower()
|
||||
if "application/json" in accept and "text/html" not in accept:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def track_visit(request, status_code: int) -> None:
|
||||
if not should_track(request, status_code):
|
||||
return
|
||||
bucket = hour_bucket(now_utc())
|
||||
page_group = _page_group(request)
|
||||
referrer_group = _referrer_group(request)
|
||||
key = (bucket, page_group, referrer_group)
|
||||
slot = _view_buffer[key]
|
||||
slot["views"] += 1
|
||||
user = get_current_user(request)
|
||||
if user:
|
||||
slot["member_views"] += 1
|
||||
member_uid = user["uid"]
|
||||
else:
|
||||
slot["guest_views"] += 1
|
||||
member_uid = None
|
||||
visitor_hash = _visitor_hash(request)
|
||||
_unique_buffer.append((bucket, visitor_hash, page_group, member_uid))
|
||||
|
||||
|
||||
def _flush_views() -> None:
|
||||
if "visit_stats_hourly" not in db.tables:
|
||||
return
|
||||
if not _view_buffer:
|
||||
return
|
||||
pending = dict(_view_buffer)
|
||||
_view_buffer.clear()
|
||||
with db:
|
||||
for (bucket, page_group, referrer_group), counts in pending.items():
|
||||
db.query(
|
||||
"INSERT INTO visit_stats_hourly "
|
||||
"(bucket_start, page_group, referrer_group, views, member_views, guest_views) "
|
||||
"VALUES (:bucket, :page, :ref, :views, :members, :guests) "
|
||||
"ON CONFLICT(bucket_start, page_group, referrer_group) DO UPDATE SET "
|
||||
"views = views + excluded.views, "
|
||||
"member_views = member_views + excluded.member_views, "
|
||||
"guest_views = guest_views + excluded.guest_views",
|
||||
bucket=bucket,
|
||||
page=page_group,
|
||||
ref=referrer_group,
|
||||
views=counts["views"],
|
||||
members=counts["member_views"],
|
||||
guests=counts["guest_views"],
|
||||
)
|
||||
|
||||
|
||||
def _flush_uniques() -> None:
|
||||
if "visit_unique_slots" not in db.tables:
|
||||
return
|
||||
if not _unique_buffer:
|
||||
return
|
||||
pending = list(_unique_buffer)
|
||||
_unique_buffer.clear()
|
||||
with db:
|
||||
for bucket, visitor_hash, page_group, member_uid in pending:
|
||||
db.query(
|
||||
"INSERT OR IGNORE INTO visit_unique_slots "
|
||||
"(bucket_start, visitor_hash, page_group, user_uid) "
|
||||
"VALUES (:bucket, :hash, :page, :uid)",
|
||||
bucket=bucket,
|
||||
hash=visitor_hash,
|
||||
page=page_group,
|
||||
uid=member_uid,
|
||||
)
|
||||
|
||||
|
||||
def _prune_old() -> None:
|
||||
cutoff = (now_utc() - timedelta(days=RETENTION_DAYS)).isoformat(timespec="seconds")
|
||||
with db:
|
||||
if "visit_stats_hourly" in db.tables:
|
||||
db.query("DELETE FROM visit_stats_hourly WHERE bucket_start < :cutoff", cutoff=cutoff)
|
||||
if "visit_unique_slots" in db.tables:
|
||||
db.query("DELETE FROM visit_unique_slots WHERE bucket_start < :cutoff", cutoff=cutoff)
|
||||
|
||||
|
||||
def flush_visits(*, prune: bool = False) -> None:
|
||||
try:
|
||||
_flush_views()
|
||||
_flush_uniques()
|
||||
if prune:
|
||||
_prune_old()
|
||||
except Exception:
|
||||
logger.exception("visit statistics flush failed")
|
||||
|
||||
|
||||
async def _flush_loop() -> None:
|
||||
ticks = 0
|
||||
while True:
|
||||
await asyncio.sleep(FLUSH_SECONDS)
|
||||
ticks += 1
|
||||
flush_visits(prune=ticks % 120 == 0)
|
||||
|
||||
|
||||
def start_visit_flusher() -> None:
|
||||
global _flush_task
|
||||
if _flush_task is not None:
|
||||
return
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
return
|
||||
_flush_task = loop.create_task(_flush_loop())
|
||||
@ -40,6 +40,30 @@
|
||||
margin-bottom: 1.25rem;
|
||||
}
|
||||
|
||||
.admin-tabs[data-overflow-tabs] {
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius-lg);
|
||||
padding: 0.25rem;
|
||||
}
|
||||
|
||||
.admin-tabs[data-overflow-tabs] .overflow-tabs-more {
|
||||
background: none;
|
||||
border: none;
|
||||
box-shadow: none;
|
||||
cursor: pointer;
|
||||
padding: 0.4rem 0.85rem;
|
||||
font-size: 0.8125rem;
|
||||
font-weight: 600;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.admin-tabs[data-overflow-tabs] .overflow-tabs-more:hover {
|
||||
color: var(--text-primary);
|
||||
border-color: transparent;
|
||||
background: none;
|
||||
}
|
||||
|
||||
.admin-tab {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
|
||||
107
devplacepy/static/css/awards.css
Normal file
107
devplacepy/static/css/awards.css
Normal file
@ -0,0 +1,107 @@
|
||||
.award-prominent {
|
||||
margin-bottom: 1.25rem;
|
||||
padding: 1.25rem;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
background: var(--bg-card);
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.award-prominent-image {
|
||||
width: min(256px, 100%);
|
||||
height: auto;
|
||||
object-fit: contain;
|
||||
margin: 0 auto 0.75rem;
|
||||
}
|
||||
|
||||
.award-prominent-desc {
|
||||
font-size: 1.05rem;
|
||||
margin: 0 0 0.5rem;
|
||||
}
|
||||
|
||||
.award-prominent-meta {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.5rem 1rem;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
color: var(--text-muted);
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.award-prominent-giver {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
|
||||
.awards-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
.awards-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
|
||||
.award-tile {
|
||||
position: relative;
|
||||
padding: 1rem;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
background: var(--bg-card);
|
||||
}
|
||||
|
||||
.award-tile-image {
|
||||
width: 100%;
|
||||
max-width: 256px;
|
||||
height: auto;
|
||||
object-fit: contain;
|
||||
display: block;
|
||||
margin: 0 auto 0.75rem;
|
||||
}
|
||||
|
||||
.award-tile-desc {
|
||||
margin: 0 0 0.75rem;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.award-tile-meta {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.5rem 1rem;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
color: var(--text-muted);
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.award-tile-giver {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
|
||||
.award-revoke-form {
|
||||
margin-top: 0.75rem;
|
||||
}
|
||||
|
||||
.award-revoke-btn {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
#award-give-modal textarea {
|
||||
width: 100%;
|
||||
min-height: 5rem;
|
||||
resize: vertical;
|
||||
}
|
||||
|
||||
.award-char-count {
|
||||
display: block;
|
||||
text-align: right;
|
||||
color: var(--text-muted);
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
@ -30,6 +30,34 @@
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
[data-overflow-tabs] {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.25rem;
|
||||
width: 100%;
|
||||
max-width: 100%;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.overflow-tabs-strip {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
align-items: center;
|
||||
flex-wrap: nowrap;
|
||||
gap: inherit;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.overflow-tabs-strip > * {
|
||||
flex-shrink: 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
[data-overflow-tabs] .overflow-tabs-more {
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.modal-card {
|
||||
width: 100%;
|
||||
max-width: 560px;
|
||||
@ -394,22 +422,14 @@ img {
|
||||
display: block;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar {
|
||||
width: 6px;
|
||||
height: 6px;
|
||||
* {
|
||||
scrollbar-width: none;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-track {
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb {
|
||||
background: var(--border);
|
||||
border-radius: 3px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb:hover {
|
||||
background: var(--border-light);
|
||||
*::-webkit-scrollbar {
|
||||
display: none;
|
||||
width: 0;
|
||||
height: 0;
|
||||
}
|
||||
|
||||
.btn {
|
||||
@ -559,6 +579,16 @@ img {
|
||||
|
||||
.presence-dot.online { background: var(--success, #2f9e44); }
|
||||
|
||||
.award-badge {
|
||||
position: absolute;
|
||||
left: 0;
|
||||
bottom: 0;
|
||||
width: 25%;
|
||||
height: 25%;
|
||||
object-fit: contain;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.topnav {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
|
||||
@ -57,6 +57,18 @@
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
.gw-badge {
|
||||
display: inline-block;
|
||||
padding: 0.125rem 0.5rem;
|
||||
border-radius: 999px;
|
||||
background: var(--bg-card-hover);
|
||||
border: 1px solid var(--border);
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-secondary);
|
||||
margin-right: 0.25rem;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.gw-actions {
|
||||
text-align: right;
|
||||
white-space: nowrap;
|
||||
@ -82,6 +94,17 @@
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.gw-form-subtitle {
|
||||
margin-top: var(--space-sm);
|
||||
padding-top: var(--space-sm);
|
||||
border-top: 1px dashed var(--border);
|
||||
}
|
||||
|
||||
.gw-form .gw-section-hint {
|
||||
grid-column: 1 / -1;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.gw-field {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
|
||||
@ -205,31 +205,20 @@ a.profile-stat-value:hover {
|
||||
}
|
||||
|
||||
.profile-tabs {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: nowrap;
|
||||
gap: 0.25rem;
|
||||
margin-bottom: 1rem;
|
||||
background: var(--bg-card);
|
||||
border-radius: var(--radius-lg);
|
||||
padding: 0.25rem;
|
||||
border: 1px solid var(--border);
|
||||
overflow-x: auto;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scroll-margin-top: calc(var(--nav-height) + 1rem);
|
||||
}
|
||||
|
||||
.profile-tabs.tabs-managed {
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.profile-tab {
|
||||
flex-shrink: 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.profile-tabs-more {
|
||||
margin-left: auto;
|
||||
.profile-tabs .overflow-tabs-more {
|
||||
background: none;
|
||||
border: none;
|
||||
cursor: pointer;
|
||||
|
||||
229
devplacepy/static/css/statistics.css
Normal file
229
devplacepy/static/css/statistics.css
Normal file
@ -0,0 +1,229 @@
|
||||
/* retoor <retoor@molodetz.nl> */
|
||||
|
||||
.statistics-toolbar {
|
||||
flex-wrap: wrap;
|
||||
align-items: flex-start;
|
||||
gap: var(--space-md);
|
||||
}
|
||||
|
||||
.statistics-toolbar h2 {
|
||||
flex: 1 1 auto;
|
||||
min-width: 8rem;
|
||||
}
|
||||
|
||||
.statistics-controls {
|
||||
flex: 1 1 16rem;
|
||||
min-width: 0;
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
justify-content: flex-end;
|
||||
gap: var(--space-sm) var(--space-lg);
|
||||
}
|
||||
|
||||
.statistics-controls-primary {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
gap: var(--space-sm) var(--space-lg);
|
||||
}
|
||||
|
||||
.statistics-controls label,
|
||||
.statistics-compare-label {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--text-secondary);
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--space-xs);
|
||||
}
|
||||
|
||||
.statistics-compare-label span {
|
||||
white-space: normal;
|
||||
line-height: 1.35;
|
||||
}
|
||||
|
||||
.statistics-generated {
|
||||
flex: 1 1 12rem;
|
||||
margin-left: 0;
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-muted);
|
||||
text-align: right;
|
||||
white-space: normal;
|
||||
word-break: break-word;
|
||||
line-height: 1.35;
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
.statistics-toolbar {
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
.statistics-controls {
|
||||
justify-content: flex-start;
|
||||
}
|
||||
|
||||
.statistics-generated {
|
||||
flex-basis: 100%;
|
||||
text-align: left;
|
||||
}
|
||||
}
|
||||
|
||||
.statistics-section {
|
||||
margin: var(--space-xl) 0;
|
||||
}
|
||||
|
||||
.statistics-section h3 {
|
||||
margin: 0 0 var(--space-base);
|
||||
font-size: 0.9375rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.statistics-delta {
|
||||
font-size: 0.6875rem;
|
||||
font-weight: 600;
|
||||
margin-top: 0.125rem;
|
||||
}
|
||||
|
||||
.statistics-delta.up {
|
||||
color: var(--success);
|
||||
}
|
||||
|
||||
.statistics-delta.down {
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
.statistics-delta.flat {
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
.statistics-charts {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr;
|
||||
gap: var(--space-md);
|
||||
margin-bottom: var(--space-md);
|
||||
}
|
||||
|
||||
@media (min-width: 768px) {
|
||||
.statistics-charts {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
}
|
||||
}
|
||||
|
||||
.statistics-chart-panel {
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: var(--space-md);
|
||||
min-height: 16rem;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.statistics-chart-panel h4 {
|
||||
margin: 0 0 var(--space-sm);
|
||||
font-size: 0.75rem;
|
||||
font-weight: 600;
|
||||
color: var(--text-secondary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
|
||||
.statistics-chart-wrap {
|
||||
position: relative;
|
||||
flex: 1;
|
||||
min-height: 12rem;
|
||||
}
|
||||
|
||||
.statistics-breakdowns-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
|
||||
gap: var(--space-md);
|
||||
}
|
||||
|
||||
.statistics-table-panel {
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.statistics-table-panel.is-wide {
|
||||
grid-column: 1 / -1;
|
||||
}
|
||||
|
||||
.statistics-table-panel-head {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: var(--space-sm);
|
||||
padding: var(--space-md) var(--space-md) var(--space-sm);
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.statistics-table-panel-head h4 {
|
||||
margin: 0;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 600;
|
||||
color: var(--text-secondary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
|
||||
.statistics-table-panel-count {
|
||||
font-size: 0.6875rem;
|
||||
color: var(--text-muted);
|
||||
font-variant-numeric: tabular-nums;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.statistics-table-panel .admin-table-wrap {
|
||||
overflow: visible;
|
||||
max-height: none;
|
||||
}
|
||||
|
||||
.statistics-table-panel .admin-table td.statistics-col-name {
|
||||
font-weight: 600;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.statistics-table-panel .admin-table td.statistics-col-key {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.statistics-table-panel .admin-table th.statistics-col-num,
|
||||
.statistics-table-panel .admin-table td.statistics-col-num {
|
||||
text-align: right;
|
||||
font-variant-numeric: tabular-nums;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.statistics-table-panel .admin-table td.statistics-col-num {
|
||||
font-weight: 600;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.statistics-table-panel .admin-table th.statistics-col-text,
|
||||
.statistics-table-panel .admin-table td.statistics-col-text {
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.statistics-note {
|
||||
font-size: 0.75rem;
|
||||
color: var(--text-muted);
|
||||
margin: var(--space-sm) 0 var(--space-lg);
|
||||
}
|
||||
|
||||
.statistics-link {
|
||||
font-size: 0.8125rem;
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
.statistics-link:hover {
|
||||
color: var(--accent-hover);
|
||||
}
|
||||
@ -44,6 +44,7 @@ import { LocalTime } from "./LocalTime.js";
|
||||
import { ScrollMemory } from "./ScrollMemory.js";
|
||||
import { GameFarm } from "./GameFarm.js";
|
||||
import { Accessibility } from "./Accessibility.js";
|
||||
import { OverflowTabs } from "./OverflowTabs.js";
|
||||
|
||||
class Application {
|
||||
constructor() {
|
||||
@ -95,6 +96,7 @@ class Application {
|
||||
this.localTime = new LocalTime();
|
||||
this.scrollMemory = new ScrollMemory();
|
||||
this.gameFarm = new GameFarm();
|
||||
this.overflowTabs = new OverflowTabs();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
66
devplacepy/static/js/AwardGiver.js
Normal file
66
devplacepy/static/js/AwardGiver.js
Normal file
@ -0,0 +1,66 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
import { Http } from "./Http.js";
|
||||
|
||||
class AwardGiver {
|
||||
constructor() {
|
||||
this.btn = document.querySelector("[data-award-give]");
|
||||
this.modal = document.getElementById("award-give-modal");
|
||||
if (!this.btn || !this.modal) return;
|
||||
this.form = this.modal.querySelector("form");
|
||||
this.textarea = this.modal.querySelector('textarea[name="description"]');
|
||||
this.counter = this.modal.querySelector("[data-award-char-count]");
|
||||
this.btn.addEventListener("click", () => this.open());
|
||||
if (this.form) {
|
||||
this.form.addEventListener("submit", (event) => this.submit(event));
|
||||
}
|
||||
if (this.textarea) {
|
||||
this.textarea.addEventListener("input", () => this.updateCounter());
|
||||
this.updateCounter();
|
||||
}
|
||||
}
|
||||
|
||||
open() {
|
||||
if (this.btn.disabled) return;
|
||||
this.modal.classList.add("visible");
|
||||
if (this.textarea) {
|
||||
this.textarea.focus();
|
||||
}
|
||||
}
|
||||
|
||||
updateCounter() {
|
||||
if (!this.counter || !this.textarea) return;
|
||||
const max = Number(this.textarea.getAttribute("maxlength") || 125);
|
||||
const length = this.textarea.value.length;
|
||||
this.counter.textContent = `${length}/${max}`;
|
||||
}
|
||||
|
||||
async submit(event) {
|
||||
event.preventDefault();
|
||||
const username = this.btn.dataset.username;
|
||||
const description = (this.textarea?.value || "").trim();
|
||||
if (!description) return;
|
||||
const submitBtn = this.form.querySelector('button[type="submit"]');
|
||||
if (submitBtn) submitBtn.disabled = true;
|
||||
try {
|
||||
await Http.postJson(`/profile/${username}/award`, { description });
|
||||
this.modal.classList.remove("visible");
|
||||
if (this.form) this.form.reset();
|
||||
this.updateCounter();
|
||||
await window.app.dialog.alert({
|
||||
title: "Award submitted",
|
||||
message: `Your award for @${username} is being created. They will be notified when it is ready.`,
|
||||
});
|
||||
} catch {
|
||||
await window.app.dialog.alert({
|
||||
title: "Award failed",
|
||||
message: "Could not submit the award. Try again later.",
|
||||
});
|
||||
} finally {
|
||||
if (submitBtn) submitBtn.disabled = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
window.AwardGiver = AwardGiver;
|
||||
export { AwardGiver };
|
||||
@ -15,9 +15,25 @@ export class GatewayAdmin {
|
||||
|
||||
async start() {
|
||||
this.bind();
|
||||
this.syncKindFields();
|
||||
await this.reload();
|
||||
}
|
||||
|
||||
syncKindFields() {
|
||||
const kind = this.modelForm.kind.value || "chat";
|
||||
const priceInputLabel = this.root.querySelector("#gw-model-price-input-label");
|
||||
if (priceInputLabel) {
|
||||
priceInputLabel.textContent =
|
||||
kind === "image"
|
||||
? "Price per image ($)"
|
||||
: "Price input / 1M ($) (embed/vision)";
|
||||
}
|
||||
this.root.querySelectorAll("[data-kind-field]").forEach((field) => {
|
||||
const kinds = (field.dataset.kindField || "").split(/\s+/).filter(Boolean);
|
||||
field.style.display = kinds.includes(kind) ? "" : "none";
|
||||
});
|
||||
}
|
||||
|
||||
bind() {
|
||||
this.providerForm.addEventListener("submit", (event) => {
|
||||
event.preventDefault();
|
||||
@ -27,6 +43,7 @@ export class GatewayAdmin {
|
||||
event.preventDefault();
|
||||
this.saveModel();
|
||||
});
|
||||
this.modelForm.kind.addEventListener("change", () => this.syncKindFields());
|
||||
this.providersBody.addEventListener("click", (event) => this.onProviderClick(event));
|
||||
this.modelsBody.addEventListener("click", (event) => this.onModelClick(event));
|
||||
}
|
||||
@ -71,7 +88,9 @@ export class GatewayAdmin {
|
||||
el.innerHTML = `
|
||||
<strong>default</strong> (from <a href="/admin/services">Services config</a>):
|
||||
chat <code class="gw-code">${this.escape(def.model)}</code> at <code class="gw-code">${this.escape(def.base_url)}</code>,
|
||||
embed <code class="gw-code">${this.escape(def.embed_model)}</code>, vision <code class="gw-code">${this.escape(def.vision_model)}</code>`;
|
||||
embed <code class="gw-code">${this.escape(def.embed_model)}</code>,
|
||||
image <code class="gw-code">${this.escape(def.image_model)}</code>,
|
||||
vision <code class="gw-code">${this.escape(def.vision_model)}</code>`;
|
||||
}
|
||||
|
||||
renderProviders() {
|
||||
@ -109,7 +128,7 @@ export class GatewayAdmin {
|
||||
const data = await Http.getJson("/admin/gateway/models");
|
||||
const models = data.models || [];
|
||||
if (!models.length) {
|
||||
this.modelsBody.innerHTML = `<tr><td colspan="6" class="admin-empty">No model routes. Requests fall through to the default upstream.</td></tr>`;
|
||||
this.modelsBody.innerHTML = `<tr><td colspan="7" class="admin-empty">No model routes. Requests fall through to the default upstream.</td></tr>`;
|
||||
return 0;
|
||||
}
|
||||
this.modelsBody.innerHTML = models
|
||||
@ -118,12 +137,26 @@ export class GatewayAdmin {
|
||||
? `<code class="gw-code">${this.escape(m.vision_provider || m.provider || "default")}/${this.escape(m.vision_model)}</code>`
|
||||
: `<span class="gw-muted">-</span>`;
|
||||
const provider = m.provider || "default";
|
||||
const badges = [];
|
||||
if (m.context_tier_threshold_tokens) {
|
||||
badges.push(`<span class="gw-badge" title="Tier-2 rates above ${m.context_tier_threshold_tokens} input tokens">tiered</span>`);
|
||||
}
|
||||
if (m.off_peak_start_minute !== null && m.off_peak_start_minute !== undefined) {
|
||||
const start = this.minutesToTime(m.off_peak_start_minute);
|
||||
const end = this.minutesToTime(m.off_peak_end_minute);
|
||||
badges.push(`<span class="gw-badge" title="${m.off_peak_discount_pct}% off ${start}-${end} UTC">off-peak</span>`);
|
||||
}
|
||||
if (m.kind === "image" && m.price_input_per_m) {
|
||||
badges.push(`<span class="gw-badge" title="Flat price per generated image">$${m.price_input_per_m}/img</span>`);
|
||||
}
|
||||
const economy = badges.length ? badges.join(" ") : `<span class="gw-muted">-</span>`;
|
||||
return `<tr>
|
||||
<td>${this.escape(m.source_model)}</td>
|
||||
<td>${this.escape(provider)}</td>
|
||||
<td><code class="gw-code">${this.escape(m.target_model)}</code></td>
|
||||
<td>${this.escape(m.kind)}</td>
|
||||
<td>${vision}</td>
|
||||
<td>${economy}</td>
|
||||
<td class="gw-actions">
|
||||
<button class="admin-btn admin-btn-sm" data-edit-model='${this.attr(JSON.stringify(m))}'>Edit</button>
|
||||
<button class="admin-btn admin-btn-sm admin-btn-danger" data-del-model="${this.attr(m.source_model)}">Delete</button>
|
||||
@ -142,6 +175,28 @@ export class GatewayAdmin {
|
||||
return values;
|
||||
}
|
||||
|
||||
timeToMinutes(value) {
|
||||
if (!value) return null;
|
||||
const [hours, minutes] = value.split(":").map((part) => parseInt(part, 10));
|
||||
if (Number.isNaN(hours) || Number.isNaN(minutes)) return null;
|
||||
return hours * 60 + minutes;
|
||||
}
|
||||
|
||||
minutesToTime(minutes) {
|
||||
if (minutes === null || minutes === undefined || minutes === "") return "";
|
||||
const total = parseInt(minutes, 10);
|
||||
if (Number.isNaN(total)) return "";
|
||||
const hours = Math.floor(total / 60) % 24;
|
||||
const mins = total % 60;
|
||||
return `${String(hours).padStart(2, "0")}:${String(mins).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
optionalFloat(value) {
|
||||
if (value === "" || value === null || value === undefined) return null;
|
||||
const parsed = parseFloat(value);
|
||||
return Number.isNaN(parsed) ? null : parsed;
|
||||
}
|
||||
|
||||
async saveProvider() {
|
||||
const values = this.formValues(this.providerForm);
|
||||
const payload = {
|
||||
@ -174,6 +229,14 @@ export class GatewayAdmin {
|
||||
price_cache_miss_per_m: parseFloat(values.price_cache_miss_per_m) || 0,
|
||||
price_output_per_m: parseFloat(values.price_output_per_m) || 0,
|
||||
price_input_per_m: parseFloat(values.price_input_per_m) || 0,
|
||||
context_tier_threshold_tokens: parseInt(values.context_tier_threshold_tokens, 10) || 0,
|
||||
price_cache_hit_per_m_tier2: this.optionalFloat(values.price_cache_hit_per_m_tier2),
|
||||
price_cache_miss_per_m_tier2: this.optionalFloat(values.price_cache_miss_per_m_tier2),
|
||||
price_output_per_m_tier2: this.optionalFloat(values.price_output_per_m_tier2),
|
||||
price_input_per_m_tier2: this.optionalFloat(values.price_input_per_m_tier2),
|
||||
off_peak_start_minute: this.timeToMinutes(values.off_peak_start),
|
||||
off_peak_end_minute: this.timeToMinutes(values.off_peak_end),
|
||||
off_peak_discount_pct: parseFloat(values.off_peak_discount_pct) || 0,
|
||||
is_active: values.is_active === "1",
|
||||
};
|
||||
try {
|
||||
@ -243,6 +306,7 @@ export class GatewayAdmin {
|
||||
form.provider.value = model.provider || "";
|
||||
form.target_model.value = model.target_model || "";
|
||||
form.kind.value = model.kind || "chat";
|
||||
this.syncKindFields();
|
||||
form.vision_provider.value = model.vision_provider || "";
|
||||
form.vision_model.value = model.vision_model || "";
|
||||
form.context_window.value = model.context_window || 0;
|
||||
@ -250,6 +314,14 @@ export class GatewayAdmin {
|
||||
form.price_cache_miss_per_m.value = model.price_cache_miss_per_m || 0;
|
||||
form.price_output_per_m.value = model.price_output_per_m || 0;
|
||||
form.price_input_per_m.value = model.price_input_per_m || 0;
|
||||
form.context_tier_threshold_tokens.value = model.context_tier_threshold_tokens || 0;
|
||||
form.price_cache_hit_per_m_tier2.value = model.price_cache_hit_per_m_tier2 ?? "";
|
||||
form.price_cache_miss_per_m_tier2.value = model.price_cache_miss_per_m_tier2 ?? "";
|
||||
form.price_output_per_m_tier2.value = model.price_output_per_m_tier2 ?? "";
|
||||
form.price_input_per_m_tier2.value = model.price_input_per_m_tier2 ?? "";
|
||||
form.off_peak_start.value = this.minutesToTime(model.off_peak_start_minute);
|
||||
form.off_peak_end.value = this.minutesToTime(model.off_peak_end_minute);
|
||||
form.off_peak_discount_pct.value = model.off_peak_discount_pct || 0;
|
||||
form.is_active.value = model.is_active ? "1" : "0";
|
||||
form.source_model.scrollIntoView({ block: "center" });
|
||||
}
|
||||
|
||||
@ -22,6 +22,13 @@ export class LocalTime {
|
||||
const date = new Date(iso);
|
||||
if (isNaN(date.getTime())) return null;
|
||||
if (mode === "ago") return this.relative(date);
|
||||
if (mode === "award") {
|
||||
const weekday = date.toLocaleDateString([], { weekday: "long" });
|
||||
const day = String(date.getDate()).padStart(2, "0");
|
||||
const month = String(date.getMonth() + 1).padStart(2, "0");
|
||||
const year = date.getFullYear();
|
||||
return `${weekday}, ${day}-${month}-${year}`;
|
||||
}
|
||||
if (mode === "date") return date.toLocaleDateString();
|
||||
if (mode === "time") {
|
||||
return date.toLocaleTimeString([], { hour: "2-digit", minute: "2-digit" });
|
||||
|
||||
@ -49,6 +49,14 @@ export class OnlineUsers {
|
||||
dot.setAttribute("aria-hidden", "true");
|
||||
badge.appendChild(img);
|
||||
badge.appendChild(dot);
|
||||
if (user.award_prominent && user.last_award_slug) {
|
||||
const award = document.createElement("img");
|
||||
award.className = "award-badge";
|
||||
award.src = `/awards/${encodeURIComponent(user.last_award_slug)}/64`;
|
||||
award.alt = "Latest award";
|
||||
award.loading = "lazy";
|
||||
badge.appendChild(award);
|
||||
}
|
||||
link.appendChild(badge);
|
||||
return link;
|
||||
}
|
||||
|
||||
115
devplacepy/static/js/OverflowTabs.js
Normal file
115
devplacepy/static/js/OverflowTabs.js
Normal file
@ -0,0 +1,115 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
export class OverflowTabs {
|
||||
constructor() {
|
||||
this.instances = [];
|
||||
document.querySelectorAll("[data-overflow-tabs]").forEach((root) => {
|
||||
const instance = this.create(root);
|
||||
if (instance) {
|
||||
this.instances.push(instance);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
create(root) {
|
||||
const more = root.querySelector(".overflow-tabs-more");
|
||||
const strip = root.querySelector(".overflow-tabs-strip");
|
||||
if (!more || !strip) {
|
||||
return null;
|
||||
}
|
||||
const tabs = Array.from(strip.children).filter(
|
||||
(el) => !el.classList.contains("overflow-tabs-more")
|
||||
);
|
||||
if (!tabs.length) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const state = {
|
||||
root,
|
||||
strip,
|
||||
more,
|
||||
tabs,
|
||||
overflow: [],
|
||||
};
|
||||
|
||||
state.sync = () => {
|
||||
void strip.offsetWidth;
|
||||
};
|
||||
|
||||
state.stripOverflows = () => strip.scrollWidth > strip.clientWidth + 1;
|
||||
|
||||
state.layout = () => {
|
||||
tabs.forEach((tab) => {
|
||||
tab.hidden = false;
|
||||
});
|
||||
more.hidden = true;
|
||||
state.sync();
|
||||
|
||||
if (!state.stripOverflows()) {
|
||||
state.overflow = [];
|
||||
return;
|
||||
}
|
||||
|
||||
more.hidden = false;
|
||||
state.sync();
|
||||
|
||||
while (state.stripOverflows()) {
|
||||
const victim = tabs
|
||||
.filter((tab) => !tab.hidden && !tab.classList.contains("active"))
|
||||
.pop();
|
||||
if (!victim) {
|
||||
break;
|
||||
}
|
||||
victim.hidden = true;
|
||||
state.sync();
|
||||
}
|
||||
|
||||
state.overflow = tabs.filter((tab) => tab.hidden);
|
||||
if (state.overflow.length > 0 || state.stripOverflows()) {
|
||||
more.hidden = false;
|
||||
}
|
||||
};
|
||||
|
||||
state.open = (event) => {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
const items = state.overflow.map((tab) => ({
|
||||
icon: tab.dataset.menuIcon || "",
|
||||
label: tab.dataset.menuLabel || tab.textContent.trim(),
|
||||
onSelect: () => tab.click(),
|
||||
}));
|
||||
if (!items.length) {
|
||||
return;
|
||||
}
|
||||
const rect = more.getBoundingClientRect();
|
||||
window.app.contextMenu.open(rect.left, rect.bottom + 4, items);
|
||||
};
|
||||
|
||||
root.classList.add("tabs-managed");
|
||||
more.addEventListener("click", (event) => state.open(event));
|
||||
for (const tab of tabs) {
|
||||
tab.addEventListener("click", () => {
|
||||
requestAnimationFrame(() => state.layout());
|
||||
});
|
||||
}
|
||||
|
||||
const scheduleLayout = () => {
|
||||
requestAnimationFrame(() => state.layout());
|
||||
};
|
||||
|
||||
window.addEventListener("resize", scheduleLayout);
|
||||
if (typeof ResizeObserver !== "undefined") {
|
||||
state.resizeObserver = new ResizeObserver(scheduleLayout);
|
||||
state.resizeObserver.observe(root);
|
||||
state.resizeObserver.observe(strip);
|
||||
}
|
||||
|
||||
scheduleLayout();
|
||||
if (document.fonts?.ready) {
|
||||
document.fonts.ready.then(scheduleLayout);
|
||||
}
|
||||
window.addEventListener("load", scheduleLayout, { once: true });
|
||||
|
||||
return state;
|
||||
}
|
||||
}
|
||||
@ -1,64 +0,0 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
export class ProfileTabs {
|
||||
constructor(root) {
|
||||
this.root = root;
|
||||
this.more = root.querySelector(".profile-tabs-more");
|
||||
this.tabs = Array.from(root.querySelectorAll(".profile-tab")).filter(
|
||||
(tab) => tab !== this.more
|
||||
);
|
||||
if (!this.more || !this.tabs.length) {
|
||||
return;
|
||||
}
|
||||
this.overflow = [];
|
||||
this.root.classList.add("tabs-managed");
|
||||
this.more.addEventListener("click", (event) => this.open(event));
|
||||
this._onResize = () => this.layout();
|
||||
window.addEventListener("resize", this._onResize);
|
||||
this.layout();
|
||||
}
|
||||
|
||||
layout() {
|
||||
this.tabs.forEach((tab) => {
|
||||
tab.hidden = false;
|
||||
});
|
||||
this.more.hidden = false;
|
||||
|
||||
if (this._fits()) {
|
||||
this.more.hidden = true;
|
||||
this.overflow = [];
|
||||
return;
|
||||
}
|
||||
|
||||
while (!this._fits()) {
|
||||
const victim = this.tabs
|
||||
.filter((tab) => !tab.hidden && !tab.classList.contains("active"))
|
||||
.pop();
|
||||
if (!victim) {
|
||||
break;
|
||||
}
|
||||
victim.hidden = true;
|
||||
}
|
||||
|
||||
this.overflow = this.tabs.filter((tab) => tab.hidden);
|
||||
}
|
||||
|
||||
_fits() {
|
||||
return this.root.scrollWidth <= this.root.clientWidth;
|
||||
}
|
||||
|
||||
open(event) {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
const items = this.overflow.map((tab) => ({
|
||||
icon: tab.dataset.menuIcon || "",
|
||||
label: tab.dataset.menuLabel || tab.textContent.trim(),
|
||||
onSelect: () => tab.click(),
|
||||
}));
|
||||
if (!items.length) {
|
||||
return;
|
||||
}
|
||||
const rect = this.more.getBoundingClientRect();
|
||||
window.app.contextMenu.open(rect.left, rect.bottom + 4, items);
|
||||
}
|
||||
}
|
||||
111
devplacepy/static/js/StatisticsCharts.js
Normal file
111
devplacepy/static/js/StatisticsCharts.js
Normal file
@ -0,0 +1,111 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
const CHART_COLORS = [
|
||||
"#ff6b35",
|
||||
"#7c4dff",
|
||||
"#00bfa5",
|
||||
"#448aff",
|
||||
"#ffab00",
|
||||
"#ff5252",
|
||||
"#42a5f5",
|
||||
"#ff4f8b",
|
||||
];
|
||||
|
||||
export class StatisticsCharts {
|
||||
constructor() {
|
||||
this.instances = [];
|
||||
}
|
||||
|
||||
destroy() {
|
||||
for (const chart of this.instances) {
|
||||
try {
|
||||
chart.destroy();
|
||||
} catch {
|
||||
// ignore destroy errors
|
||||
}
|
||||
}
|
||||
this.instances = [];
|
||||
}
|
||||
|
||||
render(container, seriesList) {
|
||||
this.destroy();
|
||||
if (!container || !window.Chart || !Array.isArray(seriesList)) return;
|
||||
const panels = [];
|
||||
for (const series of seriesList) {
|
||||
const panel = document.createElement("div");
|
||||
panel.className = "statistics-chart-panel";
|
||||
const title = document.createElement("h4");
|
||||
title.textContent = series.label || series.key || "Series";
|
||||
panel.appendChild(title);
|
||||
const wrap = document.createElement("div");
|
||||
wrap.className = "statistics-chart-wrap";
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.setAttribute("role", "img");
|
||||
canvas.setAttribute("aria-label", `${series.label || "Chart"} over time`);
|
||||
wrap.appendChild(canvas);
|
||||
panel.appendChild(wrap);
|
||||
container.appendChild(panel);
|
||||
panels.push({ canvas, series });
|
||||
}
|
||||
for (let index = 0; index < panels.length; index += 1) {
|
||||
const { canvas, series } = panels[index];
|
||||
const labels = (series.points || []).map((point) => point.t);
|
||||
const values = (series.points || []).map((point) => Number(point.v || 0));
|
||||
const color = CHART_COLORS[index % CHART_COLORS.length];
|
||||
const chart = new window.Chart(canvas, {
|
||||
type: "line",
|
||||
data: {
|
||||
labels,
|
||||
datasets: [
|
||||
{
|
||||
label: series.label || series.key,
|
||||
data: values,
|
||||
borderColor: color,
|
||||
backgroundColor: `${color}22`,
|
||||
fill: true,
|
||||
tension: 0.25,
|
||||
pointRadius: labels.length > 40 ? 0 : 2,
|
||||
pointHoverRadius: 4,
|
||||
},
|
||||
],
|
||||
},
|
||||
options: {
|
||||
responsive: true,
|
||||
maintainAspectRatio: false,
|
||||
interaction: { mode: "index", intersect: false },
|
||||
plugins: {
|
||||
legend: { display: false },
|
||||
tooltip: {
|
||||
callbacks: {
|
||||
label(context) {
|
||||
const value = context.parsed.y;
|
||||
return `${series.label}: ${Number(value).toLocaleString("en-GB")}`;
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
scales: {
|
||||
x: {
|
||||
ticks: {
|
||||
maxRotation: 0,
|
||||
autoSkip: true,
|
||||
maxTicksLimit: 8,
|
||||
},
|
||||
grid: { color: "rgba(255, 255, 255, 0.08)" },
|
||||
ticks: { color: "#7a6a90" },
|
||||
},
|
||||
y: {
|
||||
beginAtZero: true,
|
||||
ticks: {
|
||||
precision: 0,
|
||||
color: "#7a6a90",
|
||||
},
|
||||
grid: { color: "rgba(255, 255, 255, 0.08)" },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
this.instances.push(chart);
|
||||
}
|
||||
}
|
||||
}
|
||||
327
devplacepy/static/js/StatisticsDashboard.js
Normal file
327
devplacepy/static/js/StatisticsDashboard.js
Normal file
@ -0,0 +1,327 @@
|
||||
// retoor <retoor@molodetz.nl>
|
||||
|
||||
import { Http } from "./Http.js";
|
||||
import { StatisticsCharts } from "./StatisticsCharts.js";
|
||||
|
||||
class StatisticsDashboard {
|
||||
constructor(options = {}) {
|
||||
this.root = document.getElementById(options.rootId || "statistics-root");
|
||||
this.initialNode = document.getElementById(options.initialId || "statistics-initial");
|
||||
this.windowSelect = document.getElementById("statistics-window");
|
||||
this.compareToggle = document.getElementById("statistics-compare");
|
||||
this.generated = document.querySelector("[data-meta='generated']");
|
||||
this.tabLinks = Array.from(document.querySelectorAll(".admin-tabs .admin-tab[data-tab]"));
|
||||
this.activeTab = options.activeTab || "overview";
|
||||
this.windowHours = Number(options.windowHours || 168);
|
||||
this.charts = new StatisticsCharts();
|
||||
this.cache = new Map();
|
||||
this.loading = false;
|
||||
}
|
||||
|
||||
start() {
|
||||
if (!this.root) return;
|
||||
if (this.windowSelect) {
|
||||
this.windowSelect.addEventListener("change", () => {
|
||||
const parsed = parseInt(this.windowSelect.value, 10);
|
||||
this.windowHours = Number.isNaN(parsed) ? 168 : parsed;
|
||||
this.cache.clear();
|
||||
this.loadTab(this.activeTab, true);
|
||||
});
|
||||
}
|
||||
if (this.compareToggle) {
|
||||
this.compareToggle.addEventListener("change", () => {
|
||||
this.cache.clear();
|
||||
this.loadTab(this.activeTab, true);
|
||||
});
|
||||
}
|
||||
for (const link of this.tabLinks) {
|
||||
link.addEventListener("click", (event) => {
|
||||
event.preventDefault();
|
||||
const tab = link.dataset.tab;
|
||||
if (!tab || tab === this.activeTab) return;
|
||||
for (const node of this.tabLinks) {
|
||||
node.classList.toggle("active", node.dataset.tab === tab);
|
||||
}
|
||||
this.activeTab = tab;
|
||||
const url = new URL(window.location.href);
|
||||
url.searchParams.set("tab", tab);
|
||||
url.searchParams.set("hours", String(this.windowHours));
|
||||
window.history.replaceState({}, "", url);
|
||||
this.loadTab(tab, false);
|
||||
});
|
||||
}
|
||||
const initial = this.readInitial();
|
||||
if (initial) {
|
||||
this.cache.set(this.cacheKey(this.activeTab), initial);
|
||||
this.render(initial);
|
||||
} else {
|
||||
this.loadTab(this.activeTab, true);
|
||||
}
|
||||
}
|
||||
|
||||
readInitial() {
|
||||
if (!this.initialNode) return null;
|
||||
try {
|
||||
return JSON.parse(this.initialNode.textContent || "null");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
cacheKey(tab) {
|
||||
const compare = this.compareToggle && this.compareToggle.checked ? 1 : 0;
|
||||
return `${tab}:${this.windowHours}:${compare}`;
|
||||
}
|
||||
|
||||
async loadTab(tab, force) {
|
||||
const key = this.cacheKey(tab);
|
||||
if (!force && this.cache.has(key)) {
|
||||
this.render(this.cache.get(key));
|
||||
return;
|
||||
}
|
||||
if (this.loading) return;
|
||||
this.loading = true;
|
||||
this.root.innerHTML = '<p class="admin-empty">Loading statistics...</p>';
|
||||
try {
|
||||
const compare = this.compareToggle && this.compareToggle.checked ? 1 : 0;
|
||||
const data = await Http.getJson(
|
||||
`/admin/statistics/data?tab=${encodeURIComponent(tab)}&hours=${this.windowHours}&compare=${compare}`
|
||||
);
|
||||
this.cache.set(key, data);
|
||||
this.render(data);
|
||||
} catch {
|
||||
this.root.innerHTML = '<p class="admin-empty">Could not load statistics.</p>';
|
||||
} finally {
|
||||
this.loading = false;
|
||||
}
|
||||
}
|
||||
|
||||
formatValue(card) {
|
||||
const value = card.value;
|
||||
const kind = card.format || "int";
|
||||
if (kind === "money") {
|
||||
const number = Number(value || 0);
|
||||
return number < 1 ? `$${number.toFixed(4)}` : `$${number.toFixed(2)}`;
|
||||
}
|
||||
if (kind === "decimal") {
|
||||
return Number(value || 0).toLocaleString("en-GB", { maximumFractionDigits: 1 });
|
||||
}
|
||||
return Number(value || 0).toLocaleString("en-GB");
|
||||
}
|
||||
|
||||
deltaMarkup(card) {
|
||||
if (card.delta === undefined || card.delta === null) return null;
|
||||
const direction = card.direction || "flat";
|
||||
const arrow = direction === "up" ? "\u2191" : direction === "down" ? "\u2193" : "\u2192";
|
||||
const node = document.createElement("span");
|
||||
node.className = `statistics-delta ${direction}`;
|
||||
node.textContent = `${arrow} ${Math.abs(Number(card.delta || 0)).toLocaleString("en-GB")}%`;
|
||||
return node;
|
||||
}
|
||||
|
||||
isNumericValue(value) {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return true;
|
||||
const text = String(value ?? "").trim().replace(/,/g, "");
|
||||
if (!text) return false;
|
||||
return /^-?\d+(\.\d+)?$/.test(text);
|
||||
}
|
||||
|
||||
columnIsNumeric(columnIndex, columns, rows) {
|
||||
const header = (columns[columnIndex] || "").toLowerCase();
|
||||
const numericHeaders = [
|
||||
"count", "events", "views", "gists", "posts", "votes", "actions", "stars",
|
||||
"turns", "requests", "cost", "coins", "level", "restarts", "followers",
|
||||
"upvotes", "downvotes", "members", "guests", "tokens", "errors",
|
||||
];
|
||||
if (numericHeaders.some((token) => header.includes(token))) return true;
|
||||
const sample = rows.slice(0, 12);
|
||||
if (!sample.length) return columnIndex > 0;
|
||||
return sample.every((row) => this.isNumericValue(row[columnIndex]));
|
||||
}
|
||||
|
||||
formatTableCell(value) {
|
||||
if (typeof value === "number" && Number.isFinite(value)) {
|
||||
if (Number.isInteger(value)) return value.toLocaleString("en-GB");
|
||||
return value.toLocaleString("en-GB", { maximumFractionDigits: 2 });
|
||||
}
|
||||
const text = String(value ?? "");
|
||||
const plain = text.trim().replace(/,/g, "");
|
||||
if (/^-?\d+(\.\d+)?$/.test(plain)) {
|
||||
const number = Number(plain);
|
||||
if (Number.isInteger(number)) return number.toLocaleString("en-GB");
|
||||
return number.toLocaleString("en-GB", { maximumFractionDigits: 2 });
|
||||
}
|
||||
return text;
|
||||
}
|
||||
|
||||
columnClass(columnIndex, columns, rows) {
|
||||
const header = (columns[columnIndex] || "").toLowerCase();
|
||||
if (columnIndex === 0) {
|
||||
if (
|
||||
header.includes("username")
|
||||
|| header === "user"
|
||||
|| header.includes("instance")
|
||||
|| header === "name"
|
||||
) {
|
||||
return "statistics-col-name";
|
||||
}
|
||||
return "statistics-col-key";
|
||||
}
|
||||
if (this.columnIsNumeric(columnIndex, columns, rows)) return "statistics-col-num";
|
||||
return "statistics-col-text";
|
||||
}
|
||||
|
||||
renderBreakdownTable(table) {
|
||||
const columns = table.columns || [];
|
||||
const rows = table.rows || [];
|
||||
const panel = document.createElement("article");
|
||||
panel.className = "statistics-table-panel";
|
||||
if (columns.length > 2) panel.classList.add("is-wide");
|
||||
|
||||
const head = document.createElement("div");
|
||||
head.className = "statistics-table-panel-head";
|
||||
const title = document.createElement("h4");
|
||||
title.textContent = table.title || "";
|
||||
head.appendChild(title);
|
||||
const count = document.createElement("span");
|
||||
count.className = "statistics-table-panel-count";
|
||||
count.textContent = `${rows.length.toLocaleString("en-GB")} rows`;
|
||||
head.appendChild(count);
|
||||
panel.appendChild(head);
|
||||
|
||||
const wrap = document.createElement("div");
|
||||
wrap.className = "admin-table-wrap";
|
||||
|
||||
const node = document.createElement("table");
|
||||
node.className = "admin-table";
|
||||
|
||||
const thead = document.createElement("thead");
|
||||
const headRow = document.createElement("tr");
|
||||
for (let index = 0; index < columns.length; index += 1) {
|
||||
const th = document.createElement("th");
|
||||
th.className = this.columnClass(index, columns, rows);
|
||||
th.textContent = columns[index];
|
||||
headRow.appendChild(th);
|
||||
}
|
||||
thead.appendChild(headRow);
|
||||
node.appendChild(thead);
|
||||
|
||||
const tbody = document.createElement("tbody");
|
||||
for (const row of rows) {
|
||||
const tr = document.createElement("tr");
|
||||
for (let index = 0; index < columns.length; index += 1) {
|
||||
const td = document.createElement("td");
|
||||
const cellClass = this.columnClass(index, columns, rows);
|
||||
td.className = cellClass;
|
||||
const cell = row[index];
|
||||
td.textContent = cellClass === "statistics-col-num"
|
||||
? this.formatTableCell(cell)
|
||||
: String(cell ?? "");
|
||||
tr.appendChild(td);
|
||||
}
|
||||
tbody.appendChild(tr);
|
||||
}
|
||||
node.appendChild(tbody);
|
||||
wrap.appendChild(node);
|
||||
panel.appendChild(wrap);
|
||||
return panel;
|
||||
}
|
||||
|
||||
render(data) {
|
||||
if (!data) return;
|
||||
if (this.generated && data.generated_at) {
|
||||
this.generated.textContent = `Generated ${data.generated_at.replace("T", " ").slice(0, 19)} UTC`;
|
||||
}
|
||||
this.root.innerHTML = "";
|
||||
if (Array.isArray(data.highlights) && data.highlights.length) {
|
||||
const wrap = document.createElement("div");
|
||||
wrap.className = "metric-cards";
|
||||
for (const item of data.highlights) {
|
||||
const card = document.createElement("div");
|
||||
card.className = "metric-card";
|
||||
const value = document.createElement("span");
|
||||
value.className = "metric-value";
|
||||
value.textContent = String(item.value ?? "");
|
||||
const label = document.createElement("span");
|
||||
label.className = "metric-label";
|
||||
label.textContent = item.label || "";
|
||||
card.appendChild(value);
|
||||
card.appendChild(label);
|
||||
wrap.appendChild(card);
|
||||
}
|
||||
this.root.appendChild(wrap);
|
||||
}
|
||||
if (Array.isArray(data.cards) && data.cards.length) {
|
||||
const grid = document.createElement("div");
|
||||
grid.className = "metric-cards";
|
||||
for (const card of data.cards) {
|
||||
const node = document.createElement("div");
|
||||
node.className = "metric-card";
|
||||
const value = document.createElement("span");
|
||||
value.className = "metric-value";
|
||||
value.textContent = this.formatValue(card);
|
||||
const label = document.createElement("span");
|
||||
label.className = "metric-label";
|
||||
label.textContent = card.label || "";
|
||||
node.appendChild(value);
|
||||
node.appendChild(label);
|
||||
const delta = this.deltaMarkup(card);
|
||||
if (delta) node.appendChild(delta);
|
||||
grid.appendChild(node);
|
||||
}
|
||||
this.root.appendChild(grid);
|
||||
}
|
||||
if (Array.isArray(data.series) && data.series.length) {
|
||||
const section = document.createElement("div");
|
||||
section.className = "statistics-section";
|
||||
const heading = document.createElement("h3");
|
||||
heading.textContent = "Trends";
|
||||
section.appendChild(heading);
|
||||
const charts = document.createElement("div");
|
||||
charts.className = "statistics-charts";
|
||||
section.appendChild(charts);
|
||||
this.root.appendChild(section);
|
||||
const renderCharts = () => this.charts.render(charts, data.series);
|
||||
if (window.Chart) {
|
||||
renderCharts();
|
||||
} else {
|
||||
window.addEventListener("load", renderCharts, { once: true });
|
||||
}
|
||||
} else {
|
||||
this.charts.destroy();
|
||||
}
|
||||
if (Array.isArray(data.tables) && data.tables.length) {
|
||||
const section = document.createElement("div");
|
||||
section.className = "statistics-section statistics-breakdowns";
|
||||
const heading = document.createElement("h3");
|
||||
heading.textContent = "Breakdowns";
|
||||
section.appendChild(heading);
|
||||
const grid = document.createElement("div");
|
||||
grid.className = "statistics-breakdowns-grid";
|
||||
for (const table of data.tables) {
|
||||
grid.appendChild(this.renderBreakdownTable(table));
|
||||
}
|
||||
section.appendChild(grid);
|
||||
this.root.appendChild(section);
|
||||
}
|
||||
if (data.notes && data.notes.full_page) {
|
||||
const note = document.createElement("p");
|
||||
note.className = "statistics-note";
|
||||
const link = document.createElement("a");
|
||||
link.className = "statistics-link";
|
||||
link.href = data.notes.full_page;
|
||||
link.textContent = "Open full detail page";
|
||||
note.appendChild(link);
|
||||
this.root.appendChild(note);
|
||||
}
|
||||
if (data.notes && data.notes.retention_days) {
|
||||
const note = document.createElement("p");
|
||||
note.className = "statistics-note";
|
||||
note.textContent = `Visitor data retention: ${data.notes.retention_days} days.`;
|
||||
this.root.appendChild(note);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
window.StatisticsDashboard = StatisticsDashboard;
|
||||
export { StatisticsDashboard };
|
||||
20
devplacepy/static/vendor/chartjs/chart.umd.min.js
vendored
Normal file
20
devplacepy/static/vendor/chartjs/chart.umd.min.js
vendored
Normal file
File diff suppressed because one or more lines are too long
@ -6,6 +6,7 @@ import logging
|
||||
import re
|
||||
import ssl
|
||||
from dataclasses import dataclass, field
|
||||
from os import environ
|
||||
from pathlib import Path
|
||||
from typing import Iterable, Mapping, Sequence
|
||||
from urllib.parse import unquote, urlsplit
|
||||
@ -14,6 +15,8 @@ import httpx
|
||||
|
||||
logger = logging.getLogger("stealth")
|
||||
|
||||
OUTBOUND_PROXY_URL: str | None = environ.get("DEVPLACE_OUTBOUND_PROXY_URL", "").strip() or None
|
||||
|
||||
|
||||
def _supported_accept_encoding() -> str:
|
||||
encodings = ["gzip", "deflate"]
|
||||
@ -55,7 +58,7 @@ CHROME_CIPHER_SUITES: str = ":".join(
|
||||
|
||||
ALPN_PROTOCOLS: tuple[str, ...] = ("h2", "http/1.1")
|
||||
|
||||
DEFAULT_CHROME_VERSION: str = "149"
|
||||
DEFAULT_CHROME_VERSION: str = "146"
|
||||
|
||||
DEFAULT_CHUNK_SIZE: int = 65536
|
||||
|
||||
@ -155,6 +158,24 @@ def resource_kind_for_url(url: str) -> str:
|
||||
return "image" if path.endswith(IMAGE_EXTENSIONS) else "empty"
|
||||
|
||||
|
||||
CONSENT_GATE_MARKERS: tuple[str, ...] = ("privacy gate", "privacy-gate", "cmpproperties", "consent.js")
|
||||
|
||||
CONSENT_GATE_CALLBACK_RE = re.compile(r"decodeURIComponent\(\s*['\"]([^'\"]+)['\"]\s*\)")
|
||||
|
||||
|
||||
def detect_consent_gate(html: str) -> str | None:
|
||||
lowered = html.lower()
|
||||
if not any(marker in lowered for marker in CONSENT_GATE_MARKERS):
|
||||
return None
|
||||
match = CONSENT_GATE_CALLBACK_RE.search(html)
|
||||
if not match:
|
||||
return None
|
||||
candidate = unquote(match.group(1))
|
||||
if not candidate.startswith(("http://", "https://")):
|
||||
return None
|
||||
return candidate
|
||||
|
||||
|
||||
def build_chrome_ssl_context() -> ssl.SSLContext:
|
||||
context = ssl.create_default_context()
|
||||
context.check_hostname = True
|
||||
@ -207,13 +228,26 @@ def chrome_base_headers(profile: ChromeProfile | None = None) -> dict[str, str]:
|
||||
}
|
||||
|
||||
|
||||
def configured_proxy_url() -> str | None:
|
||||
try:
|
||||
from devplacepy.database import get_setting
|
||||
|
||||
configured = get_setting("outbound_proxy_url", "").strip()
|
||||
except ImportError:
|
||||
configured = ""
|
||||
return configured or OUTBOUND_PROXY_URL
|
||||
|
||||
|
||||
def stealth_transport(
|
||||
profile: ChromeProfile | None = None,
|
||||
*,
|
||||
proxy: str | None = None,
|
||||
**kwargs: object,
|
||||
) -> httpx.AsyncBaseTransport:
|
||||
effective_proxy = proxy or configured_proxy_url()
|
||||
if CURL_AVAILABLE:
|
||||
return CurlTransport()
|
||||
return httpx.AsyncHTTPTransport(verify=chrome_ssl_context(), http2=True, **kwargs)
|
||||
return CurlTransport(proxy=effective_proxy)
|
||||
return httpx.AsyncHTTPTransport(verify=chrome_ssl_context(), http2=True, proxy=effective_proxy, **kwargs)
|
||||
|
||||
|
||||
def stealth_async_client(
|
||||
@ -221,13 +255,14 @@ def stealth_async_client(
|
||||
profile: ChromeProfile | None = None,
|
||||
headers: Mapping[str, str] | None = None,
|
||||
transport: httpx.AsyncBaseTransport | None = None,
|
||||
proxy: str | None = None,
|
||||
**kwargs: object,
|
||||
) -> httpx.AsyncClient:
|
||||
merged_headers: dict[str, str] = {} if CURL_AVAILABLE else chrome_base_headers(profile)
|
||||
if headers:
|
||||
merged_headers.update({key.lower(): value for key, value in headers.items()})
|
||||
if transport is None:
|
||||
transport = stealth_transport(profile)
|
||||
transport = stealth_transport(profile, proxy=proxy)
|
||||
return httpx.AsyncClient(
|
||||
transport=transport,
|
||||
trust_env=True,
|
||||
@ -240,6 +275,7 @@ def stealth_sync_client(
|
||||
*,
|
||||
profile: ChromeProfile | None = None,
|
||||
headers: Mapping[str, str] | None = None,
|
||||
proxy: str | None = None,
|
||||
**kwargs: object,
|
||||
) -> httpx.Client:
|
||||
merged_headers = chrome_base_headers(profile)
|
||||
@ -250,6 +286,7 @@ def stealth_sync_client(
|
||||
verify=chrome_ssl_context(),
|
||||
trust_env=True,
|
||||
headers=merged_headers,
|
||||
proxy=proxy or configured_proxy_url(),
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@ -627,4 +664,6 @@ __all__ = [
|
||||
"curl_active",
|
||||
"resource_kind_for_url",
|
||||
"slugify_filename",
|
||||
"detect_consent_gate",
|
||||
"configured_proxy_url",
|
||||
]
|
||||
|
||||
@ -2,5 +2,6 @@
|
||||
<a href="/profile/{{ _user['username'] }}" class="user-avatar-link">
|
||||
<img src="{{ avatar_url('multiavatar', avatar_seed(_user), _size) }}" class="avatar-img avatar-{{ _size_class }}" alt="{{ _user['username'] }}" loading="lazy">
|
||||
{% include "_presence_dot.html" %}
|
||||
{% include "_award_badge.html" %}
|
||||
</a>
|
||||
{% endif %}
|
||||
3
devplacepy/templates/_award_badge.html
Normal file
3
devplacepy/templates/_award_badge.html
Normal file
@ -0,0 +1,3 @@
|
||||
{% if award_is_prominent(_user) %}
|
||||
<img src="/awards/{{ _user['last_award_slug'] }}/64" class="award-badge" alt="Latest award" loading="lazy">
|
||||
{% endif %}
|
||||
31
devplacepy/templates/_awards_gallery.html
Normal file
31
devplacepy/templates/_awards_gallery.html
Normal file
@ -0,0 +1,31 @@
|
||||
<div class="awards-grid">
|
||||
{% for award in awards %}
|
||||
<article class="award-tile" id="award-{{ award['slug'] }}">
|
||||
<img src="{{ award['image_url'] }}" alt="{{ award['description'] }}" title="{{ award['description'] }}" class="award-tile-image" loading="lazy">
|
||||
<div class="award-tile-desc rendered-content">{{ render_content(award['description']) }}</div>
|
||||
<div class="award-tile-meta">
|
||||
{% if award.get('giver') %}
|
||||
<a href="/profile/{{ award['giver']['username'] }}" class="award-tile-giver">
|
||||
<img src="{{ avatar_url('multiavatar', avatar_seed(award['giver']), 24) }}" class="avatar-img avatar-xs" alt="{{ award['giver']['username'] }}" loading="lazy">
|
||||
<span>@{{ award['giver']['username'] }}</span>
|
||||
</a>
|
||||
{% endif %}
|
||||
{% if award.get('generated_at') %}
|
||||
<span class="award-tile-date">{{ award_date(award['generated_at']) }}</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if viewer_is_admin %}
|
||||
<form method="POST" action="/admin/awards/{{ award['uid'] }}/revoke" class="award-revoke-form">
|
||||
<button type="submit"
|
||||
class="award-revoke-btn"
|
||||
data-confirm="Revoke this award from @{{ profile_user['username'] }}? It will disappear from their profile, media gallery, and avatar badge."
|
||||
data-confirm-danger
|
||||
data-confirm-title="Revoke award"
|
||||
aria-label="Revoke award">Revoke</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
</article>
|
||||
{% else %}
|
||||
<div class="empty-state">No awards yet.</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
@ -35,6 +35,9 @@
|
||||
<a href="/admin/ai-usage" class="sidebar-link {% if admin_section == 'ai-usage' %}active{% endif %}">
|
||||
<span class="sidebar-icon">📊</span> AI usage
|
||||
</a>
|
||||
<a href="/admin/statistics" class="sidebar-link {% if admin_section == 'statistics' %}active{% endif %}">
|
||||
<span class="sidebar-icon">📈</span> Statistics
|
||||
</a>
|
||||
<a href="/admin/audit-log" class="sidebar-link {% if admin_section == 'audit-log' %}active{% endif %}">
|
||||
<span class="sidebar-icon">📋</span> Audit Log
|
||||
</a>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<h2>Gateway routing</h2>
|
||||
<span class="admin-count" id="gw-count" role="status" aria-live="polite"></span>
|
||||
</div>
|
||||
<p class="gw-intro">Map any requested model name onto a provider and target model, each with its own pricing economy and an optional vision model for image to text merging. Unmapped requests fall through to the default upstream unchanged.</p>
|
||||
<p class="gw-intro">Map any requested model name onto a provider and target model, each with its own pricing economy and an optional vision model for image to text merging. Image routes use a flat per-image price. Unmapped requests fall through to the default upstream unchanged.</p>
|
||||
|
||||
<section class="gw-section">
|
||||
<div class="gw-section-head">
|
||||
@ -40,12 +40,12 @@
|
||||
<div class="gw-section-head">
|
||||
<h3>Model routes</h3>
|
||||
</div>
|
||||
<p class="gw-section-hint">Each source model maps to a provider and target model with its own economy (USD per 1M tokens). Chat uses cache-hit, cache-miss and output prices; embeddings use the input price; a vision model adds input and output pricing for the image description merge.</p>
|
||||
<p class="gw-section-hint">Each source model maps to a provider and target model with its own economy. Chat uses cache-hit, cache-miss and output prices (USD per 1M tokens); embeddings use the input price; image routes use the input price as a flat USD per image; a vision model adds input and output pricing for the image description merge.</p>
|
||||
<div class="admin-table-wrap">
|
||||
<table class="admin-table">
|
||||
<caption class="sr-only">Model routes</caption>
|
||||
<thead>
|
||||
<tr><th scope="col">Source</th><th scope="col">Provider</th><th scope="col">Target</th><th scope="col">Kind</th><th scope="col">Vision</th><th scope="col" class="gw-actions">Actions</th></tr>
|
||||
<tr><th scope="col">Source</th><th scope="col">Provider</th><th scope="col">Target</th><th scope="col">Kind</th><th scope="col">Vision</th><th scope="col">Economy</th><th scope="col" class="gw-actions">Actions</th></tr>
|
||||
</thead>
|
||||
<tbody id="gw-models"></tbody>
|
||||
</table>
|
||||
@ -55,15 +55,25 @@
|
||||
<div class="gw-field"><label for="gw-model-source">Source model (requested)</label><input type="text" id="gw-model-source" name="source_model" placeholder="gpt-4o" required aria-required="true"></div>
|
||||
<div class="gw-field"><label for="gw-model-provider">Provider</label><select id="gw-model-provider" name="provider" data-provider-select></select></div>
|
||||
<div class="gw-field"><label for="gw-model-target">Target model (upstream)</label><input type="text" id="gw-model-target" name="target_model" placeholder="openai/gpt-4o" required aria-required="true"></div>
|
||||
<div class="gw-field"><label for="gw-model-kind">Kind</label><select id="gw-model-kind" name="kind"><option value="chat">chat</option><option value="embed">embed</option></select></div>
|
||||
<div class="gw-field"><label for="gw-model-vision-provider">Vision provider</label><select id="gw-model-vision-provider" name="vision_provider" data-provider-select></select></div>
|
||||
<div class="gw-field"><label for="gw-model-vision-model">Vision model</label><input type="text" id="gw-model-vision-model" name="vision_model" placeholder="(optional) google/gemma-3-12b-it"></div>
|
||||
<div class="gw-field"><label for="gw-model-kind">Kind</label><select id="gw-model-kind" name="kind"><option value="chat">chat</option><option value="embed">embed</option><option value="image">image</option></select></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-vision-provider">Vision provider</label><select id="gw-model-vision-provider" name="vision_provider" data-provider-select></select></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-vision-model">Vision model</label><input type="text" id="gw-model-vision-model" name="vision_model" placeholder="(optional) google/gemma-3-12b-it"></div>
|
||||
<div class="gw-field"><label for="gw-model-context">Context window</label><input type="number" id="gw-model-context" name="context_window" min="0" value="0"></div>
|
||||
<div class="gw-field"><label for="gw-model-active">Active</label><select id="gw-model-active" name="is_active"><option value="1">Yes</option><option value="0">No</option></select></div>
|
||||
<div class="gw-field"><label for="gw-model-price-cache-hit">Price cache-hit / 1M ($)</label><input type="number" id="gw-model-price-cache-hit" name="price_cache_hit_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field"><label for="gw-model-price-cache-miss">Price cache-miss / 1M ($)</label><input type="number" id="gw-model-price-cache-miss" name="price_cache_miss_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field"><label for="gw-model-price-output">Price output / 1M ($)</label><input type="number" id="gw-model-price-output" name="price_output_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field"><label for="gw-model-price-input">Price input / 1M ($) (embed/vision)</label><input type="number" id="gw-model-price-input" name="price_input_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-price-cache-hit">Price cache-hit / 1M ($)</label><input type="number" id="gw-model-price-cache-hit" name="price_cache_hit_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-price-cache-miss">Price cache-miss / 1M ($)</label><input type="number" id="gw-model-price-cache-miss" name="price_cache_miss_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-price-output">Price output / 1M ($)</label><input type="number" id="gw-model-price-output" name="price_output_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<div class="gw-field" data-kind-field="embed chat"><label for="gw-model-price-input" id="gw-model-price-input-label">Price input / 1M ($) (embed/vision)</label><input type="number" id="gw-model-price-input" name="price_input_per_m" min="0" step="0.0001" value="0"></div>
|
||||
<p class="gw-form-title gw-form-subtitle" data-kind-field="chat embed">Tiered / off-peak pricing (optional)</p>
|
||||
<p class="gw-section-hint" data-kind-field="chat embed">Some providers charge a different rate once a request crosses a context-length threshold, or discount a fixed time-of-day window. Leave blank/zero to keep the flat rates above at all times.</p>
|
||||
<div class="gw-field" data-kind-field="chat embed"><label for="gw-model-tier-threshold">Tier-2 threshold (input tokens)</label><input type="number" id="gw-model-tier-threshold" name="context_tier_threshold_tokens" min="0" value="0" title="0 disables tiered pricing"></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-price-cache-hit-tier2">Tier-2 price cache-hit / 1M ($)</label><input type="number" id="gw-model-price-cache-hit-tier2" name="price_cache_hit_per_m_tier2" min="0" step="0.0001" placeholder="same as tier 1"></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-price-cache-miss-tier2">Tier-2 price cache-miss / 1M ($)</label><input type="number" id="gw-model-price-cache-miss-tier2" name="price_cache_miss_per_m_tier2" min="0" step="0.0001" placeholder="same as tier 1"></div>
|
||||
<div class="gw-field" data-kind-field="chat"><label for="gw-model-price-output-tier2">Tier-2 price output / 1M ($)</label><input type="number" id="gw-model-price-output-tier2" name="price_output_per_m_tier2" min="0" step="0.0001" placeholder="same as tier 1"></div>
|
||||
<div class="gw-field" data-kind-field="chat embed"><label for="gw-model-price-input-tier2">Tier-2 price input / 1M ($) (embed/vision)</label><input type="number" id="gw-model-price-input-tier2" name="price_input_per_m_tier2" min="0" step="0.0001" placeholder="same as tier 1"></div>
|
||||
<div class="gw-field" data-kind-field="chat embed image"><label for="gw-model-off-peak-start">Off-peak start (UTC)</label><input type="time" id="gw-model-off-peak-start" name="off_peak_start" title="Leave blank to disable off-peak discount"></div>
|
||||
<div class="gw-field" data-kind-field="chat embed image"><label for="gw-model-off-peak-end">Off-peak end (UTC)</label><input type="time" id="gw-model-off-peak-end" name="off_peak_end" title="Leave blank to disable off-peak discount"></div>
|
||||
<div class="gw-field" data-kind-field="chat embed image"><label for="gw-model-off-peak-discount">Off-peak discount (%)</label><input type="number" id="gw-model-off-peak-discount" name="off_peak_discount_pct" min="0" max="100" step="0.1" value="0"></div>
|
||||
<div class="gw-form-actions"><button type="submit" class="admin-btn admin-btn-primary">Save model route</button></div>
|
||||
</form>
|
||||
</section>
|
||||
|
||||
@ -113,6 +113,12 @@
|
||||
<small class="hint-text">Cookie lifetime when "remember me" is checked at login.</small>
|
||||
</div>
|
||||
|
||||
<div class="admin-field">
|
||||
<label for="outbound_proxy_url">Outbound Proxy URL</label>
|
||||
<input type="text" id="outbound_proxy_url" name="outbound_proxy_url" value="{{ settings.get('outbound_proxy_url', '') }}" maxlength="500" placeholder="http://user:pass@host:port" class="admin-settings-select-wide">
|
||||
<small class="hint-text">Routes every outbound stealth-client request (Devii fetch, DeepSearch, news, attachments, and more) through this proxy. Leave empty to connect directly. Needed only when a destination blocks this server's own IP/network (not its browser fingerprint) - most sites never need this.</small>
|
||||
</div>
|
||||
|
||||
<h3 class="admin-settings-group-title admin-settings-group-title-spaced">Custom Code</h3>
|
||||
|
||||
<div class="admin-field">
|
||||
|
||||
58
devplacepy/templates/admin_statistics.html
Normal file
58
devplacepy/templates/admin_statistics.html
Normal file
@ -0,0 +1,58 @@
|
||||
{% extends "admin_base.html" %}
|
||||
{% block extra_head %}
|
||||
{{ super() }}
|
||||
<link rel="stylesheet" href="{{ static_url('/static/css/services.css') }}">
|
||||
<link rel="stylesheet" href="{{ static_url('/static/css/statistics.css') }}">
|
||||
{% endblock %}
|
||||
{% block admin_content %}
|
||||
<div class="admin-toolbar statistics-toolbar">
|
||||
<h2>Statistics</h2>
|
||||
<div class="statistics-controls">
|
||||
<div class="statistics-controls-primary">
|
||||
<label for="statistics-window">Window
|
||||
<select id="statistics-window">
|
||||
<option value="24" {% if window_hours == 24 %}selected{% endif %}>Last 24 hours</option>
|
||||
<option value="168" {% if window_hours == 168 %}selected{% endif %}>Last 7 days</option>
|
||||
<option value="720" {% if window_hours == 720 %}selected{% endif %}>Last 30 days</option>
|
||||
<option value="2160" {% if window_hours == 2160 %}selected{% endif %}>Last 90 days</option>
|
||||
<option value="0" {% if window_hours == 0 %}selected{% endif %}>All time</option>
|
||||
</select>
|
||||
</label>
|
||||
<label class="statistics-compare-label">
|
||||
<input type="checkbox" id="statistics-compare" checked>
|
||||
<span>Compare previous period</span>
|
||||
</label>
|
||||
</div>
|
||||
<span class="statistics-generated" data-meta="generated" role="status" aria-live="polite">-</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<nav class="admin-tabs" data-overflow-tabs aria-label="Statistics categories">
|
||||
<div class="overflow-tabs-strip">
|
||||
{% for tab in tabs %}
|
||||
<a href="/admin/statistics?tab={{ tab['key'] }}&hours={{ window_hours }}" class="admin-tab {% if tab['active'] %}active{% endif %}" data-tab="{{ tab['key'] }}" data-menu-icon="{{ tab['icon'] }}" data-menu-label="{{ tab['label'] }}">
|
||||
<span class="icon">{{ tab['icon'] }}</span> {{ tab['label'] }}
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
<button type="button" class="admin-tab overflow-tabs-more" aria-haspopup="menu" aria-label="More tabs" hidden><span class="icon">⋯</span> More</button>
|
||||
</nav>
|
||||
|
||||
<div id="statistics-root" class="statistics" role="region" aria-label="Platform statistics" aria-live="polite">
|
||||
<p class="admin-empty" data-empty>Loading statistics...</p>
|
||||
</div>
|
||||
|
||||
<script type="application/json" id="statistics-initial">{{ initial | tojson }}</script>
|
||||
{% endblock %}
|
||||
{% block extra_js %}
|
||||
<script src="{{ static_url('/static/vendor/chartjs/chart.umd.min.js') }}" defer></script>
|
||||
<script type="module" src="{{ static_url('/static/js/StatisticsDashboard.js') }}"></script>
|
||||
<script type="module">
|
||||
new window.StatisticsDashboard({
|
||||
rootId: "statistics-root",
|
||||
initialId: "statistics-initial",
|
||||
activeTab: {{ active_tab | tojson }},
|
||||
windowHours: {{ window_hours }},
|
||||
}).start();
|
||||
</script>
|
||||
{% endblock %}
|
||||
@ -5,12 +5,15 @@
|
||||
<span class="admin-count">{{ pagination.total }} soft-deleted</span>
|
||||
</div>
|
||||
|
||||
<nav class="admin-tabs" aria-label="Trash categories">
|
||||
{% for tab in tables %}
|
||||
<a href="/admin/trash?table={{ tab['key'] }}" class="admin-tab {% if tab['active'] %}active{% endif %}">
|
||||
{{ tab['label'] }} <span class="admin-tab-count">{{ tab['count'] }}</span>
|
||||
</a>
|
||||
{% endfor %}
|
||||
<nav class="admin-tabs" data-overflow-tabs aria-label="Trash categories">
|
||||
<div class="overflow-tabs-strip">
|
||||
{% for tab in tables %}
|
||||
<a href="/admin/trash?table={{ tab['key'] }}" class="admin-tab {% if tab['active'] %}active{% endif %}" data-menu-icon="{{ tab['icon'] }}" data-menu-label="{{ tab['label'] }} ({{ tab['count'] }})">
|
||||
<span class="icon">{{ tab['icon'] }}</span> {{ tab['label'] }} <span class="admin-tab-count">{{ tab['count'] }}</span>
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
<button type="button" class="admin-tab overflow-tabs-more" aria-haspopup="menu" aria-label="More tabs" hidden><span class="icon">⋯</span> More</button>
|
||||
</nav>
|
||||
|
||||
<table class="admin-table">
|
||||
|
||||
@ -106,7 +106,7 @@
|
||||
{% endif %}
|
||||
<div class="topnav-user-dropdown">
|
||||
<button type="button" class="topnav-user" aria-label="User menu" aria-expanded="false" aria-controls="user-menu">
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(user), 32) }}" class="avatar-img avatar-sm" alt="{{ user['username'] }}" loading="lazy">{% set _user = user %}{% include "_presence_dot.html" %}</span>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(user), 32) }}" class="avatar-img avatar-sm" alt="{{ user['username'] }}" loading="lazy">{% set _user = user %}{% include "_presence_dot.html" %}{% include "_award_badge.html" %}</span>
|
||||
<div class="topnav-user-info">
|
||||
<span class="topnav-user-name">{{ user['username'] }}</span>
|
||||
<span class="topnav-user-level">Level {{ user.get('level', 1) }}</span>
|
||||
@ -164,7 +164,7 @@
|
||||
{% endif %}
|
||||
<div class="topnav-mobile-divider"></div>
|
||||
<div class="topnav-mobile-user">
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(user), 32) }}" class="avatar-img avatar-sm" alt="{{ user['username'] }}" loading="lazy">{% set _user = user %}{% include "_presence_dot.html" %}</span>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(user), 32) }}" class="avatar-img avatar-sm" alt="{{ user['username'] }}" loading="lazy">{% set _user = user %}{% include "_presence_dot.html" %}{% include "_award_badge.html" %}</span>
|
||||
<div class="topnav-mobile-user-info">
|
||||
<span>{{ user['username'] }}</span>
|
||||
<span class="topnav-mobile-user-level">Level {{ user.get('level', 1) }}</span>
|
||||
|
||||
26
devplacepy/templates/docs/awards.html
Normal file
26
devplacepy/templates/docs/awards.html
Normal file
@ -0,0 +1,26 @@
|
||||
<div class="docs-content" data-render>
|
||||
# Profile awards
|
||||
|
||||
Members can give one award per cooldown window to another member on their profile. The giver writes a short message (up to 125 characters); the server enqueues durable image generation and returns immediately. When the job finishes, the receiver gets a notification and the award appears on their profile gallery, a prominent banner, and a small badge on their avatar for a configurable display window.
|
||||
|
||||
Guests can browse awards read-only on profiles. Muting a user suppresses award notifications only; it does not block giving.
|
||||
|
||||
## Giving an award
|
||||
|
||||
On another member's profile, use **Give Award** when you are not on cooldown. Submit a short message in the modal. The giver's API key pays for gateway image generation. You cannot award yourself, and block rules match direct messages.
|
||||
|
||||
## Viewing awards
|
||||
|
||||
- **Awards tab** on a profile lists published awards newest first (first tab when the user has at least one award).
|
||||
- **Prominent banner** shows the latest award for a limited time after publish.
|
||||
- **Avatar badge** appears bottom-left on avatars while the latest award is prominent.
|
||||
- Images are served at `/awards/{slug}/{size}` with sizes `512`, `256`, or `64`.
|
||||
|
||||
## Admin revoke
|
||||
|
||||
Administrators can revoke a published award from the gallery or prominent banner. Revocation soft-deletes the award and attachments and recomputes receiver stats. Restored rows from **Admin - Trash** bring the award back.
|
||||
|
||||
## API
|
||||
|
||||
See [Profiles & Social Graph](/docs/profiles.html) for `POST /profile/{username}/award` and the awards tab on `GET /profile/{username}?tab=awards`. Image redirects are documented on the same page. Admin revoke is under [Admin API](/docs/admin.html).
|
||||
</div>
|
||||
@ -4,7 +4,7 @@
|
||||
|
||||
DevPlace notifies you when something involves you: a comment on your post, a reply to your comment, a
|
||||
mention, an upvote on your work, a new follower, a direct message, a badge, a level-up, or an update
|
||||
on an issue you filed. You decide which reach you, and how.
|
||||
on an issue you filed, or someone gives you an award on your profile. You decide which reach you, and how.
|
||||
|
||||
Each notification type is delivered on three independent channels:
|
||||
|
||||
@ -45,6 +45,7 @@ immediately - there is no separate save button.
|
||||
| Issue tracker | there is an update on an issue report you filed |
|
||||
| Reminders | a reminder or scheduled task you asked Devii to run fires |
|
||||
| Farm raids | someone steals a ready build from your Code Farm |
|
||||
| Awards | someone gives you an award on your profile |
|
||||
|
||||
## Defaults
|
||||
|
||||
|
||||
@ -14,10 +14,11 @@ It forwards chat-completions and other `/v1/*` calls to a configured upstream (D
|
||||
```
|
||||
POST /openai/v1/chat/completions
|
||||
POST /openai/v1/embeddings
|
||||
POST /openai/v1/images/generations
|
||||
GET|POST|PUT|DELETE|PATCH /openai/v1/{path} passthrough
|
||||
```
|
||||
|
||||
Chat requests are augmented (vision), forwarded, retried, priced, and ledgered. Embeddings forward to the configured embeddings upstream. Other paths pass through to the upstream.
|
||||
Chat requests are augmented (vision), forwarded, retried, priced, and ledgered. Embeddings forward to the configured embeddings upstream. Image generation forwards to the configured images upstream. Other paths pass through to the upstream.
|
||||
|
||||
## Authentication and attribution
|
||||
|
||||
@ -37,9 +38,11 @@ Chat requests are augmented (vision), forwarded, retried, priced, and ledgered.
|
||||
|
||||
**Vision:** `gateway_vision_enabled` (default on), `gateway_vision_url` (OpenRouter by default), `gateway_vision_model` (`google/gemma-3-12b-it`), `gateway_vision_key` (auto-migrated from `OPENROUTER_API_KEY`), `gateway_vision_cache_size` (LRU entries, default 256, 0 disables).
|
||||
|
||||
**Images:** `gateway_image_enabled` (default on), `gateway_image_url` (OpenRouter by default), `gateway_image_model` (`black-forest-labs/flux-1.1-pro`), `gateway_image_key` (falls back to `gateway_api_key` / `OPENROUTER_API_KEY`).
|
||||
|
||||
**Access:** `gateway_require_auth` (default on), `gateway_allow_admins`, `gateway_allow_users` (both default on), `gateway_access_key` (secret, write-only), `gateway_internal_key` (auto-generated on boot; clear and restart to rotate).
|
||||
|
||||
**Pricing (USD per 1M tokens):** `gateway_price_cache_hit_per_m` (0.0028), `gateway_price_cache_miss_per_m` (0.14), `gateway_price_output_per_m` (0.28), `gateway_vision_price_input_per_m` (0.0), `gateway_vision_price_output_per_m` (0.0). Used only when the upstream returns no native cost.
|
||||
**Pricing:** `gateway_price_cache_hit_per_m` (0.0028), `gateway_price_cache_miss_per_m` (0.14), `gateway_price_output_per_m` (0.28), `gateway_vision_price_input_per_m` (0.0), `gateway_vision_price_output_per_m` (0.0), `gateway_embed_price_input_per_m` (0.01), `gateway_image_price_per_call` (0.04). Token rates are USD per 1M tokens; the image price is a flat USD per generated image. Used only when the upstream returns no native cost.
|
||||
|
||||
**Reliability:** `gateway_max_retries` (default 2, 0 to 10), `gateway_retry_backoff_ms` (250, linear multiplied by attempt number), `gateway_circuit_threshold` (consecutive failures before opening, default 5, 0 disables), `gateway_circuit_cooldown_seconds` (default 30).
|
||||
|
||||
@ -49,20 +52,23 @@ Chat requests are augmented (vision), forwarded, retried, priced, and ledgered.
|
||||
|
||||
The single upstream above is the implicit `default` provider. On top of it, an administrator can register additional named **providers** and map any number of requested model names onto them, so one gateway can front many models across many backends. Routing is managed on the **Gateway** page (`/admin/gateway`) and stored in two tables (`gateway_providers`, `gateway_models`); it is admin configuration, not user content.
|
||||
|
||||
- A **provider** is a named upstream: `name`, a chat-completions `base_url`, and an `api_key`. The embeddings URL is derived from the base URL by swapping `/chat/completions` for `/embeddings`.
|
||||
- A **model route** maps a `source_model` (what a client requests) to a `provider` (blank uses the default upstream) and a `target_model` (sent upstream), with a `kind` of `chat` or `embed`, an optional `context_window`, an optional vision provider and model, and its own per-model **economy** (USD per 1M tokens): `price_cache_hit_per_m`, `price_cache_miss_per_m`, `price_output_per_m`, and `price_input_per_m` (the input price covers embeddings and the vision description).
|
||||
- A **provider** is a named upstream: `name`, a chat-completions `base_url`, and an `api_key`. The embeddings URL is derived from the base URL by swapping `/chat/completions` for `/embeddings`; the images URL swaps `/chat/completions` for `/images/generations`.
|
||||
- A **model route** maps a `source_model` (what a client requests) to a `provider` (blank uses the default upstream) and a `target_model` (sent upstream), with a `kind` of `chat`, `embed`, or `image`, an optional `context_window`, an optional vision provider and model, and its own per-model **economy**: `price_cache_hit_per_m`, `price_cache_miss_per_m`, `price_output_per_m`, and `price_input_per_m` (USD per 1M tokens for chat/embed/vision; for image routes the input price is a flat USD per generated image).
|
||||
- A route can optionally add two further, independent pricing dimensions on top of that flat economy, for providers that charge more than one rate for the same token type: **tiered (context-length) pricing** - `context_tier_threshold_tokens` (0 disables it) plus a tier-2 override for any of the four rates above (`price_cache_hit_per_m_tier2`, `price_cache_miss_per_m_tier2`, `price_output_per_m_tier2`, `price_input_per_m_tier2`); once a request's input tokens exceed the threshold, the tier-2 rate is used for any component that has one set, while a component left blank keeps its tier-1 rate at any size - and **off-peak discounting** - `off_peak_start_minute`/`off_peak_end_minute` (UTC minutes since midnight; both must be set together, and an end before the start wraps past midnight) plus `off_peak_discount_pct`, a percentage knocked off whichever rate (tier-1 or tier-2) is currently active. Both dimensions default to off, so an unconfigured route's pricing is unaffected.
|
||||
|
||||
When a request's model matches an active route of the right kind, the gateway forwards to that route's provider and target model and meters the call against that route's economy. A route with a vision model turns on the **text-and-vision merge** for that route, so a text-only model can answer about images. When no route matches, the request falls through to the default upstream unchanged, so `molodetz`, `molodetz~embed`, and every existing client keep behaving exactly as before. Resolution is a per-request overlay onto the service config, so all of the behavior below (authentication, system-message composition, vision augmentation, retries, the circuit breaker, the ledger, and the `X-Gateway-*` headers) applies identically to routed requests.
|
||||
When a request's model matches an active route of the right kind, the gateway forwards to that route's provider and target model and meters the call against that route's economy (tiered/off-peak included, if configured). A route with a vision model turns on the **text-and-vision merge** for that route, so a text-only model can answer about images. When no route matches, the request falls through to the default upstream unchanged, so `molodetz`, `molodetz~embed`, `molodetz-img-small`, and every existing client keep behaving exactly as before - and the service-level pricing fields above never gain the tiered/off-peak dimensions, since only a model route can enable them. Resolution is a per-request overlay onto the service config, so all of the behavior below (authentication, system-message composition, vision augmentation, retries, the circuit breaker, the ledger, and the `X-Gateway-*` headers, whose format is unchanged by tiered/off-peak pricing) applies identically to routed requests.
|
||||
|
||||
Routes and providers are also managed conversationally by an administrator through Devii (`gateway_providers`, `gateway_models` to read; `gateway_provider_set`, `gateway_provider_delete`, `gateway_model_set`, `gateway_model_delete` to manage, with deletes confirmation-gated).
|
||||
|
||||
DeepSeek's real billing (verified against its published API pricing) is exactly the flat cache-hit/cache-miss/output shape above, with no context-length tier or off-peak window on its current models - two ready-made routes, `deepseek-v4-flash` (`$0.0028` / `$0.14` / `$0.28` per 1M tokens, 1M-token context) and `deepseek-v4-pro` (`$0.003625` / `$0.435` / `$0.87` per 1M tokens, 1M-token context), are seeded automatically (idempotently, never overwriting a customized route) so either model can be requested by name with correct pricing regardless of what the single `gateway_model` default above is set to.
|
||||
|
||||
## Cost computation
|
||||
|
||||
If the upstream response carries a numeric `cost` field, that value is used verbatim and split proportionally into input and output (the row is flagged `native_cost`). Otherwise cost is computed from the per-1M pricing: for chat, `cache_hit_tokens` and `cache_miss_tokens` are priced separately on input plus `completion_tokens` on output; for vision, input and output use the vision rates. When the requested model matches a model route, that route's per-model prices are used instead of the service-level pricing fields, so each model carries its own economy. Prompt-cache hit rate directly drives cost: at the default rates cache hits are roughly fifty times cheaper than misses.
|
||||
If the upstream response carries a numeric `cost` field, that value is used verbatim and split proportionally into input and output (the row is flagged `native_cost`). Otherwise cost is computed from the per-1M pricing: for chat, `cache_hit_tokens` and `cache_miss_tokens` are priced separately on input plus `completion_tokens` on output; for vision, input and output use the vision rates. When the requested model matches a model route, that route's per-model prices are used instead of the service-level pricing fields, so each model carries its own economy - including, when configured, the tiered and off-peak dimensions described above (tier selection first, then the off-peak discount, then the same input/output split). Prompt-cache hit rate directly drives cost: at the default rates cache hits are roughly fifty times cheaper than misses.
|
||||
|
||||
## Response headers
|
||||
|
||||
Every gateway response (chat, embeddings, and passthrough; success and error) carries `X-Gateway-*` headers built from the same computed ledger row, so any caller can read the token usage and dollar cost of its own call straight from the response without querying the admin analytics. `GatewayUsageLedger.record()` returns the computed row and each handler maps it through `usage.usage_response_headers()` onto the response:
|
||||
Every gateway response (chat, embeddings, images, and passthrough; success and error) carries `X-Gateway-*` headers built from the same computed ledger row, so any caller can read the token usage and dollar cost of its own call straight from the response without querying the admin analytics. `GatewayUsageLedger.record()` returns the computed row and each handler maps it through `usage.usage_response_headers()` onto the response:
|
||||
|
||||
`X-Gateway-Model`, `X-Gateway-Backend`, `X-Gateway-Prompt-Tokens`, `X-Gateway-Completion-Tokens`, `X-Gateway-Total-Tokens`, `X-Gateway-Cache-Hit-Tokens`, `X-Gateway-Cache-Miss-Tokens`, `X-Gateway-Reasoning-Tokens`, `X-Gateway-Cost-USD`, `X-Gateway-Input-Cost-USD`, `X-Gateway-Output-Cost-USD`, `X-Gateway-Cost-Native` (`1` when the dollar cost is the upstream's own value, `0` when computed from the per-1M pricing), `X-Gateway-Tokens-Per-Second`, `X-Gateway-Upstream-Latency-Ms`, `X-Gateway-Total-Latency-Ms` (full end-to-end gateway time), `X-Gateway-Gateway-Overhead-Ms` (gateway processing minus upstream and queue wait), `X-Gateway-Queue-Wait-Ms` (time waiting on the concurrency semaphore), `X-Gateway-Connect-Ms` (upstream connection establishment), and, when the model's window is known, `X-Gateway-Context-Window` and `X-Gateway-Context-Utilization`. The denied embeddings-disabled path makes no upstream call and returns no usage headers.
|
||||
|
||||
@ -86,7 +92,7 @@ Before forwarding a chat request, image blocks are described by the vision model
|
||||
|
||||
## Data it offers
|
||||
|
||||
**Live metrics** (in `describe()`, at `GET /admin/services/data`): runtime counters (`requests`, `errors`, `in_flight`, `peak_in_flight`, `vision_calls`, `last_status`, `last_latency_ms`, `pool`, `circuit_open`) and a 24h summary (`requests`, `success_pct`, `error_pct`, `cost_hour`, `cost_24h`, `tokens_24h`, `avg_latency_ms`, `avg_tps`, `peak_concurrency`, `top_model`, `top_caller`).
|
||||
**Live metrics** (in `describe()`, at `GET /admin/services/data`): runtime counters (`requests`, `errors`, `in_flight`, `peak_in_flight`, `vision_calls`, `embed_calls`, `image_calls`, `last_status`, `last_latency_ms`, `pool`, `circuit_open`) and a 24h summary (`requests`, `success_pct`, `error_pct`, `cost_hour`, `cost_24h`, `tokens_24h`, `avg_latency_ms`, `avg_tps`, `peak_concurrency`, `top_model`, `top_caller`).
|
||||
|
||||
**Ledger** `gateway_usage_ledger`: one row per upstream call with owner, backend, requested and actual model, status, success, error category, the full latency breakdown (`upstream_latency_ms`, `gateway_overhead_ms`, `queue_wait_ms`, `connect_ms`, `total_latency_ms`), token counts (prompt, completion, cache hit and miss, reasoning, total), `tokens_per_second`, context window and utilization, the cost fields (`cost_usd`, `input_cost_usd`, `output_cost_usd`, `native_cost`), request shape (`stream_requested`, `temperature`, `top_p`, `max_tokens`, `has_tools`), and reliability flags (`retries_attempted`, `retry_succeeded`, `circuit_open`). Pruned to `gateway_usage_retention_hours`.
|
||||
|
||||
|
||||
@ -66,7 +66,7 @@
|
||||
<div class="online-users-list" data-online-users-list data-empty-text="No one online right now">
|
||||
{% for ou in online_users %}
|
||||
<a href="/profile/{{ ou['username'] }}" class="online-user" title="{{ ou['username'] }}">
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(ou), 32) }}" class="avatar-img avatar-sm" alt="{{ ou['username'] }}" loading="lazy"><span class="presence-dot online" aria-hidden="true"></span></span>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(ou), 32) }}" class="avatar-img avatar-sm" alt="{{ ou['username'] }}" loading="lazy"><span class="presence-dot online" aria-hidden="true"></span>{% set _user = ou %}{% include "_award_badge.html" %}</span>
|
||||
</a>
|
||||
{% else %}
|
||||
<span class="online-empty">No one online right now</span>
|
||||
|
||||
@ -15,7 +15,7 @@
|
||||
<div class="messages-conversations" role="list" aria-label="Conversation list">
|
||||
{% for conv in conversations %}
|
||||
<a href="/messages?with_uid={{ conv.other_user['uid'] }}" class="conversation-item {% if current_conversation == conv.other_user['uid'] %}active{% endif %}" data-conv-uid="{{ conv.other_user['uid'] }}" role="listitem"{% if current_conversation == conv.other_user['uid'] %} aria-current="true"{% endif %}>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(conv.other_user), 32) }}" class="avatar-img avatar-sm" alt="{{ conv.other_user['username'] }}" loading="lazy">{% set _user = conv.other_user %}{% include "_presence_dot.html" %}</span>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(conv.other_user), 32) }}" class="avatar-img avatar-sm" alt="{{ conv.other_user['username'] }}" loading="lazy">{% set _user = conv.other_user %}{% include "_presence_dot.html" %}{% include "_award_badge.html" %}</span>
|
||||
<div class="conversation-info">
|
||||
<div class="conversation-name">{{ conv.other_user['username'] }}</div>
|
||||
<div class="conversation-preview">{{ content_preview(conv.last_message, 60) }}</div>
|
||||
|
||||
@ -4,13 +4,14 @@
|
||||
<link rel="stylesheet" href="{{ static_url('/static/css/profile.css') }}">
|
||||
<link rel="stylesheet" href="{{ static_url('/static/css/projects.css') }}">
|
||||
<link rel="stylesheet" href="{{ static_url('/static/css/gists.css') }}">
|
||||
<link rel="stylesheet" href="{{ static_url('/static/css/awards.css') }}">
|
||||
{% endblock %}
|
||||
{% block content %}
|
||||
<div class="profile-layout">
|
||||
<aside class="profile-sidebar">
|
||||
<div class="profile-card">
|
||||
<div class="profile-avatar-wrap">
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(profile_user), 80) }}" class="avatar-img avatar-lg" alt="{{ profile_user['username'] }}" id="profile-avatar-preview" loading="lazy">{% set _user = profile_user %}{% include "_presence_dot.html" %}</span>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(profile_user), 80) }}" class="avatar-img avatar-lg" alt="{{ profile_user['username'] }}" id="profile-avatar-preview" loading="lazy">{% set _user = profile_user %}{% include "_presence_dot.html" %}{% include "_award_badge.html" %}</span>
|
||||
{% if is_owner or viewer_is_admin %}
|
||||
<button type="button" class="btn btn-sm avatar-regen-btn" data-regenerate-avatar data-username="{{ profile_user['username'] }}">Regenerate avatar</button>
|
||||
{% endif %}
|
||||
@ -89,6 +90,9 @@
|
||||
<button type="submit" class="btn btn-secondary btn-sm full-width" data-confirm="Mute {{ profile_user['username'] }}? They will no longer create notifications for you." data-confirm-danger{{ guest_disabled(user) }}><span class="icon">🔇</span>Mute</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
{% if can_give_award %}
|
||||
<button type="button" class="btn btn-primary btn-sm full-width" data-award-give data-username="{{ profile_user['username'] }}" data-modal="award-give-modal"{{ guest_disabled(user) }}><span class="icon">🏆</span>Give Award</button>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
@ -473,18 +477,51 @@
|
||||
{% endfor %}
|
||||
</details>
|
||||
|
||||
<div class="profile-tabs" id="profile-tabs" data-profile-tabs>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=posts#profile-tabs" class="profile-tab {% if current_tab == 'posts' %}active{% endif %}" data-menu-icon="📝" data-menu-label="Posts"><span class="icon">📝</span> Posts</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=projects#profile-tabs" class="profile-tab {% if current_tab == 'projects' %}active{% endif %}" data-menu-icon="🚀" data-menu-label="Projects"><span class="icon">🚀</span> Projects</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=gists#profile-tabs" class="profile-tab {% if current_tab == 'gists' %}active{% endif %}" data-menu-icon="📝" data-menu-label="Gists"><span class="icon">📝</span> Gists</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=media#profile-tabs" class="profile-tab {% if current_tab == 'media' %}active{% endif %}" data-menu-icon="🖼️" data-menu-label="Media"><span class="icon">🖼️</span> Media</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=activity#profile-tabs" class="profile-tab {% if current_tab == 'activity' %}active{% endif %}" data-menu-icon="📊" data-menu-label="Activity"><span class="icon">📊</span> Activity</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=followers#profile-tabs" class="profile-tab {% if current_tab == 'followers' %}active{% endif %}" data-menu-icon="👥" data-menu-label="Followers ({{ followers_count }})"><span class="icon">👥</span> Followers ({{ followers_count }})</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=following#profile-tabs" class="profile-tab {% if current_tab == 'following' %}active{% endif %}" data-menu-icon="👤" data-menu-label="Following ({{ following_count }})"><span class="icon">👤</span> Following ({{ following_count }})</a>
|
||||
{% if is_owner or viewer_is_admin %}
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=notifications#profile-tabs" class="profile-tab {% if current_tab == 'notifications' %}active{% endif %}" data-menu-icon="🔔" data-menu-label="Notifications"><span class="icon">🔔</span> Notifications</a>
|
||||
{% if prominent_award %}
|
||||
<section class="award-prominent" aria-label="Latest award">
|
||||
<img src="{{ prominent_award['image_url'] }}" alt="{{ prominent_award['description'] }}" title="{{ prominent_award['description'] }}" class="award-prominent-image" loading="lazy">
|
||||
<div class="award-prominent-desc rendered-content">{{ render_content(prominent_award['description']) }}</div>
|
||||
<div class="award-prominent-meta">
|
||||
{% if prominent_award.get('giver') %}
|
||||
<a href="/profile/{{ prominent_award['giver']['username'] }}" class="award-prominent-giver">
|
||||
<img src="{{ avatar_url('multiavatar', avatar_seed(prominent_award['giver']), 24) }}" class="avatar-img avatar-xs" alt="{{ prominent_award['giver']['username'] }}" loading="lazy">
|
||||
<span>Given by @{{ prominent_award['giver']['username'] }}</span>
|
||||
</a>
|
||||
{% endif %}
|
||||
{% if prominent_award.get('generated_at') %}
|
||||
<span>{{ award_date(prominent_award['generated_at']) }}</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if viewer_is_admin %}
|
||||
<form method="POST" action="/admin/awards/{{ prominent_award['uid'] }}/revoke" class="award-revoke-form">
|
||||
<button type="submit"
|
||||
class="award-revoke-btn btn btn-secondary btn-sm"
|
||||
data-confirm="Revoke this award from @{{ profile_user['username'] }}? It will disappear from their profile, media gallery, and avatar badge."
|
||||
data-confirm-danger
|
||||
data-confirm-title="Revoke award"
|
||||
aria-label="Revoke award">Revoke award</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
<button type="button" class="profile-tab profile-tabs-more" aria-haspopup="menu" aria-label="More tabs" hidden><span class="icon">⋯</span> More</button>
|
||||
</section>
|
||||
{% endif %}
|
||||
|
||||
<div class="profile-tabs" id="profile-tabs" data-overflow-tabs>
|
||||
<div class="overflow-tabs-strip">
|
||||
{% if awards_count > 0 %}
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=awards#profile-tabs" class="profile-tab {% if current_tab == 'awards' %}active{% endif %}" data-menu-icon="🏆" data-menu-label="Awards ({{ awards_count }})"><span class="icon">🏆</span> Awards ({{ awards_count }})</a>
|
||||
{% endif %}
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=posts#profile-tabs" class="profile-tab {% if current_tab == 'posts' %}active{% endif %}" data-menu-icon="📝" data-menu-label="Posts"><span class="icon">📝</span> Posts</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=projects#profile-tabs" class="profile-tab {% if current_tab == 'projects' %}active{% endif %}" data-menu-icon="🚀" data-menu-label="Projects"><span class="icon">🚀</span> Projects</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=gists#profile-tabs" class="profile-tab {% if current_tab == 'gists' %}active{% endif %}" data-menu-icon="📝" data-menu-label="Gists"><span class="icon">📝</span> Gists</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=media#profile-tabs" class="profile-tab {% if current_tab == 'media' %}active{% endif %}" data-menu-icon="🖼️" data-menu-label="Media"><span class="icon">🖼️</span> Media</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=activity#profile-tabs" class="profile-tab {% if current_tab == 'activity' %}active{% endif %}" data-menu-icon="📊" data-menu-label="Activity"><span class="icon">📊</span> Activity</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=followers#profile-tabs" class="profile-tab {% if current_tab == 'followers' %}active{% endif %}" data-menu-icon="👥" data-menu-label="Followers ({{ followers_count }})"><span class="icon">👥</span> Followers ({{ followers_count }})</a>
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=following#profile-tabs" class="profile-tab {% if current_tab == 'following' %}active{% endif %}" data-menu-icon="👤" data-menu-label="Following ({{ following_count }})"><span class="icon">👤</span> Following ({{ following_count }})</a>
|
||||
{% if is_owner or viewer_is_admin %}
|
||||
<a href="/profile/{{ profile_user['username'] }}?tab=notifications#profile-tabs" class="profile-tab {% if current_tab == 'notifications' %}active{% endif %}" data-menu-icon="🔔" data-menu-label="Notifications"><span class="icon">🔔</span> Notifications</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
<button type="button" class="profile-tab overflow-tabs-more" aria-haspopup="menu" aria-label="More tabs" hidden><span class="icon">⋯</span> More</button>
|
||||
</div>
|
||||
|
||||
<div class="profile-posts">
|
||||
@ -544,6 +581,11 @@
|
||||
{% set pagination = media_pagination %}
|
||||
{% set pagination_query = "tab=media&" %}
|
||||
{% include "_pagination.html" %}
|
||||
{% elif current_tab == 'awards' %}
|
||||
{% include "_awards_gallery.html" %}
|
||||
{% set pagination = awards_pagination %}
|
||||
{% set pagination_query = "tab=awards&" %}
|
||||
{% include "_pagination.html" %}
|
||||
{% elif current_tab == 'followers' or current_tab == 'following' %}
|
||||
{% if people %}
|
||||
<div class="profile-panel">
|
||||
@ -551,7 +593,7 @@
|
||||
{% for person in people %}
|
||||
<div class="follow-row">
|
||||
<a href="/profile/{{ person['username'] }}" class="follow-user">
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(person), 40) }}" class="avatar-img avatar-md" alt="{{ person['username'] }}" loading="lazy">{% set _user = person %}{% include "_presence_dot.html" %}</span>
|
||||
<span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(person), 40) }}" class="avatar-img avatar-md" alt="{{ person['username'] }}" loading="lazy">{% set _user = person %}{% include "_presence_dot.html" %}{% include "_award_badge.html" %}</span>
|
||||
<div class="follow-user-info">
|
||||
<span class="follow-user-name">{{ person['username'] }}</span>
|
||||
{% if person['bio'] %}<span class="follow-user-bio rendered-title">{{ render_title(person['bio']) }}</span>{% endif %}
|
||||
@ -663,13 +705,24 @@
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% if can_give_award %}
|
||||
{% from "_macros.html" import modal %}
|
||||
{% call modal('award-give-modal', 'Give award') %}
|
||||
<form id="award-give-form">
|
||||
<div class="auth-field auth-field-gap">
|
||||
<label for="award-description">Message for @{{ profile_user['username'] }}</label>
|
||||
<textarea id="award-description" name="description" maxlength="125" rows="3" required placeholder="What did they do well?"></textarea>
|
||||
<small class="award-char-count" data-award-char-count>0/125</small>
|
||||
</div>
|
||||
<button type="submit" class="btn btn-primary full-width">Submit award</button>
|
||||
</form>
|
||||
{% endcall %}
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
{% block extra_js %}
|
||||
<script type="module">
|
||||
import { ProfileTabs } from "{{ static_url('/static/js/ProfileTabs.js') }}";
|
||||
const tabs = document.querySelector("[data-profile-tabs]");
|
||||
if (tabs) {
|
||||
new ProfileTabs(tabs);
|
||||
}
|
||||
import { AwardGiver } from "{{ static_url('/static/js/AwardGiver.js') }}";
|
||||
new AwardGiver();
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
@ -155,6 +155,31 @@ def local_dt(dt_str: str, mode: str = "datetime") -> Markup:
|
||||
|
||||
templates.env.globals["local_dt"] = local_dt
|
||||
templates.env.globals["dt_ago"] = lambda dt_str: local_dt(dt_str, "ago")
|
||||
|
||||
|
||||
def award_date(dt_str: str) -> Markup:
|
||||
if not dt_str:
|
||||
return Markup("")
|
||||
iso = _normalize_iso(dt_str)
|
||||
if iso is None:
|
||||
return Markup(escape(dt_str))
|
||||
try:
|
||||
parsed = datetime.fromisoformat(iso)
|
||||
if parsed.tzinfo is None:
|
||||
parsed = parsed.replace(tzinfo=timezone.utc)
|
||||
fallback = parsed.strftime("%A, %d-%m-%Y")
|
||||
except (ValueError, TypeError):
|
||||
fallback = _format_date(dt_str)
|
||||
return Markup(
|
||||
f'<time datetime="{escape(iso)}" data-dt data-dt-mode="award">'
|
||||
f"{escape(fallback)}</time>"
|
||||
)
|
||||
|
||||
|
||||
from devplacepy.database.awards import award_is_prominent
|
||||
|
||||
templates.env.globals["award_date"] = award_date
|
||||
templates.env.globals["award_is_prominent"] = award_is_prominent
|
||||
templates.env.globals["badge_info"] = get_badge
|
||||
templates.env.globals["TOPICS"] = TOPICS
|
||||
templates.env.globals["REACTION_EMOJI"] = REACTION_EMOJI
|
||||
|
||||
@ -65,6 +65,76 @@ def test_model_route_create_and_economy(seeded_db):
|
||||
assert admin.delete(f"{BASE_URL}/admin/gateway/models/{source}").status_code == 404
|
||||
|
||||
|
||||
def test_model_route_tiered_and_off_peak_pricing_round_trip(seeded_db):
|
||||
admin = admin_session(seeded_db)
|
||||
source = _unique_gateway("tiered")
|
||||
|
||||
created = admin.post(
|
||||
f"{BASE_URL}/admin/gateway/models",
|
||||
json={
|
||||
"source_model": source,
|
||||
"target_model": "vendor/tiered",
|
||||
"kind": "chat",
|
||||
"price_cache_hit_per_m": 0.0028,
|
||||
"price_cache_miss_per_m": 0.14,
|
||||
"price_output_per_m": 0.28,
|
||||
"context_tier_threshold_tokens": 128000,
|
||||
"price_output_per_m_tier2": 0.56,
|
||||
"off_peak_start_minute": 990,
|
||||
"off_peak_end_minute": 30,
|
||||
"off_peak_discount_pct": 25.0,
|
||||
},
|
||||
)
|
||||
assert created.status_code == 200, created.text[:300]
|
||||
model = created.json()["model"]
|
||||
assert model["context_tier_threshold_tokens"] == 128000
|
||||
assert model["price_output_per_m_tier2"] == 0.56
|
||||
assert model["price_cache_hit_per_m_tier2"] is None
|
||||
assert model["off_peak_start_minute"] == 990
|
||||
assert model["off_peak_end_minute"] == 30
|
||||
assert model["off_peak_discount_pct"] == 25.0
|
||||
|
||||
listed = admin.get(f"{BASE_URL}/admin/gateway/models").json()
|
||||
row = next(m for m in listed["models"] if m["source_model"] == source)
|
||||
assert row["context_tier_threshold_tokens"] == 128000
|
||||
assert row["price_output_per_m_tier2"] == 0.56
|
||||
assert row["off_peak_start_minute"] == 990
|
||||
assert row["off_peak_end_minute"] == 30
|
||||
assert row["off_peak_discount_pct"] == 25.0
|
||||
|
||||
deleted = admin.delete(f"{BASE_URL}/admin/gateway/models/{source}")
|
||||
assert deleted.status_code == 200
|
||||
|
||||
|
||||
def test_model_route_off_peak_requires_both_start_and_end(seeded_db):
|
||||
admin = admin_session(seeded_db)
|
||||
source = _unique_gateway("badwindow")
|
||||
|
||||
only_start = admin.post(
|
||||
f"{BASE_URL}/admin/gateway/models",
|
||||
json={
|
||||
"source_model": source,
|
||||
"target_model": "vendor/w",
|
||||
"kind": "chat",
|
||||
"off_peak_start_minute": 60,
|
||||
},
|
||||
)
|
||||
assert only_start.status_code == 400
|
||||
assert only_start.json()["ok"] is False
|
||||
|
||||
only_end = admin.post(
|
||||
f"{BASE_URL}/admin/gateway/models",
|
||||
json={
|
||||
"source_model": source,
|
||||
"target_model": "vendor/w",
|
||||
"kind": "chat",
|
||||
"off_peak_end_minute": 120,
|
||||
},
|
||||
)
|
||||
assert only_end.status_code == 400
|
||||
assert only_end.json()["ok"] is False
|
||||
|
||||
|
||||
def test_model_route_validation(seeded_db):
|
||||
admin = admin_session(seeded_db)
|
||||
missing_target = admin.post(
|
||||
|
||||
@ -103,3 +103,44 @@ def test_restore_revives_post_and_records_audit(seeded_db):
|
||||
entry = _audit_find(admin, "admin.trash.restore", post["uid"])
|
||||
assert entry is not None
|
||||
assert entry["result"] == "success"
|
||||
|
||||
|
||||
def test_restore_revoked_award_recomputes_stats(seeded_db):
|
||||
from datetime import datetime, timezone
|
||||
from devplacepy.database.awards import revoke_award
|
||||
from devplacepy.utils import generate_uid, make_combined_slug
|
||||
|
||||
admin = _admin_trash(seeded_db)
|
||||
bob = get_table("users").find_one(username="bob_test")
|
||||
alice = get_table("users").find_one(username="alice_test")
|
||||
uid = generate_uid()
|
||||
slug = make_combined_slug("Trash restore", uid)
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
get_table("awards").insert(
|
||||
{
|
||||
"uid": uid,
|
||||
"slug": slug,
|
||||
"description": "Trash restore",
|
||||
"giver_uid": alice["uid"],
|
||||
"receiver_uid": bob["uid"],
|
||||
"attachment_uid_512": "",
|
||||
"attachment_uid_256": "",
|
||||
"attachment_uid_64": "",
|
||||
"generated_at": now,
|
||||
"created_at": now,
|
||||
"job_uid": "",
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
revoke_award(uid, alice["uid"])
|
||||
r = admin.post(
|
||||
f"{BASE_URL}/admin/trash/awards/{uid}/restore",
|
||||
headers=JSON_trash,
|
||||
allow_redirects=False,
|
||||
)
|
||||
assert r.status_code == 200, r.text[:300]
|
||||
row = get_table("users").find_one(uid=bob["uid"])
|
||||
assert row.get("award_count") == 1
|
||||
assert row.get("last_award_uid") == uid
|
||||
assert requests.get(f"{BASE_URL}/awards/{slug}/256").status_code in (302, 404)
|
||||
|
||||
69
tests/e2e/admin/awards.py
Normal file
69
tests/e2e/admin/awards.py
Normal file
@ -0,0 +1,69 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from playwright.sync_api import expect
|
||||
from tests.conftest import BASE_URL
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.utils import generate_uid, make_combined_slug
|
||||
|
||||
AWARD_REVOKE = ".award-revoke-btn"
|
||||
|
||||
|
||||
def _confirm(page):
|
||||
page.locator(".dialog-overlay.visible .dialog-confirm").click()
|
||||
|
||||
|
||||
def _seed_published(receiver_username, giver_username):
|
||||
receiver = get_table("users").find_one(username=receiver_username)
|
||||
giver = get_table("users").find_one(username=giver_username)
|
||||
uid = generate_uid()
|
||||
slug = make_combined_slug("Revoke me", uid)
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
get_table("awards").insert(
|
||||
{
|
||||
"uid": uid,
|
||||
"slug": slug,
|
||||
"description": "Revoke me",
|
||||
"giver_uid": giver["uid"],
|
||||
"receiver_uid": receiver["uid"],
|
||||
"attachment_uid_512": "",
|
||||
"attachment_uid_256": "",
|
||||
"attachment_uid_64": "",
|
||||
"generated_at": now,
|
||||
"created_at": now,
|
||||
"job_uid": "",
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
get_table("users").update(
|
||||
{
|
||||
"uid": receiver["uid"],
|
||||
"award_count": 1,
|
||||
"last_award_at": now,
|
||||
"last_award_slug": slug,
|
||||
"last_award_uid": uid,
|
||||
},
|
||||
["uid"],
|
||||
)
|
||||
return slug
|
||||
|
||||
|
||||
def test_admin_sees_revoke_button(alice, bob):
|
||||
_seed_published("bob_test", "alice_test")
|
||||
admin_page, _ = alice
|
||||
member_page, _ = bob
|
||||
admin_page.goto(f"{BASE_URL}/profile/bob_test?tab=awards", wait_until="domcontentloaded")
|
||||
expect(admin_page.locator(AWARD_REVOKE)).to_be_visible()
|
||||
member_page.goto(f"{BASE_URL}/profile/bob_test?tab=awards", wait_until="domcontentloaded")
|
||||
expect(member_page.locator(AWARD_REVOKE)).to_have_count(0)
|
||||
|
||||
|
||||
def test_admin_revoke_confirmation_removes_tile(alice):
|
||||
page, _ = alice
|
||||
slug = _seed_published("bob_test", "alice_test")
|
||||
page.goto(f"{BASE_URL}/profile/bob_test?tab=awards", wait_until="domcontentloaded")
|
||||
page.locator(AWARD_REVOKE).first.click()
|
||||
_confirm(page)
|
||||
page.wait_for_url("**/profile/bob_test**", wait_until="domcontentloaded")
|
||||
expect(page.locator(f"#award-{slug}")).to_have_count(0)
|
||||
@ -80,6 +80,32 @@ def test_global_default_applies_only_to_untouched_users(alice, bob):
|
||||
assert get_notification_default("badge", "push") is True
|
||||
|
||||
|
||||
def test_award_type_listed_in_prefs(bob):
|
||||
_, user = bob
|
||||
uid = _user_notification_prefs(user["username"])["uid"]
|
||||
reset_notification_prefs(uid)
|
||||
prefs = get_notification_prefs(uid)
|
||||
assert any(pref["key"] == "award" for pref in prefs)
|
||||
|
||||
|
||||
def test_create_notification_honors_award_pref(bob):
|
||||
_, user = bob
|
||||
uid = _user_notification_prefs(user["username"])["uid"]
|
||||
notifications = get_table("notifications")
|
||||
reset_notification_prefs(uid)
|
||||
try:
|
||||
set_notification_pref(uid, "award", "in_app", False)
|
||||
suppressed = f"award suppressed {generate_uid()}"
|
||||
create_notification(uid, "award", suppressed, uid, "/profile/bob_test?tab=awards")
|
||||
assert notifications.find_one(user_uid=uid, message=suppressed) is None
|
||||
set_notification_pref(uid, "award", "in_app", True)
|
||||
delivered = f"award delivered {generate_uid()}"
|
||||
create_notification(uid, "award", delivered, uid, "/profile/bob_test?tab=awards")
|
||||
assert notifications.find_one(user_uid=uid, message=delivered) is not None
|
||||
finally:
|
||||
reset_notification_prefs(uid)
|
||||
|
||||
|
||||
def test_create_notification_honors_in_app_pref(bob):
|
||||
_, user = bob
|
||||
uid = _user_notification_prefs(user["username"])["uid"]
|
||||
|
||||
109
tests/e2e/profile/awards.py
Normal file
109
tests/e2e/profile/awards.py
Normal file
@ -0,0 +1,109 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import re
|
||||
from datetime import datetime, timezone
|
||||
from playwright.sync_api import expect
|
||||
from tests.conftest import BASE_URL
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.utils import generate_uid, make_combined_slug
|
||||
|
||||
GIVE_AWARD_BTN = "[data-award-give]"
|
||||
AWARD_MODAL = "#award-give-modal"
|
||||
AWARD_TEXTAREA = "#award-give-modal textarea[name='description']"
|
||||
AWARDS_TAB = "a.profile-tab[href*='tab=awards']"
|
||||
AWARD_BADGE = ".award-badge"
|
||||
|
||||
|
||||
def _seed_published(receiver_username, giver_username, description="E2E award"):
|
||||
receiver = get_table("users").find_one(username=receiver_username)
|
||||
giver = get_table("users").find_one(username=giver_username)
|
||||
uid = generate_uid()
|
||||
slug = make_combined_slug(description, uid)
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
get_table("awards").insert(
|
||||
{
|
||||
"uid": uid,
|
||||
"slug": slug,
|
||||
"description": description,
|
||||
"giver_uid": giver["uid"],
|
||||
"receiver_uid": receiver["uid"],
|
||||
"attachment_uid_512": "",
|
||||
"attachment_uid_256": "",
|
||||
"attachment_uid_64": "",
|
||||
"generated_at": now,
|
||||
"created_at": now,
|
||||
"job_uid": "",
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
get_table("users").update(
|
||||
{
|
||||
"uid": receiver["uid"],
|
||||
"award_count": 1,
|
||||
"last_award_at": now,
|
||||
"last_award_slug": slug,
|
||||
"last_award_uid": uid,
|
||||
},
|
||||
["uid"],
|
||||
)
|
||||
return slug
|
||||
|
||||
|
||||
def test_give_award_button_visible_on_other_profile(alice):
|
||||
page, user = alice
|
||||
page.goto(f"{BASE_URL}/profile/bob_test", wait_until="domcontentloaded")
|
||||
if page.is_visible(GIVE_AWARD_BTN):
|
||||
expect(page.locator(GIVE_AWARD_BTN)).to_be_visible()
|
||||
page.goto(f"{BASE_URL}/profile/{user['username']}", wait_until="domcontentloaded")
|
||||
expect(page.locator(GIVE_AWARD_BTN)).to_have_count(0)
|
||||
|
||||
|
||||
def test_award_modal_char_counter_and_submit_dialog(alice):
|
||||
page, _ = alice
|
||||
page.goto(f"{BASE_URL}/profile/bob_test", wait_until="domcontentloaded")
|
||||
if not page.is_visible(GIVE_AWARD_BTN):
|
||||
return
|
||||
page.click(GIVE_AWARD_BTN)
|
||||
expect(page.locator(AWARD_MODAL)).to_have_class(re.compile(r"\bvisible\b"))
|
||||
page.fill(AWARD_TEXTAREA, "x" * 125)
|
||||
expect(page.locator("[data-award-char-count]")).to_contain_text("125/125")
|
||||
page.locator(f"{AWARD_MODAL} button[type='submit']").click()
|
||||
page.locator(".dialog-overlay.visible").wait_for(state="visible")
|
||||
expect(page.locator(".dialog-overlay.visible")).to_contain_text("being created")
|
||||
|
||||
|
||||
def test_awards_tab_and_gallery_anchor(alice):
|
||||
page, _ = alice
|
||||
slug = _seed_published("bob_test", "alice_test", "Gallery tile")
|
||||
page.goto(f"{BASE_URL}/profile/bob_test?tab=awards", wait_until="domcontentloaded")
|
||||
expect(page.locator(AWARDS_TAB)).to_be_visible()
|
||||
expect(page.locator(f"#award-{slug}")).to_be_visible()
|
||||
|
||||
|
||||
def test_prominent_banner_visible(alice):
|
||||
page, _ = alice
|
||||
_seed_published("bob_test", "alice_test", "Prominent banner text")
|
||||
page.goto(f"{BASE_URL}/profile/bob_test", wait_until="domcontentloaded")
|
||||
expect(page.locator(".award-prominent")).to_be_visible()
|
||||
expect(page.locator(".award-prominent-desc")).to_contain_text("Prominent banner text")
|
||||
|
||||
|
||||
def test_guest_can_read_awards_tab(browser):
|
||||
page = browser.new_page()
|
||||
try:
|
||||
slug = _seed_published("bob_test", "alice_test", "Guest gallery")
|
||||
page.goto(f"{BASE_URL}/profile/bob_test?tab=awards", wait_until="domcontentloaded")
|
||||
expect(page.locator(".awards-grid")).to_be_visible()
|
||||
expect(page.locator(f"#award-{slug}")).to_be_visible()
|
||||
expect(page.locator(GIVE_AWARD_BTN)).to_have_count(0)
|
||||
finally:
|
||||
page.close()
|
||||
|
||||
|
||||
def test_avatar_badge_on_feed_when_prominent(alice):
|
||||
page, _ = alice
|
||||
_seed_published("bob_test", "alice_test", "Badge feed")
|
||||
page.goto(f"{BASE_URL}/feed", wait_until="domcontentloaded")
|
||||
if page.locator(".online-user").count() > 0:
|
||||
expect(page.locator(".online-user .award-badge").first).to_be_visible()
|
||||
@ -331,6 +331,140 @@ def test_embeddings_disabled_returns_503(local_db, monkeypatch):
|
||||
assert resp.status_code == 503
|
||||
|
||||
|
||||
class FakeImageClient_openai_gateway:
|
||||
def __init__(self, *a, **k):
|
||||
self.calls = []
|
||||
|
||||
def build_request(self, method, url, headers=None, json=None, content=None):
|
||||
return FakeRequest(method, url, json)
|
||||
|
||||
async def send(self, request):
|
||||
self.calls.append((request.url, request.json_body))
|
||||
body = request.json_body or {}
|
||||
return FakeResp_openai_gateway(
|
||||
payload={
|
||||
"created": 1,
|
||||
"model": body.get("model"),
|
||||
"data": [{"b64_json": "aGVsbG8="}],
|
||||
"usage": {"cost": 0.05},
|
||||
}
|
||||
)
|
||||
|
||||
async def aclose(self):
|
||||
pass
|
||||
|
||||
|
||||
def test_images_remaps_alias_to_configured_model(local_db, monkeypatch):
|
||||
monkeypatch.setattr(gwmod.httpx, "AsyncClient", FakeImageClient_openai_gateway)
|
||||
svc = GatewayService()
|
||||
cfg = svc.effective_config()
|
||||
cfg["gateway_force_model"] = False
|
||||
cfg["gateway_image_enabled"] = True
|
||||
cfg["gateway_image_model"] = "black-forest-labs/flux.2-pro"
|
||||
rt = svc.runtime()
|
||||
run_async(
|
||||
rt.handle_images(
|
||||
{
|
||||
"model": "molodetz-img-small",
|
||||
"prompt": "award emblem",
|
||||
"size": "512x512",
|
||||
},
|
||||
cfg,
|
||||
("guest", "test"),
|
||||
"test",
|
||||
)
|
||||
)
|
||||
assert rt._client.calls[-1][1]["model"] == "black-forest-labs/flux.2-pro"
|
||||
|
||||
|
||||
def test_image_route_overrides_upstream(local_db, monkeypatch):
|
||||
from devplacepy.services.openai_gateway import routing
|
||||
|
||||
monkeypatch.setattr(gwmod.httpx, "AsyncClient", FakeImageClient_openai_gateway)
|
||||
routing.provider_store.set(
|
||||
routing.ProviderIn(
|
||||
name="gwimg",
|
||||
base_url="https://routed.example/v1/chat/completions",
|
||||
api_key="img-key",
|
||||
)
|
||||
)
|
||||
routing.model_store.set(
|
||||
routing.ModelRouteIn(
|
||||
source_model="custom-img",
|
||||
provider="gwimg",
|
||||
target_model="vendor/flux-pro",
|
||||
kind="image",
|
||||
price_input_per_m=0.06,
|
||||
)
|
||||
)
|
||||
try:
|
||||
svc = GatewayService()
|
||||
cfg = svc.effective_config()
|
||||
rt = svc.runtime()
|
||||
run_async(
|
||||
rt.handle_images(
|
||||
{
|
||||
"model": "custom-img",
|
||||
"prompt": "trophy",
|
||||
"response_format": "b64_json",
|
||||
},
|
||||
cfg,
|
||||
("guest", "test"),
|
||||
"test",
|
||||
)
|
||||
)
|
||||
url, body = rt._client.calls[-1]
|
||||
assert str(url) == "https://routed.example/v1/images"
|
||||
assert body["model"] == "vendor/flux-pro"
|
||||
finally:
|
||||
routing.model_store.remove("custom-img")
|
||||
routing.provider_store.remove("gwimg")
|
||||
|
||||
|
||||
def test_images_success_records_ledger(local_db, monkeypatch):
|
||||
monkeypatch.setattr(gwmod.httpx, "AsyncClient", FakeImageClient_openai_gateway)
|
||||
svc = GatewayService()
|
||||
cfg = svc.effective_config()
|
||||
cfg["gateway_image_enabled"] = True
|
||||
cfg["gateway_image_model"] = "black-forest-labs/flux.2-pro"
|
||||
rt = svc.runtime()
|
||||
before = rt.image_calls
|
||||
resp = run_async(
|
||||
rt.handle_images(
|
||||
{"model": "molodetz-img-small", "prompt": "badge"},
|
||||
cfg,
|
||||
("guest", "img_ledger"),
|
||||
"test",
|
||||
)
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
payload = json.loads(bytes(resp.body).decode())
|
||||
assert payload["data"][0]["b64_json"] == "aGVsbG8="
|
||||
assert rt.image_calls == before + 1
|
||||
row = get_table("gateway_usage_ledger").find_one(owner_id="img_ledger")
|
||||
assert row is not None
|
||||
assert row["backend"] == "image"
|
||||
assert row["endpoint"] == "images/generations"
|
||||
assert row["requested_model"] == "molodetz-img-small"
|
||||
assert row["model"] == "black-forest-labs/flux.2-pro"
|
||||
assert row["success"] == 1
|
||||
assert float(row["cost_usd"]) == 0.05
|
||||
|
||||
|
||||
def test_images_disabled_returns_503(local_db, monkeypatch):
|
||||
monkeypatch.setattr(gwmod.httpx, "AsyncClient", FakeImageClient_openai_gateway)
|
||||
svc = GatewayService()
|
||||
cfg = svc.effective_config()
|
||||
cfg["gateway_image_enabled"] = False
|
||||
rt = svc.runtime()
|
||||
resp = run_async(
|
||||
rt.handle_images(
|
||||
{"prompt": "badge"}, cfg, ("guest", "test"), "test"
|
||||
)
|
||||
)
|
||||
assert resp.status_code == 503
|
||||
|
||||
|
||||
def test_compute_cost_embed_branch():
|
||||
from devplacepy.services.openai_gateway.usage import (
|
||||
Pricing,
|
||||
|
||||
@ -1,5 +1,8 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from devplacepy.services.openai_gateway import routing as r
|
||||
from devplacepy.services.openai_gateway.usage import pricing_from_cfg
|
||||
|
||||
@ -14,6 +17,7 @@ def _cleanup(providers, models):
|
||||
def test_no_routes_pass_through(local_db):
|
||||
assert r.chat_overlay("ghost-model-xyz", {}) is None
|
||||
assert r.embed_overlay("ghost-embed-xyz", {}) is None
|
||||
assert r.image_overlay("ghost-image-xyz", {}) is None
|
||||
|
||||
|
||||
def test_chat_route_overlay_and_economy(local_db):
|
||||
@ -80,6 +84,37 @@ def test_vision_merge_overlay(local_db):
|
||||
_cleanup(["utvis"], ["ut-vision"])
|
||||
|
||||
|
||||
def test_image_route_and_kind_isolation(local_db):
|
||||
r.provider_store.set(
|
||||
r.ProviderIn(
|
||||
name="utimg",
|
||||
base_url="https://img.example/v1/chat/completions",
|
||||
api_key="ik",
|
||||
)
|
||||
)
|
||||
r.model_store.set(
|
||||
r.ModelRouteIn(
|
||||
source_model="ut-image",
|
||||
provider="utimg",
|
||||
target_model="vendor/flux",
|
||||
kind="image",
|
||||
price_input_per_m=0.05,
|
||||
)
|
||||
)
|
||||
try:
|
||||
overlay = r.image_overlay("ut-image", {})
|
||||
assert overlay["gateway_image_model"] == "vendor/flux"
|
||||
assert overlay["gateway_image_url"] == "https://img.example/v1/images"
|
||||
assert overlay["gateway_image_key"] == "ik"
|
||||
assert overlay["gateway_image_price_per_call"] == 0.05
|
||||
assert r.chat_overlay("ut-image", {}) is None
|
||||
assert r.embed_overlay("ut-image", {}) is None
|
||||
pricing = pricing_from_cfg(overlay)
|
||||
assert pricing.image_per_call == 0.05
|
||||
finally:
|
||||
_cleanup(["utimg"], ["ut-image"])
|
||||
|
||||
|
||||
def test_embed_route_and_kind_isolation(local_db):
|
||||
r.provider_store.set(
|
||||
r.ProviderIn(
|
||||
@ -140,3 +175,108 @@ def test_blank_provider_uses_default_upstream(local_db):
|
||||
assert overlay["gateway_price_output_per_m"] == 3.0
|
||||
finally:
|
||||
_cleanup([], ["ut-default"])
|
||||
|
||||
|
||||
def test_tier2_and_off_peak_fields_propagate_through_overlay(local_db):
|
||||
r.model_store.set(
|
||||
r.ModelRouteIn(
|
||||
source_model="ut-tiered",
|
||||
target_model="vendor/tiered",
|
||||
kind="chat",
|
||||
price_cache_hit_per_m=0.0028,
|
||||
price_cache_miss_per_m=0.14,
|
||||
price_output_per_m=0.28,
|
||||
context_tier_threshold_tokens=128_000,
|
||||
price_output_per_m_tier2=0.56,
|
||||
off_peak_start_minute=990,
|
||||
off_peak_end_minute=30,
|
||||
off_peak_discount_pct=25.0,
|
||||
)
|
||||
)
|
||||
try:
|
||||
overlay = r.chat_overlay("ut-tiered", {})
|
||||
assert overlay["gateway_context_tier_threshold_tokens"] == 128_000
|
||||
assert overlay["gateway_price_output_per_m_tier2"] == 0.56
|
||||
assert overlay["gateway_price_cache_hit_per_m_tier2"] is None
|
||||
assert overlay["gateway_off_peak_start_minute"] == 990
|
||||
assert overlay["gateway_off_peak_end_minute"] == 30
|
||||
assert overlay["gateway_off_peak_discount_pct"] == 25.0
|
||||
pricing = pricing_from_cfg(overlay)
|
||||
assert pricing.context_tier_threshold_tokens == 128_000
|
||||
assert pricing.chat_output_per_m_tier2 == 0.56
|
||||
assert pricing.chat_cache_hit_per_m_tier2 is None
|
||||
assert pricing.off_peak_start_minute == 990
|
||||
finally:
|
||||
_cleanup([], ["ut-tiered"])
|
||||
|
||||
|
||||
def test_unrouted_request_never_gains_tier_or_off_peak_keys(local_db):
|
||||
assert r.chat_overlay("ghost-model-untiered", {}) is None
|
||||
pricing = pricing_from_cfg({})
|
||||
assert pricing.context_tier_threshold_tokens == 0
|
||||
assert pricing.off_peak_start_minute is None
|
||||
assert pricing.off_peak_end_minute is None
|
||||
assert pricing.chat_output_per_m_tier2 is None
|
||||
|
||||
|
||||
def test_off_peak_window_requires_both_start_and_end():
|
||||
with pytest.raises(ValidationError):
|
||||
r.ModelRouteIn(
|
||||
source_model="ut-bad-window",
|
||||
target_model="vendor/x",
|
||||
off_peak_start_minute=60,
|
||||
)
|
||||
with pytest.raises(ValidationError):
|
||||
r.ModelRouteIn(
|
||||
source_model="ut-bad-window",
|
||||
target_model="vendor/x",
|
||||
off_peak_end_minute=120,
|
||||
)
|
||||
|
||||
|
||||
def test_seed_default_image_routes_is_idempotent(local_db):
|
||||
r.model_store.remove(r.IMAGE_SOURCE_MODEL)
|
||||
r.seed_default_image_routes()
|
||||
route = r.model_store.get(r.IMAGE_SOURCE_MODEL)
|
||||
assert route is not None
|
||||
assert route.kind == "image"
|
||||
assert route.target_model == r.FLUX_TARGET_MODEL
|
||||
r.seed_default_image_routes()
|
||||
assert r.model_store.get(r.IMAGE_SOURCE_MODEL).target_model == r.FLUX_TARGET_MODEL
|
||||
|
||||
|
||||
def test_seed_default_deepseek_routes_is_idempotent_and_preserves_customization(
|
||||
local_db,
|
||||
):
|
||||
r.seed_default_deepseek_routes()
|
||||
flash = r.model_store.get("deepseek-v4-flash")
|
||||
pro = r.model_store.get("deepseek-v4-pro")
|
||||
assert flash is not None and pro is not None
|
||||
assert flash.price_cache_hit_per_m == 0.0028
|
||||
assert flash.price_cache_miss_per_m == 0.14
|
||||
assert flash.price_output_per_m == 0.28
|
||||
assert flash.context_window == 1_048_576
|
||||
assert pro.price_cache_hit_per_m == 0.003625
|
||||
assert pro.price_cache_miss_per_m == 0.435
|
||||
assert pro.price_output_per_m == 0.87
|
||||
r.model_store.set(
|
||||
r.ModelRouteIn(
|
||||
source_model="deepseek-v4-pro",
|
||||
target_model="deepseek-v4-pro",
|
||||
price_output_per_m=1.23,
|
||||
)
|
||||
)
|
||||
try:
|
||||
r.seed_default_deepseek_routes()
|
||||
assert r.model_store.get("deepseek-v4-pro").price_output_per_m == 1.23
|
||||
finally:
|
||||
r.model_store.set(
|
||||
r.ModelRouteIn(
|
||||
source_model="deepseek-v4-pro",
|
||||
target_model="deepseek-v4-pro",
|
||||
price_cache_hit_per_m=0.003625,
|
||||
price_cache_miss_per_m=0.435,
|
||||
price_output_per_m=0.87,
|
||||
context_window=1_048_576,
|
||||
)
|
||||
)
|
||||
|
||||
@ -1,13 +1,28 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from dataclasses import replace
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from devplacepy.services.openai_gateway.usage import (
|
||||
USAGE_FIELDS,
|
||||
Pricing,
|
||||
accumulate_usage,
|
||||
compute_cost,
|
||||
new_usage_totals,
|
||||
normalize_usage,
|
||||
parse_usage_headers,
|
||||
usage_metric_cards,
|
||||
)
|
||||
|
||||
BASE_PRICING = Pricing(
|
||||
chat_cache_hit_per_m=0.0028,
|
||||
chat_cache_miss_per_m=0.14,
|
||||
chat_output_per_m=0.28,
|
||||
vision_input_per_m=0.0,
|
||||
vision_output_per_m=0.0,
|
||||
embed_input_per_m=0.01,
|
||||
)
|
||||
|
||||
GATEWAY_HEADERS = {
|
||||
"X-Gateway-Cost-USD": "0.00010000",
|
||||
"X-Gateway-Model": "molodetz",
|
||||
@ -89,3 +104,118 @@ def test_usage_metric_cards_labels_and_formatting():
|
||||
assert by_label["Total cost"] == "$0.0492"
|
||||
assert by_label["Avg cost/call"] == "$0.012300"
|
||||
assert by_label["Avg latency"] == "4200ms"
|
||||
|
||||
|
||||
def test_compute_cost_matches_deepseek_flat_rates_when_no_tier_configured():
|
||||
norm = normalize_usage(
|
||||
{
|
||||
"prompt_tokens": 2000,
|
||||
"completion_tokens": 100,
|
||||
"prompt_cache_hit_tokens": 500,
|
||||
"prompt_cache_miss_tokens": 1500,
|
||||
}
|
||||
)
|
||||
total, input_cost, output_cost, native = compute_cost({}, norm, BASE_PRICING, "chat")
|
||||
assert native is False
|
||||
assert abs(input_cost - (500 / 1e6 * 0.0028 + 1500 / 1e6 * 0.14)) < 1e-12
|
||||
assert abs(output_cost - (100 / 1e6 * 0.28)) < 1e-12
|
||||
assert abs(total - (input_cost + output_cost)) < 1e-12
|
||||
|
||||
|
||||
def test_compute_cost_switches_to_tier2_above_threshold():
|
||||
tiered = replace(
|
||||
BASE_PRICING,
|
||||
chat_cache_hit_per_m_tier2=0.005,
|
||||
chat_cache_miss_per_m_tier2=0.25,
|
||||
chat_output_per_m_tier2=0.5,
|
||||
context_tier_threshold_tokens=1000,
|
||||
)
|
||||
below = normalize_usage({"prompt_tokens": 900, "completion_tokens": 50})
|
||||
above = normalize_usage({"prompt_tokens": 1001, "completion_tokens": 50})
|
||||
_, below_input, below_output, _ = compute_cost({}, below, tiered, "chat")
|
||||
_, above_input, above_output, _ = compute_cost({}, above, tiered, "chat")
|
||||
assert abs(below_output - (50 / 1e6 * 0.28)) < 1e-12
|
||||
assert abs(above_output - (50 / 1e6 * 0.5)) < 1e-12
|
||||
assert below_input != above_input
|
||||
|
||||
|
||||
def test_compute_cost_tier2_leaves_unset_component_at_tier1():
|
||||
tiered = replace(
|
||||
BASE_PRICING,
|
||||
chat_output_per_m_tier2=0.5,
|
||||
context_tier_threshold_tokens=100,
|
||||
)
|
||||
norm = normalize_usage({"prompt_tokens": 200, "completion_tokens": 10})
|
||||
_, input_cost, output_cost, _ = compute_cost({}, norm, tiered, "chat")
|
||||
assert abs(output_cost - (10 / 1e6 * 0.5)) < 1e-12
|
||||
assert abs(input_cost - (200 / 1e6 * 0.14)) < 1e-12
|
||||
|
||||
|
||||
def test_compute_cost_applies_off_peak_discount():
|
||||
discounted = replace(
|
||||
BASE_PRICING,
|
||||
off_peak_start_minute=60,
|
||||
off_peak_end_minute=120,
|
||||
off_peak_discount_pct=50.0,
|
||||
)
|
||||
norm = normalize_usage({"prompt_tokens": 1000, "completion_tokens": 100})
|
||||
in_window = datetime(2026, 1, 1, 1, 30, tzinfo=timezone.utc)
|
||||
outside_window = datetime(2026, 1, 1, 10, 0, tzinfo=timezone.utc)
|
||||
total_in, _, _, _ = compute_cost({}, norm, discounted, "chat", now=in_window)
|
||||
total_out, _, _, _ = compute_cost({}, norm, discounted, "chat", now=outside_window)
|
||||
total_flat, _, _, _ = compute_cost({}, norm, BASE_PRICING, "chat")
|
||||
assert abs(total_out - total_flat) < 1e-12
|
||||
assert abs(total_in - total_flat / 2) < 1e-12
|
||||
|
||||
|
||||
def test_compute_cost_off_peak_wraps_past_midnight():
|
||||
wrapped = replace(
|
||||
BASE_PRICING,
|
||||
off_peak_start_minute=23 * 60,
|
||||
off_peak_end_minute=60,
|
||||
off_peak_discount_pct=100.0,
|
||||
)
|
||||
norm = normalize_usage({"prompt_tokens": 1000, "completion_tokens": 100})
|
||||
just_after_midnight = datetime(2026, 1, 1, 0, 30, tzinfo=timezone.utc)
|
||||
midday = datetime(2026, 1, 1, 12, 0, tzinfo=timezone.utc)
|
||||
total_wrapped, _, _, _ = compute_cost(
|
||||
{}, norm, wrapped, "chat", now=just_after_midnight
|
||||
)
|
||||
total_midday, _, _, _ = compute_cost({}, norm, wrapped, "chat", now=midday)
|
||||
assert total_wrapped == 0.0
|
||||
assert total_midday > 0.0
|
||||
|
||||
|
||||
def test_compute_cost_image_branch_flat_per_call():
|
||||
image_pricing = replace(BASE_PRICING, image_per_call=0.04)
|
||||
norm = normalize_usage({})
|
||||
total, input_cost, output_cost, native = compute_cost(
|
||||
{}, norm, image_pricing, "image"
|
||||
)
|
||||
assert native is False
|
||||
assert output_cost == 0.0
|
||||
assert abs(total - 0.04) < 1e-9
|
||||
assert abs(input_cost - 0.04) < 1e-9
|
||||
|
||||
|
||||
def test_compute_cost_image_native_cost_preferred():
|
||||
from devplacepy.services.openai_gateway.usage import extract_image_usage
|
||||
|
||||
image_pricing = replace(BASE_PRICING, image_per_call=0.04)
|
||||
norm = normalize_usage({})
|
||||
usage = extract_image_usage({"usage": {"cost": 0.055}})
|
||||
total, _, _, native = compute_cost(usage, norm, image_pricing, "image")
|
||||
assert native is True
|
||||
assert total == 0.055
|
||||
|
||||
|
||||
def test_compute_cost_native_upstream_cost_unaffected_by_tiering():
|
||||
tiered = replace(
|
||||
BASE_PRICING,
|
||||
chat_output_per_m_tier2=100.0,
|
||||
context_tier_threshold_tokens=1,
|
||||
)
|
||||
norm = normalize_usage({"prompt_tokens": 1000, "completion_tokens": 100})
|
||||
total, _, _, native = compute_cost({"cost": 0.05}, norm, tiered, "chat")
|
||||
assert native is True
|
||||
assert total == 0.05
|
||||
|
||||
@ -30,11 +30,11 @@ def test_curl_mode_forwards_headers_without_chrome_base():
|
||||
run_async(client.aclose())
|
||||
|
||||
|
||||
def test_chrome_base_headers_match_chrome_149():
|
||||
def test_chrome_base_headers_match_chrome_146():
|
||||
headers = stealth.chrome_base_headers()
|
||||
assert "Chrome/149.0.0.0" in headers["user-agent"]
|
||||
assert "Chrome/146.0.0.0" in headers["user-agent"]
|
||||
assert headers["sec-ch-ua"] == (
|
||||
'"Google Chrome";v="149", "Chromium";v="149", "Not)A;Brand";v="24"'
|
||||
'"Google Chrome";v="146", "Chromium";v="146", "Not)A;Brand";v="24"'
|
||||
)
|
||||
assert headers["accept-encoding"] == "gzip, deflate, br, zstd"
|
||||
|
||||
@ -45,6 +45,29 @@ def test_fallback_path_injects_chrome_headers(monkeypatch):
|
||||
assert isinstance(transport, httpx.AsyncHTTPTransport)
|
||||
client = stealth.stealth_async_client()
|
||||
assert client.headers.get("sec-ch-ua") is not None
|
||||
assert "Chrome/149.0.0.0" in client.headers.get("user-agent", "")
|
||||
assert "Chrome/146.0.0.0" in client.headers.get("user-agent", "")
|
||||
run_async(client.aclose())
|
||||
run_async(transport.aclose())
|
||||
|
||||
|
||||
def test_detect_consent_gate_extracts_accept_url():
|
||||
html = """
|
||||
<title>DPG Media Privacy Gate</title>
|
||||
<script>
|
||||
const callbackUrl = new URL(decodeURIComponent('https%3A%2F%2Fnu.nl%2Fprivacy-gate%2Faccept%3FredirectUri%3D%252F'))
|
||||
</script>
|
||||
<script src="https://myprivacy-static.dpgmedia.net/consent.js"></script>
|
||||
"""
|
||||
gate_url = stealth.detect_consent_gate(html)
|
||||
assert gate_url == "https://nu.nl/privacy-gate/accept?redirectUri=%2F"
|
||||
|
||||
|
||||
def test_detect_consent_gate_ignores_ordinary_pages():
|
||||
assert stealth.detect_consent_gate("<html><body>hello</body></html>") is None
|
||||
|
||||
|
||||
def test_proxy_threaded_into_curl_transport():
|
||||
transport = stealth.stealth_transport(proxy="http://127.0.0.1:8080")
|
||||
assert isinstance(transport, CurlTransport)
|
||||
assert transport._proxy == "http://127.0.0.1:8080"
|
||||
run_async(transport.aclose())
|
||||
|
||||
Loading…
Reference in New Issue
Block a user