Compare commits
8 Commits
typosaurus
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| 5079f40f46 | |||
|
|
d895de1b47 | ||
| 571a0485c5 | |||
| b185574760 | |||
| 3709f4fab9 | |||
| 1a87c392bd | |||
| ff49c8342a | |||
| 76d73ccaea |
@ -65,6 +65,7 @@ devplace devii tasks prune # disable every task whose owner may not
|
||||
devplace gateway quota list # list AI gateway quota rules and current 24h spend
|
||||
devplace gateway quota set --limit-usd N [--owner-kind K] [--owner-id ID] [--app-reference APP] [--label L] [--uid UID]
|
||||
devplace gateway quota delete <uid> # delete a quota rule
|
||||
devplace gateway quota reset [--owner-kind K] [--owner-id ID] [--app-reference APP] # clear the counted 24h spend (keeps the usage history)
|
||||
devplace zips prune # delete expired zip archives + job rows
|
||||
devplace zips clear # delete every zip archive + job row
|
||||
devplace forks prune # delete expired completed fork job rows (forked projects persist)
|
||||
@ -248,7 +249,7 @@ Users and guests inject their own CSS and JS, scoped to a page type or globally,
|
||||
|
||||
### Container manager, Devii assistant, AI gateway, async jobs, audit log
|
||||
|
||||
DevPlace ships a Docker-backed **Container manager** (admin-only, one shared `ppy:latest` image, security-hardened rootless workflow via a `sudo` superclone and `aptroot`), the **Devii** in-platform agentic assistant (WebSocket terminal, per-owner-channel sessions, scheduler/reminders, self-learning lessons, user-defined virtual tools, self-configured behavior, browser-automation client tools), the **AI gateway** (`/openai/v1/*`, single point of truth for every AI call, per-user cost attribution, provider/model routing overlay), **async job services** (`JobService` base pattern: zip, fork, SEO diagnostics, SEO metadata generation, DeepSearch, AI Usage Analyzer), and an admin-only append-only **audit log** (`record`/`record_system` entrypoints, `events.md` catalogue of 223 keys, never raises into the caller). These are among the largest subsystems in the codebase - read their dedicated nested `CLAUDE.md` files (see Subsystem map) before working in any of them; do not assume prior knowledge from this summary.
|
||||
DevPlace ships a Docker-backed **Container manager** (admin-only, one shared `ppy:latest` image, security-hardened rootless workflow via a `sudo` superclone and `aptroot`), the **Devii** in-platform agentic assistant (WebSocket terminal, per-owner-channel sessions, scheduler/reminders, self-learning lessons, user-defined virtual tools, self-configured behavior, browser-automation client tools), the **AI gateway** (`/openai/v1/*`, single point of truth for every AI call, per-user cost attribution, provider/model routing overlay), **async job services** (`JobService` base pattern: zip, fork, SEO diagnostics, SEO metadata generation, DeepSearch, AI Usage Analyzer), and an admin-only append-only **audit log** (`record`/`record_system` entrypoints, `events.md` catalogue of 288 keys, never raises into the caller). These are among the largest subsystems in the codebase - read their dedicated nested `CLAUDE.md` files (see Subsystem map) before working in any of them; do not assume prior knowledge from this summary.
|
||||
|
||||
### Telegram bot, email, devRant compatibility API, issue tracker
|
||||
|
||||
@ -346,6 +347,7 @@ Gitea Actions workflow at `.gitea/workflows/test.yaml` runs on every push/PR to
|
||||
Single-host Docker Compose (`docker-compose.yml`): an **app** container (Uvicorn, 2 workers) behind an **nginx** container. Full operator reference is the admin-only `Production` docs section (`templates/docs/production*.html`); conventions that must not regress:
|
||||
|
||||
- **Shared DB and files = same as dev.** The app container bind-mounts the host project root (`.:/app`) and runs as `${DEVPLACE_UID}:${DEVPLACE_GID}` (default `1000`), so it reads/writes the same `data/devplace.db`, `data/uploads/`, `data/devii_*.db`, `data/keys/` (VAPID), and `data/locks/devplace-services.lock` as `make dev`. No `DEVPLACE_DATABASE_URL` override - `config.py` resolves an absolute path under the project's `data/` dir. WAL + the `flock` on `devplace-services.lock` make concurrent dev/prod safe and keep a single background-services owner. SQLite is local-file, so prod and dev must be the **same host**.
|
||||
- **Code updates need no rebuild** (bind-mounted source); rebuild only on `pyproject.toml` dependency changes. `.env` is git-ignored; `.env.example` is the committed template. The app reads `DEVPLACE_DATABASE_URL`, never `DATABASE_URL`.
|
||||
- **Code updates need no rebuild** (bind-mounted source); rebuild only on `pyproject.toml` dependency changes. Use **`make docker-reload`** (`restart app` + `up -d --wait`) to pick up new source - a bare `make docker-up` does **not** restart an unchanged container, so the running uvicorn keeps serving the code it imported at boot. `.env` is git-ignored; `.env.example` is the committed template. The app reads `DEVPLACE_DATABASE_URL`, never `DATABASE_URL`.
|
||||
- **Dockerfile layer order is load-bearing for build time.** The dependency layer (`pip install ".[bots]"` + `playwright install --with-deps chromium`, ~3GB and ~2.5 min) must depend on `pyproject.toml` **only**. `COPY devplacepy/` therefore comes *after* it, and the project itself is installed last with `pip install --no-deps --force-reinstall .`. hatchling needs the package directory to exist to build a wheel, so the dependency layer creates a placeholder `devplacepy/__init__.py` that the real `COPY` overwrites (verified: site-packages holds the full 39-entry package, not the stub). Copying source before the install inverts this and makes **every source edit** reinstall every dependency and re-download Chromium - measured 2m36s per source-only rebuild versus 7.4s with the correct order. Never move `COPY devplacepy/` above the dependency layer.
|
||||
- **nginx parity rules** (`nginx/nginx.conf.template`, rendered by `start.sh` via `envsubst` with an allow-list that preserves `$http_upgrade`): `/static/uploads/` must re-apply `nosniff` + a `Content-Disposition` via the `map $uri $upload_disposition` block (`inline` for safe image/video/audio extensions, `attachment` otherwise), mirroring `UploadStaticFiles.INLINE_MEDIA_EXTENSIONS` - an XSS control nginx would otherwise bypass, and the inline branch is what lets video play in production; `/devii/ws` needs the `map $http_upgrade $connection_upgrade` block and `Upgrade`/`Connection` headers or the Devii terminal cannot connect (every new WebSocket route needs its own nginx upgrade location - the catch-all `location /` strips upgrade headers); `client_max_body_size` comes from `NGINX_MAX_BODY_SIZE` (default `50m`) and must be `>= max_upload_size_mb` or uploads 413. nginx serves `devplacepy/static` via a read-only bind mount, so assets stay current without an image rebuild.
|
||||
- **Healthcheck start period** (`start_period: 120s` in `docker-compose.yml`, `--start-period=120s` in `Dockerfile`): full startup takes ~110s (DB init, services, uvicorn workers). The start period must stay above that. Bump both files if startup grows.
|
||||
- **Healthcheck cadence** (`docker-compose.yml` + `Dockerfile`, keep both in step): `start_period: 120s` is the grace window in which a failing probe does not count against `retries`; `start_interval: 2s` is how often the probe runs *inside* that window. Without `start_interval` the first probe only fires after the full `interval: 30s`, so a container ready in 5s still reports healthy at 30s and `depends_on: service_healthy` holds nginx back for no reason. The generous 120s start period is deliberate headroom for a cold page cache on a multi-GB database, not a measure of normal startup - normal startup is a few seconds. **Startup work is a per-worker, lock-serialized cost:** `lifespan` runs `init_db()` under an exclusive `init_lock()`, so every uvicorn worker pays it end to end, one after another, and total time-to-serving is `workers x init_db`. Never put a per-user or per-row scan in `init_db` - see the backfill convergence rule in `devplacepy/database/CLAUDE.md`.
|
||||
|
||||
14
Dockerfile
14
Dockerfile
@ -18,22 +18,24 @@ RUN if [ "$INSTALL_DOCKER_CLI" = "true" ]; then \
|
||||
rm -rf /var/lib/apt/lists/* ; \
|
||||
fi
|
||||
|
||||
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
|
||||
|
||||
COPY pyproject.toml .
|
||||
RUN mkdir -p devplacepy && touch devplacepy/__init__.py \
|
||||
&& pip install --no-cache-dir ".[bots]" \
|
||||
&& python -m playwright install --with-deps chromium \
|
||||
&& chmod -R a+rx /ms-playwright
|
||||
|
||||
COPY devplacepy/ devplacepy/
|
||||
|
||||
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
|
||||
|
||||
RUN pip install --no-cache-dir ".[bots]" \
|
||||
&& python -m playwright install --with-deps chromium \
|
||||
&& chmod -R a+rx /ms-playwright
|
||||
RUN pip install --no-cache-dir --no-deps --force-reinstall .
|
||||
|
||||
EXPOSE 10500
|
||||
|
||||
ENV DEVPLACE_WEB_WORKERS=2
|
||||
ENV DEVPLACE_TEMPLATE_AUTO_RELOAD=0
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=120s \
|
||||
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=120s --start-interval=2s \
|
||||
CMD curl -f http://localhost:10500/ || exit 1
|
||||
|
||||
CMD ["sh", "-c", "DEVPLACE_STATIC_VERSION=${DEVPLACE_STATIC_VERSION:-$(date +%s)} exec uvicorn devplacepy.main:app --host 0.0.0.0 --port 10500 --workers 2 --backlog 8192 --proxy-headers --forwarded-allow-ips '*'"]
|
||||
|
||||
6
Makefile
6
Makefile
@ -138,7 +138,7 @@ DOCKER_GID ?= $(shell stat -c '%g' /var/run/docker.sock 2>/dev/null)
|
||||
export DEVPLACE_DATA_DIR
|
||||
export DOCKER_GID
|
||||
|
||||
.PHONY: docker-build docker-up docker-down docker-logs docker-clean docker-prep ppy
|
||||
.PHONY: docker-build docker-up docker-reload docker-down docker-logs docker-clean docker-prep ppy
|
||||
|
||||
# Build the single shared container image every instance runs. Build once;
|
||||
# rebuild only when ppy.Dockerfile, the sudo shim, or pagent change.
|
||||
@ -154,6 +154,10 @@ docker-build: docker-prep
|
||||
docker-up: docker-prep
|
||||
$(COMPOSE) up -d
|
||||
|
||||
docker-reload:
|
||||
$(COMPOSE) restart app
|
||||
$(COMPOSE) up -d --wait
|
||||
|
||||
docker-down:
|
||||
$(COMPOSE) down
|
||||
|
||||
|
||||
15
README.md
15
README.md
@ -543,6 +543,17 @@ disclosed only to administrators, while members and guests can see only the perc
|
||||
24-hour quota used. The `/devii/usage` endpoint returns that percentage and the day's turn count to
|
||||
everyone, and includes dollar figures only for administrators.
|
||||
|
||||
**Spend limits (Gateway page, `/admin/gateway`).** An administrator caps the rolling 24-hour
|
||||
gateway spend with quota rules scoped by any combination of caller role, individual user, and
|
||||
application reference (the `X-App-Reference` header), so a single application belonging to one
|
||||
user can be limited independently of that user's other traffic. The most specific matching rule
|
||||
wins; a caller over its cap gets `429`. Because a cap otherwise only lifts with the passage of
|
||||
time, each rule has a **Reset spend** action that clears what has been counted against it
|
||||
without deleting anything from the usage history the cost analytics are built on - the
|
||||
figures on the AI usage page stay intact, only the amount counted towards the limit is cleared.
|
||||
The **Reset all quotas** button on the AI usage page clears the assistant quotas and the gateway
|
||||
spend together. From the terminal: `devplace gateway quota list|set|delete|reset`.
|
||||
|
||||
Configuration on the Services tab:
|
||||
|
||||
| Parameter | Default | Purpose |
|
||||
@ -984,11 +995,13 @@ Open `http://<host>:${PORT}` (default 10500). `make docker-logs` tails output; `
|
||||
|
||||
```bash
|
||||
git pull
|
||||
make docker-up # restart with new code (bind-mounted, no rebuild)
|
||||
make docker-reload # restart workers on the new code (bind-mounted, no rebuild)
|
||||
make docker-build && \
|
||||
make docker-up # only when dependencies in pyproject.toml change
|
||||
```
|
||||
|
||||
`make docker-reload` is the target for a source-only change: `docker compose up -d` leaves an unchanged container running, so the workers would keep serving the code they imported at boot. A rebuild after a source-only change costs about 7 seconds because the Dockerfile installs dependencies from `pyproject.toml` in a layer that no source edit invalidates.
|
||||
|
||||
The `make deploy` target fast-forwards the `production` branch (`git checkout production && git merge master && git push origin production`); pull that branch on the server.
|
||||
|
||||
### Container Manager wiring (what the overlay does)
|
||||
|
||||
@ -70,6 +70,35 @@ def cmd_gateway_quota_delete(args):
|
||||
print(f"Deleted quota rule {args.uid}")
|
||||
|
||||
|
||||
def cmd_gateway_quota_reset(args):
|
||||
from devplacepy.services.openai_gateway import quota
|
||||
|
||||
try:
|
||||
payload = quota.QuotaResetIn(
|
||||
owner_kind=args.owner_kind,
|
||||
owner_id=args.owner_id,
|
||||
app_reference=args.app_reference,
|
||||
)
|
||||
except Exception as exc:
|
||||
print(f"Error: {exc}")
|
||||
sys.exit(1)
|
||||
scope = quota.reset(payload, created_by="cli")
|
||||
label = quota.scope_label(scope, fallback="every caller")
|
||||
_audit_cli(
|
||||
"gateway.quota.reset",
|
||||
f"CLI reset the gateway 24h spend for {label}",
|
||||
target_type="gateway_quota",
|
||||
target_uid=scope["uid"],
|
||||
metadata={
|
||||
"owner_kind": scope["owner_kind"],
|
||||
"owner_id": scope["owner_id"],
|
||||
"app_reference": scope["app_reference"],
|
||||
"reset_at": scope["reset_at"],
|
||||
},
|
||||
)
|
||||
print(f"Reset the rolling 24h spend for {label}")
|
||||
|
||||
|
||||
def register_gateway(subparsers):
|
||||
gateway = subparsers.add_parser("gateway", help="AI gateway management")
|
||||
gateway_sub = gateway.add_subparsers(title="action", dest="action")
|
||||
@ -101,3 +130,16 @@ def register_gateway(subparsers):
|
||||
quota_delete = quota_sub.add_parser("delete", help="Delete a quota rule by uid")
|
||||
quota_delete.add_argument("uid", help="Quota rule uid")
|
||||
quota_delete.set_defaults(func=cmd_gateway_quota_delete)
|
||||
|
||||
quota_reset = quota_sub.add_parser(
|
||||
"reset",
|
||||
help="Clear the rolling-24h spend so a capped caller can call again (keeps the usage history)",
|
||||
)
|
||||
quota_reset.add_argument(
|
||||
"--owner-kind",
|
||||
choices=("internal", "key", "user", "admin", "anonymous"),
|
||||
help="Role to scope by. Omit to reset every role",
|
||||
)
|
||||
quota_reset.add_argument("--owner-id", help="Specific user uid to scope by. Omit for every caller")
|
||||
quota_reset.add_argument("--app-reference", help="App label to scope by. Omit for every app")
|
||||
quota_reset.set_defaults(func=cmd_gateway_quota_reset)
|
||||
|
||||
@ -13,6 +13,8 @@ from devplacepy.database import (
|
||||
resolve_by_slug,
|
||||
get_users_by_uids,
|
||||
get_vote_counts,
|
||||
get_comment_counts_by_post_uids,
|
||||
paginate,
|
||||
STAR_TARGETS,
|
||||
get_user_votes,
|
||||
get_reactions_by_targets,
|
||||
@ -718,3 +720,24 @@ def enrich_items(
|
||||
)
|
||||
enriched.append(entry)
|
||||
return enriched
|
||||
|
||||
|
||||
def get_project_devlog(
|
||||
project_uid: str, before: str | None = None, viewer: dict | None = None
|
||||
) -> tuple[list, str | None]:
|
||||
posts, next_cursor = paginate(
|
||||
get_table("posts"),
|
||||
before=before,
|
||||
viewer_uid=viewer["uid"] if viewer else None,
|
||||
project_uid=project_uid,
|
||||
)
|
||||
if not posts:
|
||||
return [], None
|
||||
|
||||
authors = get_users_by_uids([post["user_uid"] for post in posts])
|
||||
counts = get_comment_counts_by_post_uids([post["uid"] for post in posts])
|
||||
|
||||
enriched = enrich_items(
|
||||
posts, "post", authors, {"comment_count": counts}, user=viewer
|
||||
)
|
||||
return enriched, next_cursor
|
||||
|
||||
@ -105,6 +105,17 @@ if "comments" not in db.tables:
|
||||
|
||||
`init_db()` ends with `_refresh_query_planner_stats()`: a one-time `ANALYZE` when `sqlite_stat1` is absent, then `PRAGMA optimize` every boot (both inside `with db:`). Validate any index change with `EXPLAIN QUERY PLAN` against `data/devplace.db` (or a copy) - a correct plan reads `SEARCH ... USING INDEX <name>` with no temp b-tree.
|
||||
|
||||
## Startup backfills must converge (hard rule)
|
||||
|
||||
`init_db()` runs inside `lifespan` under an exclusive `init_lock()`, **before the worker accepts a single request**, and every uvicorn worker runs it in turn. Time-to-serving is therefore `workers x init_db`, so anything added there is paid N times on every boot and every deploy. Two rules follow:
|
||||
|
||||
- **A backfill must be able to finish.** A backfill selects the rows that still need migrating and must leave them *not* selected afterwards. If a row can stay in the candidate set after a successful pass, the "one-time migration" is really a permanent per-boot scan that grows with the table. `_backfill_gamification` had exactly this bug: it selected `users.find(xp=0)` and then ran the full milestone sweep over *every* one of them, but a user with no content is awarded no XP, so they stayed at `xp=0` and were re-swept forever. At 7814 such users that was 36s of the 37.5s boot - about 140k queries that provably could not award anything, on every worker, on every restart.
|
||||
- **Never fan a per-row query out over a whole table at boot.** Compute the candidate set with a few set-based `GROUP BY`/`DISTINCT` queries first, then do per-row work only for rows that survive. `_milestone_candidates()` is the pattern: one `SELECT DISTINCT` per milestone source table (`MILESTONE_SOURCES`), unioned into a set, intersected with the pending users. A user absent from all of those tables scores 0 on every milestone metric and the lowest threshold is 5, so skipping them cannot change any award - verified by diffing the full `badges` table between the full sweep and the narrowed one (identical, 9800 rows, 22.6s -> 0.9s).
|
||||
|
||||
Watch for the same shape in `dataset` internals: `db.tables` is a live SQLAlchemy reflection, not a cached attribute. `get_user_stars` does one `in db.tables` check per `STAR_TARGETS` entry, so a per-user loop calling it re-reflects the whole table list on every iteration - 78667 reflections costing 17.7s in the profile above. Hoist `db.tables` into a local when looping.
|
||||
|
||||
Profile with the real database before and after any change here (`cProfile` around `init_db()` against a copy of `data/devplace.db`); a synthetic or empty DB hides every one of these costs.
|
||||
|
||||
## Project-wide soft delete (hard rule)
|
||||
|
||||
Attachments (see "Profile media gallery and soft-deleted attachments" below) are one instance of a platform-wide model: **every removal is a soft delete; only garbage collection is a hard delete.** Every removable row carries two columns, `deleted_at` (ISO timestamp) and `deleted_by` (actor uid, or `system`). A live row has `deleted_at = NULL`, and every list/count read filters `deleted_at IS NULL`.
|
||||
|
||||
@ -8,7 +8,7 @@ from .users import get_users_by_uids, _admins_cache, invalidate_admins_cache, ge
|
||||
from .relations import _relations_cache, get_user_relations, get_blocked_uids, get_muted_uids, get_silenced_uids, invalidate_user_relations
|
||||
from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, clear_user_post_count, build_pagination
|
||||
from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns, soft_delete, soft_delete_in, restore, purge, list_deleted, count_deleted, restore_event, purge_event
|
||||
from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_project_devlog, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_polls_by_post_uids, get_poll_for_post
|
||||
from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_polls_by_post_uids, get_poll_for_post
|
||||
from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage, AWARD_USAGE_KEY, add_award_usage, get_award_usage
|
||||
from .awards import (
|
||||
AWARDS_PER_PAGE,
|
||||
@ -115,7 +115,6 @@ __all__ = [
|
||||
"_comment_count_cache",
|
||||
"get_comment_counts_by_post_uids",
|
||||
"get_post_counts_by_user_uids",
|
||||
"get_project_devlog",
|
||||
"get_vote_counts",
|
||||
"get_user_votes",
|
||||
"get_reactions_by_targets",
|
||||
|
||||
@ -1,9 +1,6 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .core import TTLCache, _in_clause, db, defaultdict, get_table
|
||||
from .pagination import paginate
|
||||
from devplacepy.content import enrich_items
|
||||
from .users import get_users_by_uids
|
||||
from .core import TTLCache, _in_clause, db, defaultdict
|
||||
|
||||
|
||||
_comment_count_cache = TTLCache(ttl=15, max_size=10000)
|
||||
@ -190,23 +187,3 @@ def get_poll_for_post(post_uid, user=None):
|
||||
return get_polls_by_post_uids([post_uid], user).get(post_uid)
|
||||
|
||||
|
||||
def get_project_devlog(project_uid: str, before: str | None = None, viewer: dict | None = None) -> tuple[list, str | None]:
|
||||
posts_table = get_table("posts")
|
||||
posts, next_cursor = paginate(
|
||||
posts_table,
|
||||
before=before,
|
||||
viewer_uid=viewer["uid"] if viewer else None,
|
||||
project_uid=project_uid,
|
||||
)
|
||||
if not posts:
|
||||
return [], None
|
||||
|
||||
authors = get_users_by_uids([p["user_uid"] for p in posts])
|
||||
counts = get_comment_counts_by_post_uids([p["uid"] for p in posts])
|
||||
|
||||
result = enrich_items(
|
||||
posts, "post", authors, {"comment_count": counts}, user=viewer
|
||||
)
|
||||
return result, next_cursor
|
||||
|
||||
|
||||
|
||||
@ -32,12 +32,13 @@ def _ranked_authors() -> list:
|
||||
cached = _authors_cache.get("ranked")
|
||||
if cached is not None:
|
||||
return cached
|
||||
tables = db.tables
|
||||
sources = [
|
||||
(target_type, table_name)
|
||||
for target_type, table_name in VOTABLE_TARGETS.items()
|
||||
if table_name in db.tables
|
||||
if table_name in tables
|
||||
]
|
||||
if "votes" not in db.tables or not sources:
|
||||
if "votes" not in tables or not sources:
|
||||
_authors_cache.set("ranked", [])
|
||||
return []
|
||||
target_union = " UNION ALL ".join(
|
||||
@ -101,12 +102,13 @@ def get_user_stars(user_uid: str) -> int:
|
||||
cached = _stars_cache.get(user_uid)
|
||||
if cached is not None:
|
||||
return cached
|
||||
if "votes" not in db.tables:
|
||||
tables = db.tables
|
||||
if "votes" not in tables:
|
||||
return 0
|
||||
target_union = " UNION ALL ".join(
|
||||
f"SELECT uid, '{target_type}' AS target_type FROM {table_name} WHERE user_uid = :u AND deleted_at IS NULL"
|
||||
for target_type, table_name in VOTABLE_TARGETS.items()
|
||||
if table_name in db.tables
|
||||
if table_name in tables
|
||||
)
|
||||
if not target_union:
|
||||
return 0
|
||||
@ -154,7 +156,8 @@ def delete_engagement(target_type: str, target_uids: list) -> None:
|
||||
uids = [uid for uid in (target_uids or []) if uid]
|
||||
if not uids:
|
||||
return
|
||||
if "reactions" in db.tables:
|
||||
tables = db.tables
|
||||
if "reactions" in tables:
|
||||
placeholders, params = _in_clause(uids)
|
||||
params["tt"] = target_type
|
||||
with db:
|
||||
@ -162,7 +165,7 @@ def delete_engagement(target_type: str, target_uids: list) -> None:
|
||||
f"DELETE FROM reactions WHERE target_type=:tt AND target_uid IN ({placeholders})",
|
||||
**params,
|
||||
)
|
||||
if "bookmarks" in db.tables:
|
||||
if "bookmarks" in tables:
|
||||
placeholders, params = _in_clause(uids)
|
||||
params["tt"] = target_type
|
||||
with db:
|
||||
@ -170,12 +173,12 @@ def delete_engagement(target_type: str, target_uids: list) -> None:
|
||||
f"DELETE FROM bookmarks WHERE target_type=:tt AND target_uid IN ({placeholders})",
|
||||
**params,
|
||||
)
|
||||
if target_type == "post" and "polls" in db.tables:
|
||||
if target_type == "post" and "polls" in tables:
|
||||
for uid in uids:
|
||||
for poll in db["polls"].find(post_uid=uid):
|
||||
if "poll_votes" in db.tables:
|
||||
if "poll_votes" in tables:
|
||||
db["poll_votes"].delete(poll_uid=poll["uid"])
|
||||
if "poll_options" in db.tables:
|
||||
if "poll_options" in tables:
|
||||
db["poll_options"].delete(poll_uid=poll["uid"])
|
||||
db["polls"].delete(post_uid=uid)
|
||||
|
||||
|
||||
@ -1761,6 +1761,29 @@ def backfill_api_keys() -> int:
|
||||
return updated
|
||||
|
||||
|
||||
MILESTONE_SOURCES = (
|
||||
("posts", "user_uid"),
|
||||
("comments", "user_uid"),
|
||||
("projects", "user_uid"),
|
||||
("gists", "user_uid"),
|
||||
("follows", "follower_uid"),
|
||||
("follows", "following_uid"),
|
||||
("user_activity", "user_uid"),
|
||||
)
|
||||
|
||||
|
||||
def _milestone_candidates() -> set:
|
||||
tables = db.tables
|
||||
candidates = set()
|
||||
for table, column in MILESTONE_SOURCES:
|
||||
if table not in tables:
|
||||
continue
|
||||
for row in db.query(f"SELECT DISTINCT {column} AS uid FROM {table}"):
|
||||
if row["uid"]:
|
||||
candidates.add(row["uid"])
|
||||
return candidates
|
||||
|
||||
|
||||
def _backfill_gamification():
|
||||
if "users" not in db.tables:
|
||||
return
|
||||
@ -1824,7 +1847,12 @@ def _backfill_gamification():
|
||||
)
|
||||
|
||||
_authors_cache.clear()
|
||||
for user in pending:
|
||||
candidates = _milestone_candidates()
|
||||
checked = [user for user in pending if user["uid"] in candidates]
|
||||
for user in checked:
|
||||
check_milestone_badges(user["uid"])
|
||||
logger.info(f"Gamification backfill processed {len(pending)} users")
|
||||
logger.info(
|
||||
f"Gamification backfill processed {len(pending)} users, "
|
||||
f"{len(checked)} with milestone-eligible activity"
|
||||
)
|
||||
|
||||
|
||||
@ -77,7 +77,10 @@ def get_online_users(cutoff_iso: str, limit: int = 30) -> list:
|
||||
def _can_hold_primary_admin(row, tracks_active):
|
||||
if row.get("deleted_at"):
|
||||
return False
|
||||
return not tracks_active or bool(row.get("is_active"))
|
||||
if not tracks_active:
|
||||
return True
|
||||
is_active = row.get("is_active")
|
||||
return is_active is None or bool(is_active)
|
||||
|
||||
|
||||
def get_primary_admin_uid():
|
||||
|
||||
@ -648,6 +648,26 @@ four ways to sign requests.
|
||||
field("label", "json", "string", False, "", "Optional admin-facing note."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="admin-gateway-quota-reset",
|
||||
method="POST",
|
||||
path="/admin/gateway/quota-resets",
|
||||
title="Reset the AI gateway 24h spend",
|
||||
summary=(
|
||||
"Clear the counted rolling-24h spend for a scope so a capped caller can call "
|
||||
"again, without deleting any usage history (the cost analytics stay intact). "
|
||||
"Scope it exactly like a quota rule; leaving all three dimensions blank resets "
|
||||
"every caller. Only spend recorded before the reset is cleared - new calls "
|
||||
"count again immediately against the same limit."
|
||||
),
|
||||
auth="admin",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("owner_kind", "json", "string", False, "user", "internal, key, user, admin, or anonymous. Blank = every role."),
|
||||
field("owner_id", "json", "string", False, "", "Specific user uid. Blank = every caller."),
|
||||
field("app_reference", "json", "string", False, "typosaurus", "App label (the X-App-Reference header). Blank = every app."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="admin-gateway-quota-rule-delete",
|
||||
method="DELETE",
|
||||
|
||||
@ -41,9 +41,12 @@ four ways to sign requests.
|
||||
method="GET",
|
||||
path="/profile/{username}",
|
||||
title="View a profile",
|
||||
summary="Render a user profile, including an online-presence indicator (JSON exposes profile_online and profile_user.last_seen). Returns an HTML page.",
|
||||
summary="Render a user profile, including an online-presence indicator (JSON exposes profile_online, profile_user.last_seen, xp_next_level, and xp_progress_pct). Returns an HTML page.",
|
||||
auth="public",
|
||||
interactive=True,
|
||||
notes=[
|
||||
"Level progress: `xp_next_level = level * 100` (total XP needed), `xp_progress_pct = xp % 100` (percentage towards next level). Both are also embedded in `profile_user`.",
|
||||
],
|
||||
params=[
|
||||
field(
|
||||
"username",
|
||||
@ -793,3 +796,4 @@ four ways to sign requests.
|
||||
),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@ -6,6 +6,7 @@ from devplacepy.utils import require_admin
|
||||
from devplacepy.responses import action_result
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.services.openai_gateway import quota
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@ -34,14 +35,20 @@ async def admin_reset_all_ai_quota(request: Request):
|
||||
admin = require_admin(request)
|
||||
devii = service_manager.get_service("devii")
|
||||
removed = devii.reset_all_quotas() if devii is not None else 0
|
||||
gateway = quota.reset(created_by=admin["uid"])
|
||||
logger.info(
|
||||
f"Admin {admin['username']} reset ALL AI quotas ({removed} ledger rows)"
|
||||
f"Admin {admin['username']} reset ALL AI quotas "
|
||||
f"({removed} Devii ledger rows, gateway watermark {gateway['reset_at']})"
|
||||
)
|
||||
audit.record(
|
||||
request,
|
||||
"admin.ai_quota.reset_all",
|
||||
user=admin,
|
||||
metadata={"rows_removed": removed},
|
||||
summary=f"admin {admin['username']} reset all AI quotas",
|
||||
metadata={"rows_removed": removed, "gateway_reset_at": gateway["reset_at"]},
|
||||
summary=(
|
||||
f"admin {admin['username']} reset all AI quotas "
|
||||
"(Devii assistant and AI gateway)"
|
||||
),
|
||||
)
|
||||
return action_result(request, "/admin/ai-usage")
|
||||
|
||||
|
||||
@ -197,14 +197,7 @@ def _quota_defaults_summary() -> dict:
|
||||
|
||||
|
||||
def _rule_label(rule: dict) -> str:
|
||||
parts = []
|
||||
if rule.get("owner_kind"):
|
||||
parts.append(f"role={rule['owner_kind']}")
|
||||
if rule.get("owner_id"):
|
||||
parts.append(f"user={rule['owner_id']}")
|
||||
if rule.get("app_reference"):
|
||||
parts.append(f"app={rule['app_reference']}")
|
||||
return ", ".join(parts) or rule.get("uid", "")
|
||||
return quota.scope_label(rule, fallback=rule.get("uid", ""))
|
||||
|
||||
|
||||
@router.get("/gateway/quota-rules")
|
||||
@ -253,6 +246,34 @@ async def save_quota_rule(request: Request):
|
||||
return JSONResponse({"ok": True, "rule": saved})
|
||||
|
||||
|
||||
@router.post("/gateway/quota-resets")
|
||||
async def reset_quota_spend(request: Request):
|
||||
admin = require_admin(request)
|
||||
body = await _payload(request)
|
||||
try:
|
||||
payload = quota.QuotaResetIn(**body)
|
||||
except ValidationError as exc:
|
||||
return _validation_error(exc)
|
||||
scope = quota.reset(payload, created_by=admin["uid"])
|
||||
label = quota.scope_label(scope, fallback="every caller")
|
||||
audit.record(
|
||||
request,
|
||||
"gateway.quota.reset",
|
||||
user=admin,
|
||||
target_type="gateway_quota",
|
||||
target_uid=scope["uid"],
|
||||
target_label=label,
|
||||
summary=f"admin {admin['username']} reset the gateway 24h spend for {label}",
|
||||
metadata={
|
||||
"owner_kind": scope["owner_kind"],
|
||||
"owner_id": scope["owner_id"],
|
||||
"app_reference": scope["app_reference"],
|
||||
"reset_at": scope["reset_at"],
|
||||
},
|
||||
)
|
||||
return JSONResponse({"ok": True, "reset": scope})
|
||||
|
||||
|
||||
@router.delete("/gateway/quota-rules/{uid}")
|
||||
async def delete_quota_rule(request: Request, uid: str):
|
||||
admin = require_admin(request)
|
||||
|
||||
@ -7,7 +7,6 @@ from devplacepy.models import ProfileForm
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
db,
|
||||
get_customization_prefs,
|
||||
get_notification_prefs,
|
||||
get_user_stars,
|
||||
@ -37,6 +36,7 @@ from devplacepy.database.awards import (
|
||||
from devplacepy.content import can_view_project, enrich_items
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
get_badge,
|
||||
require_user,
|
||||
require_user_api,
|
||||
time_ago,
|
||||
@ -46,6 +46,7 @@ from devplacepy.utils import (
|
||||
track_action,
|
||||
build_achievements,
|
||||
)
|
||||
from devplacepy.utils.rewards import LEVEL_XP
|
||||
from devplacepy.responses import respond, action_result, wants_json
|
||||
from devplacepy.schemas import ProfileOut
|
||||
from devplacepy.avatar import avatar_url, avatar_seed
|
||||
@ -139,6 +140,12 @@ async def profile_page(
|
||||
current_user["uid"], f"/profile/{profile_user['username']}"
|
||||
)
|
||||
profile_user["stars"] = get_user_stars(profile_user["uid"])
|
||||
xp_raw = profile_user.get("xp") or 0
|
||||
level_raw = profile_user.get("level") or 1
|
||||
xp_progress_pct = xp_raw % LEVEL_XP
|
||||
xp_next_level = level_raw * LEVEL_XP
|
||||
profile_user["xp_progress_pct"] = xp_progress_pct
|
||||
profile_user["xp_next_level"] = xp_next_level
|
||||
rank = get_user_rank(profile_user["uid"])
|
||||
follow_counts = get_follow_counts(profile_user["uid"])
|
||||
|
||||
@ -195,6 +202,8 @@ async def profile_page(
|
||||
item["poll"] = polls_map.get(uid)
|
||||
|
||||
badges = list(get_table("badges").find(user_uid=profile_user["uid"]))
|
||||
for b in badges:
|
||||
b["icon"] = get_badge(b["badge_name"]).get("icon")
|
||||
achievements = build_achievements({b["badge_name"] for b in badges})
|
||||
badge_total = sum(group["total"] for group in achievements)
|
||||
badge_earned = sum(group["earned"] for group in achievements)
|
||||
@ -440,6 +449,8 @@ async def profile_page(
|
||||
"awards_count": awards_count,
|
||||
"prominent_award": prominent_award,
|
||||
"can_give_award": can_give,
|
||||
"xp_next_level": xp_next_level,
|
||||
"xp_progress_pct": xp_progress_pct,
|
||||
},
|
||||
model=ProfileOut,
|
||||
)
|
||||
@ -499,3 +510,7 @@ async def regenerate_api_key(request: Request):
|
||||
links=[audit.target("user", user["uid"], user["username"])],
|
||||
)
|
||||
return JSONResponse({"api_key": new_key})
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@ -17,7 +17,6 @@ from devplacepy.database import (
|
||||
get_reactions_by_targets,
|
||||
get_user_bookmarks,
|
||||
get_polls_by_post_uids,
|
||||
get_project_devlog,
|
||||
paginate,
|
||||
text_search_clause,
|
||||
resolve_by_slug,
|
||||
@ -40,6 +39,7 @@ from devplacepy.content import (
|
||||
is_owner,
|
||||
can_view_project,
|
||||
can_view_project_containers,
|
||||
get_project_devlog,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
|
||||
@ -19,6 +19,8 @@ class UserOut(_Out):
|
||||
website: Optional[str] = None
|
||||
level: Optional[int] = None
|
||||
xp: Optional[int] = None
|
||||
xp_progress_pct: Optional[int] = None
|
||||
xp_next_level: Optional[int] = None
|
||||
stars: Optional[int] = None
|
||||
created_at: Optional[str] = None
|
||||
last_seen: Optional[str] = None
|
||||
@ -65,6 +67,7 @@ class PollOut(_Out):
|
||||
|
||||
class BadgeOut(_Out):
|
||||
name: Optional[str] = Field(None, alias="badge_name")
|
||||
icon: Optional[str] = None
|
||||
created_at: Optional[str] = None
|
||||
model_config = ConfigDict(populate_by_name=True)
|
||||
|
||||
|
||||
@ -72,6 +72,8 @@ class ProfileOut(_Out):
|
||||
followers_count: Optional[int] = None
|
||||
following_count: Optional[int] = None
|
||||
viewer_is_admin: bool = False
|
||||
xp_next_level: int = 0
|
||||
xp_progress_pct: int = 0
|
||||
media: list[MediaItemOut] = []
|
||||
media_pagination: Optional[Any] = None
|
||||
notification_prefs: list[Any] = []
|
||||
@ -88,3 +90,4 @@ class TelegramPairOut(_Out):
|
||||
code: Optional[str] = None
|
||||
expires_at: Optional[str] = None
|
||||
ttl_minutes: Optional[int] = None
|
||||
|
||||
|
||||
@ -161,6 +161,30 @@ GATEWAY_ACTIONS: tuple[Action, ...] = (
|
||||
body("label", "Optional admin-facing note describing what this rule is for."),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="gateway_quota_reset",
|
||||
method="POST",
|
||||
path="/admin/gateway/quota-resets",
|
||||
summary="Reset the rolling-24h AI gateway spend so a capped caller can call again (admin only)",
|
||||
description=(
|
||||
"Clears the counted spend for a scope without deleting any usage history, so the "
|
||||
"cost analytics on /admin/ai-usage stay intact. Scope it exactly like a quota rule: "
|
||||
"owner_kind (internal/key/user/admin/anonymous), a specific owner_id (user uid), and "
|
||||
"app_reference (the X-App-Reference header). Leaving all three blank resets every "
|
||||
"caller. A reset only clears spend recorded BEFORE it - new calls start counting "
|
||||
"again immediately against the same limit. Use this when an app is stuck on "
|
||||
"'AI gateway daily quota exceeded' and you want it running again without raising "
|
||||
"its cap."
|
||||
),
|
||||
handler="http",
|
||||
requires_admin=True,
|
||||
params=(
|
||||
body("owner_kind", "Role to scope by: internal, key, user, admin, or anonymous. Blank = every role."),
|
||||
body("owner_id", "Specific user uid to scope by. Blank = every caller."),
|
||||
body("app_reference", "App label to scope by (the X-App-Reference header). Blank = every app."),
|
||||
confirm(),
|
||||
),
|
||||
),
|
||||
Action(
|
||||
name="gateway_quota_rule_delete",
|
||||
method="DELETE",
|
||||
|
||||
@ -59,6 +59,7 @@ CONFIRM_REQUIRED = {
|
||||
"gateway_provider_delete",
|
||||
"gateway_model_delete",
|
||||
"gateway_quota_rule_delete",
|
||||
"gateway_quota_reset",
|
||||
"email_account_delete",
|
||||
"email_delete_message",
|
||||
"game_prestige",
|
||||
|
||||
@ -73,7 +73,11 @@ The gateway records one row per upstream call (chat, vision, passthrough) and su
|
||||
|
||||
**429 body never carries a dollar figure**, admin or not (`{"detail": "AI gateway daily quota exceeded"}`) - mirrors Devii's own over-limit WS message, which likewise never states a number. The admin-only services log line and the `ai.quota.exceeded` audit row (`GatewayService._audit_quota_exceeded`, reusing `usage.audit_actor_for`) do carry the spend/limit/matched-rule-uid, since those are admin-only surfaces.
|
||||
|
||||
**CRUD.** Admin JSON at `/admin/gateway/quota-rules` (`routers/admin/gateway_configs.py`, list returns each rule's live `spent_24h_usd` plus the Layer A defaults for context), audited `gateway.quota_rule.update`/`gateway.quota_rule.delete` (category `ai`, both already in `events.md`), rendered in the **Quota rules** section of `/admin/gateway` (`GatewayAdmin.js`, mirrors the providers/models CRUD tables). Devii tools `gateway_quota_rules`/`gateway_quota_rule_set`/`gateway_quota_rule_delete` (`requires_admin=True`, delete is `CONFIRM_REQUIRED`) proxy the same endpoints via `handler="http"`, same as the provider/model tools. CLI: `devplace gateway quota list|set|delete`.
|
||||
**Resetting the counted spend (`gateway_quota_resets`).** A cap is only lifted by *time* otherwise, so there is a reset that clears what has been counted **without deleting any ledger row** - `gateway_usage_ledger` is the cost-analytics source for `/admin/ai-usage`, so a reset must never truncate it. `quota.reset(QuotaResetIn, created_by=)` upserts one watermark row into `gateway_quota_resets` (same `ensure_tables()`/`"gateway_quota"` cache-version/hard-CRUD shape as the rules table, same two indexes) scoped by the SAME three nullable dimensions as a rule, and `quota.spent_24h` sums from `max(24h cutoff, reset_watermark(scope))`. A reset row applies to a queried scope when each of its non-null dimensions equals that scope's - so an all-null reset clears everyone, while a reset scoped to one app deliberately does NOT clear a broader per-user-all-apps scope (clearing a narrower window can only over-credit). Spend recorded after the reset counts again immediately against the same limit. `QuotaScopeIn` is the shared base holding the three dimensions and their validators; `QuotaRuleIn` and `QuotaResetIn` both extend it, so scope parsing exists once.
|
||||
|
||||
**The two AI quotas are separate systems and the reset surfaces must say so.** `/admin/ai-usage`'s *Reset all quotas* clears the Devii `devii_usage_ledger` AND now also stamps a global gateway watermark, because a caller hitting `429 AI gateway daily quota exceeded` had no reset at all before and the button looked global. *Reset guest quotas* stays Devii-only (guest gateway calls ride the shared internal key, so there is no per-guest gateway scope to clear).
|
||||
|
||||
**CRUD.** Admin JSON at `/admin/gateway/quota-rules` (`routers/admin/gateway_configs.py`, list returns each rule's live `spent_24h_usd` plus the Layer A defaults for context), audited `gateway.quota_rule.update`/`gateway.quota_rule.delete` (category `ai`, both already in `events.md`), rendered in the **Quota rules** section of `/admin/gateway` (`GatewayAdmin.js`, mirrors the providers/models CRUD tables). Devii tools `gateway_quota_rules`/`gateway_quota_rule_set`/`gateway_quota_rule_delete` (`requires_admin=True`, delete is `CONFIRM_REQUIRED`) proxy the same endpoints via `handler="http"`, same as the provider/model tools. Reset is `POST /admin/gateway/quota-resets` (same file, `_payload`/`ValidationError` shape as the rule CRUD), audited `gateway.quota.reset` (category `ai`), surfaced as a per-rule **Reset spend** button in the Quota rules table (`GatewayAdmin.js`), and exposed as the Devii tool `gateway_quota_reset` (`requires_admin=True`, in `CONFIRM_REQUIRED` with a declared `confirm` param, like the other quota-lifting admin resets). CLI: `devplace gateway quota list|set|delete|reset`.
|
||||
|
||||
## Image generation
|
||||
|
||||
|
||||
@ -16,6 +16,7 @@ from devplacepy.database import bump_cache_version, db, get_table, sync_local_ca
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
RULES_TABLE = "gateway_quota_rules"
|
||||
RESETS_TABLE = "gateway_quota_resets"
|
||||
CACHE_NAME = "gateway_quota"
|
||||
APP_REFERENCE_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]{1,30}$")
|
||||
OWNER_KINDS = ("internal", "key", "user", "admin", "anonymous")
|
||||
@ -62,22 +63,38 @@ def ensure_tables() -> None:
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("gateway quota rule index creation failed: %s", exc)
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS "
|
||||
+ RESETS_TABLE
|
||||
+ " (id INTEGER PRIMARY KEY, uid TEXT, owner_kind TEXT, owner_id TEXT, "
|
||||
"app_reference TEXT, reset_at TEXT, created_by TEXT)"
|
||||
)
|
||||
try:
|
||||
db.query(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS idx_gateway_quota_resets_uid ON "
|
||||
+ RESETS_TABLE
|
||||
+ " (uid)"
|
||||
)
|
||||
db.query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_gateway_quota_resets_lookup ON "
|
||||
+ RESETS_TABLE
|
||||
+ " (owner_kind, owner_id, app_reference)"
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("gateway quota reset index creation failed: %s", exc)
|
||||
|
||||
|
||||
class QuotaRuleIn(BaseModel):
|
||||
class QuotaScopeIn(BaseModel):
|
||||
owner_kind: Optional[str] = None
|
||||
owner_id: Optional[str] = Field(default=None, max_length=64)
|
||||
app_reference: Optional[str] = Field(default=None, max_length=30)
|
||||
limit_usd: float = Field(default=0.0, ge=0)
|
||||
is_active: bool = True
|
||||
label: str = Field(default="", max_length=200)
|
||||
|
||||
@field_validator("owner_kind")
|
||||
@classmethod
|
||||
def _clean_owner_kind(cls, value: Optional[str]) -> Optional[str]:
|
||||
value = (value or "").strip().lower()
|
||||
if not value:
|
||||
return None
|
||||
value = value.strip().lower()
|
||||
if value not in OWNER_KINDS:
|
||||
raise ValueError(f"owner_kind must be one of {', '.join(OWNER_KINDS)}")
|
||||
return value
|
||||
@ -85,20 +102,24 @@ class QuotaRuleIn(BaseModel):
|
||||
@field_validator("owner_id")
|
||||
@classmethod
|
||||
def _clean_owner_id(cls, value: Optional[str]) -> Optional[str]:
|
||||
if not value:
|
||||
return None
|
||||
return value.strip()
|
||||
return (value or "").strip() or None
|
||||
|
||||
@field_validator("app_reference")
|
||||
@classmethod
|
||||
def _clean_app_reference(cls, value: Optional[str]) -> Optional[str]:
|
||||
value = (value or "").strip()
|
||||
if not value:
|
||||
return None
|
||||
value = value.strip()
|
||||
if not APP_REFERENCE_PATTERN.match(value):
|
||||
raise ValueError("app_reference must match ^[a-zA-Z0-9_.-]{1,30}$")
|
||||
return value
|
||||
|
||||
|
||||
class QuotaRuleIn(QuotaScopeIn):
|
||||
limit_usd: float = Field(default=0.0, ge=0)
|
||||
is_active: bool = True
|
||||
label: str = Field(default="", max_length=200)
|
||||
|
||||
@field_validator("label")
|
||||
@classmethod
|
||||
def _clean_label(cls, value: str) -> str:
|
||||
@ -114,6 +135,10 @@ class QuotaRuleIn(BaseModel):
|
||||
return self
|
||||
|
||||
|
||||
class QuotaResetIn(QuotaScopeIn):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QuotaRule:
|
||||
uid: str
|
||||
@ -250,6 +275,83 @@ class QuotaRuleStore:
|
||||
quota_rule_store = QuotaRuleStore()
|
||||
|
||||
|
||||
def _load_resets() -> list[dict]:
|
||||
sync_local_cache(CACHE_NAME, _QUOTA_CACHE)
|
||||
if "resets" not in _QUOTA_CACHE:
|
||||
resets: list[dict] = []
|
||||
try:
|
||||
if RESETS_TABLE in db.tables:
|
||||
for row in get_table(RESETS_TABLE).all():
|
||||
if row.get("reset_at"):
|
||||
resets.append(
|
||||
{
|
||||
"owner_kind": row.get("owner_kind") or None,
|
||||
"owner_id": row.get("owner_id") or None,
|
||||
"app_reference": row.get("app_reference") or None,
|
||||
"reset_at": str(row["reset_at"]),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("gateway quota reset load failed: %s", exc)
|
||||
_QUOTA_CACHE["resets"] = resets
|
||||
return _QUOTA_CACHE["resets"]
|
||||
|
||||
|
||||
def _reset_applies(reset: dict, scope: dict) -> bool:
|
||||
for field in ("owner_kind", "owner_id", "app_reference"):
|
||||
wanted = reset.get(field)
|
||||
if wanted is None:
|
||||
continue
|
||||
if scope.get(field) is None or scope[field] != wanted:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def reset_watermark(
|
||||
owner_kind: Optional[str], owner_id: Optional[str], app_reference: Optional[str]
|
||||
) -> str:
|
||||
scope = {
|
||||
"owner_kind": owner_kind,
|
||||
"owner_id": owner_id,
|
||||
"app_reference": app_reference,
|
||||
}
|
||||
stamps = [r["reset_at"] for r in _load_resets() if _reset_applies(r, scope)]
|
||||
return max(stamps) if stamps else ""
|
||||
|
||||
|
||||
def reset(payload: Optional[QuotaResetIn] = None, *, created_by: str = "") -> dict:
|
||||
ensure_tables()
|
||||
payload = payload or QuotaResetIn()
|
||||
table = get_table(RESETS_TABLE)
|
||||
scope = {
|
||||
"owner_kind": payload.owner_kind,
|
||||
"owner_id": payload.owner_id,
|
||||
"app_reference": payload.app_reference,
|
||||
}
|
||||
stamp = _now()
|
||||
existing = table.find_one(**scope)
|
||||
if existing:
|
||||
table.update({"id": existing["id"], "reset_at": stamp, "created_by": created_by}, ["id"])
|
||||
uid = existing.get("uid") or uuid.uuid4().hex
|
||||
else:
|
||||
uid = uuid.uuid4().hex
|
||||
table.insert({**scope, "uid": uid, "reset_at": stamp, "created_by": created_by})
|
||||
bump_cache_version(CACHE_NAME)
|
||||
_QUOTA_CACHE.clear()
|
||||
return {**scope, "uid": uid, "reset_at": stamp}
|
||||
|
||||
|
||||
def scope_label(scope: dict, fallback: str = "") -> str:
|
||||
parts = []
|
||||
if scope.get("owner_kind"):
|
||||
parts.append(f"role={scope['owner_kind']}")
|
||||
if scope.get("owner_id"):
|
||||
parts.append(f"user={scope['owner_id']}")
|
||||
if scope.get("app_reference"):
|
||||
parts.append(f"app={scope['app_reference']}")
|
||||
return ", ".join(parts) or fallback
|
||||
|
||||
|
||||
def default_limit(owner_kind: str, cfg: dict) -> float:
|
||||
field = _DEFAULT_FIELD_BY_KIND.get(owner_kind, FIELD_DEFAULT_USER)
|
||||
return float(cfg.get(field, 0.0) or 0.0)
|
||||
@ -294,10 +396,10 @@ def spent_24h(
|
||||
|
||||
if GATEWAY_LEDGER not in db.tables:
|
||||
return 0.0
|
||||
cutoff = (datetime.now(timezone.utc) - timedelta(hours=24)).isoformat()
|
||||
watermark = reset_watermark(owner_kind, owner_id, app_reference)
|
||||
clauses = ["created_at >= :cutoff"]
|
||||
params: dict = {
|
||||
"cutoff": (datetime.now(timezone.utc) - timedelta(hours=24)).isoformat()
|
||||
}
|
||||
params: dict = {"cutoff": max(cutoff, watermark) if watermark else cutoff}
|
||||
if owner_kind is not None:
|
||||
clauses.append("owner_kind = :owner_kind")
|
||||
params["owner_kind"] = owner_kind
|
||||
|
||||
@ -221,7 +221,7 @@ for `target_type == "quiz"`, soft-deleting questions, options, attempts and answ
|
||||
| SEO | `seo.quiz_schema`, `/quizzes` + the newest published quizzes in the sitemap; hub and detail `index,follow`, builder/player/results `noindex,follow`, a draft detail `noindex,nofollow` |
|
||||
| Notifications | `quiz_attempt`, fired once on the finish transition to the author, never per answer |
|
||||
| Gamification | `XP_QUIZ`/`XP_QUIZ_PUBLISH`/`XP_QUIZ_COMPLETE`, achievements `quiz_publish`/`quiz_complete`/`quiz_perfect`, the **Quizzes** badge group |
|
||||
| Audit | prefix `quiz` -> category `content`; eleven keys in `events.md` |
|
||||
| Audit | prefix `quiz` -> category `content`; thirteen keys in `events.md` |
|
||||
| Devii | `actions/catalog/quizzes.py`; `publish_quiz`, `delete_quiz` and `delete_quiz_question` are in `dispatcher.CONFIRM_REQUIRED` and each declares a `confirm` param |
|
||||
| CLI | `devplace quiz prune` - hard-deletes abandoned and expired attempts older than `QUIZ_ATTEMPT_RETENTION_DAYS`. Completed attempts are never pruned; they are the player's record |
|
||||
| Frontend | `dp-quiz-player`/`dp-quiz-builder` (light DOM, adopt the server-rendered markup), `static/css/quiz.css` (adds only what is new - the layout and card chrome come from `feed.css`/`sidebar.css`) |
|
||||
|
||||
@ -279,9 +279,9 @@ export class GatewayAdmin {
|
||||
form.name.scrollIntoView({ block: "center" });
|
||||
}
|
||||
|
||||
async confirmDelete(message) {
|
||||
async confirmAction(message, confirmLabel = "Delete") {
|
||||
if (window.app && window.app.dialog) {
|
||||
return window.app.dialog.confirm({ message, danger: true, confirmLabel: "Delete" });
|
||||
return window.app.dialog.confirm({ message, danger: true, confirmLabel });
|
||||
}
|
||||
return window.confirm(message);
|
||||
}
|
||||
@ -294,7 +294,7 @@ export class GatewayAdmin {
|
||||
}
|
||||
|
||||
async deleteProvider(name) {
|
||||
if (!(await this.confirmDelete(`Delete provider "${name}"?`))) return;
|
||||
if (!(await this.confirmAction(`Delete provider "${name}"?`))) return;
|
||||
try {
|
||||
await this.remove(`/admin/gateway/providers/${encodeURIComponent(name)}`);
|
||||
this.notify("Provider deleted", "success");
|
||||
@ -338,7 +338,7 @@ export class GatewayAdmin {
|
||||
}
|
||||
|
||||
async deleteModel(source) {
|
||||
if (!(await this.confirmDelete(`Delete model route "${source}"?`))) return;
|
||||
if (!(await this.confirmAction(`Delete model route "${source}"?`))) return;
|
||||
try {
|
||||
await this.remove(`/admin/gateway/models/${encodeURIComponent(source)}`);
|
||||
this.notify("Model route deleted", "success");
|
||||
@ -394,6 +394,7 @@ export class GatewayAdmin {
|
||||
<td>${this.escape(r.label || "")}</td>
|
||||
<td class="gw-actions">
|
||||
<button class="admin-btn admin-btn-sm" data-edit-quota='${this.attr(JSON.stringify(r))}'>Edit</button>
|
||||
<button class="admin-btn admin-btn-sm" data-reset-quota='${this.attr(JSON.stringify(r))}'>Reset spend</button>
|
||||
<button class="admin-btn admin-btn-sm admin-btn-danger" data-del-quota="${this.attr(r.uid)}">Delete</button>
|
||||
</td>
|
||||
</tr>`;
|
||||
@ -443,13 +444,31 @@ export class GatewayAdmin {
|
||||
|
||||
onQuotaRuleClick(event) {
|
||||
const editRaw = event.target.dataset.editQuota;
|
||||
const resetRaw = event.target.dataset.resetQuota;
|
||||
const delUid = event.target.dataset.delQuota;
|
||||
if (editRaw) this.fillQuotaForm(JSON.parse(editRaw));
|
||||
if (resetRaw) this.resetQuotaSpend(JSON.parse(resetRaw));
|
||||
if (delUid) this.deleteQuotaRule(delUid);
|
||||
}
|
||||
|
||||
async resetQuotaSpend(rule) {
|
||||
const label = this.scopeLabel(rule);
|
||||
if (!(await this.confirmAction(`Reset the counted 24h spend for ${label}? The usage history is kept.`, "Reset"))) return;
|
||||
try {
|
||||
await Http.postJson("/admin/gateway/quota-resets", {
|
||||
owner_kind: rule.owner_kind || "",
|
||||
owner_id: rule.owner_id || "",
|
||||
app_reference: rule.app_reference || "",
|
||||
});
|
||||
this.notify("Quota spend reset", "success");
|
||||
await this.reload();
|
||||
} catch (err) {
|
||||
this.notify(err.message || "Reset failed", "error");
|
||||
}
|
||||
}
|
||||
|
||||
async deleteQuotaRule(uid) {
|
||||
if (!(await this.confirmDelete("Delete this quota rule?"))) return;
|
||||
if (!(await this.confirmAction("Delete this quota rule?"))) return;
|
||||
try {
|
||||
await this.remove(`/admin/gateway/quota-rules/${encodeURIComponent(uid)}`);
|
||||
this.notify("Quota rule deleted", "success");
|
||||
|
||||
@ -42,7 +42,7 @@ Open `http://<host>:<PORT>`. Useful targets: `make docker-logs` (tail), `make do
|
||||
|
||||
The make targets always apply the `docker-compose.containers.yml` overlay, so the admin-only Container Manager works with no extra setup. Always update through them: a plain `docker compose up -d` drops the overlay and silently removes the docker socket and CLI the manager needs.
|
||||
|
||||
The healthcheck gives the app a 120s start window before any failure counts (`start_period: 120s` in `docker-compose.yml`). The full startup (DB init, background service registration, uvicorn workers) takes roughly 110s, so the start period must stay well above that. If you add heavyweight init work, verify the app boots within 120s or bump the start period to match.
|
||||
The healthcheck gives the app a 120s start window in which a failing probe does not count against `retries` (`start_period: 120s`), and probes every 2s inside that window (`start_interval: 2s`) so the container is marked healthy as soon as it actually serves rather than at the next 30s tick. Both settings live in `docker-compose.yml` and the `Dockerfile` and must be changed together. Normal startup is a few seconds; the 120s window is headroom for a cold page cache on a multi-GB database. Startup cost is paid once per uvicorn worker, serialized under an exclusive init lock, so anything added to `init_db` is multiplied by the worker count. If you add heavyweight init work, measure the real boot time before relying on the existing window.
|
||||
|
||||
## Updating
|
||||
|
||||
@ -50,15 +50,19 @@ Code is bind-mounted, so most updates need no rebuild:
|
||||
|
||||
```bash
|
||||
git pull
|
||||
make docker-up # restart workers on the new code
|
||||
make docker-reload # restart workers on the new code
|
||||
```
|
||||
|
||||
Use `make docker-reload`, not `make docker-up`: `docker compose up -d` sees an unchanged container and leaves it running, so the workers keep serving the code they imported at boot. `docker-reload` restarts the app and waits for it to report healthy again.
|
||||
|
||||
Rebuild the image only when dependencies change:
|
||||
|
||||
```bash
|
||||
make docker-build && make docker-up
|
||||
```
|
||||
|
||||
A rebuild after a source-only change takes about 7 seconds. Dependencies install from `pyproject.toml` in a layer that source edits cannot invalidate, so `pip install` and the Chromium download stay cached; only the source copy and the final project install re-run.
|
||||
|
||||
## Release branch
|
||||
|
||||
`make deploy` fast-forwards the release branch:
|
||||
|
||||
@ -42,10 +42,10 @@
|
||||
<div class="profile-level-bar">
|
||||
<div class="level-label">
|
||||
<span>Progress to next level</span>
|
||||
<span>{{ (profile_user.get('xp') or 0) % 100 }}%</span>
|
||||
<span>{{ xp_progress_pct }}%</span>
|
||||
</div>
|
||||
<div class="bar">
|
||||
<div class="bar-fill" style="--bar-pct: {{ (profile_user.get('xp') or 0) % 100 }}%;"></div>
|
||||
<div class="bar-fill" style="--bar-pct: {{ xp_progress_pct }}%;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -765,3 +765,6 @@ import { AwardGiver } from "{{ static_url('/static/js/AwardGiver.js') }}";
|
||||
new AwardGiver();
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
|
||||
|
||||
|
||||
@ -25,6 +25,7 @@ services:
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 120s
|
||||
start_interval: 2s
|
||||
|
||||
nginx:
|
||||
build:
|
||||
|
||||
482
events.md
Normal file
482
events.md
Normal file
@ -0,0 +1,482 @@
|
||||
# Audit event catalogue
|
||||
|
||||
retoor <retoor@molodetz.nl>
|
||||
|
||||
Every state-changing action in DevPlace records one append-only row through `devplacepy/services/audit/record.py` (`record` for a request-scoped actor, `record_system` for a background one). This file is the authoritative catalogue of the 288 event keys currently emitted, grouped by the category `services/audit/categories.py` `category_for` resolves them to.
|
||||
|
||||
## Rules
|
||||
|
||||
- An event key is `<domain>.<noun>.<verb>` (or `<domain>.<verb>` when the domain is the noun). The domain is the first segment and MUST have an entry in `CATEGORY_BY_PREFIX`; an unmapped domain resolves to category `other` and is a bug.
|
||||
- `target_type` names the entity acted upon, not the event (`gateway_quota`, not `gateway_quota_reset`).
|
||||
- Every row carries a result: `success`, `failure`, `denied`. A refused action is recorded with `denied`, never dropped.
|
||||
- Actor kinds: `user`, `guest`, `system`, `cli`, `service`. Origins: `web`, `api`, `devii`, `cli`, `service`, `scheduler`, `devrant`.
|
||||
- The recorder never raises into its caller; a failed audit write is logged and swallowed so it can never break the action it describes.
|
||||
- Some families are built from a variable at the call site (`vote.{target_type}.{direction}`, `comment.create.{target_type}`, `quiz.question.{action}`, `profile.customization.{scope}.toggle`, and the `create`/`edit`/`delete` verbs the shared `content.py` helpers emit per content type). Every member of those families is listed below individually.
|
||||
- Adding an event means all three of: the key in this file, a `CATEGORY_BY_PREFIX` entry for a new domain, and the recorder call at the mutation point.
|
||||
|
||||
## Categories
|
||||
|
||||
| Category | Keys |
|
||||
|---|---|
|
||||
| `account` | 10 |
|
||||
| `admin` | 21 |
|
||||
| `ai` | 10 |
|
||||
| `attachment` | 5 |
|
||||
| `auth` | 9 |
|
||||
| `backup` | 2 |
|
||||
| `cli` | 42 |
|
||||
| `container` | 12 |
|
||||
| `content` | 37 |
|
||||
| `database` | 4 |
|
||||
| `devii` | 18 |
|
||||
| `email` | 6 |
|
||||
| `engagement` | 23 |
|
||||
| `game` | 6 |
|
||||
| `ingress` | 1 |
|
||||
| `message` | 2 |
|
||||
| `news` | 9 |
|
||||
| `notification` | 4 |
|
||||
| `project` | 11 |
|
||||
| `project_files` | 14 |
|
||||
| `pubsub` | 1 |
|
||||
| `push` | 2 |
|
||||
| `reward` | 4 |
|
||||
| `security` | 3 |
|
||||
| `service` | 5 |
|
||||
| `social` | 10 |
|
||||
| `telegram` | 5 |
|
||||
| `tools` | 12 |
|
||||
|
||||
**Total: 288 keys.**
|
||||
|
||||
## Account and profile (`account`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `profile.ai_correction` | `routers/profile/ai_correction.py` |
|
||||
| `profile.ai_modifier` | `routers/profile/ai_modifier.py` |
|
||||
| `profile.api_key.regenerate` | `routers/profile/index.py` |
|
||||
| `profile.avatar.regenerate` | `routers/profile/avatar.py` |
|
||||
| `profile.customization.global.toggle` | `routers/profile/customization.py` |
|
||||
| `profile.customization.pagetype.toggle` | `routers/profile/customization.py` |
|
||||
| `profile.interactions` | `routers/profile/interactions.py`, `services/devii/actions/dispatcher.py` |
|
||||
| `profile.notification.reset` | `routers/profile/notifications.py` |
|
||||
| `profile.notification.toggle` | `routers/profile/notifications.py` |
|
||||
| `profile.update` | `routers/devrant/auth.py`, `routers/profile/index.py` |
|
||||
|
||||
## Administration (`admin`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `admin.ai_quota.reset_all` | `routers/admin/aiquota.py` |
|
||||
| `admin.ai_quota.reset_guests` | `routers/admin/aiquota.py` |
|
||||
| `admin.backup.delete` | `routers/admin/backups.py` |
|
||||
| `admin.backup.run` | `routers/admin/backups.py` |
|
||||
| `admin.backup_schedule.create` | `routers/admin/backups.py` |
|
||||
| `admin.backup_schedule.delete` | `routers/admin/backups.py` |
|
||||
| `admin.backup_schedule.toggle` | `routers/admin/backups.py` |
|
||||
| `admin.backup_schedule.update` | `routers/admin/backups.py` |
|
||||
| `admin.devii_task.delete` | `routers/admin/devii_tasks.py` |
|
||||
| `admin.devii_task.disable` | `routers/admin/devii_tasks.py` |
|
||||
| `admin.game.era_end` | `routers/admin/game.py` |
|
||||
| `admin.game.era_start` | `routers/admin/game.py` |
|
||||
| `admin.notification.default` | `routers/admin/notifications.py` |
|
||||
| `admin.setting.update` | `docs_api/groups/admin.py`, `routers/admin/settings.py` |
|
||||
| `admin.trash.purge` | `routers/admin/trash.py` |
|
||||
| `admin.trash.restore` | `routers/admin/trash.py` |
|
||||
| `admin.user.active.disable` | `routers/admin/users.py` |
|
||||
| `admin.user.active.enable` | `routers/admin/users.py` |
|
||||
| `admin.user.ai_quota.reset` | `routers/admin/users.py` |
|
||||
| `admin.user.password.reset` | `routers/admin/users.py` |
|
||||
| `admin.user.role.change` | `routers/admin/users.py` |
|
||||
|
||||
## AI gateway (`ai`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `ai.clippy.chat` | `routers/devii.py` |
|
||||
| `ai.gateway.call` | `services/openai_gateway/gateway.py`, `services/openai_gateway/usage.py` |
|
||||
| `ai.quota.exceeded` | `routers/devii.py`, `services/openai_gateway/service.py` |
|
||||
| `gateway.model.delete` | `routers/admin/gateway_configs.py` |
|
||||
| `gateway.model.update` | `routers/admin/gateway_configs.py` |
|
||||
| `gateway.provider.delete` | `routers/admin/gateway_configs.py` |
|
||||
| `gateway.provider.update` | `routers/admin/gateway_configs.py` |
|
||||
| `gateway.quota.reset` | `cli/gateway.py`, `routers/admin/gateway_configs.py` |
|
||||
| `gateway.quota_rule.delete` | `cli/gateway.py`, `routers/admin/gateway_configs.py` |
|
||||
| `gateway.quota_rule.update` | `cli/gateway.py`, `routers/admin/gateway_configs.py` |
|
||||
|
||||
## Attachments (`attachment`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `attachment.delete` | `routers/admin/media.py`, `routers/media.py` |
|
||||
| `attachment.rename` | `routers/uploads.py` |
|
||||
| `attachment.restore` | `routers/media.py` |
|
||||
| `attachment.upload` | `routers/uploads.py` |
|
||||
| `attachment.upload_url` | `routers/uploads.py` |
|
||||
|
||||
## Authentication (`auth`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `auth.account.disable` | `routers/devrant/auth.py` |
|
||||
| `auth.login.failure` | `routers/auth/login.py`, `routers/devrant/auth.py` |
|
||||
| `auth.login.success` | `routers/auth/login.py`, `routers/devrant/auth.py` |
|
||||
| `auth.logout` | `routers/auth/logout.py` |
|
||||
| `auth.password.forgot_request` | `routers/auth/forgotpassword.py` |
|
||||
| `auth.password.reset_complete` | `routers/auth/resetpassword.py` |
|
||||
| `auth.signup` | `routers/auth/signup.py`, `routers/devrant/auth.py` |
|
||||
| `auth.token.failure` | `routers/auth/token.py` |
|
||||
| `auth.token.issued` | `routers/auth/token.py` |
|
||||
|
||||
## Backups (`backup`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `job.backup.complete` | `services/backup/service.py` |
|
||||
| `job.backup.failed` | `services/backup/service.py` |
|
||||
|
||||
## Command line (`cli`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `cli.apikey.backfill` | `cli/apikeys.py` |
|
||||
| `cli.apikey.reset` | `cli/apikeys.py` |
|
||||
| `cli.attachments.prune` | `cli/attachments.py` |
|
||||
| `cli.backups.clear` | `cli/backups.py` |
|
||||
| `cli.backups.prune` | `cli/backups.py` |
|
||||
| `cli.backups.run` | `cli/backups.py` |
|
||||
| `cli.containers.gc_workspaces` | `cli/containers.py` |
|
||||
| `cli.containers.prune` | `cli/containers.py` |
|
||||
| `cli.containers.prune_builds` | `cli/containers.py` |
|
||||
| `cli.containers.reconcile` | `cli/containers.py` |
|
||||
| `cli.deepsearch.clear` | `cli/jobs.py` |
|
||||
| `cli.deepsearch.prune` | `cli/jobs.py` |
|
||||
| `cli.devii.lessons.clear` | `cli/devii.py` |
|
||||
| `cli.devii.lessons.prune` | `cli/devii.py` |
|
||||
| `cli.devii.quota.reset` | `cli/devii.py` |
|
||||
| `cli.devii.task.disable` | `cli/devii.py` |
|
||||
| `cli.devii.task.prune` | `cli/devii.py` |
|
||||
| `cli.emoji.sync` | `cli/migrate.py` |
|
||||
| `cli.forks.clear` | `cli/jobs.py` |
|
||||
| `cli.forks.prune` | `cli/jobs.py` |
|
||||
| `cli.game.era.end` | `cli/game.py` |
|
||||
| `cli.game.era.start` | `cli/game.py` |
|
||||
| `cli.game.market.prune` | `cli/game.py` |
|
||||
| `cli.game.steals.prune` | `cli/game.py` |
|
||||
| `cli.isslop.analyze` | `cli/jobs.py` |
|
||||
| `cli.isslop.clear` | `cli/jobs.py` |
|
||||
| `cli.isslop.prune` | `cli/jobs.py` |
|
||||
| `cli.messaging.prune_tickets` | `cli/messaging.py` |
|
||||
| `cli.news.clear` | `cli/news.py` |
|
||||
| `cli.news.sanitize` | `cli/news.py` |
|
||||
| `cli.quiz.prune` | `cli/quiz.py` |
|
||||
| `cli.role.set` | `cli/roles.py` |
|
||||
| `cli.seo.clear` | `cli/jobs.py` |
|
||||
| `cli.seo.prune` | `cli/jobs.py` |
|
||||
| `cli.seo_meta.clear` | `cli/jobs.py` |
|
||||
| `cli.seo_meta.prune` | `cli/jobs.py` |
|
||||
| `cli.token.issue` | `cli/tokens.py` |
|
||||
| `cli.token.prune` | `cli/tokens.py` |
|
||||
| `cli.token.revoke` | `cli/tokens.py` |
|
||||
| `cli.token.revoke_all` | `cli/tokens.py` |
|
||||
| `cli.zips.clear` | `cli/jobs.py` |
|
||||
| `cli.zips.prune` | `cli/jobs.py` |
|
||||
|
||||
## Containers (`container`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `container.instance.configure` | `routers/admin/containers.py`, `services/devii/actions/dispatcher.py` |
|
||||
| `container.instance.create` | `routers/admin/containers.py`, `routers/projects/containers/instances.py` |
|
||||
| `container.instance.delete` | `routers/admin/containers.py`, `routers/projects/containers/instances.py` |
|
||||
| `container.instance.exec` | `routers/projects/containers/instances.py`, `services/devii/actions/dispatcher.py` |
|
||||
| `container.instance.shell.close` | `routers/projects/containers/instances.py` |
|
||||
| `container.instance.shell.open` | `routers/projects/containers/instances.py` |
|
||||
| `container.instance.start` | `docs_api/groups/admin.py` |
|
||||
| `container.instance.status` | `services/containers/service.py` |
|
||||
| `container.instance.sync` | `routers/admin/containers.py`, `routers/projects/containers/instances.py` |
|
||||
| `container.reconcile.action` | `services/containers/service.py` |
|
||||
| `container.schedule.create` | `routers/projects/containers/schedules.py`, `services/devii/actions/dispatcher.py` |
|
||||
| `container.schedule.delete` | `routers/projects/containers/schedules.py` |
|
||||
|
||||
## Content (`content`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `comment.create.gist` | `content.py` |
|
||||
| `comment.create.issue` | `content.py` |
|
||||
| `comment.create.news` | `content.py` |
|
||||
| `comment.create.post` | `content.py` |
|
||||
| `comment.create.project` | `content.py` |
|
||||
| `comment.create.quiz` | `content.py` |
|
||||
| `comment.delete` | `content.py`, `routers/comments.py` |
|
||||
| `comment.edit` | `content.py`, `routers/comments.py` |
|
||||
| `gist.create` | `content.py` |
|
||||
| `gist.delete` | `content.py` |
|
||||
| `gist.edit` | `content.py` |
|
||||
| `issue.attachment.add` | `routers/issues/attachments.py` |
|
||||
| `issue.attachment.delete` | `routers/issues/attachments.py` |
|
||||
| `issue.comment` | `routers/issues/comment.py` |
|
||||
| `issue.create` | `services/jobs/issue_create_service.py` |
|
||||
| `issue.create.request` | `routers/issues/create.py` |
|
||||
| `issue.planning.generate` | `services/jobs/planning_service.py` |
|
||||
| `issue.planning.request` | `routers/issues/planning.py` |
|
||||
| `issue.status` | `routers/issues/status.py` |
|
||||
| `issue.sync.reply` | `services/gitea/service.py` |
|
||||
| `issue.sync.status` | `services/gitea/service.py` |
|
||||
| `post.create` | `content.py` |
|
||||
| `post.delete` | `content.py` |
|
||||
| `post.edit` | `content.py` |
|
||||
| `quiz.attempt.answer` | `routers/quizzes/attempts.py` |
|
||||
| `quiz.attempt.finish` | `routers/quizzes/attempts.py` |
|
||||
| `quiz.attempt.start` | `routers/quizzes/attempts.py` |
|
||||
| `quiz.create` | `content.py` |
|
||||
| `quiz.delete` | `content.py` |
|
||||
| `quiz.edit` | `content.py` |
|
||||
| `quiz.grade.failed` | `routers/quizzes/attempts.py` |
|
||||
| `quiz.import` | `routers/quizzes/index.py` |
|
||||
| `quiz.publish` | `routers/quizzes/index.py` |
|
||||
| `quiz.question.create` | `routers/quizzes/questions.py` |
|
||||
| `quiz.question.delete` | `routers/quizzes/questions.py` |
|
||||
| `quiz.question.edit` | `routers/quizzes/questions.py` |
|
||||
| `quiz.question.reorder` | `routers/quizzes/questions.py` |
|
||||
|
||||
## Database API (`database`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `database.access.denied` | `routers/dbapi/_shared.py` |
|
||||
| `database.nl.design` | `routers/dbapi/nl.py` |
|
||||
| `database.query` | `routers/dbapi/query.py` |
|
||||
| `database.query.async` | `routers/dbapi/query.py` |
|
||||
|
||||
## Devii assistant (`devii`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `devii.behavior.update` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.customization.css.set` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.customization.js.set` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.customization.reset` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.lesson.forget` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.lesson.reflect` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.notification.reset` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.notification.set` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.task.blocked` | `services/devii/tasks/scheduler.py` |
|
||||
| `devii.task.create` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.task.deferred` | `services/devii/tasks/scheduler.py` |
|
||||
| `devii.task.delete` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.task.execute` | `services/devii/tasks/scheduler.py` |
|
||||
| `devii.task.update` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.tool.create` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.tool.delete` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.tool.update` | `services/devii/actions/dispatcher.py` |
|
||||
| `devii.turn` | `services/devii/session/core.py` |
|
||||
|
||||
## Email (`email`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `email.account.delete` | `services/devii/actions/dispatcher.py` |
|
||||
| `email.account.set` | `services/devii/actions/dispatcher.py` |
|
||||
| `email.message.delete` | `services/devii/actions/dispatcher.py` |
|
||||
| `email.message.flag` | `services/devii/actions/dispatcher.py` |
|
||||
| `email.message.move` | `services/devii/actions/dispatcher.py` |
|
||||
| `email.send` | `services/devii/actions/dispatcher.py` |
|
||||
|
||||
## Engagement (`engagement`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `bookmark.add` | `content.py`, `routers/bookmarks.py` |
|
||||
| `bookmark.remove` | `content.py`, `routers/bookmarks.py` |
|
||||
| `poll.create` | `routers/posts.py` |
|
||||
| `poll.vote.cast` | `routers/polls.py` |
|
||||
| `poll.vote.change` | `routers/polls.py` |
|
||||
| `poll.vote.clear` | `routers/polls.py` |
|
||||
| `reaction.add` | `routers/reactions.py` |
|
||||
| `reaction.remove` | `routers/reactions.py` |
|
||||
| `vote.comment.clear` | `content.py` |
|
||||
| `vote.comment.down` | `content.py` |
|
||||
| `vote.comment.up` | `content.py` |
|
||||
| `vote.gist.clear` | `content.py` |
|
||||
| `vote.gist.down` | `content.py` |
|
||||
| `vote.gist.up` | `content.py` |
|
||||
| `vote.post.clear` | `content.py` |
|
||||
| `vote.post.down` | `content.py` |
|
||||
| `vote.post.up` | `content.py` |
|
||||
| `vote.project.clear` | `content.py` |
|
||||
| `vote.project.down` | `content.py` |
|
||||
| `vote.project.up` | `content.py` |
|
||||
| `vote.quiz.clear` | `content.py` |
|
||||
| `vote.quiz.down` | `content.py` |
|
||||
| `vote.quiz.up` | `content.py` |
|
||||
|
||||
## Code Farm (`game`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `game.cosmetic.buy` | `routers/game/index.py` |
|
||||
| `game.defense.downgrade` | `routers/game/index.py` |
|
||||
| `game.defense.upgrade` | `routers/game/index.py` |
|
||||
| `game.grant.claim` | `routers/game/index.py` |
|
||||
| `game.infrastructure.buy` | `routers/game/index.py` |
|
||||
| `game.prestige` | `routers/game/index.py` |
|
||||
|
||||
## Container ingress (`ingress`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `proxy.access` | `routers/proxy.py` |
|
||||
|
||||
## Direct messages (`message`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `message.read_on_view` | `routers/messages.py` |
|
||||
| `message.send` | `services/messaging/persist.py` |
|
||||
|
||||
## News (`news`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `news.delete` | `routers/admin/news.py` |
|
||||
| `news.featured.toggle` | `routers/admin/news.py` |
|
||||
| `news.landing.toggle` | `routers/admin/news.py` |
|
||||
| `news.publish.toggle` | `routers/admin/news.py` |
|
||||
| `news.service.draft` | `services/news/service.py` |
|
||||
| `news.service.ingest` | `services/news/service.py` |
|
||||
| `news.service.landing` | `services/news/service.py` |
|
||||
| `news.service.publish` | `services/news/service.py` |
|
||||
| `news.service.reject` | `services/news/service.py` |
|
||||
|
||||
## Notifications (`notification`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `notification.create` | `utils/notifications.py` |
|
||||
| `notification.open` | `routers/notifications.py` |
|
||||
| `notification.read.all` | `routers/devrant/notifs.py`, `routers/notifications.py` |
|
||||
| `notification.read.one` | `routers/notifications.py` |
|
||||
|
||||
## Projects (`project`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `project.create` | `content.py` |
|
||||
| `project.delete` | `content.py` |
|
||||
| `project.edit` | `content.py` |
|
||||
| `project.fork.complete` | `services/jobs/fork_service.py` |
|
||||
| `project.fork.failed` | `services/jobs/fork_service.py` |
|
||||
| `project.fork.request` | `routers/projects/index.py` |
|
||||
| `project.readonly.disable` | `routers/projects/index.py` |
|
||||
| `project.readonly.enable` | `routers/projects/index.py` |
|
||||
| `project.visibility.private` | `routers/projects/index.py` |
|
||||
| `project.visibility.public` | `routers/projects/index.py` |
|
||||
| `project.zip.request` | `routers/projects/index.py` |
|
||||
|
||||
## Project files (`project_files`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `dir.create` | `routers/projects/files.py` |
|
||||
| `file.append` | `routers/projects/files.py` |
|
||||
| `file.delete` | `routers/projects/files.py` |
|
||||
| `file.delete_lines` | `routers/projects/files.py` |
|
||||
| `file.insert_lines` | `routers/projects/files.py` |
|
||||
| `file.move` | `routers/projects/files.py` |
|
||||
| `file.replace_lines` | `routers/projects/files.py` |
|
||||
| `file.upload` | `routers/projects/files.py` |
|
||||
| `file.write` | `routers/projects/files.py` |
|
||||
| `file.write.create` | `routers/projects/files.py` |
|
||||
| `file.write.overwrite` | `routers/projects/files.py` |
|
||||
| `files.zip.request` | `routers/projects/files.py` |
|
||||
| `job.zip.complete` | `services/jobs/zip_service.py` |
|
||||
| `job.zip.failed` | `services/jobs/zip_service.py` |
|
||||
|
||||
## Pub/sub (`pubsub`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `pubsub.publish` | `routers/pubsub.py` |
|
||||
|
||||
## Web push (`push`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `push.subscribe` | `routers/push.py` |
|
||||
| `push.update` | `routers/push.py` |
|
||||
|
||||
## Gamification (`reward`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `reward.badge.award` | `utils/badges.py` |
|
||||
| `reward.level.up` | `utils/rewards.py` |
|
||||
| `reward.streak.milestone` | `utils/rewards.py` |
|
||||
| `reward.xp.grant` | `utils/rewards.py` |
|
||||
|
||||
## Security (`security`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `security.authz.denied` | `routers/admin/backups.py`, `services/devii/actions/dispatcher.py` |
|
||||
| `security.maintenance.block` | `main.py` |
|
||||
| `security.rate_limit.block` | `main.py` |
|
||||
|
||||
## Background services (`service`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `service.config.update` | `routers/admin/services.py` |
|
||||
| `service.logs.clear` | `routers/admin/services.py` |
|
||||
| `service.run_now` | `routers/admin/services.py` |
|
||||
| `service.start` | `routers/admin/services.py` |
|
||||
| `service.stop` | `routers/admin/services.py` |
|
||||
|
||||
## Social graph (`social`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `award.complete` | `services/jobs/award_service.py` |
|
||||
| `award.failed` | `services/jobs/award_service.py` |
|
||||
| `award.give` | `routers/profile/award.py` |
|
||||
| `award.revoke` | `routers/admin/awards.py` |
|
||||
| `follow.follow` | `routers/follow.py` |
|
||||
| `follow.unfollow` | `routers/follow.py` |
|
||||
| `relation.block` | `routers/relations.py` |
|
||||
| `relation.mute` | `routers/relations.py` |
|
||||
| `relation.unblock` | `routers/relations.py` |
|
||||
| `relation.unmute` | `routers/relations.py` |
|
||||
|
||||
## Telegram (`telegram`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `telegram.pair.failure` | `services/telegram/bridge.py` |
|
||||
| `telegram.pair.request` | `routers/profile/telegram.py` |
|
||||
| `telegram.pair.success` | `services/telegram/bridge.py` |
|
||||
| `telegram.send` | `services/devii/actions/dispatcher.py` |
|
||||
| `telegram.unpair` | `routers/profile/telegram.py` |
|
||||
|
||||
## Developer tools (`tools`)
|
||||
|
||||
| Event key | Recorded in |
|
||||
|---|---|
|
||||
| `deepsearch.chat` | `routers/tools/deepsearch.py` |
|
||||
| `deepsearch.run.complete` | `services/jobs/deepsearch/service.py` |
|
||||
| `deepsearch.run.failed` | `services/jobs/deepsearch/service.py` |
|
||||
| `deepsearch.run.request` | `routers/tools/deepsearch.py` |
|
||||
| `isslop.run.complete` | `services/jobs/isslop/service.py` |
|
||||
| `isslop.run.failed` | `services/jobs/isslop/service.py` |
|
||||
| `isslop.run.request` | `routers/tools/isslop.py` |
|
||||
| `seo.meta.failed` | `services/jobs/seo_meta_service.py` |
|
||||
| `seo.meta.generate` | `services/jobs/seo_meta_service.py` |
|
||||
| `seo.run.complete` | `services/jobs/seo/service.py` |
|
||||
| `seo.run.failed` | `services/jobs/seo/service.py` |
|
||||
| `seo.run.request` | `routers/tools/seo.py` |
|
||||
|
||||
## Mapped domains with no recorder
|
||||
|
||||
The `backup` prefix has a `CATEGORY_BY_PREFIX` entry but no call site emits a bare `backup.*` key - backup activity is recorded as `admin.backup*` (operator actions) and `job.backup.*` (worker outcomes), both of which resolve to the `backup` category through their own prefixes. Keep the mapping: it is what routes `job.backup.*` correctly.
|
||||
78
tests/api/admin/aiquota/resetall.py
Normal file
78
tests/api/admin/aiquota/resetall.py
Normal file
@ -0,0 +1,78 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import requests
|
||||
|
||||
from devplacepy.database import get_table, refresh_snapshot
|
||||
from devplacepy.services.openai_gateway.usage import GATEWAY_LEDGER
|
||||
from devplacepy.utils import generate_uid
|
||||
from tests.api.admin.gateway.index import JSON_gateway, admin_session
|
||||
from tests.conftest import BASE_URL
|
||||
|
||||
RESET_ALL_URL = f"{BASE_URL}/admin/ai-quota/reset-all"
|
||||
|
||||
_counter = [0]
|
||||
|
||||
|
||||
def _owner():
|
||||
_counter[0] += 1
|
||||
return f"resetall{_counter[0]}-{generate_uid()}"
|
||||
|
||||
|
||||
def _burn(owner_id, cost):
|
||||
refresh_snapshot()
|
||||
get_table(GATEWAY_LEDGER).insert(
|
||||
{
|
||||
"uid": generate_uid(),
|
||||
"owner_kind": "user",
|
||||
"owner_id": owner_id,
|
||||
"app_reference": "typosaurus",
|
||||
"cost_usd": cost,
|
||||
"created_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
)
|
||||
refresh_snapshot()
|
||||
|
||||
|
||||
def _spent(owner_id):
|
||||
from devplacepy.services.openai_gateway import quota
|
||||
|
||||
refresh_snapshot()
|
||||
quota._QUOTA_CACHE.clear()
|
||||
return quota.spent_24h("user", owner_id, "typosaurus")
|
||||
|
||||
|
||||
def test_reset_all_clears_the_gateway_spend(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, 3.0)
|
||||
assert _spent(owner) == 3.0
|
||||
response = admin_session(seeded_db).post(RESET_ALL_URL, allow_redirects=False)
|
||||
assert response.status_code == 200
|
||||
assert _spent(owner) == 0.0
|
||||
|
||||
|
||||
def test_reset_all_keeps_the_gateway_usage_history(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, 3.0)
|
||||
before = get_table(GATEWAY_LEDGER).count(owner_id=owner)
|
||||
admin_session(seeded_db).post(RESET_ALL_URL, allow_redirects=False)
|
||||
refresh_snapshot()
|
||||
assert get_table(GATEWAY_LEDGER).count(owner_id=owner) == before
|
||||
|
||||
|
||||
def test_gateway_spend_after_reset_all_counts_again(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, 3.0)
|
||||
admin_session(seeded_db).post(RESET_ALL_URL, allow_redirects=False)
|
||||
_burn(owner, 0.25)
|
||||
assert _spent(owner) == 0.25
|
||||
|
||||
|
||||
def test_reset_all_requires_admin(seeded_db):
|
||||
assert (
|
||||
requests.post(
|
||||
RESET_ALL_URL, headers=JSON_gateway, allow_redirects=False
|
||||
).status_code
|
||||
== 401
|
||||
)
|
||||
162
tests/api/admin/gateway/quota.py
Normal file
162
tests/api/admin/gateway/quota.py
Normal file
@ -0,0 +1,162 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import requests
|
||||
|
||||
from devplacepy.database import get_table, refresh_snapshot
|
||||
from devplacepy.services.openai_gateway.usage import GATEWAY_LEDGER
|
||||
from devplacepy.utils import generate_uid
|
||||
from tests.api.admin.gateway.index import (
|
||||
JSON_gateway,
|
||||
admin_session,
|
||||
member_key,
|
||||
)
|
||||
from tests.conftest import BASE_URL
|
||||
|
||||
RESETS_URL = f"{BASE_URL}/admin/gateway/quota-resets"
|
||||
|
||||
_counter = [0]
|
||||
|
||||
|
||||
def _owner():
|
||||
_counter[0] += 1
|
||||
return f"gwquota{_counter[0]}-{generate_uid()}"
|
||||
|
||||
|
||||
def _burn(owner_id, app_reference, cost):
|
||||
refresh_snapshot()
|
||||
get_table(GATEWAY_LEDGER).insert(
|
||||
{
|
||||
"uid": generate_uid(),
|
||||
"owner_kind": "user",
|
||||
"owner_id": owner_id,
|
||||
"app_reference": app_reference,
|
||||
"cost_usd": cost,
|
||||
"created_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
)
|
||||
refresh_snapshot()
|
||||
|
||||
|
||||
def _spent(owner_id, app_reference):
|
||||
from devplacepy.services.openai_gateway import quota
|
||||
|
||||
refresh_snapshot()
|
||||
quota._QUOTA_CACHE.clear()
|
||||
return quota.spent_24h("user", owner_id, app_reference)
|
||||
|
||||
|
||||
def test_quota_reset_requires_admin(seeded_db):
|
||||
assert (
|
||||
requests.post(RESETS_URL, headers=JSON_gateway, allow_redirects=False).status_code
|
||||
== 401
|
||||
)
|
||||
key = member_key()
|
||||
assert (
|
||||
requests.post(
|
||||
RESETS_URL,
|
||||
headers={**JSON_gateway, "X-API-KEY": key},
|
||||
json={},
|
||||
allow_redirects=False,
|
||||
).status_code
|
||||
== 403
|
||||
)
|
||||
|
||||
|
||||
def test_admin_can_reset_a_scoped_spend(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 2.5)
|
||||
assert _spent(owner, "appa") == 2.5
|
||||
response = admin_session(seeded_db).post(
|
||||
RESETS_URL,
|
||||
json={"owner_kind": "user", "owner_id": owner, "app_reference": "appa"},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json()["ok"] is True
|
||||
assert _spent(owner, "appa") == 0.0
|
||||
|
||||
|
||||
def test_the_reset_response_echoes_the_scope(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.0)
|
||||
payload = admin_session(seeded_db).post(
|
||||
RESETS_URL,
|
||||
json={"owner_kind": "user", "owner_id": owner, "app_reference": "appa"},
|
||||
).json()["reset"]
|
||||
assert payload["owner_kind"] == "user"
|
||||
assert payload["owner_id"] == owner
|
||||
assert payload["app_reference"] == "appa"
|
||||
assert payload["reset_at"]
|
||||
|
||||
|
||||
def test_a_reset_leaves_another_caller_capped(seeded_db):
|
||||
first, second = _owner(), _owner()
|
||||
_burn(first, "appa", 2.0)
|
||||
_burn(second, "appa", 2.0)
|
||||
admin_session(seeded_db).post(
|
||||
RESETS_URL,
|
||||
json={"owner_kind": "user", "owner_id": first, "app_reference": "appa"},
|
||||
)
|
||||
assert _spent(first, "appa") == 0.0
|
||||
assert _spent(second, "appa") == 2.0
|
||||
|
||||
|
||||
def test_a_reset_keeps_the_usage_history(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 2.0)
|
||||
before = get_table(GATEWAY_LEDGER).count(owner_id=owner)
|
||||
admin_session(seeded_db).post(
|
||||
RESETS_URL,
|
||||
json={"owner_kind": "user", "owner_id": owner, "app_reference": "appa"},
|
||||
)
|
||||
refresh_snapshot()
|
||||
assert get_table(GATEWAY_LEDGER).count(owner_id=owner) == before
|
||||
|
||||
|
||||
def test_an_unscoped_reset_clears_every_caller(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 2.0)
|
||||
assert admin_session(seeded_db).post(RESETS_URL, json={}).status_code == 200
|
||||
assert _spent(owner, "appa") == 0.0
|
||||
|
||||
|
||||
def test_an_unknown_owner_kind_is_rejected(seeded_db):
|
||||
response = admin_session(seeded_db).post(RESETS_URL, json={"owner_kind": "wizard"})
|
||||
assert response.status_code == 400
|
||||
assert response.json()["ok"] is False
|
||||
|
||||
|
||||
def test_a_malformed_app_reference_is_rejected(seeded_db):
|
||||
response = admin_session(seeded_db).post(
|
||||
RESETS_URL, json={"app_reference": "not a valid app!"}
|
||||
)
|
||||
assert response.status_code == 400
|
||||
|
||||
|
||||
def test_the_reset_is_audited(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.0)
|
||||
admin = admin_session(seeded_db)
|
||||
admin.post(
|
||||
RESETS_URL,
|
||||
json={"owner_kind": "user", "owner_id": owner, "app_reference": "appa"},
|
||||
)
|
||||
refresh_snapshot()
|
||||
entries = admin.get(
|
||||
f"{BASE_URL}/admin/audit-log",
|
||||
headers=JSON_gateway,
|
||||
params={"event_key": "gateway.quota.reset"},
|
||||
).json()["entries"]
|
||||
assert any(entry.get("target_type") == "gateway_quota" for entry in entries)
|
||||
|
||||
|
||||
def test_spend_recorded_after_a_reset_counts_again(seeded_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 2.0)
|
||||
admin_session(seeded_db).post(
|
||||
RESETS_URL,
|
||||
json={"owner_kind": "user", "owner_id": owner, "app_reference": "appa"},
|
||||
)
|
||||
_burn(owner, "appa", 0.5)
|
||||
assert _spent(owner, "appa") == 0.5
|
||||
@ -234,7 +234,6 @@ def test_activity_comment_url_matches_notification_target(app_server):
|
||||
|
||||
|
||||
def test_activity_post_exposes_url(app_server):
|
||||
from datetime import datetime, timezone
|
||||
from devplacepy.database import get_table, refresh_snapshot
|
||||
|
||||
owner = _seed_owner()
|
||||
@ -407,3 +406,96 @@ def test_viewing_profile_marks_notification_read(app_server):
|
||||
|
||||
refresh_snapshot()
|
||||
assert bool(get_table("notifications").find_one(uid=notif_uid)["read"]) is True
|
||||
|
||||
|
||||
def test_own_profile_json_exposes_xp_fields(app_server, seeded_db):
|
||||
"""GET /profile (own) with Accept: application/json includes xp_next_level and xp_progress_pct."""
|
||||
import requests
|
||||
|
||||
session = requests.Session()
|
||||
session.post(
|
||||
f"{BASE_URL}/auth/login",
|
||||
data={
|
||||
"email": seeded_db["alice"]["email"],
|
||||
"password": seeded_db["alice"]["password"],
|
||||
},
|
||||
allow_redirects=True,
|
||||
)
|
||||
r = session.get(f"{BASE_URL}/profile", headers={"Accept": "application/json"})
|
||||
assert r.status_code == 200
|
||||
data = r.json()
|
||||
pu = data["profile_user"]
|
||||
assert "xp_next_level" in pu, "xp_next_level missing from own profile JSON"
|
||||
assert "xp_progress_pct" in pu, "xp_progress_pct missing from own profile JSON"
|
||||
assert isinstance(pu["xp_next_level"], int)
|
||||
assert isinstance(pu["xp_progress_pct"], int)
|
||||
|
||||
|
||||
def test_other_profile_json_exposes_xp_fields(app_server):
|
||||
"""GET /profile/{username} with Accept: application/json includes xp_next_level and xp_progress_pct."""
|
||||
from devplacepy.database import get_table, refresh_snapshot
|
||||
|
||||
owner = _seed_owner()
|
||||
username = get_table("users").find_one(uid=owner)["username"]
|
||||
refresh_snapshot()
|
||||
|
||||
r = requests.get(
|
||||
f"{BASE_URL}/profile/{username}", headers={"Accept": "application/json"}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
data = r.json()
|
||||
pu = data["profile_user"]
|
||||
assert "xp_next_level" in pu, "xp_next_level missing from other profile JSON"
|
||||
assert "xp_progress_pct" in pu, "xp_progress_pct missing from other profile JSON"
|
||||
assert isinstance(pu["xp_next_level"], int)
|
||||
assert isinstance(pu["xp_progress_pct"], int)
|
||||
|
||||
|
||||
def test_profile_json_xp_fields_zero_xp(app_server):
|
||||
"""User with 0 XP returns xp_next_level=100 (level 1) and xp_progress_pct=0."""
|
||||
from devplacepy.database import get_table, refresh_snapshot
|
||||
from devplacepy.utils.rewards import LEVEL_XP
|
||||
|
||||
owner = _seed_owner()
|
||||
username = get_table("users").find_one(uid=owner)["username"]
|
||||
refresh_snapshot()
|
||||
|
||||
r = requests.get(
|
||||
f"{BASE_URL}/profile/{username}", headers={"Accept": "application/json"}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
pu = r.json()["profile_user"]
|
||||
assert pu["xp"] == 0 or pu["xp"] is None, f"expected 0 xp, got {pu['xp']}"
|
||||
assert pu["xp_next_level"] == LEVEL_XP, (
|
||||
f"expected xp_next_level={LEVEL_XP} for level 1, got {pu['xp_next_level']}"
|
||||
)
|
||||
assert pu["xp_progress_pct"] == 0, (
|
||||
f"expected xp_progress_pct=0 for 0 XP, got {pu['xp_progress_pct']}"
|
||||
)
|
||||
|
||||
|
||||
def test_profile_json_xp_fields_boundary_xp(app_server):
|
||||
"""User with exactly 100 XP (level 2) returns xp_next_level=200 and xp_progress_pct=0."""
|
||||
from devplacepy.database import get_table, refresh_snapshot
|
||||
from devplacepy.utils.rewards import award_xp
|
||||
|
||||
owner = _seed_owner()
|
||||
username = get_table("users").find_one(uid=owner)["username"]
|
||||
award_xp(owner, 100)
|
||||
refresh_snapshot()
|
||||
|
||||
r = requests.get(
|
||||
f"{BASE_URL}/profile/{username}", headers={"Accept": "application/json"}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
pu = r.json()["profile_user"]
|
||||
assert pu["xp"] == 100, f"expected 100 xp, got {pu['xp']}"
|
||||
assert pu["level"] == 2, f"expected level 2, got {pu['level']}"
|
||||
assert pu["xp_next_level"] == 200, (
|
||||
f"expected xp_next_level=200 for level 2, got {pu['xp_next_level']}"
|
||||
)
|
||||
assert pu["xp_progress_pct"] == 0, (
|
||||
f"expected xp_progress_pct=0 at boundary, got {pu['xp_progress_pct']}"
|
||||
)
|
||||
|
||||
|
||||
|
||||
@ -101,7 +101,7 @@ def _create_post_direct(project_uid, user_uid, order, marker=None):
|
||||
"project_uid": project_uid,
|
||||
"image": None,
|
||||
"stars": 0,
|
||||
"created_at": (datetime(2026, 1, 1, tzinfo=timezone.utc) - timedelta(seconds=order)).isoformat(),
|
||||
"created_at": (datetime(2026, 1, 1, tzinfo=timezone.utc) + timedelta(seconds=order)).isoformat(),
|
||||
}
|
||||
)
|
||||
refresh_snapshot()
|
||||
|
||||
@ -60,7 +60,7 @@ def _seed_project_post(project_uid, user_uid, order, title=None):
|
||||
"project_uid": project_uid,
|
||||
"image": None,
|
||||
"stars": 0,
|
||||
"created_at": (datetime(2026, 1, 1, tzinfo=timezone.utc) - timedelta(seconds=order)).isoformat(),
|
||||
"created_at": (datetime(2026, 1, 1, tzinfo=timezone.utc) + timedelta(seconds=order)).isoformat(),
|
||||
}
|
||||
)
|
||||
return marker, uid
|
||||
@ -129,7 +129,9 @@ def test_devlog_shows_vote_and_comment_buttons(alice):
|
||||
page.goto(f"{BASE_URL}/projects/{slug}", wait_until="domcontentloaded")
|
||||
|
||||
post_card = page.locator(".project-devlog .post-card").first
|
||||
expect(post_card.locator(".post-action-btn")).to_be_visible()
|
||||
expect(post_card.locator(".post-action-btn.vote-up")).to_be_visible()
|
||||
expect(post_card.locator(".post-action-btn.vote-down")).to_be_visible()
|
||||
expect(post_card.locator(".post-vote-count")).to_be_visible()
|
||||
expect(post_card.locator("form[action*='/posts/delete/']")).to_be_visible()
|
||||
|
||||
|
||||
|
||||
@ -9,7 +9,7 @@ def test_primary_admin_ignores_accounts_without_a_signup_time(local_db):
|
||||
"uid": "pa-real",
|
||||
"username": "pa-real",
|
||||
"role": "Admin",
|
||||
"created_at": "2020-01-01T00:00:00",
|
||||
"created_at": "1990-01-01T00:00:00",
|
||||
"deleted_at": None,
|
||||
}
|
||||
)
|
||||
@ -45,9 +45,9 @@ def test_primary_admin_skips_deactivated_and_deleted_founders(local_db):
|
||||
|
||||
users = local_db["users"]
|
||||
seeded = [
|
||||
("pa-deleted", "2001-01-01T00:00:00", True, "2020-01-01T00:00:00"),
|
||||
("pa-inactive", "2002-01-01T00:00:00", False, None),
|
||||
("pa-usable", "2003-01-01T00:00:00", True, None),
|
||||
("pa-deleted", "1991-01-01T00:00:00", True, "2020-01-01T00:00:00"),
|
||||
("pa-inactive", "1992-01-01T00:00:00", False, None),
|
||||
("pa-usable", "1993-01-01T00:00:00", True, None),
|
||||
]
|
||||
for uid, created_at, active, deleted_at in seeded:
|
||||
users.insert(
|
||||
|
||||
164
tests/unit/services/openai_gateway/quota.py
Normal file
164
tests/unit/services/openai_gateway/quota.py
Normal file
@ -0,0 +1,164 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.services.openai_gateway import quota as q
|
||||
from devplacepy.services.openai_gateway.usage import GATEWAY_LEDGER
|
||||
from devplacepy.utils import generate_uid
|
||||
|
||||
_counter = [0]
|
||||
|
||||
|
||||
def _owner():
|
||||
_counter[0] += 1
|
||||
return f"quotauser{_counter[0]}-{generate_uid()}"
|
||||
|
||||
|
||||
def _burn(owner_id, app_reference, cost, minutes_ago=0):
|
||||
stamp = datetime.now(timezone.utc) - timedelta(minutes=minutes_ago)
|
||||
get_table(GATEWAY_LEDGER).insert(
|
||||
{
|
||||
"uid": generate_uid(),
|
||||
"owner_kind": "user",
|
||||
"owner_id": owner_id,
|
||||
"app_reference": app_reference,
|
||||
"cost_usd": cost,
|
||||
"created_at": stamp.isoformat(),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_reset_requires_no_dimension(local_db):
|
||||
assert q.QuotaResetIn().owner_kind is None
|
||||
assert q.QuotaResetIn().owner_id is None
|
||||
assert q.QuotaResetIn().app_reference is None
|
||||
|
||||
|
||||
def test_reset_rejects_an_unknown_owner_kind(local_db):
|
||||
with pytest.raises(ValidationError):
|
||||
q.QuotaResetIn(owner_kind="wizard")
|
||||
|
||||
|
||||
def test_reset_rejects_a_malformed_app_reference(local_db):
|
||||
with pytest.raises(ValidationError):
|
||||
q.QuotaResetIn(app_reference="not a valid app!")
|
||||
|
||||
|
||||
def test_reset_normalizes_blanks_to_wildcards(local_db):
|
||||
payload = q.QuotaResetIn(owner_kind="", owner_id=" ", app_reference="")
|
||||
assert payload.owner_kind is None
|
||||
assert payload.owner_id is None
|
||||
assert payload.app_reference is None
|
||||
|
||||
|
||||
def test_spend_counts_before_any_reset(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
assert q.spent_24h("user", owner, "appa") == 1.5
|
||||
|
||||
|
||||
def test_a_scoped_reset_clears_that_scope(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=owner, app_reference="appa"))
|
||||
assert q.spent_24h("user", owner, "appa") == 0.0
|
||||
|
||||
|
||||
def test_a_scoped_reset_leaves_another_caller_alone(local_db):
|
||||
first, second = _owner(), _owner()
|
||||
_burn(first, "appa", 1.5)
|
||||
_burn(second, "appa", 2.0)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=first, app_reference="appa"))
|
||||
assert q.spent_24h("user", first, "appa") == 0.0
|
||||
assert q.spent_24h("user", second, "appa") == 2.0
|
||||
|
||||
|
||||
def test_a_scoped_reset_leaves_another_app_alone(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
_burn(owner, "appb", 2.0)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=owner, app_reference="appa"))
|
||||
assert q.spent_24h("user", owner, "appa") == 0.0
|
||||
assert q.spent_24h("user", owner, "appb") == 2.0
|
||||
|
||||
|
||||
def test_a_global_reset_clears_every_scope(local_db):
|
||||
first, second = _owner(), _owner()
|
||||
_burn(first, "appa", 1.5)
|
||||
_burn(second, "appb", 2.0)
|
||||
q.reset(q.QuotaResetIn())
|
||||
assert q.spent_24h("user", first, "appa") == 0.0
|
||||
assert q.spent_24h("user", second, "appb") == 0.0
|
||||
|
||||
|
||||
def test_spend_after_a_reset_counts_again(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=owner, app_reference="appa"))
|
||||
_burn(owner, "appa", 0.75)
|
||||
assert q.spent_24h("user", owner, "appa") == 0.75
|
||||
|
||||
|
||||
def test_a_reset_keeps_the_usage_history(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
before = get_table(GATEWAY_LEDGER).count(owner_id=owner)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=owner, app_reference="appa"))
|
||||
assert get_table(GATEWAY_LEDGER).count(owner_id=owner) == before
|
||||
|
||||
|
||||
def test_a_narrower_reset_does_not_clear_a_broader_scope(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=owner, app_reference="appa"))
|
||||
assert q.spent_24h("user", owner, None) == 1.5
|
||||
|
||||
|
||||
def test_a_broader_reset_clears_a_narrower_scope(local_db):
|
||||
owner = _owner()
|
||||
_burn(owner, "appa", 1.5)
|
||||
q.reset(q.QuotaResetIn(owner_kind="user", owner_id=owner))
|
||||
assert q.spent_24h("user", owner, "appa") == 0.0
|
||||
|
||||
|
||||
def test_resetting_the_same_scope_twice_reuses_one_row(local_db):
|
||||
owner = _owner()
|
||||
scope = q.QuotaResetIn(owner_kind="user", owner_id=owner, app_reference="appa")
|
||||
first = q.reset(scope)
|
||||
second = q.reset(scope)
|
||||
assert first["uid"] == second["uid"]
|
||||
assert second["reset_at"] >= first["reset_at"]
|
||||
|
||||
|
||||
def test_a_rule_scope_is_unblocked_by_a_reset(local_db):
|
||||
owner = _owner()
|
||||
q.quota_rule_store.set(
|
||||
q.QuotaRuleIn(
|
||||
owner_kind="user", owner_id=owner, app_reference="appa", limit_usd=1.0
|
||||
),
|
||||
created_by="test",
|
||||
)
|
||||
_burn(owner, "appa", 1.5)
|
||||
limit, scope, rule = q.resolve("user", owner, "appa", {})
|
||||
assert q.spent_24h(*scope) >= limit
|
||||
q.reset(
|
||||
q.QuotaResetIn(
|
||||
owner_kind=scope[0], owner_id=scope[1], app_reference=scope[2]
|
||||
)
|
||||
)
|
||||
assert q.spent_24h(*scope) < limit
|
||||
q.quota_rule_store.remove(rule.uid)
|
||||
|
||||
|
||||
def test_scope_label_reads_like_the_rule_label(local_db):
|
||||
scope = {"owner_kind": "user", "owner_id": "u1", "app_reference": "appa"}
|
||||
assert q.scope_label(scope) == "role=user, user=u1, app=appa"
|
||||
|
||||
|
||||
def test_scope_label_falls_back_for_a_wildcard_scope(local_db):
|
||||
empty = {"owner_kind": None, "owner_id": None, "app_reference": None}
|
||||
assert q.scope_label(empty, fallback="every caller") == "every caller"
|
||||
Loading…
Reference in New Issue
Block a user