feat: add admin/internal database API with CRUD, read-only query, and natural-language SQL endpoints
Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:
- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes
Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
2026-06-15 01:00:30 +02:00
|
|
|
# retoor <retoor@molodetz.nl>
|
|
|
|
|
|
|
|
|
|
from datetime import datetime, timezone
|
|
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
|
from fastapi.testclient import TestClient
|
|
|
|
|
|
2026-06-19 10:06:09 +02:00
|
|
|
from devplacepy.database import get_table, purge, refresh_snapshot
|
feat: add admin/internal database API with CRUD, read-only query, and natural-language SQL endpoints
Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:
- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes
Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
2026-06-15 01:00:30 +02:00
|
|
|
from devplacepy.main import app
|
|
|
|
|
from devplacepy.services.manager import service_manager
|
|
|
|
|
from devplacepy.utils import generate_uid
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _seed_schema():
|
|
|
|
|
now = datetime.now(timezone.utc).isoformat()
|
|
|
|
|
users = get_table("users")
|
|
|
|
|
if "username" not in users.columns:
|
|
|
|
|
users.insert(
|
|
|
|
|
{
|
|
|
|
|
"uid": generate_uid(),
|
|
|
|
|
"username": "dbapi_seed",
|
Add the trust and safety subsystem and the App Store compliance work
Implements the moderation and consent obligations a social platform carries,
so the web version and any client that speaks to it enforce the same rules.
Moderation core (services/moderation/, database/moderation.py): a reportable
target registry, the content filter and its choke points, the report queue with
atomic resolution, enforcement actions, consent tracking, maturity gating, and
account deletion with a grace window.
Surfaces: POST /reports plus the member report list, /admin/moderation and the
per-report admin view, /workspaces, terms acceptance at /auth/terms, consent and
account deletion under /profile, the report button and dialog partials, the
maturity gate, and the moderation stylesheet and ReportDialog client.
Every user-generated surface stays reportable by construction: new content tables
are registered in REPORTABLE_TARGETS or listed in UNREPORTABLE_TABLES with a
reason, and the registry test fails the suite on anything left unclassified.
Docs: community guidelines, content moderation, intellectual property, privacy,
terms, contact, and the admin-only moderation operations page, plus the
moderation API group and the Devii moderation actions.
Compliance record: applecomp.md is the requirement register, applechanges.md the
gap analysis against this codebase, and appleimpl.md the implementation design
they resolve to.
Tests cover the report flow, admin moderation, consent, account deletion, terms
acceptance, workspaces, and the registry invariant across the unit, api, and e2e
tiers.
2026-08-09 00:18:20 +02:00
|
|
|
"terms_version": "1",
|
feat: add admin/internal database API with CRUD, read-only query, and natural-language SQL endpoints
Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:
- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes
Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
2026-06-15 01:00:30 +02:00
|
|
|
"email": "seed@dbapi.test",
|
|
|
|
|
"role": "Member",
|
|
|
|
|
"xp": 0,
|
|
|
|
|
"level": 1,
|
|
|
|
|
"created_at": now,
|
|
|
|
|
}
|
|
|
|
|
)
|
|
|
|
|
bookmarks = get_table("bookmarks")
|
|
|
|
|
if "user_uid" not in bookmarks.columns:
|
|
|
|
|
seed = generate_uid()
|
|
|
|
|
bookmarks.insert(
|
|
|
|
|
{
|
|
|
|
|
"uid": seed,
|
|
|
|
|
"user_uid": "_seedcols",
|
|
|
|
|
"target_type": "post",
|
|
|
|
|
"target_uid": "_s",
|
|
|
|
|
"created_at": now,
|
|
|
|
|
"deleted_at": None,
|
|
|
|
|
"deleted_by": None,
|
|
|
|
|
}
|
|
|
|
|
)
|
|
|
|
|
purge("bookmarks", uid=seed)
|
|
|
|
|
refresh_snapshot()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
|
|
|
def client():
|
|
|
|
|
with TestClient(app) as test_client:
|
|
|
|
|
service_manager.set_lock_owner(True)
|
|
|
|
|
_seed_schema()
|
|
|
|
|
yield test_client
|
|
|
|
|
service_manager.set_lock_owner(False)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
def auth(client):
|
2026-06-19 10:06:09 +02:00
|
|
|
from devplacepy.database import get_primary_admin_uid, invalidate_admins_cache
|
|
|
|
|
from devplacepy.utils import clear_user_cache
|
|
|
|
|
|
|
|
|
|
users = get_table("users")
|
|
|
|
|
uid = get_primary_admin_uid()
|
|
|
|
|
if uid is None:
|
|
|
|
|
uid = generate_uid()
|
|
|
|
|
users.insert(
|
|
|
|
|
{
|
|
|
|
|
"uid": uid,
|
|
|
|
|
"username": f"dbapi_admin_{uid[:8]}",
|
Add the trust and safety subsystem and the App Store compliance work
Implements the moderation and consent obligations a social platform carries,
so the web version and any client that speaks to it enforce the same rules.
Moderation core (services/moderation/, database/moderation.py): a reportable
target registry, the content filter and its choke points, the report queue with
atomic resolution, enforcement actions, consent tracking, maturity gating, and
account deletion with a grace window.
Surfaces: POST /reports plus the member report list, /admin/moderation and the
per-report admin view, /workspaces, terms acceptance at /auth/terms, consent and
account deletion under /profile, the report button and dialog partials, the
maturity gate, and the moderation stylesheet and ReportDialog client.
Every user-generated surface stays reportable by construction: new content tables
are registered in REPORTABLE_TARGETS or listed in UNREPORTABLE_TABLES with a
reason, and the registry test fails the suite on anything left unclassified.
Docs: community guidelines, content moderation, intellectual property, privacy,
terms, contact, and the admin-only moderation operations page, plus the
moderation API group and the Devii moderation actions.
Compliance record: applecomp.md is the requirement register, applechanges.md the
gap analysis against this codebase, and appleimpl.md the implementation design
they resolve to.
Tests cover the report flow, admin moderation, consent, account deletion, terms
acceptance, workspaces, and the registry invariant across the unit, api, and e2e
tiers.
2026-08-09 00:18:20 +02:00
|
|
|
"terms_version": "1",
|
2026-06-19 10:06:09 +02:00
|
|
|
"email": f"{uid[:8]}@dbapi.test",
|
|
|
|
|
"role": "Admin",
|
|
|
|
|
"api_key": f"dbapikey_{uid}",
|
|
|
|
|
"xp": 0,
|
|
|
|
|
"level": 1,
|
|
|
|
|
"created_at": datetime.now(timezone.utc).isoformat(),
|
|
|
|
|
}
|
|
|
|
|
)
|
|
|
|
|
invalidate_admins_cache()
|
|
|
|
|
refresh_snapshot()
|
|
|
|
|
admin = users.find_one(uid=uid)
|
|
|
|
|
key = admin.get("api_key")
|
|
|
|
|
if not key:
|
|
|
|
|
key = f"dbapikey_{uid}"
|
|
|
|
|
users.update({"uid": uid, "api_key": key}, ["uid"])
|
|
|
|
|
clear_user_cache(uid)
|
|
|
|
|
refresh_snapshot()
|
|
|
|
|
return {"X-API-KEY": key}
|