feat: add admin/internal database API with CRUD, read-only query, and natural-language SQL endpoints

Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:

- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes

Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
This commit is contained in:
2026-06-14 23:00:30 +00:00
parent 96521e8757
commit d31ccba6c1
69 changed files with 3278 additions and 12 deletions
+2
View File
@@ -25,6 +25,7 @@ ZIPS_DIR = DATA_DIR / "zips"
ZIP_STAGING_DIR = DATA_DIR / "zip_staging"
FORK_STAGING_DIR = DATA_DIR / "fork_staging"
SEO_REPORTS_DIR = DATA_DIR / "seo_reports"
DBAPI_DIR = DATA_DIR / "dbapi"
DEEPSEARCH_DIR = DATA_DIR / "deepsearch"
DEEPSEARCH_CHROMA_DIR = DEEPSEARCH_DIR / "chroma"
KEYS_DIR = DATA_DIR / "keys"
@@ -102,6 +103,7 @@ DATA_PATHS: dict[str, Path] = {
"zip_staging": ZIP_STAGING_DIR,
"fork_staging": FORK_STAGING_DIR,
"seo_reports": SEO_REPORTS_DIR,
"dbapi": DBAPI_DIR,
"deepsearch": DEEPSEARCH_DIR,
"deepsearch_chroma": DEEPSEARCH_CHROMA_DIR,
"keys": KEYS_DIR,