blindxfish/devplacepy
1
0
Fork 0
forked from retoor/devplacepy
Code Pull Requests Activity

feat: add AI markdown reformatting and usage metering to NewsService

Browse Source 
Add AI-powered body reformatting for valid articles in the news pipeline, converting raw text walls into clean Markdown with paragraphs, headings, and lists. Introduce `news_usage` database table and `add_news_usage`/`get_news_usage` helpers to track per-cycle gateway costs (calls, tokens, latency, USD) from response headers, reported on the admin Services page. Remove unused `.sidebar-more` CSS and apply `render_title()` to poll question/label fields.
This commit is contained in:
retoor
2026-06-18 23:46:53 +00:00
parent 6ceca3d0d4
commit f3a4667fce
13 changed files with 427 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -1
View File
@@ -322,7 +322,7 @@ and its full configuration are documented automatically - including future servi
- **`ConfigField`** - declarative parameter spec (type, default, validation, secret) a service uses to declare its editable settings
- **`BaseService`** - abstract class with a reconciling run loop that honors the persisted `enabled`/command/interval state, plus `config_fields`, `get_config()`, `describe()`, and a log buffer
- **`ServiceManager`** - singleton: `register`, `describe_all`, `set_enabled`, `send_command`, `save_config`, `supervise`, `shutdown_all`
- **`NewsService`** - a fully automatic, zero-maintenance news pipeline: fetches news from `news.app.molodetz.nl/api`, cleans each article, fetches and perceptually compares the images to reject placeholders and detect uniqueness, grades each one deterministically, and auto-rotates the best articles to Featured and the landing page
- **`NewsService`** - a fully automatic, zero-maintenance news pipeline: fetches news from `news.app.molodetz.nl/api`, cleans each article, fetches and perceptually compares the images to reject placeholders and detect uniqueness, grades each one deterministically, reformats every valid article into clean Markdown (paragraphs, headings, lists) with the AI so the source wall of text reads as a proper article, and auto-rotates the best articles to Featured and the landing page. Its AI spend is metered from the gateway response headers and reported on the admin Services page (calls, tokens, total cost, and per-call averages)
- **`BotsService`** - Playwright fleet of AI personas that browse and interact with a DevPlace instance, with live cost/usage metrics and a live screenshot monitor at `/admin/bots` (opt-in; install the `bots` extra)
- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected
- **`JobService` / `ZipService` / `ForkService`** - generic async job framework (`services/jobs/`) for heavy, blocking work run off the request path; `ZipService` builds project zip archives in a subprocess, `ForkService` copies a project into a new project owned by the forking user
@@ -369,6 +369,8 @@ A fully automatic, zero-maintenance import pipeline. It fetches news from a conf
Admin can manually publish/draft, toggle Featured, toggle landing-page appearance, and delete; a manual Featured or landing toggle **locks** that article so the service no longer auto-manages it.
The Featured badge and the numeric Grade are editorial signals shown only to administrators. On the public news pages (listing, article, and landing-page cards) members and guests see only the article, its source, and its time; the badges remain visible to administrators and in the admin Manage News area.
Configuration on the Services tab (`/admin/services`):
| Parameter | Default | Purpose |