This commit is contained in:
2026-07-19 18:57:43 +02:00
parent 48bb6c2ec2
commit c53e2a3319
179 changed files with 9307 additions and 897 deletions
+55 -1
View File
@@ -35,6 +35,12 @@ Conversation history persists to `devii_conversations` (rehydrated on reconnect,
**Per-owner self-learning memory is privacy-critical.** The `LessonStore` (reflect/recall) is **owner-scoped, never shared**: `LessonStore(db, owner_kind, owner_id)` filters every read/write by owner. The hub builds one per session over the same `owned_db` as the task store - the main `db` (table `devii_lessons`) for signed-in users (persistent, isolated, survives restarts) and a fresh `memory_db()` for guests (ephemeral, scoped to that web session). `forget_lessons` (agentic tool -> `LessonStore.clear()`/`delete()`) lets the user purge them; the system prompt forbids storing credentials/secrets in lessons. **Regression to avoid: do NOT share one `LessonStore` across sessions** - that leaked one user's reflected lessons (including credentials) into every other user's recall.
**Lesson retention and deduplication.** Every `add()` deduplicates against existing active lessons via Jaccard similarity (threshold 0.70): a near-duplicate bumps the original's `hits` counter and refreshes `created_at` instead of inserting a new row. A per-owner cap (`devii_lessons_max_per_owner`, default 500, configurable on `/admin/services` and `site_settings`) is enforced on every insert: when exceeded, the oldest lessons are soft-deleted (`deleted_by="retention"`). Age-based pruning runs on `DeviiService.run_once()` (every 60s on the lock-owner worker): lessons older than `devii_lessons_max_age_days` (default 90, configurable) are soft-deleted across all owners. Both settings persist in `site_settings` and are seeded in `init_db`. The `devii_lessons` table is in `SOFT_DELETE_TABLES` for admin Trash restore/purge.
**Quality signals.** Each lesson has a `rating` column (integer, default 0). The `lesson_rate(uid, value)` agentic tool (value 1 for useful, -1 for unhelpful) lets the agent self-rate lessons. In `_rebuild()`, lessons with `rating <= -3` are excluded from the BM25 index and therefore never returned by `recall()`. The `lesson_count()` tool reports active lesson count.
**CLI:** `devplace devii lessons count` reports active/soft-deleted totals; `devplace devii lessons prune --all-owners|--username USER` soft-deletes old lessons; `devplace devii lessons clear --force` hard-deletes all rows. Rate-limiting guard: `run_once` swallows all exceptions so a bad schema never stops housekeeping.
## Reminders and scheduled tasks (persistent across reboot)
`create_task` queues a self-contained prompt that a fresh agent runs later (`services/devii/tasks/`): `kind=once` (`delay_seconds` for relative, `run_at` UTC for absolute), `interval` (`every_seconds`), or `cron`. The per-session `Scheduler` ticks every 1s and executes due rows through the session's executor, so the result is broadcast to any connected tab and buffered (`type:"task"` frame) when the terminal is closed.
@@ -142,9 +148,57 @@ Devii can partially configure its OWN system message: every system prompt ends w
- **Store** (`behavior/store.py`). `BehaviorStore(db, owner_kind, owner_id)` over `devii_behavior` (one upserted row per owner keyed on `owner_kind`/`owner_id`; `text()` reads, `set()` upserts; index `idx_devii_behavior_owner`). Persistent for users, `memory_db()` for guests (built in `hub.get_or_create` from the shared `owned_db`, like the other owner stores).
- **Controller** (`behavior/controller.py`). `BehaviorController(store)`, `dispatch("update_behavior", args)` -> `store.set(behavior)`. Built in `DeviiSession` and passed to `Dispatcher(behavior=...)`; the dispatcher routes `handler="behavior"` to it and degrades gracefully ("not available in this context") when unwired (e.g. the standalone CLI, same as `virtual_tools`/`avatar`).
- **Injection and refresh** (`session.py`). `_compose_system_prompt()` = base prompt (`_system_prompt_for(is_admin)`) + `\n\n` + `BEHAVIOR_HEADER` (+ `\n` + body when non-empty; just the header when empty). Used to seed the `Agent` and the scheduler executor worker, and `_refresh_system_prompt()` rewrites `agent._messages[0]["content"]` **at the top of every `_run_turn`** (next to `_refresh_tools`), so a mid-conversation `update_behavior` takes effect on the following turn and the live system message is never overridden by the stale base.
- **Injection and refresh** (`session.py`). `_compose_system_prompt()` = base prompt (`_system_prompt_for(is_admin)`) + CA-IWP fragment + live `CHANNEL` block + `\n\n` + `BEHAVIOR_HEADER` (+ `\n` + body when non-empty; just the header when empty). Used to seed the `Agent` and the scheduler executor worker, and `_refresh_system_prompt()` rewrites `agent._messages[0]["content"]` **at the top of every `_run_turn`** (next to `_refresh_tools`), so a mid-conversation `update_behavior` takes effect on the following turn and the live system message is never overridden by the stale base.
- Registered via `BEHAVIOR_ACTIONS` (`registry.py`); the system-prompt **SELF-CONFIGURED BEHAVIOR (TRUTH RULES)** section in `agent.py` steers when/how to call it.
## Channel-Aware Interactive Widget Protocol (CA-IWP / Speak With Buttons)
Devii can ask the user for decisions through a gated tool surface instead of inventing HTML or channel-specific prose. Spec lives as the CA-IWP document; implementation is Devii-only under `services/devii/interaction/`.
### Layers
| Piece | Role |
|-------|------|
| `interaction/capabilities.py` | Maps session channel (`main`/`docs` -> `site-chat`, `telegram`, `cli`) to capability flags + limits; builds the compact `CHANNEL` / `CAPS` / `LIMITS` / `TOOLS` / `INTERACTION` fragment injected every turn. |
| `interaction/schema.py` | Pydantic validation for `ui_prompt` args (widget catalog: confirm, choice, choice_multi, text, number, date, select, `//`, group). Untrusted model input is sanitized and capped. |
| `interaction/broker.py` | Validates, assigns `interaction_id`, single-flight open interactions, routes to site / telegram / plain adapters, returns structured `{status, values, meta}`. |
| `interaction/controller.py` | Dispatches `ui_prompt` / `ui_cancel` / `ui_notify` (`handler="interaction"`). |
| `interaction/actions.py` | Catalog entries registered in `registry.py` as `INTERACTION_ACTIONS`. |
| `interaction/markdown.py` + `parse.py` | Dual-surface markdown projection and plain/CLI reply parsers. |
### Tool gating and preferences (admin default + user override)
`DeviiSession._builtin_tools()` filters UI tools through `channel_context().tools`. Docs channel stays search-only (no UI tools). Open interaction (single-flight) exposes only `ui_cancel` plus preference tools.
**Admin default:** Devii service ConfigField `devii_interactions_default` (bool, default on) on `/admin/services`. Guests always use this default.
**User override:** column `users.interactions_enabled` (`-1` = inherit default, `0` = off, `1` = on). New signups insert `-1`. Resolution: `interaction/prefs.py` `effective_for(owner_kind, owner_id)`.
**Surfaces (same fan-out as AI correction):**
- Devii tools `interactions_get` / `interactions_set` (`requires_auth=True`; set accepts `enabled` or `reset=true` to inherit again). Always offered to signed-in users even when widgets are off, so they can re-enable.
- HTTP `POST /profile/{username}/interactions` (owner-or-admin, form `InteractionsForm`), profile card + `InteractionsPref.js`, `docs_api` endpoint, audit `profile.interactions`, profile JSON keys on `ProfileOut`.
- When effective is off, `ui_prompt` / `ui_notify` are absent from the tool list and the controller refuses them; the model must use degraded markdown menus.
### Site-chat path
`ui_prompt` blocks like client tools: session `_interaction_wait` emits `{type:"interaction", id, args}` on the WebSocket; `devii-terminal.js` mounts an `<ai-interaction>` tree via `createElement` (never model HTML), lazy-loads widget modules through `AiAutoload` (`static/js/autoload/AiAutoload.js`, allowlisted tag -> module map only), and replies with `{type:"interaction_result"}`. Router resolves via `session.resolve_interaction`. Custom elements live under `static/js/components/Ai*.js` with per-component CSS under `static/css/components/ai-*.css`. Light DOM only; `Application` boots the shell (`AiInteraction`, `AiStatus`, `AiActions`, `AiHelp`, `AiOption`) and starts the autoloader.
### Telegram path
`TelegramConnection` handles `type:"interaction"`: sends dual-surface plain text plus inline keyboard for confirm / single choice (`callback_data` short tokens). `TelegramBridge` registers a pending future **before** the chat lock so the next message or `callback_query` can resolve mid-turn. Worker `allowed_updates` includes `callback_query`; service routes `type:"callback"` to the bridge.
### Plain / CLI path
Broker uses `present_plain` or emits a plain menu and waits; `parse_plain_reply` accepts y/n, indices, value tokens, cancel.
### System prompt
Every non-docs turn appends `CA_IWP_SYSTEM_FRAGMENT` plus the live channel fragment from `_compose_system_prompt()`. Model rules: prefer `ui_prompt` when gated on; never invent HTML/CE tags; branch on `status` (submitted / cancelled / timeout / superseded / error).
### Tests
Unit coverage under `tests/unit/services/devii/interaction/` (capabilities, schema, markdown, parse, broker, controller) plus session tool-gating assertions and telegram callback emission. Mirror this layout for any extension.
## Related Devii tool wrappers (customization, container)
Two more Devii-side controllers live under `services/devii/` but their full mechanism is documented in their owning subsystem's file, not here:
@@ -61,7 +61,7 @@ AI_CORRECTION_ACTIONS: tuple[Action, ...] = (
),
arg(
"prompt",
"The correction instruction (max 2000 chars). Omit to keep the current one.",
"The correction instruction (max 20000 chars). Omit to keep the current one.",
),
),
),
@@ -64,7 +64,7 @@ AI_MODIFIER_ACTIONS: tuple[Action, ...] = (
),
arg(
"prompt",
"The modifier instruction (max 2000 chars). Omit to keep the current one.",
"The modifier instruction (max 20000 chars). Omit to keep the current one.",
),
),
),
@@ -28,7 +28,7 @@ COMMENTS_ACTIONS: tuple[Action, ...] = (
summary="Edit the body of one of your own comments",
params=(
path("comment_uid", "Uid of the comment."),
body("content", "New comment body, 3-1000 characters.", required=True),
body("content", "New comment body, 3-125000 characters.", required=True),
),
),
Action(
@@ -116,6 +116,7 @@ _DEVII_MECHANIC_EVENTS = {
"email_mark": "email.message.flag",
"email_set_flags": "email.message.flag",
"telegram_send": "telegram.send",
"interactions_set": "profile.interactions",
}
_DEVII_CONTAINER_EVENTS = {
@@ -268,6 +269,7 @@ class Dispatcher:
owner_id: str = "",
virtual_tools: Any = None,
behavior: Any = None,
interaction: Any = None,
) -> None:
self._actions = catalog.by_name()
self._client = client
@@ -308,6 +310,7 @@ class Dispatcher:
self._telegram = TelegramSendController(owner_kind, owner_id)
self._virtual_tools = virtual_tools
self._behavior = behavior
self._interaction = interaction
self._read_files: set[tuple[str, str]] = set()
@staticmethod
@@ -500,6 +503,15 @@ class Dispatcher:
)
return await self._behavior.dispatch(action.name, arguments)
if action.handler == "interaction":
if self._interaction is None:
return error_result(
ToolInputError(
"Interactive prompts are not available in this context."
)
)
return await self._interaction.dispatch(action.name, arguments)
if action.handler == "virtual_tool":
if self._virtual_tools is None:
return error_result(
+20 -7
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
from dataclasses import dataclass, field
from functools import lru_cache
from typing import Any, Literal
ParamLocation = Literal["path", "query", "body", "file"]
@@ -49,6 +50,7 @@ class Action:
"virtual_tool",
"email",
"telegram",
"interaction",
] = "http"
freeform_body: bool = False
ajax: bool = False
@@ -117,16 +119,27 @@ class Catalog:
def tool_schemas(self) -> list[dict[str, Any]]:
return [action.tool_schema() for action in self.actions]
@lru_cache(maxsize=16)
def _schemas_for(
self,
authenticated: bool,
is_admin: bool,
is_primary_admin: bool,
) -> tuple[dict[str, Any], ...]:
return tuple(
action.tool_schema()
for action in self.actions
if (authenticated or not action.requires_auth)
and (is_admin or not action.requires_admin)
and (is_primary_admin or not action.requires_primary_admin)
)
def tool_schemas_for(
self,
authenticated: bool,
is_admin: bool = False,
is_primary_admin: bool = False,
) -> list[dict[str, Any]]:
return [
action.tool_schema()
for action in self.actions
if (authenticated or not action.requires_auth)
and (is_admin or not action.requires_admin)
and (is_primary_admin or not action.requires_primary_admin)
]
return list(
self._schemas_for(bool(authenticated), bool(is_admin), bool(is_primary_admin))
)
@@ -150,6 +150,34 @@ AGENTIC_ACTIONS: tuple[Action, ...] = (
),
),
),
Action(
name="lesson_rate",
method="LOCAL",
path="",
summary="Vote on a lesson's quality to help the system keep the best ones and drop bad ones",
description=(
"Rate a lesson recorded by reflect(). A positive vote (1) marks it as useful; "
"a negative vote (-1) marks it as unhelpful. Lessons with a cumulative rating of "
"-3 or lower are excluded from recall() results. Use this when you notice a lesson "
"was particularly helpful or misleading."
),
handler="agentic",
requires_auth=False,
params=(
arg("uid", "The uid of the lesson to rate, from a recall() result.", required=True),
arg("value", "1 for useful, -1 for unhelpful.", required=True, kind="integer"),
),
),
Action(
name="lesson_count",
method="LOCAL",
path="",
summary="Report how many learned lessons are stored for this session or account",
description="Returns the number of active lessons with an optional per-tag breakdown.",
handler="agentic",
requires_auth=False,
params=(),
),
Action(
name="eval",
method="LOCAL",
@@ -6,13 +6,17 @@ import json
import logging
from typing import Any
from ..text import normalize_newlines
logger = logging.getLogger("devii.agentic.compaction")
SUMMARY_INPUT_CAP = 600_000
SUMMARY_PROMPT = (
"Summarize the following assistant conversation segment as a concise factual log of "
"actions taken, tools called, entities created or changed, conclusions reached, and "
"outstanding work. Keep identifiers, slugs, uids, and decisions verbatim. Maximum 800 words.\n\n"
"outstanding work. Keep identifiers, slugs, uids, and decisions verbatim. "
"Write plain markdown with real line breaks (never the two-character sequence \\n). "
"Use headings and bullet lists where helpful. Maximum 800 words.\n\n"
"---\n\n"
)
@@ -32,6 +36,37 @@ def find_compaction_split(messages: list[dict[str, Any]], keep_tail: int) -> int
return 1
def _segment_plain(messages: list[dict[str, Any]]) -> str:
parts: list[str] = []
for message in messages:
role = str(message.get("role") or "unknown")
content = message.get("content")
if isinstance(content, str) and content.strip():
parts.append(f"{role}:\n{normalize_newlines(content)}")
continue
if isinstance(content, list):
chunks: list[str] = []
for part in content:
if isinstance(part, dict) and part.get("type") == "text":
chunks.append(str(part.get("text") or ""))
elif isinstance(part, str):
chunks.append(part)
text = normalize_newlines("\n".join(c for c in chunks if c))
if text.strip():
parts.append(f"{role}:\n{text}")
continue
tool_calls = message.get("tool_calls")
if tool_calls:
names = []
for call in tool_calls:
fn = (call or {}).get("function") or {}
name = fn.get("name") or "tool"
names.append(str(name))
if names:
parts.append(f"{role}: called {', '.join(names)}")
return "\n\n".join(parts)
async def compact_messages(
llm: Any, messages: list[dict[str, Any]], keep_tail: int
) -> list[dict[str, Any]]:
@@ -46,16 +81,24 @@ async def compact_messages(
if not middle:
return messages
segment = json.dumps(middle, default=str)[:SUMMARY_INPUT_CAP]
segment = _segment_plain(middle)[:SUMMARY_INPUT_CAP]
if not segment.strip():
segment = json.dumps(middle, default=str)[:SUMMARY_INPUT_CAP]
try:
summary = await llm.summarize(SUMMARY_PROMPT + segment)
except Exception: # noqa: BLE001 - compaction must never break the loop
logger.exception("Compaction summary failed; keeping full context")
return messages
summary = normalize_newlines(summary or "").strip()
if not summary:
return messages
logger.info("Compacted %d messages into a summary", len(middle))
return [
system_message,
{"role": "assistant", "content": f"[compacted earlier turns]\n\n{summary}"},
{
"role": "assistant",
"content": f"[compacted earlier turns]\n\n{summary}",
},
*tail,
]
@@ -63,6 +63,8 @@ class AgenticController:
"reflect": self._reflect,
"recall": self._recall,
"forget_lessons": self._forget,
"lesson_rate": self._lesson_rate,
"lesson_count": self._lesson_count,
"verify": self._verify,
"delegate": self._delegate,
"eval": self._eval,
@@ -162,6 +164,31 @@ class AgenticController:
ensure_ascii=False,
)
async def _lesson_rate(self, arguments: dict[str, Any]) -> str:
uid = str(arguments.get("uid", "")).strip()
if not uid:
raise ToolInputError("lesson_rate requires a lesson uid.")
raw_value = arguments.get("value")
if raw_value is None:
raise ToolInputError("lesson_rate requires a value (1 or -1).")
try:
value = int(raw_value)
except (ValueError, TypeError):
raise ToolInputError("lesson_rate value must be an integer (1 or -1).") from None
if value not in (1, -1):
raise ToolInputError("lesson_rate value must be 1 (useful) or -1 (unhelpful).")
ok = self._lessons.rate(uid, value)
if not ok:
raise ToolInputError(f"No lesson found with uid '{uid}'.")
return json.dumps({"status": "success", "lesson_uid": uid, "rated": value}, ensure_ascii=False)
async def _lesson_count(self, arguments: dict[str, Any]) -> str:
count = self._lessons.count()
return json.dumps(
{"status": "success", "active_lessons": count},
ensure_ascii=False,
)
async def _verify(self, arguments: dict[str, Any]) -> str:
summary = str(arguments.get("summary", "")).strip()
if not summary:
+227 -10
View File
@@ -7,6 +7,7 @@ import logging
import math
import re
import uuid
from datetime import timedelta
from typing import Any
from ..tasks.schedule import now_utc, to_iso
@@ -18,6 +19,11 @@ TOKEN_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_]*|\d+")
BM25_K1 = 1.5
BM25_B = 0.75
DEDUP_JACCARD_THRESHOLD = 0.70
DEFAULT_MAX_PER_OWNER = 500
DEFAULT_MAX_AGE_DAYS = 90
LOW_QUALITY_THRESHOLD = -3
def tokenize(text: str) -> list[str]:
tokens = TOKEN_RE.findall((text or "").lower())
@@ -27,6 +33,33 @@ def tokenize(text: str) -> list[str]:
return list(dict.fromkeys(tokens + extra))
def _jaccard(tokens_a: set[str], tokens_b: set[str]) -> float:
if not tokens_a and not tokens_b:
return 0.0
if not tokens_a or not tokens_b:
return 0.0
return len(tokens_a & tokens_b) / len(tokens_a | tokens_b)
def _read_retention_settings(db: Any) -> tuple[int, int]:
max_per = DEFAULT_MAX_PER_OWNER
max_age = DEFAULT_MAX_AGE_DAYS
if "site_settings" not in db.tables:
return max_per, max_age
for row in db["site_settings"].find(key={"in": ["devii_lessons_max_per_owner", "devii_lessons_max_age_days"]}):
if row["key"] == "devii_lessons_max_per_owner":
try:
max_per = int(row["value"])
except (ValueError, TypeError):
pass
elif row["key"] == "devii_lessons_max_age_days":
try:
max_age = int(row["value"])
except (ValueError, TypeError):
pass
return max_per, max_age
class LessonStore:
def __init__(self, db: Any, owner_kind: str, owner_id: str) -> None:
self._db = db
@@ -39,9 +72,10 @@ class LessonStore:
self._idf: dict[str, float] = {}
self._avgdl = 0.0
self._n = 0
self._ensure_columns()
self._ensure_indexes()
def _ensure_indexes(self) -> None:
def _ensure_columns(self) -> None:
if TABLE not in self._db.tables:
return
table = self._db[TABLE]
@@ -49,7 +83,15 @@ class LessonStore:
table.create_column_by_example("deleted_at", "")
if not table.has_column("deleted_by"):
table.create_column_by_example("deleted_by", "")
if not table.has_column("rating"):
table.create_column_by_example("rating", 0)
def _ensure_indexes(self) -> None:
if TABLE not in self._db.tables:
return
table = self._db[TABLE]
table.create_index(["owner_kind", "owner_id"])
table.create_index(["owner_kind", "owner_id", "created_at"])
@property
def _table(self) -> Any:
@@ -64,17 +106,99 @@ class LessonStore:
return 0
return self._table.count(deleted_at=None, **self._scope)
def _row_text(self, row: dict[str, Any]) -> str:
return " ".join(
str(row.get(field) or "")
for field in ("observation", "conclusion", "next_action", "tags")
)
def _find_similar(self, text: str, threshold: float = DEDUP_JACCARD_THRESHOLD) -> dict[str, Any] | None:
query_tokens = set(tokenize(text))
if not query_tokens:
return None
all_rows = self.all()
best_row: dict[str, Any] | None = None
best_score = 0.0
for row in all_rows:
row_tokens = set(tokenize(self._row_text(row)))
score = _jaccard(query_tokens, row_tokens)
if score > best_score and score >= threshold:
best_score = score
best_row = row
return best_row
def _enforce_cap(self, max_per_owner: int) -> int:
soft_deleted = 0
while True:
current = self.count()
if current <= max_per_owner:
break
excess = current - max_per_owner
rows = list(
self._table.find(
deleted_at=None,
order_by=["created_at"],
_limit=excess,
**self._scope,
)
)
if not rows:
break
now = to_iso(now_utc())
for row in rows:
self._table.update(
{
"id": row["id"],
"deleted_at": now,
"deleted_by": "retention",
},
["id"],
)
soft_deleted += 1
self._dirty = True
if soft_deleted:
logger.info(
"Retention cap pruned %d lesson(s) for owner=%s/%s",
soft_deleted,
self._owner_kind,
self._owner_id,
)
return soft_deleted
def add(
self, observation: str, conclusion: str, next_action: str, tags: str = ""
) -> dict[str, Any]:
text = " ".join([observation, conclusion, next_action, tags])
similar = self._find_similar(text)
if similar and similar.get("id") is not None:
hits = (similar.get("hits") or 0) + 1
self._table.update(
{
"id": similar["id"],
"hits": hits,
"created_at": to_iso(now_utc()),
},
["id"],
)
self._dirty = True
logger.info(
"Lesson deduplicated owner=%s/%s hits=%d",
self._owner_kind,
self._owner_id,
hits,
)
return {**similar, "hits": hits, "deduplicated": True}
uid = uuid.uuid4().hex
record = {
"uid": uuid.uuid4().hex,
"uid": uid,
"observation": observation,
"conclusion": conclusion,
"next_action": next_action,
"tags": tags,
"created_at": to_iso(now_utc()),
"hits": 0,
"rating": 0,
"deleted_at": None,
"deleted_by": None,
**self._scope,
@@ -84,6 +208,8 @@ class LessonStore:
logger.info(
"Lesson stored owner=%s/%s tags=%s", self._owner_kind, self._owner_id, tags
)
max_per, _ = _read_retention_settings(self._db)
self._enforce_cap(max_per)
return record
def all(self) -> list[dict[str, Any]]:
@@ -118,17 +244,108 @@ class LessonStore:
)
return n
def rate(self, uid: str, value: int) -> bool:
if TABLE not in self._db.tables:
return False
row = self._table.find_one(uid=uid, deleted_at=None, **self._scope)
if not row:
return False
current = row.get("rating") or 0
self._table.update(
{"id": row["id"], "rating": current + value},
["id"],
)
self._dirty = True
logger.info(
"Lesson %s rated %+d (now %d) owner=%s/%s",
uid,
value,
current + value,
self._owner_kind,
self._owner_id,
)
return True
def prune(self, max_age_days: int | None = None) -> int:
if TABLE not in self._db.tables:
return 0
if max_age_days is None:
_, max_age_days = _read_retention_settings(self._db)
cutoff = now_utc() - timedelta(days=max_age_days)
cutoff_iso = to_iso(cutoff)
rows = list(
self._table.find(
deleted_at=None,
created_at={"<": cutoff_iso},
**self._scope,
)
)
if not rows:
return 0
now = to_iso(now_utc())
soft_deleted = 0
for row in rows:
self._table.update(
{
"id": row["id"],
"deleted_at": now,
"deleted_by": "retention",
},
["id"],
)
soft_deleted += 1
if soft_deleted:
self._dirty = True
logger.info(
"Pruned %d old lesson(s) for owner=%s/%s",
soft_deleted,
self._owner_kind,
self._owner_id,
)
return soft_deleted
def prune_all_owners(self, max_age_days: int | None = None) -> int:
if TABLE not in self._db.tables:
return 0
if max_age_days is None:
_, max_age_days = _read_retention_settings(self._db)
cutoff = now_utc() - timedelta(days=max_age_days)
cutoff_iso = to_iso(cutoff)
rows = list(
self._table.find(
deleted_at=None,
created_at={"<": cutoff_iso},
)
)
if not rows:
return 0
now = to_iso(now_utc())
soft_deleted = 0
for row in rows:
self._table.update(
{
"id": row["id"],
"deleted_at": now,
"deleted_by": "retention",
},
["id"],
)
soft_deleted += 1
if soft_deleted:
logger.info("Pruned %d old lesson(s) across all owners", soft_deleted)
return soft_deleted
def _rebuild(self) -> None:
rows = self.all()
df: collections.Counter = collections.Counter()
docs: list[dict[str, Any]] = []
tf_list: list[collections.Counter] = []
dl: list[int] = []
dl_list: list[int] = []
for row in rows:
text = " ".join(
str(row.get(field) or "")
for field in ("observation", "conclusion", "next_action", "tags")
)
rating = row.get("rating") or 0
if rating <= LOW_QUALITY_THRESHOLD:
continue
text = self._row_text(row)
tokens = tokenize(text)
if not tokens:
continue
@@ -137,12 +354,12 @@ class LessonStore:
df[term] += 1
docs.append(row)
tf_list.append(tf)
dl.append(len(tokens))
dl_list.append(len(tokens))
self._docs = docs
self._tf = tf_list
self._dl = dl
self._dl = dl_list
self._n = len(docs)
self._avgdl = sum(dl) / max(self._n, 1)
self._avgdl = sum(dl_list) / max(self._n, 1)
self._idf = {
term: math.log((self._n - freq + 0.5) / (freq + 0.5) + 1)
for term, freq in df.items()
@@ -70,7 +70,7 @@ class AiCorrectionController:
if prompt_raw is None:
prompt = user.get("ai_correction_prompt") or DEFAULT_CORRECTION_PROMPT
else:
prompt = str(prompt_raw).strip()[:2000] or DEFAULT_CORRECTION_PROMPT
prompt = str(prompt_raw).strip()[:20000] or DEFAULT_CORRECTION_PROMPT
get_table("users").update(
{
"uid": self._owner_id,
@@ -70,7 +70,7 @@ class AiModifierController:
if prompt_raw is None:
prompt = user.get("ai_modifier_prompt") or DEFAULT_MODIFIER_PROMPT
else:
prompt = str(prompt_raw).strip()[:2000] or DEFAULT_MODIFIER_PROMPT
prompt = str(prompt_raw).strip()[:20000] or DEFAULT_MODIFIER_PROMPT
get_table("users").update(
{
"uid": self._owner_id,
+1
View File
@@ -178,6 +178,7 @@ FIELD_PRICE_CACHE_HIT = "devii_price_cache_hit"
FIELD_PRICE_CACHE_MISS = "devii_price_cache_miss"
FIELD_PRICE_OUTPUT = "devii_price_output"
FIELD_ALLOW_EVAL = "devii_allow_eval"
FIELD_INTERACTIONS_DEFAULT = "devii_interactions_default"
FIELD_BROWSER_CONTROL = "devii_browser_control"
FIELD_TIMEOUT = "devii_timeout"
FIELD_FETCH_TIMEOUT = "devii_fetch_timeout"
@@ -0,0 +1,37 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
from .capabilities import (
CHANNEL_SITE_CHAT,
CHANNEL_TELEGRAM,
CHANNEL_CLI,
CHANNEL_API,
CHANNEL_UNKNOWN,
ChannelContext,
channel_id_for_session,
context_for,
fragment_for,
interactions_enabled,
)
from .controller import InteractionController
from . import prefs
from .schema import InteractionRequest, InteractionResult, validate_prompt_args
__all__ = [
"CHANNEL_SITE_CHAT",
"CHANNEL_TELEGRAM",
"CHANNEL_CLI",
"CHANNEL_API",
"CHANNEL_UNKNOWN",
"ChannelContext",
"channel_id_for_session",
"context_for",
"fragment_for",
"interactions_enabled",
"InteractionController",
"InteractionRequest",
"InteractionResult",
"validate_prompt_args",
"prefs",
]
@@ -0,0 +1,130 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
from ..actions.spec import Action, Param
def arg(
name: str, description: str, required: bool = False, kind: str = "string"
) -> Param:
return Param(
name=name,
location="body",
description=description,
required=required,
type=kind,
)
UI = (
"Presents a channel-aware interactive prompt (CA-IWP). The host renders the best "
"affordance for the current channel (site custom elements, Telegram inline keys, or "
"plain numbered menus). Blocks until the user submits, cancels, or the timeout fires. "
"Never invent HTML or custom-element tags in your prose - call this tool instead."
)
INTERACTION_ACTIONS: tuple[Action, ...] = (
Action(
name="ui_prompt",
method="LOCAL",
path="",
summary="Ask the user an interactive question with widgets (confirm, choice, text, form)",
description=(
UI
+ " Prefer this for decisions and short forms. Returns structured "
"{interaction_id, status, values, meta}. Branch on status: submitted, cancelled, "
"timeout, superseded, or error. Keep labels short and value tokens stable."
),
handler="interaction",
requires_auth=False,
params=(
arg("title", "Short question title (≤120 chars).", required=True),
arg("description", "Optional help prose (≤500 chars)."),
arg(
"widgets",
"Ordered list of widget objects. Types: confirm, choice, choice_multi, "
"text, number, date, select, // (help), group. Each input needs name+label; "
"choice types need options[{value,label}].",
required=True,
kind="array",
),
arg("submit_label", "Primary action label (default Confirm)."),
arg("cancel_label", "Cancel action label (default Cancel)."),
arg("cancelable", "Whether the user may cancel (default true).", kind="boolean"),
arg(
"timeout_sec",
"Optional timeout in seconds (0 = none, max 86400).",
kind="integer",
),
arg("id", "Optional stable interaction id (a-z, digits, hyphens)."),
),
),
Action(
name="ui_cancel",
method="LOCAL",
path="",
summary="Cancel the open interactive prompt",
description="Closes the current open interaction (or the one named by id) with status cancelled.",
handler="interaction",
requires_auth=False,
params=(
arg("id", "Optional interaction id to cancel; defaults to the open one."),
arg("reason", "Optional cancel reason."),
),
),
Action(
name="ui_notify",
method="LOCAL",
path="",
summary="Show a non-blocking ephemeral status on channels that support it",
description=(
"Sends a short status line to the user without waiting. Only effective when the "
"channel advertises ephemeral_status (site-chat). Elsewhere it is skipped."
),
handler="interaction",
requires_auth=False,
params=(arg("text", "Status text to show (≤200 chars).", required=True),),
),
Action(
name="interactions_get",
method="LOCAL",
path="",
summary="Show whether interactive widgets (ui_prompt) are enabled for the user",
description=(
"Returns the effective interactive-widgets preference: enabled (bool), source "
"('user' override or 'default'), the admin default, and the user's override if set. "
"The site default is set by administrators on the Devii service; users may override "
"it on their profile or with interactions_set."
),
handler="interaction",
requires_auth=True,
read_only=True,
),
Action(
name="interactions_set",
method="LOCAL",
path="",
summary="Enable, disable, or reset interactive widgets for the user",
description=(
"Sets whether this account accepts interactive ui_prompt widgets. Pass enabled=true "
"or false to override the admin default; pass reset=true (or omit enabled and set "
"reset) to clear the override and inherit the admin default again. Guests cannot "
"set a preference."
),
handler="interaction",
requires_auth=True,
params=(
arg(
"enabled",
"true to enable interactive widgets, false to disable them. Omit when reset=true.",
kind="boolean",
),
arg(
"reset",
"true to clear the user override and inherit the administrator default again.",
kind="boolean",
),
),
),
)
@@ -0,0 +1,513 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
import asyncio
import logging
import time
from typing import Any, Awaitable, Callable, Optional
import uuid_utils
from .capabilities import (
CHANNEL_CLI,
CHANNEL_SITE_CHAT,
CHANNEL_TELEGRAM,
ChannelContext,
channel_id_for_session,
context_for,
)
from .markdown import project_markdown, project_plain_menu
from .parse import parse_plain_reply
from .schema import (
InteractionRequest,
InteractionResult,
count_input_fields,
validate_prompt_args,
)
logger = logging.getLogger("devii.interaction")
EmitFn = Callable[[dict[str, Any]], Awaitable[None]]
WaitFn = Callable[[str, dict[str, Any], float], Awaitable[Any]]
TelegramPresentFn = Callable[[dict[str, Any]], Awaitable[Any]]
PlainPresentFn = Callable[[str], Awaitable[str]]
class InteractionBroker:
def __init__(
self,
session_channel: str = "main",
*,
owner_kind: str = "guest",
owner_id: str = "",
emit: Optional[EmitFn] = None,
wait_site: Optional[WaitFn] = None,
present_telegram: Optional[TelegramPresentFn] = None,
present_plain: Optional[PlainPresentFn] = None,
) -> None:
self._session_channel = session_channel
self._channel_id = channel_id_for_session(session_channel)
self._owner_kind = owner_kind
self._owner_id = owner_id
self._emit = emit
self._wait_site = wait_site
self._present_telegram = present_telegram
self._present_plain = present_plain
self._open: dict[str, dict[str, Any]] = {}
self._lock = asyncio.Lock()
self._single_flight = True
@property
def channel_id(self) -> str:
return self._channel_id
def set_channel_id(self, channel_id: str) -> None:
self._channel_id = channel_id
def open_id(self) -> str | None:
if not self._open:
return None
return next(iter(self._open))
def open_request(self) -> InteractionRequest | None:
open_id = self.open_id()
if not open_id:
return None
payload = self._open.get(open_id) or {}
request = payload.get("request_model")
if isinstance(request, InteractionRequest):
return request
raw = payload.get("request")
if isinstance(raw, dict):
try:
return InteractionRequest.model_validate(raw)
except Exception:
return None
return None
def answer_text(self, text: str) -> dict[str, Any] | None:
open_id = self.open_id()
if not open_id:
return None
request = self.open_request()
if request is None:
return {
"handled": True,
"status": "error",
"error": "Open interaction has no parseable schema.",
"interaction_id": open_id,
}
status, values, error = parse_plain_reply(request, text)
if status == "error":
return {
"handled": True,
"status": "error",
"error": error or "Could not parse reply.",
"interaction_id": open_id,
}
result = {
"status": status,
"interaction_id": open_id,
"values": values if status == "submitted" else {},
"meta": {
"adapter": self._channel_id,
"degraded": True,
"via": "text",
},
"error": error,
}
return {
"handled": True,
"status": status,
"interaction_id": open_id,
"result": result,
}
def channel_context(self) -> ChannelContext:
owner_kind = getattr(self, "_owner_kind", "guest")
owner_id = getattr(self, "_owner_id", "")
return context_for(
self._channel_id,
open_interaction_id=self.open_id(),
owner_kind=owner_kind,
owner_id=owner_id,
)
def bind(
self,
*,
emit: Optional[EmitFn] = None,
wait_site: Optional[WaitFn] = None,
present_telegram: Optional[TelegramPresentFn] = None,
present_plain: Optional[PlainPresentFn] = None,
) -> None:
if emit is not None:
self._emit = emit
if wait_site is not None:
self._wait_site = wait_site
if present_telegram is not None:
self._present_telegram = present_telegram
if present_plain is not None:
self._present_plain = present_plain
async def prompt(self, arguments: dict[str, Any]) -> InteractionResult:
request = validate_prompt_args(arguments)
async with self._lock:
if self._single_flight and self._open:
for open_id in list(self._open):
await self._finish(
open_id,
status="superseded",
values={},
error=None,
)
interaction_id = request.id or f"ix-{uuid_utils.uuid7().hex[:8]}"
if not interaction_id.startswith("ix-") and not request.id:
interaction_id = f"ix-{interaction_id}"
started = time.monotonic()
payload = {
"interaction_id": interaction_id,
"request": request.model_dump(),
"request_model": request,
"markdown": project_markdown(request, interaction_id),
"plain": project_plain_menu(request),
"channel_id": self._channel_id,
"started": started,
"future": asyncio.get_event_loop().create_future(),
}
self._open[interaction_id] = payload
logger.info(
"interaction open id=%s channel=%s widgets=%d",
interaction_id,
self._channel_id,
count_input_fields(request.widgets),
)
try:
result = await self._present(interaction_id, request, payload)
except Exception as exc:
logger.exception("interaction failed id=%s", interaction_id)
result = InteractionResult(
interaction_id=interaction_id,
status="error",
channel_id=self._channel_id,
values={},
meta={"adapter": self._channel_id, "degraded": True},
error=str(exc),
)
await self._resolve_local(interaction_id, result)
return result
latency_ms = int((time.monotonic() - started) * 1000)
if "latency_ms" not in result.meta:
result.meta["latency_ms"] = latency_ms
if "adapter" not in result.meta:
result.meta["adapter"] = self._channel_id
self._open.pop(interaction_id, None)
logger.info(
"interaction closed id=%s status=%s latency_ms=%s",
interaction_id,
result.status,
result.meta.get("latency_ms"),
)
return result
async def cancel(
self, interaction_id: str = "", reason: str = "cancelled"
) -> InteractionResult:
target = interaction_id or self.open_id()
if not target or target not in self._open:
return InteractionResult(
interaction_id=target or "",
status="error",
channel_id=self._channel_id,
error="No open interaction to cancel.",
)
status = reason if reason in (
"cancelled",
"timeout",
"superseded",
"error",
"channel_changed",
) else "cancelled"
result = InteractionResult(
interaction_id=target,
status=status, # type: ignore[arg-type]
channel_id=self._channel_id,
values={},
meta={"adapter": self._channel_id, "reason": reason},
)
await self._resolve_local(target, result)
return result
def resolve(
self,
interaction_id: str,
*,
status: str,
values: dict[str, Any] | None = None,
error: str | None = None,
meta: dict[str, Any] | None = None,
) -> bool:
payload = self._open.get(interaction_id)
if payload is None:
return False
result = InteractionResult(
interaction_id=interaction_id,
status=status, # type: ignore[arg-type]
channel_id=self._channel_id,
values=values or {},
meta=meta or {"adapter": self._channel_id},
error=error,
)
future = payload.get("future")
if future is not None and not future.done():
future.set_result(result)
return True
async def _present(
self,
interaction_id: str,
request: InteractionRequest,
payload: dict[str, Any],
) -> InteractionResult:
timeout = float(request.timeout_sec or 0)
if self._wait_site is not None and self._channel_id in (
CHANNEL_SITE_CHAT,
CHANNEL_TELEGRAM,
):
return await self._present_site(interaction_id, request, payload, timeout)
if self._channel_id == CHANNEL_TELEGRAM:
return await self._present_telegram_path(
interaction_id, request, payload, timeout
)
return await self._present_plain_path(
interaction_id, request, payload, timeout
)
async def _present_site(
self,
interaction_id: str,
request: InteractionRequest,
payload: dict[str, Any],
timeout: float,
) -> InteractionResult:
if self._wait_site is None:
return await self._present_plain_path(
interaction_id, request, payload, timeout, degraded=True
)
frame = {
"interaction_id": interaction_id,
"channel": self._channel_id,
"title": request.title,
"description": request.description,
"widgets": [w.model_dump() for w in request.widgets],
"submit_label": request.submit_label,
"cancel_label": request.cancel_label,
"cancelable": request.cancelable,
"timeout_sec": request.timeout_sec,
"markdown": payload["markdown"],
"plain": payload["plain"],
"request": request.model_dump(),
}
try:
raw = await self._wait_site(interaction_id, frame, timeout)
except asyncio.TimeoutError:
return InteractionResult(
interaction_id=interaction_id,
status="timeout",
channel_id=self._channel_id,
meta={
"adapter": self._channel_id,
"degraded": self._channel_id != CHANNEL_SITE_CHAT,
},
)
return self._normalize_site_result(
interaction_id,
raw,
adapter=self._channel_id,
degraded=self._channel_id != CHANNEL_SITE_CHAT,
)
async def _present_telegram_path(
self,
interaction_id: str,
request: InteractionRequest,
payload: dict[str, Any],
timeout: float,
) -> InteractionResult:
if self._present_telegram is not None:
try:
raw = await self._present_telegram(
{
"interaction_id": interaction_id,
"request": request.model_dump(),
"markdown": payload["markdown"],
"plain": payload["plain"],
"timeout_sec": request.timeout_sec,
}
)
return self._normalize_site_result(
interaction_id, raw, adapter=CHANNEL_TELEGRAM, degraded=True
)
except asyncio.TimeoutError:
return InteractionResult(
interaction_id=interaction_id,
status="timeout",
channel_id=self._channel_id,
meta={"adapter": CHANNEL_TELEGRAM, "degraded": True},
)
except Exception as exc:
logger.exception("telegram adapter failed, falling back to plain")
payload["plain_error"] = str(exc)
return await self._present_plain_path(
interaction_id, request, payload, timeout, degraded=True
)
async def _present_plain_path(
self,
interaction_id: str,
request: InteractionRequest,
payload: dict[str, Any],
timeout: float,
*,
degraded: bool = False,
) -> InteractionResult:
menu = payload["plain"]
if self._present_plain is not None:
try:
if timeout > 0:
reply = await asyncio.wait_for(
self._present_plain(menu), timeout=timeout
)
else:
reply = await self._present_plain(menu)
except asyncio.TimeoutError:
return InteractionResult(
interaction_id=interaction_id,
status="timeout",
channel_id=self._channel_id,
meta={
"adapter": self._channel_id or CHANNEL_CLI,
"degraded": True,
},
)
status, values, error = parse_plain_reply(request, reply)
return InteractionResult(
interaction_id=interaction_id,
status=status, # type: ignore[arg-type]
channel_id=self._channel_id,
values=values,
meta={
"adapter": self._channel_id or CHANNEL_CLI,
"degraded": True,
},
error=error,
)
future: asyncio.Future = payload["future"]
if self._emit is not None:
await self._emit(
{
"type": "interaction",
"id": interaction_id,
"mode": "plain",
"text": menu,
"markdown": payload["markdown"],
}
)
try:
if timeout > 0:
result = await asyncio.wait_for(future, timeout=timeout)
else:
result = await future
if isinstance(result, InteractionResult):
result.meta.setdefault("degraded", degraded or True)
return result
return self._normalize_site_result(
interaction_id, result, adapter=self._channel_id, degraded=True
)
except asyncio.TimeoutError:
return InteractionResult(
interaction_id=interaction_id,
status="timeout",
channel_id=self._channel_id,
meta={"adapter": self._channel_id, "degraded": True},
)
def _normalize_site_result(
self,
interaction_id: str,
raw: Any,
*,
adapter: str = CHANNEL_SITE_CHAT,
degraded: bool = False,
) -> InteractionResult:
if isinstance(raw, InteractionResult):
return raw
if not isinstance(raw, dict):
return InteractionResult(
interaction_id=interaction_id,
status="error",
channel_id=self._channel_id,
error="Invalid interaction result payload.",
meta={"adapter": adapter, "degraded": degraded},
)
if raw.get("error") and not raw.get("status"):
return InteractionResult(
interaction_id=interaction_id,
status="error",
channel_id=self._channel_id,
error=str(raw.get("error")),
meta={"adapter": adapter, "degraded": degraded},
)
status = str(raw.get("status") or "submitted")
if status not in (
"submitted",
"cancelled",
"timeout",
"superseded",
"error",
"channel_changed",
):
status = "error"
values = raw.get("values") if isinstance(raw.get("values"), dict) else {}
meta = raw.get("meta") if isinstance(raw.get("meta"), dict) else {}
meta.setdefault("adapter", adapter)
meta.setdefault("degraded", degraded)
return InteractionResult(
interaction_id=str(raw.get("interaction_id") or interaction_id),
status=status, # type: ignore[arg-type]
channel_id=self._channel_id,
values=values,
meta=meta,
error=str(raw["error"]) if raw.get("error") else None,
)
async def _finish(
self,
interaction_id: str,
*,
status: str,
values: dict[str, Any],
error: str | None,
) -> None:
result = InteractionResult(
interaction_id=interaction_id,
status=status, # type: ignore[arg-type]
channel_id=self._channel_id,
values=values,
meta={"adapter": self._channel_id},
error=error,
)
await self._resolve_local(interaction_id, result)
async def _resolve_local(
self, interaction_id: str, result: InteractionResult
) -> None:
payload = self._open.pop(interaction_id, None)
if payload is None:
return
future = payload.get("future")
if future is not None and not future.done():
future.set_result(result)
@@ -0,0 +1,217 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
CHANNEL_SITE_CHAT = "site-chat"
CHANNEL_TELEGRAM = "telegram"
CHANNEL_CLI = "cli"
CHANNEL_API = "api"
CHANNEL_UNKNOWN = "unknown"
UI_TOOL_NAMES = frozenset(
{
"ui_prompt",
"ui_cancel",
"ui_notify",
"interactions_get",
"interactions_set",
}
)
DEFAULT_LIMITS = {
"max_options": 32,
"max_fields": 24,
"max_label_chars": 200,
"max_help_chars": 500,
"callback_data_bytes": 0,
}
_SITE_CAPS = {
"rich_widgets": True,
"markdown": True,
"confirm": True,
"choice_single": True,
"choice_multi": True,
"text_input": True,
"number_input": True,
"date_input": True,
"file_input": False,
"inline_buttons": True,
"polls": False,
"web_app": False,
"streaming_partial": True,
"ephemeral_status": True,
}
_TELEGRAM_CAPS = {
"rich_widgets": False,
"markdown": True,
"confirm": True,
"choice_single": True,
"choice_multi": True,
"text_input": True,
"number_input": True,
"date_input": True,
"file_input": False,
"inline_buttons": True,
"polls": True,
"web_app": False,
"streaming_partial": False,
"ephemeral_status": False,
}
_PLAIN_CAPS = {
"rich_widgets": False,
"markdown": True,
"confirm": True,
"choice_single": True,
"choice_multi": True,
"text_input": True,
"number_input": True,
"date_input": True,
"file_input": False,
"inline_buttons": False,
"polls": False,
"web_app": False,
"streaming_partial": False,
"ephemeral_status": False,
}
_EMPTY_CAPS = {key: False for key in _SITE_CAPS}
_MATRICES: dict[str, dict[str, bool]] = {
CHANNEL_SITE_CHAT: _SITE_CAPS,
CHANNEL_TELEGRAM: _TELEGRAM_CAPS,
CHANNEL_CLI: _PLAIN_CAPS,
CHANNEL_API: _PLAIN_CAPS,
CHANNEL_UNKNOWN: _EMPTY_CAPS,
}
_SESSION_CHANNEL_MAP = {
"main": CHANNEL_SITE_CHAT,
"docs": CHANNEL_SITE_CHAT,
"telegram": CHANNEL_TELEGRAM,
"cli": CHANNEL_CLI,
"api": CHANNEL_API,
}
@dataclass
class ChannelContext:
channel_id: str
capabilities: dict[str, bool] = field(default_factory=dict)
limits: dict[str, int] = field(default_factory=dict)
tools: list[str] = field(default_factory=list)
open_interaction_id: str | None = None
def to_dict(self) -> dict[str, Any]:
return {
"channel_id": self.channel_id,
"capabilities": dict(self.capabilities),
"limits": dict(self.limits),
"tools": list(self.tools),
"open_interaction_id": self.open_interaction_id,
}
def channel_id_for_session(session_channel: str) -> str:
return _SESSION_CHANNEL_MAP.get(str(session_channel or "").strip().lower(), CHANNEL_UNKNOWN)
def capabilities_for(channel_id: str) -> dict[str, bool]:
return dict(_MATRICES.get(channel_id, _EMPTY_CAPS))
def limits_for(channel_id: str) -> dict[str, int]:
limits = dict(DEFAULT_LIMITS)
if channel_id == CHANNEL_TELEGRAM:
limits["callback_data_bytes"] = 64
limits["max_options"] = 16
return limits
def interactions_enabled(
owner_kind: str = "guest", owner_id: str = ""
) -> bool:
from .prefs import effective_for
return effective_for(owner_kind, owner_id)
def context_for(
channel_id: str,
*,
open_interaction_id: str | None = None,
tools_allowed: bool = True,
owner_kind: str = "guest",
owner_id: str = "",
) -> ChannelContext:
caps = capabilities_for(channel_id)
limits = limits_for(channel_id)
tools: list[str] = []
pref_tools = ["interactions_get", "interactions_set"]
if owner_kind == "user":
tools.extend(pref_tools)
if (
tools_allowed
and interactions_enabled(owner_kind, owner_id)
and channel_id in (CHANNEL_SITE_CHAT, CHANNEL_TELEGRAM, CHANNEL_CLI, CHANNEL_API)
):
if open_interaction_id:
tools = [t for t in tools if t in pref_tools] + ["ui_cancel"]
else:
tools = list(dict.fromkeys(tools + ["ui_prompt", "ui_cancel"]))
if caps.get("ephemeral_status"):
tools.append("ui_notify")
return ChannelContext(
channel_id=channel_id,
capabilities=caps,
limits=limits,
tools=tools,
open_interaction_id=open_interaction_id,
)
def fragment_for(ctx: ChannelContext) -> str:
active_caps = [name for name, on in ctx.capabilities.items() if on]
caps_text = ",".join(active_caps) if active_caps else "none"
tools_text = ",".join(ctx.tools) if ctx.tools else "none"
interaction = (
f"open:{ctx.open_interaction_id}" if ctx.open_interaction_id else "none"
)
return (
f"CHANNEL: {ctx.channel_id}\n"
f"CAPS: {caps_text}\n"
f"LIMITS: options≤{ctx.limits.get('max_options', 32)} "
f"fields≤{ctx.limits.get('max_fields', 24)}\n"
f"TOOLS: {tools_text}\n"
f"INTERACTION: {interaction}"
)
CA_IWP_SYSTEM_FRAGMENT = (
"# Interactive asks (CA-IWP)\n"
"You are channel-aware. Current channel and capabilities are injected each turn as "
"CHANNEL / CAPS / LIMITS / TOOLS / INTERACTION.\n"
"HARD RULES when TOOLS includes ui_prompt (this is the interactive mode):\n"
"1. You MUST call ui_prompt for every decision, confirm, choice, short form, yes/no, "
"pick-one, multi-select, text/number/date ask, or demo of interactive widgets. Do not "
"simulate buttons in markdown when ui_prompt is available.\n"
"2. Do NOT fall back to numbered lists, fake [Yes]/[No] prose, or 'I would show a widget' "
"while ui_prompt is in TOOLS. Only use degraded markdown menus when ui_prompt is absent "
"from TOOLS.\n"
"3. Never invent HTML, JavaScript, or custom-element tags in your visible reply. "
"The host renders UI from tool args.\n"
"4. Never claim buttons/widgets exist on a channel that lacks them; match wording to CHANNEL.\n"
"5. One open interaction at a time. Wait for the ui_prompt tool result before continuing.\n"
"6. The user may answer with the on-screen controls OR by typing a clear answer "
"(yes/no, option label/number, free text field value, DD/MM/YYYY, or cancel). "
"If they type a new request instead of an answer, treat the interaction as cancelled "
"and continue with their new request.\n"
"7. On status=cancelled or timeout, do not pretend the user agreed. Branch explicitly.\n"
"8. Keep tool arguments compact: short labels, stable value tokens, no essays inside options.\n"
"9. Do not use ui_prompt for long-form research answers or pure information delivery."
)
@@ -0,0 +1,131 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
import json
import logging
from typing import Any
from devplacepy.services.devii.errors import ToolInputError
from .broker import InteractionBroker
from .capabilities import interactions_enabled
from . import prefs
from .schema import sanitize_text
logger = logging.getLogger("devii.interaction")
class InteractionController:
def __init__(
self,
broker: InteractionBroker,
owner_kind: str = "guest",
owner_id: str = "",
) -> None:
self._broker = broker
self._owner_kind = owner_kind
self._owner_id = owner_id
@property
def broker(self) -> InteractionBroker:
return self._broker
async def dispatch(self, name: str, arguments: dict[str, Any]) -> str:
if name == "interactions_get":
return self._pref_get()
if name == "interactions_set":
return self._pref_set(arguments or {})
if name in ("ui_prompt", "ui_notify") and not interactions_enabled(
self._owner_kind, self._owner_id
):
raise ToolInputError(
"Interactive widgets are disabled for this account. "
"Enable them with interactions_set or on the profile, or use a short numbered menu."
)
if name == "ui_prompt":
return await self._prompt(arguments)
if name == "ui_cancel":
return await self._cancel(arguments)
if name == "ui_notify":
return await self._notify(arguments)
raise ToolInputError(f"Unknown interaction tool: {name}")
def _require_user(self) -> None:
if self._owner_kind != "user" or not self._owner_id:
raise ToolInputError(
"Interactive-widget preferences are only available for signed-in users."
)
def _pref_get(self) -> str:
self._require_user()
data = prefs.snapshot(self._owner_kind, self._owner_id)
return json.dumps({"status": "success", **data}, ensure_ascii=False)
def _pref_set(self, arguments: dict[str, Any]) -> str:
self._require_user()
reset = arguments.get("reset")
if isinstance(reset, bool):
do_reset = reset
else:
do_reset = str(reset or "").strip().lower() in ("true", "1", "yes", "on")
if do_reset:
data = prefs.set_user_pref(self._owner_id, None)
return json.dumps({"status": "success", **data}, ensure_ascii=False)
if "enabled" not in arguments or arguments.get("enabled") is None:
raise ToolInputError(
"Pass enabled=true/false to override, or reset=true to inherit the admin default."
)
raw = arguments.get("enabled")
if isinstance(raw, bool):
enabled = raw
else:
enabled = str(raw).strip().lower() in ("true", "1", "yes", "on")
data = prefs.set_user_pref(self._owner_id, enabled)
return json.dumps({"status": "success", **data}, ensure_ascii=False)
async def _prompt(self, arguments: dict[str, Any]) -> str:
ctx = self._broker.channel_context()
if "ui_prompt" not in ctx.tools and self._broker.open_id():
raise ToolInputError(
"An interaction is already open. Wait for it, or call ui_cancel first."
)
if "ui_prompt" not in ctx.tools:
raise ToolInputError(
"ui_prompt is not available. Enable interactive widgets with interactions_set "
"or use a short numbered markdown menu."
)
result = await self._broker.prompt(arguments or {})
return json.dumps(result.to_dict(), ensure_ascii=False)
async def _cancel(self, arguments: dict[str, Any]) -> str:
interaction_id = sanitize_text(arguments.get("id", ""), 64)
reason = sanitize_text(arguments.get("reason", "cancelled"), 40) or "cancelled"
result = await self._broker.cancel(interaction_id, reason=reason)
return json.dumps(result.to_dict(), ensure_ascii=False)
async def _notify(self, arguments: dict[str, Any]) -> str:
text = sanitize_text(arguments.get("text", ""), 200)
if not text:
raise ToolInputError("'text' is required for ui_notify.")
ctx = self._broker.channel_context()
if not ctx.capabilities.get("ephemeral_status"):
return json.dumps(
{
"status": "skipped",
"channel_id": ctx.channel_id,
"message": "ephemeral status is not supported on this channel",
},
ensure_ascii=False,
)
emit = getattr(self._broker, "_emit", None)
if emit is not None:
await emit({"type": "status", "text": text})
return json.dumps(
{
"status": "ok",
"channel_id": ctx.channel_id,
"text": text,
},
ensure_ascii=False,
)
@@ -0,0 +1,106 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
from .schema import InteractionRequest, WidgetModel, flatten_widgets
def project_markdown(request: InteractionRequest, interaction_id: str) -> str:
lines: list[str] = [f"### {request.title}", ""]
if request.description:
lines.append(request.description)
lines.append("")
for widget in request.widgets:
lines.extend(_widget_lines(widget))
actions = f"[{request.submit_label}]"
if request.cancelable:
actions += f" [{request.cancel_label}]"
lines.append("")
lines.append(actions)
lines.append(f"<!-- ai-interaction:{interaction_id} -->")
return "\n".join(lines).strip() + "\n"
def project_plain_menu(request: InteractionRequest) -> str:
lines: list[str] = [request.title]
if request.description:
lines.append(request.description)
lines.append("")
widgets = [
w
for w in flatten_widgets(request.widgets)
if w.type != "//"
]
if len(widgets) == 1 and widgets[0].type == "confirm":
lines.append(f"Reply: yes | no | cancel")
return "\n".join(lines).strip() + "\n"
if len(widgets) == 1 and widgets[0].type in ("choice", "select"):
options = widgets[0].options[:12]
lines.append(f"{widgets[0].label}")
for index, option in enumerate(options, start=1):
lines.append(f"{index}. {option.label} (`{option.value}`)")
lines.append("")
lines.append(f"Reply with a number (1-{len(options)}) or `cancel`.")
return "\n".join(lines).strip() + "\n"
for widget in widgets:
if widget.type in ("choice", "choice_multi", "select"):
lines.append(f"{widget.label}:")
for index, option in enumerate(widget.options[:12], start=1):
lines.append(f" {index}. {option.label} (`{option.value}`)")
elif widget.type == "confirm":
lines.append(f"{widget.label}: yes | no")
else:
lines.append(f"{widget.label}: (free text)")
lines.append("")
lines.append("Reply with values, or `cancel`.")
return "\n".join(lines).strip() + "\n"
def _widget_lines(widget: WidgetModel, indent: str = "") -> list[str]:
if widget.type == "//":
return [f"{indent}{widget.text or widget.label}", ""]
if widget.type == "group":
lines = [f"{indent}**{widget.label}**", ""]
for child in widget.widgets:
lines.extend(_widget_lines(child, indent + " "))
return lines
req = "required" if widget.required else "optional"
lines = [f"{indent}- [ ] **{widget.name}** ({req}) — {widget.label}"]
if widget.help:
lines.append(f"{indent} _{widget.help}_")
if widget.type == "confirm":
default = widget.default
yes = "" if default is True else " "
no = "" if default is False else " "
lines.append(f"{indent} - ({yes}) `true` — Yes")
lines.append(f"{indent} - ({no}) `false` — No")
elif widget.type in ("choice", "select"):
default = str(widget.default) if widget.default is not None else ""
for option in widget.options:
mark = "" if option.value == default else " "
desc = f"{option.description}" if option.description else ""
lines.append(
f"{indent} - ({mark}) `{option.value}` — {option.label}{desc}"
)
elif widget.type == "choice_multi":
defaults = set(widget.default or []) if isinstance(widget.default, list) else set()
for option in widget.options:
mark = "x" if option.value in defaults else " "
desc = f"{option.description}" if option.description else ""
lines.append(
f"{indent} - [{mark}] `{option.value}` — {option.label}{desc}"
)
elif widget.type == "text":
extra = f", max {widget.max_length}" if widget.max_length else ""
lines.append(f"{indent} - _free text{extra}_")
elif widget.type == "number":
parts = []
if widget.min is not None:
parts.append(f"min {widget.min}")
if widget.max is not None:
parts.append(f"max {widget.max}")
hint = (", " + ", ".join(parts)) if parts else ""
lines.append(f"{indent} - _number{hint}_")
elif widget.type == "date":
lines.append(f"{indent} - _date DD/MM/YYYY_")
return lines
@@ -0,0 +1,187 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
import re
from datetime import datetime
from typing import Any
from .schema import InteractionRequest, WidgetModel, flatten_widgets
YES = frozenset({"y", "yes", "true", "1", "ok", "confirm", "ja", "oui", "j"})
NO = frozenset({"n", "no", "false", "0", "nee", "non"})
CANCEL = frozenset({"cancel", "c", "abort", "quit", "annuleren", "stop"})
DATE_FORMATS = (
"%d/%m/%Y",
"%d-%m-%Y",
"%d.%m.%Y",
"%d/%m/%y",
"%d-%m-%y",
"%Y-%m-%d",
)
def parse_plain_reply(
request: InteractionRequest, text: str
) -> tuple[str, dict[str, Any], str | None]:
raw = (text or "").strip()
if not raw:
return "error", {}, "Empty reply."
lower = raw.lower()
if lower in CANCEL:
return "cancelled", {}, None
widgets = [w for w in flatten_widgets(request.widgets) if w.type != "//"]
if not widgets:
return "submitted", {}, None
if len(widgets) == 1:
return _parse_single(widgets[0], raw, strict=True)
if not _looks_like_multi_answer(raw, widgets):
return "error", {}, "Not a structured answer for the open form."
values: dict[str, Any] = {}
chunks = [part.strip() for part in re.split(r"[;\n]+", raw) if part.strip()]
if len(chunks) == 1 and "," in chunks[0] and len(widgets) > 1:
chunks = [part.strip() for part in chunks[0].split(",") if part.strip()]
if len(chunks) != len(widgets):
return (
"error",
{},
f"Expected {len(widgets)} values (one per field), got {len(chunks)}.",
)
for widget, chunk in zip(widgets, chunks):
status, partial, err = _parse_single(widget, chunk, strict=True)
if status != "submitted":
return status, {}, err
values.update(partial)
return "submitted", values, None
def _looks_like_multi_answer(raw: str, widgets: list[WidgetModel]) -> bool:
if ";" in raw or "\n" in raw:
return True
if "," in raw and len(widgets) > 1:
return True
return False
def _parse_single(
widget: WidgetModel, raw: str, *, strict: bool = False
) -> tuple[str, dict[str, Any], str | None]:
text = raw.strip()
lower = text.lower()
if lower in CANCEL:
return "cancelled", {}, None
name = widget.name
if widget.type == "confirm":
if lower in YES:
return "submitted", {name: True}, None
if lower in NO:
return "submitted", {name: False}, None
return "error", {}, "Reply yes/no (or ja/nee)."
if widget.type in ("choice", "select"):
value = _match_option(widget, text)
if value is None:
return "error", {}, f"Unknown option for {name}."
return "submitted", {name: value}, None
if widget.type == "choice_multi":
parts = [p.strip() for p in re.split(r"[,\s]+", text) if p.strip()]
values: list[str] = []
for part in parts:
matched = _match_option(widget, part)
if matched is None:
return "error", {}, f"Unknown option {part!r} for {name}."
if matched not in values:
values.append(matched)
if widget.required and not values:
return "error", {}, f"{name} requires at least one option."
return "submitted", {name: values}, None
if widget.type == "number":
if strict and not re.fullmatch(r"[+-]?\d+(?:[.,]\d+)?", text):
return "error", {}, f"{name} must be a number."
normalized = text.replace(",", ".")
try:
number = float(normalized) if "." in normalized else int(normalized)
except ValueError:
return "error", {}, f"{name} must be a number."
if widget.min is not None and number < widget.min:
return "error", {}, f"{name} must be ≥ {widget.min}."
if widget.max is not None and number > widget.max:
return "error", {}, f"{name} must be ≤ {widget.max}."
return "submitted", {name: number}, None
if widget.type == "date":
if strict and not _looks_like_date(text):
return "error", {}, f"{name} must be a date as DD/MM/YYYY."
european = _parse_date_european(text)
if european is None:
return "error", {}, f"{name} must be a date as DD/MM/YYYY."
return "submitted", {name: european}, None
if widget.type == "text":
if strict and _looks_like_command(text):
return "error", {}, f"{name} does not look like a field answer."
if widget.max_length and len(text) > widget.max_length:
return "error", {}, f"{name} exceeds max length {widget.max_length}."
if widget.required and not text:
return "error", {}, f"{name} is required."
return "submitted", {name: text}, None
return "error", {}, f"Unsupported widget type {widget.type}."
def _looks_like_date(text: str) -> bool:
return bool(
re.fullmatch(r"\d{1,2}[/.-]\d{1,2}[/.-]\d{2,4}", text.strip())
or re.fullmatch(r"\d{4}-\d{2}-\d{2}", text.strip())
)
def _looks_like_command(text: str) -> bool:
value = text.strip().lower()
if not value:
return False
if value.startswith("/"):
return True
starters = (
"show me ",
"laat ",
"geef ",
"toon ",
"please ",
"can you ",
"could you ",
"i want ",
"ik wil ",
"volgende",
"next ",
)
return any(value.startswith(prefix) for prefix in starters)
def _parse_date_european(text: str) -> str | None:
value = text.strip()
for fmt in DATE_FORMATS:
try:
return datetime.strptime(value, fmt).strftime("%d/%m/%Y")
except ValueError:
continue
return None
def _match_option(widget: WidgetModel, text: str) -> str | None:
raw = text.strip()
if raw.isdigit():
index = int(raw)
if 1 <= index <= len(widget.options):
option = widget.options[index - 1]
if not option.disabled:
return option.value
lower = raw.lower()
for option in widget.options:
if option.disabled:
continue
if option.value == raw or option.value.lower() == lower:
return option.value
if option.label.lower() == lower:
return option.value
return None
@@ -0,0 +1,119 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
from typing import Any
INHERIT = -1
FIELD_DEFAULT = "devii_interactions_default"
FIELD_LEGACY = "devii_interactions_enabled"
def _truthy(raw: Any, default: bool = True) -> bool:
if raw is None:
return default
if isinstance(raw, bool):
return raw
if isinstance(raw, (int, float)):
return raw != 0
text = str(raw).strip().lower()
if text in ("",):
return default
return text not in ("0", "false", "off", "no")
def admin_default() -> bool:
try:
from devplacepy.database import get_setting
raw = get_setting(FIELD_DEFAULT, "")
if raw == "" or raw is None:
raw = get_setting(FIELD_LEGACY, "1")
return _truthy(raw, True)
except Exception:
return True
def _coerce_user_pref(raw: Any) -> int | None:
if raw is None:
return None
try:
value = int(raw)
except (TypeError, ValueError):
return None
if value == INHERIT:
return None
if value in (0, 1):
return value
return None
def user_pref_raw(user: dict[str, Any] | None) -> int | None:
if not user:
return None
return _coerce_user_pref(user.get("interactions_enabled"))
def effective_for_user(user: dict[str, Any] | None) -> bool:
pref = user_pref_raw(user)
if pref is None:
return admin_default()
return bool(pref)
def effective_for(owner_kind: str, owner_id: str = "") -> bool:
if owner_kind != "user" or not owner_id:
return admin_default()
try:
from devplacepy.database import get_table
user = get_table("users").find_one(uid=owner_id)
except Exception:
return admin_default()
return effective_for_user(user)
def snapshot(owner_kind: str, owner_id: str = "", user: dict[str, Any] | None = None) -> dict[str, Any]:
default = admin_default()
if owner_kind != "user":
return {
"enabled": default,
"source": "default",
"default": default,
"override": None,
}
if user is None and owner_id:
try:
from devplacepy.database import get_table
user = get_table("users").find_one(uid=owner_id)
except Exception:
user = None
pref = user_pref_raw(user)
if pref is None:
return {
"enabled": default,
"source": "default",
"default": default,
"override": None,
}
return {
"enabled": bool(pref),
"source": "user",
"default": default,
"override": bool(pref),
}
def set_user_pref(user_uid: str, enabled: bool | None) -> dict[str, Any]:
from devplacepy.database import get_table
from devplacepy.utils import clear_user_cache
value = INHERIT if enabled is None else (1 if enabled else 0)
get_table("users").update(
{"uid": user_uid, "interactions_enabled": value},
["uid"],
)
clear_user_cache(user_uid)
user = get_table("users").find_one(uid=user_uid) or {"uid": user_uid}
return snapshot("user", user_uid, user)
@@ -0,0 +1,260 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
import re
from typing import Any, Literal
from pydantic import BaseModel, Field, field_validator, model_validator
from .capabilities import DEFAULT_LIMITS
WIDGET_TYPES = (
"confirm",
"choice",
"choice_multi",
"text",
"number",
"date",
"select",
"//",
"group",
)
VALUE_RE = re.compile(r"^[a-z0-9_][a-z0-9_-]{0,63}$")
NAME_RE = re.compile(r"^[a-zA-Z_][a-zA-Z0-9_]{0,63}$")
ID_RE = re.compile(r"^[a-z][a-z0-9-]{0,63}$")
CONTROL_CHARS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]")
def sanitize_text(value: Any, max_len: int) -> str:
text = CONTROL_CHARS.sub("", str(value if value is not None else ""))
text = " ".join(text.split())
if len(text) > max_len:
text = text[:max_len]
return text
class OptionModel(BaseModel):
value: str
label: str
description: str = ""
disabled: bool = False
@field_validator("value")
@classmethod
def _value(cls, v: str) -> str:
text = sanitize_text(v, 64).lower().replace(" ", "_")
if not VALUE_RE.match(text):
raise ValueError(
"option value must match ^[a-z0-9_][a-z0-9_-]{0,63}$"
)
return text
@field_validator("label")
@classmethod
def _label(cls, v: str) -> str:
text = sanitize_text(v, 80)
if not text:
raise ValueError("option label is required")
return text
@field_validator("description")
@classmethod
def _description(cls, v: str) -> str:
return sanitize_text(v, 120)
class WidgetModel(BaseModel):
type: Literal[
"confirm",
"choice",
"choice_multi",
"text",
"number",
"date",
"select",
"//",
"group",
]
name: str = ""
label: str = ""
help: str = ""
text: str = ""
required: bool = True
default: Any = None
display: Literal["auto", "radio", "select", "buttons"] = "auto"
options: list[OptionModel] = Field(default_factory=list)
widgets: list[WidgetModel] = Field(default_factory=list)
max_length: int | None = None
placeholder: str = ""
pattern: str = ""
min: float | None = None
max: float | None = None
step: float | None = None
min_selected: int | None = None
max_selected: int | None = None
@field_validator("name")
@classmethod
def _name(cls, v: str) -> str:
text = sanitize_text(v, 64)
return text
@field_validator("label", "help", "text", "placeholder", "pattern")
@classmethod
def _strings(cls, v: str) -> str:
return sanitize_text(v, 500)
@model_validator(mode="after")
def _shape(self) -> WidgetModel:
if self.type == "//":
if not self.text and not self.label:
raise ValueError("help widget (//) requires text")
if not self.text:
self.text = self.label
return self
if self.type == "group":
if not self.label:
raise ValueError("group requires label")
if not self.widgets:
raise ValueError("group requires nested widgets")
if len(self.widgets) > 24:
raise ValueError("group may contain at most 24 widgets")
return self
if not self.name or not NAME_RE.match(self.name):
raise ValueError(
f"widget name required and must match ^[a-zA-Z_][a-zA-Z0-9_]{{0,63}}$ (got {self.name!r})"
)
if not self.label:
raise ValueError(f"widget '{self.name}' requires label")
self.label = sanitize_text(self.label, 200)
self.help = sanitize_text(self.help, 500)
if self.type in ("choice", "choice_multi", "select"):
if not self.options:
raise ValueError(f"widget '{self.name}' requires options")
if len(self.options) > DEFAULT_LIMITS["max_options"]:
raise ValueError(
f"widget '{self.name}' has too many options (max {DEFAULT_LIMITS['max_options']})"
)
if self.type == "select":
self.display = "select"
return self
class InteractionRequest(BaseModel):
title: str
description: str = ""
widgets: list[WidgetModel]
submit_label: str = "Confirm"
cancel_label: str = "Cancel"
cancelable: bool = True
timeout_sec: int = 0
id: str = ""
@field_validator("title")
@classmethod
def _title(cls, v: str) -> str:
text = sanitize_text(v, 120)
if not text:
raise ValueError("title is required")
return text
@field_validator("description")
@classmethod
def _description(cls, v: str) -> str:
return sanitize_text(v, 500)
@field_validator("submit_label", "cancel_label")
@classmethod
def _labels(cls, v: str) -> str:
return sanitize_text(v, 40) or "Confirm"
@field_validator("timeout_sec")
@classmethod
def _timeout(cls, v: int) -> int:
n = int(v or 0)
if n < 0 or n > 86400:
raise ValueError("timeout_sec must be between 0 and 86400")
return n
@field_validator("id")
@classmethod
def _id(cls, v: str) -> str:
text = sanitize_text(v, 64).lower()
if text and not ID_RE.match(text):
raise ValueError("id must match ^[a-z][a-z0-9-]{0,63}$")
return text
@model_validator(mode="after")
def _widgets(self) -> InteractionRequest:
if not self.widgets:
raise ValueError("widgets must contain at least one item")
if len(self.widgets) > DEFAULT_LIMITS["max_fields"]:
raise ValueError(
f"at most {DEFAULT_LIMITS['max_fields']} top-level widgets"
)
depth = _max_depth(self.widgets)
if depth > 3:
raise ValueError("widget nesting depth must be ≤ 3")
return self
class InteractionResult(BaseModel):
interaction_id: str
status: Literal[
"submitted", "cancelled", "timeout", "superseded", "error", "channel_changed"
]
channel_id: str
values: dict[str, Any] = Field(default_factory=dict)
meta: dict[str, Any] = Field(default_factory=dict)
error: str | None = None
def to_dict(self) -> dict[str, Any]:
return {
"interaction_id": self.interaction_id,
"status": self.status,
"channel_id": self.channel_id,
"values": {} if self.status != "submitted" else dict(self.values),
"meta": dict(self.meta),
"error": self.error,
}
def _max_depth(widgets: list[WidgetModel], depth: int = 1) -> int:
deepest = depth
for widget in widgets:
if widget.type == "group" and widget.widgets:
deepest = max(deepest, _max_depth(widget.widgets, depth + 1))
return deepest
def validate_prompt_args(arguments: dict[str, Any]) -> InteractionRequest:
try:
return InteractionRequest.model_validate(arguments or {})
except Exception as exc:
from devplacepy.services.devii.errors import ToolInputError
raise ToolInputError(f"Invalid ui_prompt arguments: {exc}") from exc
def count_input_fields(widgets: list[WidgetModel]) -> int:
total = 0
for widget in widgets:
if widget.type == "//":
continue
if widget.type == "group":
total += count_input_fields(widget.widgets)
else:
total += 1
return total
def flatten_widgets(widgets: list[WidgetModel]) -> list[WidgetModel]:
out: list[WidgetModel] = []
for widget in widgets:
if widget.type == "group":
out.extend(flatten_widgets(widget.widgets))
else:
out.append(widget)
return out
+2
View File
@@ -19,6 +19,7 @@ from .actions.notification_actions import NOTIFICATION_ACTIONS
from .actions.rsearch_actions import RSEARCH_ACTIONS
from .actions.spec import Catalog
from .actions.telegram_actions import TELEGRAM_ACTIONS
from .interaction.actions import INTERACTION_ACTIONS
from .virtual_tools.actions import VIRTUAL_TOOL_ACTIONS
from .agentic.actions import AGENTIC_ACTIONS
from .tasks.actions import TASK_ACTIONS
@@ -42,6 +43,7 @@ CATALOG = Catalog(
+ AI_MODIFIER_ACTIONS
+ EMAIL_ACTIONS
+ TELEGRAM_ACTIONS
+ INTERACTION_ACTIONS
+ VIRTUAL_TOOL_ACTIONS
)
+46 -2
View File
@@ -126,6 +126,16 @@ class DeviiService(BaseService):
"raw JavaScript execution; read-only tools (context, discover, read) are unaffected.",
group="Agent",
),
ConfigField(
config.FIELD_INTERACTIONS_DEFAULT,
"Interactive widgets default",
type="bool",
default=True,
help="Site default for CA-IWP interactive prompts (ui_prompt). Guests always use "
"this default. Signed-in users inherit it until they override it on their profile "
"or with the interactions_set tool.",
group="Agent",
),
ConfigField(
config.FIELD_USER_DAILY_USD,
"Max USD per user / 24h",
@@ -235,6 +245,24 @@ class DeviiService(BaseService):
help="Connection/read timeout for IMAP and SMTP calls.",
group="Email",
),
ConfigField(
"devii_lessons_max_per_owner",
"Max lessons per owner",
type="int",
default=500,
minimum=0,
help="Soft cap on active lessons per user/guest. When exceeded, the oldest lessons are pruned. 0 disables the cap.",
group="Memory",
),
ConfigField(
"devii_lessons_max_age_days",
"Max lesson age (days)",
type="int",
default=90,
minimum=1,
help="Lessons older than this are soft-deleted on the periodic housekeeping pass.",
group="Memory",
),
]
def __init__(self):
@@ -316,12 +344,28 @@ class DeviiService(BaseService):
ensured = self._ensure_task_schedulers()
pruned = hub.ledger.prune(48)
removed = await hub.gc_idle()
if pruned or removed or ensured:
lessons_pruned = self._prune_old_lessons()
if pruned or removed or ensured or lessons_pruned:
self.log(
f"Housekeeping: ensured {ensured} task scheduler(s), pruned {pruned} "
f"ledger rows, closed {removed} idle sessions"
f"ledger rows, pruned {lessons_pruned} lessons, closed {removed} idle sessions"
)
def _prune_old_lessons(self) -> int:
try:
from devplacepy.database import db
from .agentic.lessons import LessonStore, _read_retention_settings
_, max_age = _read_retention_settings(db)
store = LessonStore(db, "_global", "_global")
pruned = store.prune_all_owners(max_age)
if pruned:
logger.info("Devii lessons housekeeping: pruned %d across all owners", pruned)
return pruned
except Exception:
logger.exception("Devii lessons housekeeping failed")
return 0
def _ensure_task_schedulers(self) -> int:
from devplacepy.database import db
from devplacepy.utils import is_admin, is_primary_admin
+198 -9
View File
@@ -19,10 +19,18 @@ from ..config import Settings
from ..cost import CostTracker
from ..cost.tracker import Pricing
from ..http_client import PlatformClient
from ..interaction import InteractionController
from ..interaction.broker import InteractionBroker
from ..interaction.capabilities import (
CA_IWP_SYSTEM_FRAGMENT,
UI_TOOL_NAMES,
fragment_for,
)
from ..llm import LLMClient
from ..registry import CATALOG
from ..tasks import Scheduler, TaskController, TaskStore
from ..virtual_tools import VirtualToolController, VirtualToolStore
from ..text import normalize_newlines
from ._helpers import _format_offset, _now_iso, _repair_history
from .prompts import (
BEHAVIOR_HEADER,
@@ -98,6 +106,14 @@ class DeviiSession:
)
self._behavior_store = behavior_store
self.behavior = BehaviorController(behavior_store)
self.interaction_broker = InteractionBroker(
session_channel=channel,
owner_kind=owner_kind,
owner_id=owner_id,
)
self.interaction = InteractionController(
self.interaction_broker, owner_kind=owner_kind, owner_id=owner_id
)
self.dispatcher = Dispatcher(
CATALOG,
self.client,
@@ -113,6 +129,7 @@ class DeviiSession:
owner_id=owner_id,
virtual_tools=self.virtual_tools,
behavior=self.behavior,
interaction=self.interaction,
)
self.tools = self._builtin_tools()
self._system_prompt = (
@@ -164,8 +181,16 @@ class DeviiSession:
self._turn_epoch = 0
def restore_history(self, messages: list[dict[str, Any]]) -> None:
if messages:
self.agent._messages = messages
if not messages:
return
repaired: list[dict[str, Any]] = []
for message in messages:
item = dict(message)
content = item.get("content")
if isinstance(content, str):
item["content"] = normalize_newlines(content)
repaired.append(item)
self.agent._messages = repaired
def history(self) -> list[dict[str, Any]]:
visible: list[dict[str, Any]] = []
@@ -174,11 +199,12 @@ class DeviiSession:
content = message.get("content")
if role not in ("user", "assistant") or not content:
continue
if (
role == "user"
and isinstance(content, str)
and content.startswith(INTERNAL_PREFIXES)
):
if not isinstance(content, str):
continue
content = normalize_newlines(content).strip()
if not content:
continue
if role == "user" and content.startswith(INTERNAL_PREFIXES):
continue
visible.append({"role": role, "content": content})
return visible
@@ -228,6 +254,10 @@ class DeviiSession:
self._disconnected.clear()
self.avatar.bind(self._avatar_request)
self.browser.bind(self._client_request)
self.interaction_broker.bind(
emit=self._emit_interaction,
wait_site=self._interaction_wait,
)
self.ensure_scheduler_started()
if self._buffer:
pending = self._buffer
@@ -248,6 +278,14 @@ class DeviiSession:
self._disconnected.set()
self.avatar.unbind()
self.browser.unbind()
open_id = self.interaction_broker.open_id()
if open_id:
self.interaction_broker.resolve(
open_id,
status="cancelled",
values={},
error="Browser disconnected.",
)
logger.info(
"Session %s/%s detached (%d conns)",
self.owner_kind,
@@ -457,7 +495,13 @@ class DeviiSession:
for s in schemas
if s.get("function", {}).get("name") in DOCS_TOOLS
]
return schemas
allowed = set(self.interaction_broker.channel_context().tools)
return [
s
for s in schemas
if s.get("function", {}).get("name") not in UI_TOOL_NAMES
or s.get("function", {}).get("name") in allowed
]
def _refresh_tools(self) -> None:
if self.channel == "docs":
@@ -471,7 +515,14 @@ class DeviiSession:
return self._system_prompt
body = self._behavior_store.text().strip()
section = BEHAVIOR_HEADER if not body else f"{BEHAVIOR_HEADER}\n{body}"
return f"{self._system_prompt}\n\n{self._clock_line()}\n\n{section}"
channel_block = fragment_for(self.interaction_broker.channel_context())
return (
f"{self._system_prompt}\n\n"
f"{CA_IWP_SYSTEM_FRAGMENT}\n\n"
f"{channel_block}\n\n"
f"{self._clock_line()}\n\n"
f"{section}"
)
def _clock_line(self) -> str:
from datetime import datetime, timedelta, timezone
@@ -745,6 +796,144 @@ class DeviiSession:
if len(self._buffer) > 100:
self._buffer = self._buffer[-100:]
async def _emit_interaction(self, payload: dict[str, Any]) -> None:
await self._emit(payload, buffer=False)
async def _interaction_wait(
self, interaction_id: str, frame: dict[str, Any], timeout: float
) -> Any:
loop = asyncio.get_event_loop()
deadline_seconds = timeout if timeout and timeout > 0 else BROWSER_REQUEST_DEADLINE_SECONDS
if timeout and timeout > 0:
deadline_seconds = max(timeout, 1.0)
else:
deadline_seconds = max(BROWSER_REQUEST_DEADLINE_SECONDS, 300.0)
deadline = loop.time() + deadline_seconds
request_id = f"ix:{interaction_id}"
future: asyncio.Future = loop.create_future()
self._pending[request_id] = future
wire = {
"type": "interaction",
"id": request_id,
"interaction_id": interaction_id,
"args": frame,
}
try:
while not future.done():
remaining = deadline - loop.time()
if remaining <= 0:
raise asyncio.TimeoutError()
try:
await asyncio.wait_for(self._connected.wait(), timeout=remaining)
except asyncio.TimeoutError as exc:
raise asyncio.TimeoutError() from exc
ws = self._pick_target()
if ws is None:
continue
try:
await self._send_to(ws, wire)
except Exception:
await asyncio.sleep(0.1)
continue
drop = asyncio.create_task(self._disconnected.wait())
try:
await asyncio.wait(
{future, drop},
timeout=max(0.0, deadline - loop.time()),
return_when=asyncio.FIRST_COMPLETED,
)
finally:
drop.cancel()
return future.result()
finally:
self._pending.pop(request_id, None)
def resolve_interaction(self, interaction_id: str, payload: Any) -> None:
request_id = str(interaction_id or "")
if not request_id.startswith("ix:"):
request_id = f"ix:{request_id}"
future = self._pending.get(request_id)
if future is not None and not future.done():
future.set_result(payload)
self._pending.pop(request_id, None)
bare = request_id[3:]
if isinstance(payload, dict):
self.interaction_broker.resolve(
bare,
status=str(payload.get("status") or "submitted"),
values=payload.get("values")
if isinstance(payload.get("values"), dict)
else {},
error=payload.get("error"),
meta=payload.get("meta")
if isinstance(payload.get("meta"), dict)
else None,
)
return
bare = request_id[3:] if request_id.startswith("ix:") else request_id
self.interaction_broker.resolve(
bare,
status=str((payload or {}).get("status") or "submitted")
if isinstance(payload, dict)
else "error",
values=(payload or {}).get("values")
if isinstance(payload, dict)
else {},
error=(payload or {}).get("error") if isinstance(payload, dict) else None,
meta=(payload or {}).get("meta") if isinstance(payload, dict) else None,
)
async def try_answer_interaction(self, text: str) -> bool:
open_id = self.interaction_broker.open_id()
if not open_id:
return False
pending = self._pending.get(f"ix:{open_id}")
if pending is not None and pending.done():
return False
outcome = self.interaction_broker.answer_text(text)
if not outcome or not outcome.get("handled"):
return False
if outcome.get("status") == "error":
result = {
"status": "cancelled",
"interaction_id": open_id,
"values": {},
"meta": {
"adapter": "site-chat",
"via": "text_non_answer",
"degraded": True,
},
"error": None,
}
self.resolve_interaction(open_id, result)
await self._emit(
{
"type": "interaction_closed",
"id": f"ix:{open_id}",
"interaction_id": open_id,
"result": result,
},
buffer=False,
)
return False
result = outcome.get("result") or {
"status": outcome.get("status") or "submitted",
"interaction_id": open_id,
"values": {},
"meta": {"adapter": "site-chat", "via": "text", "degraded": True},
}
self.resolve_interaction(open_id, result)
await self._emit(
{
"type": "interaction_closed",
"id": f"ix:{open_id}",
"interaction_id": open_id,
"result": result,
},
buffer=False,
)
return True
async def _avatar_request(self, action: str, args: dict[str, Any]) -> Any:
return await self._browser_request("avatar", action, args)
+19
View File
@@ -18,6 +18,25 @@ BLANK_LINES = re.compile(r"\n\s*\n\s*\n+")
TRAILING_SPACE = re.compile(r"[ \t]+\n")
SKIP_HREF_PREFIXES = ("#", "javascript:")
def normalize_newlines(text: str) -> str:
if not text or not isinstance(text, str):
return "" if text is None else str(text)
value = text
while "\\\\n" in value:
value = value.replace("\\\\n", "\\n")
while "\\\\t" in value:
value = value.replace("\\\\t", "\\t")
escaped_n = value.count("\\n")
real_n = value.count("\n")
if escaped_n > real_n:
value = value.replace("\\n", "\n")
escaped_t = value.count("\\t")
real_t = value.count("\t")
if escaped_t > real_t:
value = value.replace("\\t", "\t")
return value
HIDDEN = "[hidden]"
REDACT_FIELD_KEYS = frozenset(
{