# Service framework Two files make up the framework: `services/base.py` (the `ConfigField` and `BaseService` classes) and `services/manager.py` (the `ServiceManager` singleton). Together they give every service identical configuration, supervision, persistence, and monitoring. See also the [Services overview](/docs/services-overview.html) and [Reading service data](/docs/services-data.html). > Audience: administrators and maintainers. ## ConfigField A `ConfigField` is one typed, validated setting backed by a `site_settings` row. Fields declare a `key`, `label`, `type`, `default`, `help`, optional `group`, optional `minimum`/`maximum`, optional `options`, and a `secret` flag. Supported types and their coercion: `int` and `float` (range-checked against `minimum`/`maximum`), `bool` (stored as `"0"`/`"1"`, rendered as a select), `select` (value must be one of `options`), `url` (must start with `http://` or `https://` if non-empty), and `str`. Bad input raises a readable `ValueError`, which the save handler turns into a per-field error message. Key methods: - `read()` returns the coerced value, or the default if the row is empty or invalid. - `display_value()` returns the stored string for the form, but always returns empty for secrets so a key is never echoed back. - `spec()` is the dict the admin UI and JSON consume, including `is_set` for secrets so the UI can show "configured" without revealing the value. ## BaseService A service subclass sets class attributes and overrides `run_once`: ```python class MyService(BaseService): title = "My service" description = "What it does." default_enabled = True config_fields = [ConfigField("my_threshold", "Threshold", type="int", default=5)] async def run_once(self) -> None: ... ``` Every service automatically gets three inherited fields on top of its own: `Enabled` (bool, in group General), `Run interval (seconds)` (int, floored at `min_interval`, in General), and `Log buffer size` (int, 1 to 200, in Advanced). The setting keys are derived from the service name: `service_{name}_enabled`, `service_{name}_interval`, `service_{name}_command`, `service_{name}_log_size`. Optional overrides: - `async def on_enable(self)` / `async def on_disable(self)` run when the service transitions between enabled and disabled. - `def collect_metrics(self) -> dict` returns the service-specific data surfaced in `describe()`. The default returns `{}`. Errors here are caught and logged, never fatal. ## Lifecycle and the run loop The supervisor is an asyncio task started by `start_supervisor()` and stopped cooperatively by `request_shutdown()` plus task cancellation. The loop ticks every `TICK_SECONDS` (1s) and on each tick: 1. Syncs the log buffer size to the configured value. 2. Handles pending admin commands (`run` requests an immediate run; `clear` empties the log buffer). Commands are delivered through the `service_{name}_command` setting with a monotonic counter so a repeated command is still seen. 3. If enabled and not yet started, marks the service running, records `started_at`, and calls `on_enable()`. 4. If a run is due (`now >= next_due`) or a run was requested, calls `_execute_run()`. 5. If disabled while running, calls `on_disable()` and clears the running state. 6. Persists state (throttled). `_execute_run()` re-reads the interval live, stamps `last_run`, runs `run_once()` inside a try/except that logs any error to the buffer, then schedules `next_due = now + interval`. Because the interval is read every run, retuning it on `/admin/services` takes effect immediately. Timing constants: `TICK_SECONDS = 1`, `PERSIST_SECONDS = 3` (minimum gap between non-forced state writes), `STALE_SECONDS = 15` (a heartbeat older than this reads as `stalled`). ## State persistence and status `_persist_state()` upserts a row in `service_state` keyed by `name`, holding `status`, `last_run`, `next_run`, `started_at`, `heartbeat`, `logs` (JSON of the log deque), `metrics` (JSON of `collect_metrics()`), and `updated_at`. Writes are throttled to once per `PERSIST_SECONDS` unless forced at a run boundary. `describe()` reads that row and derives the **display status**: - `stopped` when the service is disabled. - `stalled` when enabled but the heartbeat is missing or older than `STALE_SECONDS` (the loop is not ticking). - `running` when enabled and the heartbeat is fresh. `describe()` also computes `uptime` from `started_at`, and returns `interval_seconds`, `last_run`, `next_run`, `heartbeat`, `log_buffer`, `metrics`, the flat `fields` specs, and `field_groups` (fields bucketed by their `group`, in declaration order) for the config form. ## ServiceManager The singleton `service_manager` (imported from `services/manager.py`) holds the registry and the lock-ownership flag: - `register(service)` adds a service by name. - `describe_all()` returns `describe()` for every service; this backs `GET /admin/services/data`. - `set_enabled(name, bool)` writes the enabled setting; `send_command(name, verb)` queues a `run` or `clear`. - `save_config(name, form)` coerces and validates every submitted field, skipping empty secrets, and either writes all values or returns `{"ok": False, "errors": {...}}` with no partial write. - `supervise()` starts the supervisor task for every registered service. - `shutdown_all()` requests shutdown, cancels the tasks, and awaits them. ## Single-owner supervision across workers Production runs multiple uvicorn workers. At startup each worker tries to acquire an exclusive non-blocking lock on `devplace-services.lock`. Exactly one wins, calls `service_manager.supervise()`, and sets `set_lock_owner(True)`; the rest decline and run no services. `owns_lock()` reports the result. This guarantees each service runs once across the whole deployment: news is fetched once, the bot fleet is sized once, the gateway pool and circuit breaker are per-owner. It is also why `/devii/ws` is served only by the lock owner; non-owner workers close new Devii sockets with code 1013 so the client reconnects until it reaches the owner. `DEVPLACE_DISABLE_SERVICES=1` skips all supervision (the test suite does this). ## Adding a service 1. Subclass `BaseService`, set `title`, `description`, `default_enabled`, `min_interval`, and `config_fields`. 2. Override `run_once()`; optionally `collect_metrics()`, `on_enable()`, `on_disable()`. 3. Register it in `main.py` startup with `service_manager.register(YourService())`. It then appears on `/admin/services`, configurable, supervised, persisted, and monitored, with no further work.