**Implementation Plan: XML-RPC Bridge Crash Loop (15-second fork-and-exit)** **Objective:** Eliminate the 15-second crash–restart cycle in the XML-RPC bridge process when managed by the `XmlrpcService` supervisor. **Rationale:** Two primary architectural defects cause the crash: 1. The forked child shares the parent’s process group, making it susceptible to signal propagation from the supervisor’s 15‑second `STALE_SECONDS` cycle. 2. `ForkingMixIn` + the `httpx` HTTP client used (implicitly or explicitly) in the bridge creates a thread‑safety hazard after `fork()`, leading to corruption and segfaults. The following changes address both root causes, add startup‑error logging, and introduce retry backoff to prevent rapid reconnect storms. --- ### Files and Changes **1. `devplacepy/services/xmlrpc/__init__.py` – Process isolation and backoff** - **`_spawn()` method (around line 30–40):** - Add `start_new_session=True` to the `subprocess.Popen` call. This detaches the child from the parent’s process group, insulating it from any signals emitted by the supervisor loop. - Implement exponential backoff before repawning after a crash: - Maintain a class‑level counter `_crash_count` (or an instance counter reset on successful startup). - Before calling `Popen`, sleep `min(1 * 2 **_crash_count, 60)` seconds (with ±20% random jitter). - Reset `_crash_count` to 0 when the child stays alive for at least `STALE_SECONDS` (detectable by checking that a heartbeat file or last‑check timestamp is updated). For simplicity, increment on each call and only sleep if the previous spawn exited quickly (< `STALE_SECONDS`). The exact heuristic can be: if the last spawn duration (measured inside the supervisor loop) is less than `STALE_SECONDS`, do a backoff sleep. **2. `devplacepy/services/xmlrpc/server.py` – ForkingMixIn replacement and try‑block fix** - **Line 114 (inside `main()`):** Move `server = build_server(XMLRPC_BIND, XMLRPC_PORT)` inside the `try` block so that any exception raised during server construction is logged via `logging.exception`. This eliminates the silent‑crash edge case. - **Class definition of `XMLRPCServer` (likely early in file, e.g., line 15–20):** Replace `ForkingMixIn` with `ThreadingMixIn`. ```python from socketserver import ThreadingMixIn class XMLRPCServer(ThreadingMixIn, SimpleXMLRPCServer): allow_reuse_address = True ``` (Remove any only‑style `ForkingMixIn` import; the class will now serve each request in a separate thread instead of a forked process. This is safe because the bridge volume is low and the parent event loop is not blocked.) Ensure that `allow_reuse_address = True` is kept (it often is already) so the socket is not held by a dead forked process. **3. `devplacepy/services/base.py` – (Optional but recommended)** No changes required; the supervisor loop logic remains correct. However, verify that `STALE_SECONDS = 15` is the correct threshold. It can be kept as‑is; the backoff in `_spawn()` prevents tight restart loops. --- ### Definition of Done - [ ] All three changes above have been applied to the specified files. - [ ] The project’s test command passes: `pip install -e '.[dev]' -q && make test-unit` exits with code 0. - [ ] The project’s lint command passes: `pip install -q ruff && ruff check .` reports zero errors. - [ ] A manual smoke test confirms the bridge no longer crashes after 15 seconds when running under the full uvicorn service (start `python -m devplacepy.main` and observe logs for >60 seconds – the only acceptable log lines are “Forking XML-RPC server started” once and subsequent “XML-RPC server not running” lines must not appear). *This manual step is not required for CI, but it validates the fix end‑to‑end.* **Note:** The existing 13 unit tests in `tests/api/xmlrpc/index.py` test the bridge in standalone mode only; they do not exercise the supervisor path. The changes here do not break those tests, as `start_new_session=True` and `ThreadingMixIn` are both backward‑compatible with standalone usage. The test suite provides regression coverage for the bridge API itself.