Files
tai/research/background-work-limits/celery-workers-guide.md
T
retoorandClaude Sonnet 5.5 ec9c471780 orchestration: sub agents render in live panels on the multiplexer screen
Feed panes in the multiplexer (vpane_* control commands, main-vertical layout, header row),
src/orch (board, panel, formatter, transport, CAP adapter), Agent observer and cancel flag,
supervision (idle warn/cancel, user close cancels, soft then hard stop), CAP answers collected
through the spill path. Research archived under background-work-limits.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 18:58:22 +01:00

2.2 KiB

Celery: Workers Guide (time limits, remote control)

Metadata

Field Value
Institute Molodetz Institute — Research & Retrieval Division
Subject background-work-limits
Document type Source Dossier
Tier 1
Source title Celery: Workers Guide (time limits, remote control)
Source URL https://docs.celeryq.dev/en/stable/userguide/workers.html
Source publisher / author Celery project
Source type vendor docs
Source date (published) Unknown — not stated
Date researched 2026-10-25
Researched by tai (agent), model: Claude Sonnet 5, on behalf of retoor
Research method direct page fetch
Confidence High
Status Active

Summary

The current Celery worker guide, read for time limits and revocation.

Key findings

  • Soft time limit raises SoftTimeLimitExceeded inside the task so it can clean up; the hard limit cannot be caught and force-terminates the task.
  • Time limits work in the prefork and gevent pools; the gevent pool does not implement soft limits; limits do not work on platforms without SIGUSR1.
  • revoke skips a task that has not started; it does not stop a running task unless terminate is set, which terminates the worker child (default signal TERM, configurable).
  • --max-tasks-per-child and --max-memory-per-child replace a worker process after N tasks or a resident-memory ceiling, containing leaks.
  • Autoscale keeps a minimum number of processes and grows to a maximum under load (--autoscale=10,3).

Pros / strengths

Shows the soft-before-hard ordering and that cancelling a running job needs an explicit escalation.

Cons / limitations / caveats

Process-pool specifics; time limits depend on signals, which tai's cooperative agents cannot rely on.

Relevance to tai

Pattern for tai's cancel path: first a soft cancel the worker can honor (cancel flag read between steps, cap.cancel to a CAP agent), and only then the hard stop (closing the session after ORCH_CANCEL_GRACE). Replacing a worker after a bounded amount of work is the same idea as depth and step caps.

Changelog

Date Change
2026-10-25 Initial dossier created.