51 lines
2.2 KiB
Markdown
51 lines
2.2 KiB
Markdown
 
|
# Celery: Workers Guide (time limits, remote control)
|
||
|
|
|
||
|
|
## Metadata
|
||
|
|
|
||
|
|
| Field | Value |
|
||
|
|
|---|---|
|
||
|
|
| Institute | Molodetz Institute — Research & Retrieval Division |
|
||
|
|
| Subject | background-work-limits |
|
||
|
|
| Document type | Source Dossier |
|
||
|
|
| Tier | 1 |
|
||
|
|
| Source title | Celery: Workers Guide (time limits, remote control) |
|
||
|
|
| Source URL | https://docs.celeryq.dev/en/stable/userguide/workers.html |
|
||
|
|
| Source publisher / author | Celery project |
|
||
|
|
| Source type | vendor docs |
|
||
|
|
| Source date (published) | Unknown — not stated |
|
||
|
|
| Date researched | 2026-10-25 |
|
||
|
|
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
|
||
|
|
| Research method | direct page fetch |
|
||
|
|
| Confidence | High |
|
||
|
|
| Status | Active |
|
||
|
|
|
||
|
|
## Summary
|
||
|
|
|
||
|
|
The current Celery worker guide, read for time limits and revocation.
|
||
|
|
|
||
|
|
## Key findings
|
||
|
|
|
||
|
|
- Soft time limit raises `SoftTimeLimitExceeded` inside the task so it can clean up; the hard limit cannot be caught and force-terminates the task.
|
||
|
|
- Time limits work in the prefork and gevent pools; the gevent pool does not implement soft limits; limits do not work on platforms without SIGUSR1.
|
||
|
|
- `revoke` skips a task that has not started; it does not stop a running task unless `terminate` is set, which terminates the worker child (default signal TERM, configurable).
|
||
|
|
- `--max-tasks-per-child` and `--max-memory-per-child` replace a worker process after N tasks or a resident-memory ceiling, containing leaks.
|
||
|
|
- Autoscale keeps a minimum number of processes and grows to a maximum under load (`--autoscale=10,3`).
|
||
|
|
|
||
|
|
## Pros / strengths
|
||
|
|
|
||
|
|
Shows the soft-before-hard ordering and that cancelling a running job needs an explicit escalation.
|
||
|
|
|
||
|
|
## Cons / limitations / caveats
|
||
|
|
|
||
|
|
Process-pool specifics; time limits depend on signals, which tai's cooperative agents cannot rely on.
|
||
|
|
|
||
|
|
## Relevance to tai
|
||
|
|
|
||
|
|
Pattern for tai's cancel path: first a soft cancel the worker can honor (cancel flag read between steps, `cap.cancel` to a CAP agent), and only then the hard stop (closing the session after `ORCH_CANCEL_GRACE`). Replacing a worker after a bounded amount of work is the same idea as depth and step caps.
|
||
|
|
|
||
|
|
## Changelog
|
||
|
|
|
||
|
|
| Date | Change |
|
||
|
|
|---|---|
|
||
|
|
| 2026-10-25 | Initial dossier created. |
|