refactor: replace thread-based pool with multiprocessing ProcessPoolExecutor for CPU-bound tasks

Browse Source

The previous thread pool implementation caused GIL contention under heavy CPU workloads. This change swaps concurrent.futures.ThreadPoolExecutor for ProcessPoolExecutor, enabling true parallelism across worker processes. The pool size is now configurable via environment variable and defaults to os.cpu_count(). Task submission and result collection interfaces remain backward-compatible.

...

This commit is contained in:

retoor

2024-12-14 05:33:04 +00:00

parent 98eae03f7a

commit 2f325e1448

3 changed files with 23 additions and 1 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

dist/Ragnar-1.3.37-py3-none-any.whl

Vendored

BIN

Binary file not shown.

dist/ragnar-1.3.37.tar.gz

Vendored

BIN

Binary file not shown.