Fix container sync races that leaked orphan blobs; add a system-prune CLI command

sync_workspace (user-triggered) and the reconciler's sync_bidirectional_sync
could run concurrently for the same project, and store_upload's read-then-
write on a changed path meant two racing imports each wrote their own blob
while only one ever got referenced - the loser leaked forever. Combined with
no build-artifact exclusion, an actively-compiling workspace hit this
constantly and leaked 5.9M orphan blobs (~96GB) in production before it was
caught.

Closes it at the root: api._sync_dir_bidirectional_locked serializes both
call sites per-project (non-blocking - a project already mid-sync is simply
skipped until the next tick), and IMPORT_SKIP_NAMES/IMPORT_SKIP_EXTENSIONS
keep build output (build/, dist/, *.o, *.pyc, ...) out of the walk entirely.

Recovering what already leaked is a separate concern: a new CLI subcommand
(plus matching make targets) sweeps soft-deleted attachment/project-file
blobs and any blob with zero DB reference at all, plus orphaned container
workspace directories. run_maintenance_cleanup.sh wraps the existing
prune/clear commands for routine disk upkeep.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BWJy6PrMMt5hwWxQwia2rd
This commit is contained in:
2026-09-08 03:43:49 +02:00
co-authored by Claude Sonnet 5
parent 85bd8fad47
commit 7880bf4b31
19 changed files with 1068 additions and 19 deletions
+10 -1
View File
@@ -18,7 +18,7 @@ BOOTSTRAP_PYTHON := $(shell command -v python3 2>/dev/null || command -v python
PYTHONDONTWRITEBYTECODE := 1
export PYTHONDONTWRITEBYTECODE
.PHONY: venv install dev prod clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless
.PHONY: venv install dev prod clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless prune prune-dry-run
$(PYTHON):
@test -n "$(BOOTSTRAP_PYTHON)" || { echo "python3 is required to create $(VENV)"; exit 1; }
@@ -148,6 +148,15 @@ clean:
test-cache-clean:
rm -rf .pytest_cache
# Safe but aggressive disk-space cleanup, acting on the REAL data/devplace.db
# and data/ tree (never the test database) - see CLAUDE.md "devplace system
# prune". prune-dry-run reports what would be removed without deleting.
prune: $(VENV_STAMP)
$(PYTHON) -m devplacepy.cli system prune
prune-dry-run: $(VENV_STAMP)
$(PYTHON) -m devplacepy.cli system prune --dry-run
# Container Manager works out of the box: the overlay installs the docker CLI in
# the image and mounts the host socket. DOCKER_GID is read straight from the
# socket so the UID-1000 app can use it; the data dir is the project's own data/