Fix container sync races that leaked orphan blobs; add a system-prune CLI command
sync_workspace (user-triggered) and the reconciler's sync_bidirectional_sync could run concurrently for the same project, and store_upload's read-then- write on a changed path meant two racing imports each wrote their own blob while only one ever got referenced - the loser leaked forever. Combined with no build-artifact exclusion, an actively-compiling workspace hit this constantly and leaked 5.9M orphan blobs (~96GB) in production before it was caught. Closes it at the root: api._sync_dir_bidirectional_locked serializes both call sites per-project (non-blocking - a project already mid-sync is simply skipped until the next tick), and IMPORT_SKIP_NAMES/IMPORT_SKIP_EXTENSIONS keep build output (build/, dist/, *.o, *.pyc, ...) out of the walk entirely. Recovering what already leaked is a separate concern: a new CLI subcommand (plus matching make targets) sweeps soft-deleted attachment/project-file blobs and any blob with zero DB reference at all, plus orphaned container workspace directories. run_maintenance_cleanup.sh wraps the existing prune/clear commands for routine disk upkeep. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BWJy6PrMMt5hwWxQwia2rd
This commit is contained in:
@@ -18,7 +18,7 @@ BOOTSTRAP_PYTHON := $(shell command -v python3 2>/dev/null || command -v python
|
||||
PYTHONDONTWRITEBYTECODE := 1
|
||||
export PYTHONDONTWRITEBYTECODE
|
||||
|
||||
.PHONY: venv install dev prod clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless
|
||||
.PHONY: venv install dev prod clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless prune prune-dry-run
|
||||
|
||||
$(PYTHON):
|
||||
@test -n "$(BOOTSTRAP_PYTHON)" || { echo "python3 is required to create $(VENV)"; exit 1; }
|
||||
@@ -148,6 +148,15 @@ clean:
|
||||
test-cache-clean:
|
||||
rm -rf .pytest_cache
|
||||
|
||||
# Safe but aggressive disk-space cleanup, acting on the REAL data/devplace.db
|
||||
# and data/ tree (never the test database) - see CLAUDE.md "devplace system
|
||||
# prune". prune-dry-run reports what would be removed without deleting.
|
||||
prune: $(VENV_STAMP)
|
||||
$(PYTHON) -m devplacepy.cli system prune
|
||||
|
||||
prune-dry-run: $(VENV_STAMP)
|
||||
$(PYTHON) -m devplacepy.cli system prune --dry-run
|
||||
|
||||
# Container Manager works out of the box: the overlay installs the docker CLI in
|
||||
# the image and mounts the host socket. DOCKER_GID is read straight from the
|
||||
# socket so the UID-1000 app can use it; the data dir is the project's own data/
|
||||
|
||||
Reference in New Issue
Block a user