|
<div class="docs-content" data-render>
|
|
# ZipService
|
|
|
|
`ZipService` (`services/jobs/zip_service.py`) builds downloadable zip archives off the request path and hands the caller a URL when the archive is ready. It is the first consumer of the generic [async job framework](/docs/architecture-jobs.html); read that page for the lifecycle, single-owner model, and retention that this service inherits.
|
|
|
|
> Audience: administrators and maintainers.
|
|
|
|
## What it does
|
|
|
|
A caller enqueues a job describing what to archive. The lock-owning worker materializes the source onto a temporary directory, runs a standalone Python subprocess to compress it, computes a CRC32 over the output, names the file from that checksum, and records full statistics. Callers poll the job until it is `done`, then download the archive. Archives that go unused for the retention window are deleted automatically.
|
|
|
|
## Source contract
|
|
|
|
A zip job payload carries a `source` object with a `type`. The only implemented type is `project_tree`:
|
|
|
|
```
|
|
{"source": {"type": "project_tree", "project_uid": "...", "path": "src"}}
|
|
```
|
|
|
|
An empty `path` archives the whole project; a non-empty `path` archives that file or directory subtree. `ZipService._materialize` dispatches on the type and calls `project_files.export_to_dir`, which reconstructs the virtual project filesystem (inline text rows plus on-disk binary blobs) onto a real staging directory under the runtime data dir (`config.DATA_DIR/zip_staging/{job_uid}`, default `var/`, outside the package), with a traversal guard on every written path. Materialization runs inside the worker via `asyncio.to_thread`, never in the request handler.
|
|
|
|
## Subprocess compression
|
|
|
|
`ZipService._run_worker` spawns `python -m devplacepy.services.jobs.zip_worker <staging_dir> <tmp_zip>` with `asyncio.create_subprocess_exec`. The worker is stdlib-only (no system `zip` dependency), walks the tree (sorted) into a `ZIP_DEFLATED` archive with a fixed entry timestamp so the output is **deterministic** (identical content yields identical bytes), computes a streaming `zlib.crc32` over the result, and prints a JSON stats line: `bytes_in`, `bytes_out`, `file_count`, `dir_count`, `crc32`. The service parses that JSON; a non-zero exit raises and the job is marked `failed`. Because the archive is deterministic, the CRC32 acts as a content identity: re-archiving the same content produces the same filename, and the final write simply overwrites the previous archive.
|
|
|
|
## Naming and storage
|
|
|
|
The final name is `{crc32}.{slug}.zip`, where the slug comes from the caller's preferred name with any trailing `.zip` stripped and the rest slugified. The archive lives under `config.DATA_DIR/zips/{uid[:2]}/{uid[2:4]}/` (outside the package, not under `/static`); it is served by the `/zips/{uid}/download` route via `FileResponse`, never the static mount. The temporary `{uuid4}.zip` is renamed into place with `os.replace`, so re-running a job simply overwrites the previous archive. The staging directory is removed afterwards.
|
|
|
|
## Endpoints
|
|
|
|
- `POST /projects/{project_slug}/zip` queues a whole-project archive. Authorization is checked here against the project; the response is `{uid, status_url}`.
|
|
- `POST /projects/{project_slug}/files/zip?path=...` queues a subtree archive (empty `path` means the whole project).
|
|
- `GET /zips/{uid}` returns the job status and statistics (`ZipJobOut`). `download_url` is null until the job is `done`.
|
|
- `GET /zips/{uid}/download` streams the finished archive as `application/zip` and extends the retention window on each access.
|
|
|
|
The job uid (a sortable uuid7) is the capability token: anyone with the link can download, which matches the fact that projects are publicly viewable. The owner recorded on the job is used only for attribution and metrics, never for access control.
|
|
|
|
## Frontend
|
|
|
|
`static/js/ZipDownloader.js` (instantiated as `app.zipDownloader`) drives the whole flow: it POSTs to the enqueue URL, polls `GET /zips/{uid}` through the shared `JobPoller` (`static/js/JobPoller.js`), and triggers a hidden download link once the job is `done`, with toasts for progress and errors. Any element with a `data-zip-download` attribute is wired automatically (the "Download zip" button on the project detail page and the "Download as zip" button on the files page); the files context menu calls `app.zipDownloader.start(url)` directly for a single file or folder.
|
|
|
|
## Configuration, metrics, and quotas
|
|
|
|
On `/admin/services` the service exposes the inherited job fields: artifact retention (default 7 days), maximum concurrent jobs (default 2), and job timeout. Its metrics card shows totals per status, total items zipped, bytes in and out, average duration, and a table of recent jobs.
|
|
|
|
## CLI
|
|
|
|
- `devplace zips prune` deletes expired archives and their job rows.
|
|
- `devplace zips clear` deletes every zip archive and job row.
|
|
</div>
|