Skip to content

refactor(upgrade): stream verified archives into owned staging instead of whole-archive Vecs and PID paths #1059

Description

@ScriptedAlchemy

Priority: P2 resource bounds and staging ownership. Concrete allocation/ownership finding; no measured RSS claim or local exploit is asserted.

Verified source

At #707 3d955ff998eef120cd398f163c613a54e3383ccb, crates/tracedecay-cli/src/upgrade.rs:

  • download_bytes uses as_reader().read_to_end(&mut Vec) for both SHA256SUMS and the complete release archive, without an application byte ceiling.
  • download_and_extract deliberately retains the entire archive so extraction can use a concrete Cursor<Vec<u8>>. The tar/ZIP wrappers then retain that input throughout extraction. A concrete seekable reader does not require a complete RAM buffer: std::fs::File already implements Read and Seek.
  • Extracted output is placed at temp_dir()/tracedecay_upgrade_<pid>[.exe], without reserving a new private per-attempt directory/file. ZIP uses File::create, which can open/truncate an existing path. Unix install_binary similarly copies to a predictable .tracedecay_upgrade_<pid> in the destination directory. Naming a file after a PID is not ownership proof.
  • Cleanup after replacement removes the named temp path; extraction failures before returning that path have no retained cleanup owner.

A five-minute network timeout does not bound download allocation. Counting only the final executable likewise omits the full compressed archive buffer, now including Linux runtime payloads (#1058).

Replace with existing mechanisms

  1. Keep the checksum manifest small and explicitly bounded. Preserve exact asset matching, duplicate-entry rejection, digest-format validation and checksum failure before publication.
  2. Stream the archive into an exclusively created, attempt-owned private file while hashing with the existing SHA-256 implementation and counting received bytes. Apply an explicit justified archive/disk-work bound rather than a product-wide arbitrary small constant. Rewind the same verified file for the existing tar/flate2 or ZIP readers. Do not extract/publish bytes before the whole-archive checksum has succeeded.
  3. Reuse an existing suitable private temporary-file owner or the already-used maintained tempfile dependency; no custom downloader/cache/temporary-name framework. A path existence check followed by create/copy is not exclusive creation. Keep verified bytes inaccessible to competing writers for the extraction interval; retaining a file handle alone does not make its contents immutable.
  4. Keep archive member/type/path and expanded-byte validation at extraction. Streaming the compressed body bounds one buffer, not ZIP metadata or decompression expansion. fix(upgrade): install the complete release runtime bundle, not only the executable #1058 owns which release members are required; this issue must not introduce a second payload manifest.
  5. Carry cleanup ownership through every failure and successful handoff. Remove only this attempt's scratch material; preserve pre-existing collisions and the installed target. Publication and durable-install semantics remain with the existing updater/installation owner.

The result should delete whole-archive buffering and ad-hoc temp-path lifecycle branches. Do not replace Vec with mmap and call the working set bounded, or introduce a general async transfer service for a CLI command.

Acceptance

  • Large controlled archives can be downloaded, checksum-verified, rewound and extracted in both supported archive formats without retaining an archive-sized Vec. Measure streaming-buffer allocation separately from archive-parser metadata and filesystem page cache; do not claim constant total RSS without measurement.
  • Oversized checksum manifests/archives, truncated streams, bad checksums, cancellation and disk-write errors reject before installation and release owned scratch.
  • Pre-existing temp files, directories and symlinks at colliding names remain untouched. Concurrent attempts cannot truncate/delete one another's staging. Platform-appropriate tests exercise actual creation/publication helpers, not only the random-name generator.
  • A valid archive remains byte-exact between checksum verification and extraction; invalid or duplicate members cannot overwrite unrelated paths.
  • Existing installed binary/bundle is unchanged for every pre-publication failure; supported Windows replacement semantics remain intact.

Primary references: https://doc.rust-lang.org/std/fs/struct.File.html (Read/Seek and atomic create_new); https://doc.rust-lang.org/std/io/trait.Read.html#method.read_to_end.

Separate from #1058's missing runtime companion and the package-manager ownership cut. No repository tests, downloads, benchmarks or installations executed in this audit. Keep #707 draft.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions