Visitar URL original
feat: pull artifact files into a reproducible ZIP archive by Mistyheart · Pull Request #2321 · oras-project/oras · GitHub
Skip to content

feat: pull artifact files into a reproducible ZIP archive - #2321

Open
Mistyheart wants to merge 2 commits into
oras-project:mainfrom
Mistyheart:feat/pull-output-archive
Open

feat: pull artifact files into a reproducible ZIP archive#2321
Mistyheart wants to merge 2 commits into
oras-project:mainfrom
Mistyheart:feat/pull-output-archive

Conversation

@Mistyheart

@Mistyheart Mistyheart commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown

What this PR does / why we need it:

Adds oras pull --output-archive sources.zip as an initial ZIP implementation of #2018. It walks artifact metadata, orders named descriptors, and streams verified payloads into one archive without first extracting individual files to disk. ZIP entry timestamps and permissions are normalized, so repeated pulls of the same artifact produce identical bytes.

oras pull registry.example/sources:v1 --output-archive sources.zip
oras pull --oci-layout ./layout:v1 --output-archive sources.zip
  • Preserves named aliases, regular files, packed directories (including empty directories), and executable bits in directory layers.
  • Supports platform selection, --include-subject, and manifest config selection.
  • Validates blob and directory digests and rejects unsafe or conflicting names. Legacy directory annotations remain readable.
  • Stages only the archive itself. Failed transfers remove partial output and preserve an existing destination; --keep-old-files uses exclusive creation.
  • Leaves ordinary directory pulls unchanged.

Initial scope: ZIP and text status output only. Other extensions, symbolic/hard links, path traversal, and combining archive output with --output or --format are explicitly rejected. Payload streaming is serial to keep entry order reproducible; --concurrency continues to apply to ordinary directory pulls. Additional formats and structured archive metadata can be discussed separately.

Which issue(s) this PR fixes:

Related to #2018; this implements its ZIP use case rather than claiming support for every proposed archive format.

Validation:

  • New unit tests cover reproducibility, aliases, subjects/configs, platform selection, real directory layers, metadata-only planning, invalid names, corruption, cancellation, and cleanup.
  • The full changed cmd/oras/root package passed on Windows and as a Linux test binary executed in WSL2.
  • An actual OCI-layout CLI demonstration passed eight checks, comparing ZIP contents with ordinary pull, byte-for-byte reproducibility, keep-old behavior, unsupported formats, and corrupt-transfer cleanup. The original CLI rejects the new flag.
  • Targeted golangci-lint and goimports passed.
  • go test ./... on Windows passed all packages except two existing containers-config cases that try to create registry.example:5000 as a directory. Both failures were reproduced on the unchanged base. The Kubernetes E2E suite and race detector were not run locally.

Please check the following list:

  • Does the affected code have corresponding tests, e.g. unit test, E2E test? Unit tests and a local OCI-layout CLI exercise.
  • Does this change require a documentation update? Command help includes the flag, limitations and an example; a website guide should follow the agreed interface.
  • Does this introduce breaking changes that would require an announcement or bumping the major version? No; archive output is opt-in.
  • Do all new files have an appropriate license header?

AI tools assisted with implementation, testing, and this description.

Review follow-up on 2026-11-10

Addressed the four review comments in 1284e3df6d1b83644ef71afe4e89d6c03dbe52ad: umask-respecting output modes, three focused archive E2E specs (all passed), skipped-successor reporting, and config annotation preservation. Three regressions reproduce the previous behavior and pass after the fix. Full changed-package Windows/Linux tests, vet, goimports and targeted lint passed. The complete Kubernetes E2E suite and race detector were not run locally.

@Mistyheart
feat: pull artifact files into a reproducible ZIP archive
c796805 
Signed-off-by: 石丰毓 <114632971+Mistyheart@users.noreply.github.com>
TerryHowe
TerryHowe reviewed Nov 6, 2026
View reviewed changes
Comment thread cmd/oras/root/pull_archive.go Resolved
Comment thread cmd/oras/root/pull.go Resolved
Comment thread cmd/oras/root/pull_archive.go Resolved
Comment thread cmd/oras/root/pull_archive.go Resolved
@Mistyheart
fix: align archive pulls with existing pull behavior
1284e3d 
Signed-off-by: 石丰毓 <114632971+Mistyheart@users.noreply.github.com>
@Mistyheart

Mistyheart commented Nov 10, 2026

Copy link
Copy Markdown
Author

Thanks for the review. Follow-up commit 1284e3d addresses all four points:

  • Both output paths now use exclusive OpenFile(..., 0666) creation, respecting the caller's umask. The default path stages the file in a private temporary directory before rename, so output permissions match --keep-old-files without exposing partial bytes.
  • Added three focused CLI E2E specs under test/e2e/suite/command/pull_archive.go: directory layers/empty directories and reproducibility; preservation of an existing archive; and creation of a new archive with --keep-old-files.
  • Unnamed successors are reported as skipped even when they have named descendants; nonempty configs follow the normal pull traversal.
  • Setting the requested config name now preserves existing annotations.

The full changed root package passed on Windows and Linux (WSL2), as did vet, goimports and targeted golangci-lint. All three archive E2E specs passed; suite login setup used a real local Distribution service. The three new behavioral regressions fail on the previous commit and pass with this change. The full Kubernetes E2E suite and race detector were not run locally.

AI tools assisted with these changes, tests and this response.

@codecov

codecov Bot commented Nov 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.04878% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.31%. Comparing base (f5dbbde) to head (1284e3d).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
cmd/oras/root/pull_archive_zip.go 75.92% 14 Missing and 12 partials ⚠️
cmd/oras/root/pull_archive.go 81.81% 8 Missing and 8 partials ⚠️
cmd/oras/root/pull.go 66.66% 3 Missing ⚠️

❌ Your patch check has failed because the patch coverage (78.04%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2321      +/-   ##
==========================================
+ Coverage   73.51%   74.31%   +0.79%     
==========================================
  Files         157      159       +2     
  Lines        6582     6786     +204     
==========================================
+ Hits         4839     5043     +204     
+ Misses       1454     1433      -21     
- Partials      289      310      +21

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TerryHowe

TerryHowe commented Nov 10, 2026

Copy link
Copy Markdown
Member

Interface suggestion: rather than a second output flag, keep -o as the single destination and add a format switch:

oras pull <ref> -o sources.zip --archive zip

--archive <format> lets us add tar, tar.zst, etc. later without guessing from the file extension, and -o keeps one meaning (where output goes).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TerryHowe TerryHowe TerryHowe left review comments
@sabre1041 sabre1041 Awaiting requested review from sabre1041 sabre1041 is a code owner
@sajayantony sajayantony Awaiting requested review from sajayantony sajayantony is a code owner
@shizhMSFT shizhMSFT Awaiting requested review from shizhMSFT shizhMSFT is a code owner
@Wwwsylvia Wwwsylvia Awaiting requested review from Wwwsylvia Wwwsylvia is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Mistyheart @TerryHowe