Visitar URL original
Add a pure-Rust _zstd module by youknowone · Pull Request #8917 · RustPython/RustPython · GitHub
Skip to content
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
4d68d0e
Add _zstd stdlib module backing compression.zstd
JamesClarke7283 May 23, 2026
9b517da
Fix iOS build by removing static PyMutex<PyTypeRef>
JamesClarke7283 May 23, 2026
5720b10
Fix clippy errors in zstd module
JamesClarke7283 May 23, 2026
d5037be
Retrigger CI
JamesClarke7283 May 23, 2026
6a67eaf
chore: format zstd module
JamesClarke7283 May 23, 2026
0ceea4a
Update crates/stdlib/src/zstd.rs
JamesClarke7283 May 24, 2026
e3b6888
chore: remove rustdoc comments from zstd module
JamesClarke7283 May 24, 2026
72b718c
commit message
JamesClarke7283 May 25, 2026
2575363
commit message
JamesClarke7283 May 25, 2026
c812dd8
Identify zstd parameter enums by type identity, not name
JamesClarke7283 Jun 3, 2026
4fa3ca9
Match CPython error behavior in _zstd ZstdCompressor
JamesClarke7283 Jun 3, 2026
dd03057
Address review: drop decorative separators and unused zstd crate
JamesClarke7283 Aug 9, 2026
1593954
Switch _zstd backend from zstd-safe to pure-Rust libzstd-rs-sys
JamesClarke7283 Aug 9, 2026
6d936c6
Fix data loss in ZstdDecompressor.decompress(max_length=0)
JamesClarke7283 Aug 9, 2026
12b96d8
Drop duplicate spell-checker line in zstd.rs
JamesClarke7283 Aug 9, 2026
1f5b219
Address review on _zstd constants, version attrs and dict markers
JamesClarke7283 Aug 19, 2026
116783a
Replace the _zstd flush-mode constants with a CompressMode enum
JamesClarke7283 Aug 19, 2026
692cbe6
Correct the _zstd explanatory comments and raise MemoryError on OOM
JamesClarke7283 Aug 19, 2026
97b63c5
Fix CPython divergences and a crash in the _zstd module
JamesClarke7283 Aug 19, 2026
14aeaee
Restore Lib/test/test_zstd.py to its vendored CPython state
JamesClarke7283 Aug 19, 2026
c5962fb
Rewrite _zstd against rusty_zstd
youknowone Oct 1, 2026
0507916
Use libzstd-rs-sys for _zstd
youknowone Oct 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Switch _zstd backend from zstd-safe to pure-Rust libzstd-rs-sys
Replace the C-library bindings (zstd-safe/zstd-sys) with Trifecta Tech
Foundation's libzstd-rs-sys (pinned git rev), a c2rust translation of
upstream libzstd. The module now calls its C-API-compatible functions
through small owning RAII wrappers (CCtx/DCtx/CDict/DDict) with the
same drop-order invariants as before.

The new backend builds zstdmt_compress in, so nb_workers bounds are
(0, 256) and test_zstd_multithread_compress passes for real; drop the
now-obsolete expectedFailureIf marker.
  • Loading branch information
JamesClarke7283 authored and youknowone committed Oct 1, 2026
commit 1593954961276d5712d1c98dfe741eedaa4f35cb
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,7 @@ xz-sys = "0.4.6-rc.0"
libsqlite3-sys = "0.38"
libz-rs-sys = "0.6"
zlib-rs = { version = "=0.6.8", default-features = false, features = ["rust-allocator", "__internal-api"] }
libzstd-rs-sys = { git = "https://github.com/trifectatechfoundation/libzstd-rs-sys", rev = "99fb4c096a21a9b37feebe180333a2ff8645e23b" }
lock_api = "0.4"
log = "0.4.30"
lz4_flex = "0.14"
Expand Down Expand Up @@ -320,7 +321,6 @@ which = "8"
x509-parser = "0.18"
xml = "1.3"
writeable = "0.6"
zstd-safe = { version = "7", default-features = false, features = ["std", "zdict_builder"] }

# Lints

Expand Down
5 changes: 0 additions & 5 deletions Lib/test/test_zstd.py
Original file line number Diff line number Diff line change
Expand Up @@ -321,11 +321,6 @@ def test_unknown_compression_parameter(self):
with self.assertRaisesRegex(ValueError, pattern):
ZstdCompressor(options=option)

# TODO: RUSTPYTHON
@unittest.expectedFailureIf(
not SUPPORT_MULTITHREADING,
"TODO: RUSTPYTHON; libzstd built without multi-threaded compression support",
)
def test_zstd_multithread_compress(self):
size = 40*_1M
b = THIS_FILE_BYTES * (size // len(THIS_FILE_BYTES))
Expand Down
2 changes: 1 addition & 1 deletion crates/stdlib/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ webpki-roots = { workspace = true, optional = true }

[target.'cfg(not(any(target_os = "android", target_arch = "wasm32")))'.dependencies]
libsqlite3-sys = { workspace = true, features = ["bundled"], optional = true }
libzstd-rs-sys = { workspace = true }
xz = { workspace = true }
xz-sys = { workspace = true }
zstd-safe = { workspace = true }

[target.'cfg(windows)'.dependencies]
paste = { workspace = true }
Expand Down
Loading