Repository navigation
Incompatible change in internal string representation (encountered with Rust bindings) (?) #128972
Description
Not sure if it's the right place to ask, but here it is.
I'm attempting to introduce 3.14 support in pyo3 (PyO3/pyo3#4811). I've encountered test failures involving UCS and refcounts targeting Windows and Linux. Invoking cargo test --no-default-features --lib --tests, one example of failure is here: https://github.com/clin1234/pyo3/actions/runs/12834317749/job/35791414368. The Python version used for testing 3.14 is 3.14.0a4.
However, running the same command with Python 3.14.0a3 running on Fedora Rawhide locally, I have encountered no test failures whatsoever.
And yes, I have pinged @davidhewitt on this matter.
Activity
sobolevn commentedon Jan 18, 2025
Hi, thanks for the report! Can you please try to come up with a reproducer that does not include
rust, but only C-API?davidhewitt commentedon Jan 18, 2025
I don't think it's a CPython problem.
PyO3 has to replicate the C API at the ABI level, so it's not a surprise when bumping python versions that if structures change we need to make adjustments.
The PyUnicode data access macros are a known problem for PyO3 and to be honest I think the better option might be to just remove our mapping of them. But we can discuss that downstream.
ZeroIntensity commentedon Jan 18, 2025
I suspect the "offending" change would be #128196, but yeah--we won't guarantee any stability with the internals, especially in the alpha phase.
Though, I am having some thoughts about whether or not the structure of
PyASCIIObjectis actually internal, because we do technically expose the structure in the non-limited API.cc @encukou for the stability rules here.
encukou commentedon Jan 20, 2025
By my reading of PEP-387, backwards-incompatible changes to
PyASCIIObjectneeds deprecation period (or SC exception). However, that would be C API changes: rewriting that header in Rust is, unfortunately, not covered by any guarantees.IMO, CPython should still try to keep more stable than promised. But this is a bit of a tight spot regarding backwards compatibility and free-threading support.
I see #128196 preserves C API compatibility but introduces an implementation-defined C feature (
shortbitfields) to do it. I fear that this will bite us in the future.corona10 commentedon Jan 20, 2025
If we have to guarantee object layout stuff, one possible negotiation point would be just change for the free-threaded build because it is kind of an experimental build, so some of the breaking changes will be acceptable. (And we already break ABI compatibility between the default build and the free-threaded build)
cc @colesbury
pythongh-128972: Add `_Py_ALIGN_AS` and revert `PyASCIIObject` memory…
gh-128972: Add `_Py_ALIGN_AS` and revert `PyASCIIObject` memory layou…
encukou commentedon May 2, 2025
Reverted to give everyone time to stop relying on these details. But, note that:
Py_bufferfor example).Thanks for the report!
pythongh-128972: Add `_Py_ALIGN_AS` and revert `PyASCIIObject` memory…