Repository navigation
Enable cross-interpreter sharing of tuples #111623
Description
Feature or enhancement
Proposal:
This issue is for adding support of sharing tuples (and tuples of tuples) through the crossinterp API and the interpreters module when that arrives.
The data structure will have to have a variable length.
struct _shared_tuple_data {
Py_ssize_t len;
_PyCrossInterpreterData **data;
};It will use the xid registry to lookup the type values in the tuple, then encode them using the crossinterpdatafunc callbacks. This should work recursively if the tuple contains a tuple.
PR to follow.
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Activity
gh-111623: Add Support for Cross-interpreter tuples (gh-111628)
pythongh-111623: Add Support for Cross-interpreter tuples (pythongh-1…
a-reich commentedon Dec 22, 2023
Quick question @tonybaloney , is
is_shareableintended to be reliable for containers? It now returns True for tuples even if they contain non-shareable types.ericsnowcurrently commentedon Dec 22, 2023
Excellent question! The underlying machinery for shareable objects isn't very sophisticated currently.
is_shareable()basically just checks if the object's type supports sharing. To make it more accurate, as you've implied, we would need either to try to actually share the object (like how the builtinhasattr()works) or to add a hook that each type may implement to indicate if an object is shareable.FWIW, the relevant code for the (currently internal-only) shareable-type protocol is in Include/internal/pycore_crossinterp.h and Python/crossinterp.c. For the
is_shareable()implementation look in Modules/_xxsubinterpretersmodule.c. Any changes would happen in those files.Keep in mind that, right now, the machinery for shareable objects is implemented with a global registry of types but I plan on writing a PEP in the near future targeting 3.13 to add a dedicated field to
PyTypeObjectto replace the registry. Also note thatis_shareable()is not part of PEP 734, though we may add it to the module (not the PEP) later.a-reich commentedon Dec 23, 2023
Thank you for the explanation on the current design! Yeah, it makes sense that
is_shareableis not very complex right now, and that it's not part of the official spec in PEP 734. (I just saw it intest.support.interpretersand was wondering how to use it when experimenting with the stuff there). The idea of replacing the registry with a type field/protocol sounds interesting for sure.pythongh-111623: Add Support for Cross-interpreter tuples (pythongh-1…
pythongh-111623: Add Support for Cross-interpreter tuples (pythongh-1…