Visitar URL original
Per-interpreter runtime state and a process-wide GC stop-the-world by youknowone · Pull Request #8517 · RustPython/RustPython · GitHub
Skip to content

Per-interpreter runtime state and a process-wide GC stop-the-world #8517

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 27 commits into from
Aug 16, 2026
Merged

Per-interpreter runtime state and a process-wide GC stop-the-world #8517

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5c341d2
vm: add per-interpreter runtime state and interpreter registry
youknowone Aug 13, 2026
43f4b23
gc: stop every interpreter while collecting
youknowone Aug 13, 2026
ffd3d1e
vm: scope the interpreter registry per thread without threading
youknowone Aug 13, 2026
0079994
vm: keep the interpreter registry usable across bootstrap, drop and fork
youknowone Aug 13, 2026
8f108ff
vm: attach and detach thread slots when switching interpreters
youknowone Aug 13, 2026
696a8c1
vm: list only the current interpreter's subclasses
youknowone Aug 14, 2026
562b657
gc: give each interpreter its own collector state
youknowone Aug 14, 2026
30e2020
_queue, _thread: detach before taking locks held across waits
youknowone Aug 14, 2026
5ce062e
_io, _winapi: detach on the remaining stopped-holdable lock takes
youknowone Aug 14, 2026
87b6c68
gc: size the interpreter owner tag to the header padding
youknowone Aug 15, 2026
fd27902
vm: reuse cleared frame blocks and shorten interpreter hot paths
youknowone Aug 15, 2026
c3fcc70
gc, vm: address review notes on owner tags and interpreter docs
youknowone Aug 15, 2026
57428cd
vm: gate interpreter registration on stop-the-world admission
youknowone Aug 15, 2026
8d07831
gc: keep the tracking counters off the barrier path
youknowone Aug 15, 2026
4e518d3
dict: settle lookups and iteration steps under one read guard
youknowone Aug 15, 2026
eb94daf
vm: build a call's argument vector once
youknowone Aug 15, 2026
f70db35
vm: reach an instance dict without cloning it
youknowone Aug 15, 2026
4a76424
vm: shorten the per-instruction safepoint and the call preamble
youknowone Aug 15, 2026
b0f0dcc
vm: give KwArgs a zero-sized hasher
youknowone Aug 15, 2026
f9bbc6d
vm: promote borrowed stack refs before a yield
youknowone Aug 15, 2026
b4bb43c
vm: run the exact-args vectorcall on a heap frame again
youknowone Aug 15, 2026
98d2410
vm: release a tail call's function at the callee's return
youknowone Aug 16, 2026
08bd522
vm: stop copying a running frame's locals when it materializes
youknowone Aug 16, 2026
c7c1452
vm: key data stack frame reuse on the exact frame size
youknowone Aug 16, 2026
2d58fc8
vm: store the frame object payload address in the cross-thread slot
youknowone Aug 16, 2026
e24d831
vm: copy a foreign thread's frames instead of linking them
youknowone Aug 16, 2026
5d0d9be
vm: resolve the two rustdoc links this branch added
youknowone Aug 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
vm: add per-interpreter runtime state and interpreter registry 
- vm/runtime.rs: process-global interpreter registry (monotonic ids, weak
  entries), InterpreterWhence/InterpreterInfo, process main id recording via
  main_interpreter_id(), a threading-gated owner map (store_owned_interpreter/
  take_owned_interpreter/is_owned_interpreter/owned_interpreter_count), and the
  SUPPORTS_ISOLATED_INTERPRETERS constant.
- PyGlobalState gains interpreter_id/whence/is_main and is_main_interpreter();
  PyConfig/Settings derive Clone so a subinterpreter can clone parent config.
- Interpreter: id()/whence()/is_main()/is_process_main(),
  create_subinterpreter() and create_owned_subinterpreter(); unregister on Drop.
- thread.rs: per-interpreter thread slots (INTERP_THREAD_SLOTS), slot swap when
  switching interpreters on one OS thread, cleanup keyed by interpreter id.
- Install signal handlers and init the main-thread ident only on the main
  interpreter; _thread._is_main_interpreter reflects the current interpreter.
- sys.implementation.supports_isolated_interpreters reads the constant.
- Guard the registry static for non-threading builds where rc::Weak is !Send.

Assisted-by: Claude Code:claude-opus-4-8
  • Loading branch information
@youknowone
youknowone committed Aug 15, 2026
commit 5c341d263e9b14263fa2b77046c2d46a2bd44f2c
6 changes: 5 additions & 1 deletion crates/vm/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,11 @@ pub use self::object::{
AsObject, Py, PyAtomicRef, PyExact, PyObject, PyObjectRef, PyPayload, PyRef, PyRefExact,
PyResult, PyStackRef, PyWeakRef,
};
pub use self::vm::{Context, Interpreter, InterpreterBuilder, Settings, VirtualMachine};
pub use self::vm::runtime;
pub use self::vm::{
Context, Interpreter, InterpreterBuilder, InterpreterInfo, InterpreterWhence,
MAIN_INTERPRETER_ID, Settings, VirtualMachine,
};

pub use rustpython_common as common;
pub use rustpython_compiler_core::{bytecode, frozen};
Expand Down
4 changes: 3 additions & 1 deletion crates/vm/src/stdlib/_signal.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,9 @@ pub(crate) mod _signal {
module: &Py<crate::builtins::PyModule>,
vm: &VirtualMachine,
) {
if vm.state.config.settings.install_signal_handlers {
// Process-global signal disposition is owned by the main interpreter only.
// Subinterpreters (PEP 734) must not reinstall SIGINT / probe handlers.
if vm.state.is_main_interpreter() && vm.state.config.settings.install_signal_handlers {
let sig_dfl = vm.new_pyobj(SIG_DFL as u8);
let sig_ign = vm.new_pyobj(SIG_IGN as u8);

Expand Down
5 changes: 2 additions & 3 deletions crates/vm/src/stdlib/_thread.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -796,9 +796,8 @@ pub(crate) mod _thread {
}

#[pyfunction]
fn _is_main_interpreter() -> bool {
// RustPython only has one interpreter

@ShaharNaveh ShaharNaveh Aug 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥳

true
fn _is_main_interpreter(vm: &VirtualMachine) -> bool {
vm.state.is_main_interpreter()
}

/// Initialize the main thread ident. Should be called once at interpreter startup.
Expand Down
3 changes: 2 additions & 1 deletion crates/vm/src/stdlib/sys.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -669,7 +669,8 @@ pub mod sys {
"_multiarch" => ctx.new_str(multiarch()),
"version" => PyVersionInfo::from_data(VersionInfoData::IMPLEMENTATION, vm),
"hexversion" => ctx.new_int(version::VERSION_HEX_IMPL),
"supports_isolated_interpreters" => ctx.new_bool(false),
"supports_isolated_interpreters" =>
ctx.new_bool(crate::vm::runtime::SUPPORTS_ISOLATED_INTERPRETERS),
})
}

Expand Down
Loading