Visitar URL original
Per-interpreter runtime state and a process-wide GC stop-the-world by youknowone · Pull Request #8517 · RustPython/RustPython · GitHub
Skip to content

Per-interpreter runtime state and a process-wide GC stop-the-world #8517

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 27 commits into from
Aug 16, 2026
+3,358 −989
Merged

Per-interpreter runtime state and a process-wide GC stop-the-world #8517

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5c341d2
vm: add per-interpreter runtime state and interpreter registry
youknowone Aug 13, 2026
43f4b23
gc: stop every interpreter while collecting
youknowone Aug 13, 2026
ffd3d1e
vm: scope the interpreter registry per thread without threading
youknowone Aug 13, 2026
0079994
vm: keep the interpreter registry usable across bootstrap, drop and fork
youknowone Aug 13, 2026
8f108ff
vm: attach and detach thread slots when switching interpreters
youknowone Aug 13, 2026
696a8c1
vm: list only the current interpreter's subclasses
youknowone Aug 14, 2026
562b657
gc: give each interpreter its own collector state
youknowone Aug 14, 2026
30e2020
_queue, _thread: detach before taking locks held across waits
youknowone Aug 14, 2026
5ce062e
_io, _winapi: detach on the remaining stopped-holdable lock takes
youknowone Aug 14, 2026
87b6c68
gc: size the interpreter owner tag to the header padding
youknowone Aug 15, 2026
fd27902
vm: reuse cleared frame blocks and shorten interpreter hot paths
youknowone Aug 15, 2026
c3fcc70
gc, vm: address review notes on owner tags and interpreter docs
youknowone Aug 15, 2026
57428cd
vm: gate interpreter registration on stop-the-world admission
youknowone Aug 15, 2026
8d07831
gc: keep the tracking counters off the barrier path
youknowone Aug 15, 2026
4e518d3
dict: settle lookups and iteration steps under one read guard
youknowone Aug 15, 2026
eb94daf
vm: build a call's argument vector once
youknowone Aug 15, 2026
f70db35
vm: reach an instance dict without cloning it
youknowone Aug 15, 2026
4a76424
vm: shorten the per-instruction safepoint and the call preamble
youknowone Aug 15, 2026
b0f0dcc
vm: give KwArgs a zero-sized hasher
youknowone Aug 15, 2026
f9bbc6d
vm: promote borrowed stack refs before a yield
youknowone Aug 15, 2026
b4bb43c
vm: run the exact-args vectorcall on a heap frame again
youknowone Aug 15, 2026
98d2410
vm: release a tail call's function at the callee's return
youknowone Aug 16, 2026
08bd522
vm: stop copying a running frame's locals when it materializes
youknowone Aug 16, 2026
c7c1452
vm: key data stack frame reuse on the exact frame size
youknowone Aug 16, 2026
2d58fc8
vm: store the frame object payload address in the cross-thread slot
youknowone Aug 16, 2026
e24d831
vm: copy a foreign thread's frames instead of linking them
youknowone Aug 16, 2026
5d0d9be
vm: resolve the two rustdoc links this branch added
youknowone Aug 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
vm: build a call's argument vector once 
The specialized CALL handlers collected the positional arguments into one
vector and then copied them into a second one to put `self` in front, so
every specialized builtin, method-descriptor, class and non-Python call paid
two allocations, a copy and two frees. The arguments are already laid out on
the value stack in vectorcall order, so `take_call_args` fills one vector by
index — the shape `execute_call_vectorcall` already used.

`vectorcall_native_function` and the keyword path of `vectorcall_function`
then cloned that vector again to build `FuncArgs`; both now move it in
through `from_vectorcall_owned`, as the other vectorcall slots do.

Assisted-by: Claude
  • Loading branch information
@youknowone
youknowone committed Aug 15, 2026
commit eb94dafe3fb87a1c15f05a1570af30fb2429867a
4 changes: 2 additions & 2 deletions crates/vm/src/builtins/builtin_func.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,9 @@ fn vectorcall_native_function(
let mut all_args = Vec::with_capacity(args.len() + 1);
all_args.push(self_obj);
all_args.extend(args);
FuncArgs::from_vectorcall(&all_args, nargs + 1, kwnames)
FuncArgs::from_vectorcall_owned(all_args, nargs + 1, kwnames)
} else {
FuncArgs::from_vectorcall(&args, nargs, kwnames)
FuncArgs::from_vectorcall_owned(args, nargs, kwnames)
};

(zelf.value.func)(vm, func_args)
Expand Down
4 changes: 2 additions & 2 deletions crates/vm/src/builtins/function.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1635,7 +1635,7 @@ pub(crate) fn vectorcall_function(
let has_kwargs = kwnames.is_some_and(|kw| !kw.is_empty());
if zelf.is_jitted() {
let func_args = if has_kwargs {
FuncArgs::from_vectorcall(&args, nargs, kwnames)
FuncArgs::from_vectorcall_owned(args, nargs, kwnames)
} else {
args.truncate(nargs);
FuncArgs::from(args)
Expand Down Expand Up @@ -1667,7 +1667,7 @@ pub(crate) fn vectorcall_function(

// SLOW PATH: construct FuncArgs from owned Vec and delegate to invoke()
let func_args = if has_kwargs {
FuncArgs::from_vectorcall(&args, nargs, kwnames)
FuncArgs::from_vectorcall_owned(args, nargs, kwnames)
} else {
args.truncate(nargs);
FuncArgs::from(args)
Expand Down
183 changes: 72 additions & 111 deletions crates/vm/src/frame.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6094,15 +6094,8 @@ impl ExecutingFrame<'_> {
| PyMethodFlags::O
| PyMethodFlags::KEYWORDS);
if call_conv == PyMethodFlags::O && effective_nargs == 1 {
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
let self_or_null = self.pop_value_opt();
let callable = self.pop_value();
let mut args_vec = Vec::with_capacity(effective_nargs as usize);
if let Some(self_val) = self_or_null {
args_vec.push(self_val);
}
args_vec.extend(pos_args);
let (callable, args_vec) = self.take_call_args(nargs as usize);
debug_assert_eq!(args_vec.len(), effective_nargs as usize);
let result =
callable.vectorcall(args_vec, effective_nargs as usize, None, vm)?;
self.push_value(result);
Expand All @@ -6128,15 +6121,8 @@ impl ExecutingFrame<'_> {
| PyMethodFlags::O
| PyMethodFlags::KEYWORDS);
if call_conv == PyMethodFlags::FASTCALL {
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
let self_or_null = self.pop_value_opt();
let callable = self.pop_value();
let mut args_vec = Vec::with_capacity(effective_nargs as usize);
if let Some(self_val) = self_or_null {
args_vec.push(self_val);
}
args_vec.extend(pos_args);
let (callable, args_vec) = self.take_call_args(nargs as usize);
debug_assert_eq!(args_vec.len(), effective_nargs as usize);
let result =
callable.vectorcall(args_vec, effective_nargs as usize, None, vm)?;
self.push_value(result);
Expand Down Expand Up @@ -6164,18 +6150,8 @@ impl ExecutingFrame<'_> {
if self.specialization_call_recursion_guard(vm) {
return self.execute_call_vectorcall(nargs, vm);
}
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
let self_or_null = self.pop_value_opt();
let callable = self.pop_value();
let (args_vec, effective_nargs) = if let Some(self_val) = self_or_null {
let mut v = Vec::with_capacity(nargs_usize + 1);
v.push(self_val);
v.extend(pos_args);
(v, nargs_usize + 1)
} else {
(pos_args, nargs_usize)
};
let (callable, args_vec) = self.take_call_args(nargs as usize);
let effective_nargs = args_vec.len();
let result =
vectorcall_function(&callable, args_vec, effective_nargs, None, vm)?;
self.push_value(result);
Expand Down Expand Up @@ -6214,12 +6190,11 @@ impl ExecutingFrame<'_> {
return self.execute_call_vectorcall(nargs, vm);
}
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
self.pop_value_opt(); // null (self_or_null)
self.pop_value(); // callable (bound method)
let mut args_vec = Vec::with_capacity(nargs_usize + 1);
args_vec.push(bound_self);
args_vec.extend(pos_args);
args_vec.extend(self.pop_multiple(nargs_usize));
self.pop_value_opt(); // null (self_or_null)
self.pop_value(); // callable (bound method)
let result = vectorcall_function(
&bound_function,
args_vec,
Expand Down Expand Up @@ -6301,15 +6276,8 @@ impl ExecutingFrame<'_> {
.is_some_and(|self_obj| self_obj.class().is(descr.objclass))
{
let func = descr.method.func;
let positional_args: Vec<PyObjectRef> =
self.pop_multiple(nargs as usize).collect();
let self_or_null = self.pop_value_opt();
self.pop_value(); // callable
let mut all_args = Vec::with_capacity(total_nargs as usize);
if let Some(self_val) = self_or_null {
all_args.push(self_val);
}
all_args.extend(positional_args);
let (_callable, all_args) = self.take_call_args(nargs as usize);
debug_assert_eq!(all_args.len(), total_nargs as usize);
let args = FuncArgs {
args: all_args,
kwargs: Default::default(),
Expand Down Expand Up @@ -6348,15 +6316,8 @@ impl ExecutingFrame<'_> {
.is_some_and(|self_obj| self_obj.class().is(descr.objclass))
{
let func = descr.method.func;
let positional_args: Vec<PyObjectRef> =
self.pop_multiple(nargs as usize).collect();
let self_or_null = self.pop_value_opt();
self.pop_value(); // callable
let mut all_args = Vec::with_capacity(total_nargs as usize);
if let Some(self_val) = self_or_null {
all_args.push(self_val);
}
all_args.extend(positional_args);
let (_callable, all_args) = self.take_call_args(nargs as usize);
debug_assert_eq!(all_args.len(), total_nargs as usize);
let args = FuncArgs {
args: all_args,
kwargs: Default::default(),
Expand Down Expand Up @@ -6395,15 +6356,8 @@ impl ExecutingFrame<'_> {
.is_some_and(|self_obj| self_obj.class().is(descr.objclass))
{
let func = descr.method.func;
let positional_args: Vec<PyObjectRef> =
self.pop_multiple(nargs as usize).collect();
let self_or_null = self.pop_value_opt();
self.pop_value(); // callable
let mut all_args = Vec::with_capacity(total_nargs as usize);
if let Some(self_val) = self_or_null {
all_args.push(self_val);
}
all_args.extend(positional_args);
let (_callable, all_args) = self.take_call_args(nargs as usize);
debug_assert_eq!(all_args.len(), total_nargs as usize);
let args = FuncArgs {
args: all_args,
kwargs: Default::default(),
Expand All @@ -6420,22 +6374,9 @@ impl ExecutingFrame<'_> {
if let Some(cls) = callable.downcast_ref::<PyType>()
&& cls.slots.vectorcall.load().is_some()
{
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
let self_or_null = self.pop_value_opt();
let callable = self.pop_value();
let self_is_some = self_or_null.is_some();
let mut args_vec = Vec::with_capacity(nargs_usize + usize::from(self_is_some));
if let Some(self_val) = self_or_null {
args_vec.push(self_val);
}
args_vec.extend(pos_args);
let result = callable.vectorcall(
args_vec,
nargs_usize + usize::from(self_is_some),
None,
vm,
)?;
let (callable, args_vec) = self.take_call_args(nargs as usize);
let effective_nargs = args_vec.len();
let result = callable.vectorcall(args_vec, effective_nargs, None, vm)?;
self.push_value(result);
return Ok(None);
}
Expand Down Expand Up @@ -6520,15 +6461,8 @@ impl ExecutingFrame<'_> {
.is_some_and(|self_obj| self_obj.class().is(descr.objclass))
{
let func = descr.method.func;
let positional_args: Vec<PyObjectRef> =
self.pop_multiple(nargs as usize).collect();
let self_or_null = self.pop_value_opt();
self.pop_value(); // callable
let mut all_args = Vec::with_capacity(total_nargs as usize);
if let Some(self_val) = self_or_null {
all_args.push(self_val);
}
all_args.extend(positional_args);
let (_callable, all_args) = self.take_call_args(nargs as usize);
debug_assert_eq!(all_args.len(), total_nargs as usize);
let args = FuncArgs {
args: all_args,
kwargs: Default::default(),
Expand Down Expand Up @@ -6557,15 +6491,8 @@ impl ExecutingFrame<'_> {
| PyMethodFlags::O
| PyMethodFlags::KEYWORDS);
if call_conv == (PyMethodFlags::FASTCALL | PyMethodFlags::KEYWORDS) {
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
let self_or_null = self.pop_value_opt();
let callable = self.pop_value();
let mut args_vec = Vec::with_capacity(effective_nargs as usize);
if let Some(self_val) = self_or_null {
args_vec.push(self_val);
}
args_vec.extend(pos_args);
let (callable, args_vec) = self.take_call_args(nargs as usize);
debug_assert_eq!(args_vec.len(), effective_nargs as usize);
let result =
callable.vectorcall(args_vec, effective_nargs as usize, None, vm)?;
self.push_value(result);
Expand All @@ -6589,22 +6516,13 @@ impl ExecutingFrame<'_> {
{
return self.execute_call_vectorcall(nargs, vm);
}
let nargs_usize = nargs as usize;
let pos_args: Vec<PyObjectRef> = self.pop_multiple(nargs_usize).collect();
let self_or_null = self.pop_value_opt();
let callable = self.pop_value();
let mut args_vec =
Vec::with_capacity(nargs_usize + usize::from(self_or_null_is_some));
if let Some(self_val) = self_or_null {
args_vec.push(self_val);
}
args_vec.extend(pos_args);
let result = callable.vectorcall(
args_vec,
nargs_usize + usize::from(self_or_null_is_some),
None,
vm,
)?;
let (callable, args_vec) = self.take_call_args(nargs as usize);
debug_assert_eq!(
args_vec.len(),
nargs as usize + usize::from(self_or_null_is_some)
);
let effective_nargs = args_vec.len();
let result = callable.vectorcall(args_vec, effective_nargs, None, vm)?;
self.push_value(result);
Ok(None)
}
Expand Down Expand Up @@ -11413,6 +11331,49 @@ impl ExecutingFrame<'_> {
}
}

/// Take a call's `[self_or_null, arg1, ..., argN]` off the stack as one
/// vectorcall argument list, along with the callable underneath them.
///
/// The stack already holds the arguments in vectorcall order, so filling a
/// single vector by index costs one allocation — collecting the positional
/// arguments first and then pushing `self` in front of them costs two plus
/// a copy.
fn take_call_args(&mut self, nargs: usize) -> (PyObjectRef, Vec<PyObjectRef>) {
let stack_len = self.localsplus.stack_len();
debug_assert!(
stack_len >= nargs + 2,
"CALL stack underflow: need callable + self_or_null + {nargs} args, have {stack_len}"
);
let callable_idx = stack_len - nargs - 2;
let self_or_null_idx = callable_idx + 1;

let self_or_null = self
.localsplus
.stack_index_mut(self_or_null_idx)
.take()
.map(|sr| sr.to_pyobj());
let mut args = Vec::with_capacity(nargs + usize::from(self_or_null.is_some()));
args.extend(self_or_null);
for stack_idx in self_or_null_idx + 1..stack_len {
let val = self
.localsplus
.stack_index_mut(stack_idx)
.take()
.unwrap()
.to_pyobj();
args.push(val);
}

let callable = self
.localsplus
.stack_index_mut(callable_idx)
.take()
.unwrap()
.to_pyobj();
self.localsplus.stack_truncate(callable_idx);
(callable, args)
}

/// Pop multiple values from the stack. Panics if any slot is NULL.
fn pop_multiple(&mut self, count: usize) -> impl ExactSizeIterator<Item = PyObjectRef> + '_ {
let stack_len = self.localsplus.stack_len();
Expand Down