Visitar URL original
Bytecode parity by youknowone · Pull Request #7535 · RustPython/RustPython · GitHub
Skip to content

Bytecode parity #7535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 4 commits into from
Mar 30, 2026
Merged

Bytecode parity #7535

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4959703
Add CFG block splitting, jump threading, backward jump normalization,…
youknowone Mar 29, 2026
385b719
Add ConstantData::Slice and constant slice folding
youknowone Mar 30, 2026
f8927e1
Add ConstantData::Frozenset variant (type only, folding deferred)
youknowone Mar 30, 2026
42b24cb
Add duplicate_exits_without_lineno (disabled) and Block: Clone
youknowone Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add ConstantData::Slice and constant slice folding 
- Add Slice variant to ConstantData and BorrowedConstant
- Fold constant slices (x[:3], x[1:4]) into LOAD_CONST(slice(...))
- Marshal serialization/deserialization for Slice type
- Box::leak in borrow_obj_constant for PySlice roundtrip
  • Loading branch information
@youknowone
youknowone committed Mar 30, 2026
commit 385b719876ad0b81fa77d3ae4d7464d125e117eb
45 changes: 45 additions & 0 deletions crates/codegen/src/compile.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7574,6 +7574,10 @@ impl Compiler {
ast::Expr::Slice(ast::ExprSlice {
lower, upper, step, ..
}) => {
// Try constant slice folding first
if self.try_fold_constant_slice(lower.as_deref(), upper.as_deref(), step.as_deref()) {
return Ok(());
}
Comment on lines +7577 to +7581

@coderabbitai coderabbitai Bot Mar 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

try_fold_constant_slice is narrower than the rest of the compiler’s constant logic.

Lines 8999-9016 only fold numbers, booleans, and None, and Line 9005 also limits integer folding to i64. But ExprExt::is_constant_slice() already treats string/bytes/ellipsis bounds as constant, so cases like obj["a":"z"] skip the BINARY_SLICE fast path and still never reach the new LOAD_CONST(slice) path. Large integer literals have the same gap. That leaves valid constant slices on the old bytecode path and misses the parity target for this feature.

💡 Suggested fix 
-                if self.try_fold_constant_slice(lower.as_deref(), upper.as_deref(), step.as_deref())
-                {
+                if self.try_fold_constant_slice(
+                    lower.as_deref(),
+                    upper.as_deref(),
+                    step.as_deref(),
+                )? {
                     return Ok(());
                 }
-    fn try_fold_constant_slice(
-        &mut self,
-        lower: Option<&ast::Expr>,
-        upper: Option<&ast::Expr>,
-        step: Option<&ast::Expr>,
-    ) -> bool {
-        fn to_const(expr: Option<&ast::Expr>) -> Option<ConstantData> {
-            match expr {
-                None | Some(ast::Expr::NoneLiteral(_)) => Some(ConstantData::None),
-                Some(ast::Expr::NumberLiteral(ast::ExprNumberLiteral { value, .. })) => match value
-                {
-                    ast::Number::Int(i) => Some(ConstantData::Integer {
-                        value: i.as_i64()?.into(),
-                    }),
-                    ast::Number::Float(f) => Some(ConstantData::Float { value: *f }),
-                    ast::Number::Complex { real, imag } => Some(ConstantData::Complex {
-                        value: num_complex::Complex64::new(*real, *imag),
-                    }),
-                },
-                Some(ast::Expr::BooleanLiteral(ast::ExprBooleanLiteral { value, .. })) => {
-                    Some(ConstantData::Boolean { value: *value })
-                }
-                // Only match Constant_kind nodes (no UnaryOp)
-                _ => None,
-            }
-        }
+    fn try_fold_constant_slice(
+        &mut self,
+        lower: Option<&ast::Expr>,
+        upper: Option<&ast::Expr>,
+        step: Option<&ast::Expr>,
+    ) -> CompileResult<bool> {
+        let mut to_const = |expr: Option<&ast::Expr>| -> CompileResult<Option<ConstantData>> {
+            Ok(match expr {
+                None | Some(ast::Expr::NoneLiteral(_)) => Some(ConstantData::None),
+                Some(ast::Expr::NumberLiteral(ast::ExprNumberLiteral { value, .. })) => {
+                    Some(match value {
+                        ast::Number::Int(i) => ConstantData::Integer {
+                            value: ruff_int_to_bigint(i).map_err(|e| self.error(e))?,
+                        },
+                        ast::Number::Float(f) => ConstantData::Float { value: *f },
+                        ast::Number::Complex { real, imag } => ConstantData::Complex {
+                            value: Complex::new(*real, *imag),
+                        },
+                    })
+                }
+                Some(ast::Expr::StringLiteral(s)) => Some(ConstantData::Str {
+                    value: self.compile_string_value(s),
+                }),
+                Some(ast::Expr::BytesLiteral(b)) => Some(ConstantData::Bytes {
+                    value: b.value.bytes().collect(),
+                }),
+                Some(ast::Expr::BooleanLiteral(ast::ExprBooleanLiteral { value, .. })) => {
+                    Some(ConstantData::Boolean { value: *value })
+                }
+                Some(ast::Expr::EllipsisLiteral(_)) => Some(ConstantData::Ellipsis),
+                _ => None,
+            })
+        };
 
-        let (Some(start), Some(stop), Some(step_val)) =
-            (to_const(lower), to_const(upper), to_const(step))
+        let (Some(start), Some(stop), Some(step_val)) =
+            (to_const(lower)?, to_const(upper)?, to_const(step)?)
         else {
-            return false;
+            return Ok(false);
         };
 
         self.emit_load_const(ConstantData::Slice {
             elements: Box::new([start, stop, step_val]),
         });
-        true
+        Ok(true)
     }

Also applies to: 8992-9030

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@crates/codegen/src/compile.rs` around lines 7577 - 7581,
try_fold_constant_slice currently only recognizes numbers, booleans and None and
limits integer folding to i64, which is narrower than
ExprExt::is_constant_slice; update try_fold_constant_slice to accept the same
constant kinds that ExprExt::is_constant_slice treats constant (include
string/bytes/ellipsis bounds and arbitrary-size integers, not just i64) so that
patterns like obj["a":"z"] and large integer literals are folded to
LOAD_CONST(slice) and hit the BINARY_SLICE fast path; make the same expansion in
the adjacent constant-folding block that has the i64 restriction so both places
have parity with ExprExt::is_constant_slice.

let mut compile_bound = |bound: Option<&ast::Expr>| match bound {
Some(exp) => self.compile_expression(exp),
None => {
Expand Down Expand Up @@ -8984,6 +8988,47 @@ impl Compiler {
self.emit_arg(idx, |consti| Instruction::LoadConst { consti })
}

/// Fold constant slice: if all parts are compile-time constants, emit LOAD_CONST(slice).
fn try_fold_constant_slice(
&mut self,
lower: Option<&ast::Expr>,
upper: Option<&ast::Expr>,
step: Option<&ast::Expr>,
) -> bool {
fn to_const(expr: Option<&ast::Expr>) -> Option<ConstantData> {
match expr {
None | Some(ast::Expr::NoneLiteral(_)) => Some(ConstantData::None),
Some(ast::Expr::NumberLiteral(ast::ExprNumberLiteral { value, .. })) => {
match value {
ast::Number::Int(i) => Some(ConstantData::Integer {
value: i.as_i64()?.into(),
}),
ast::Number::Float(f) => Some(ConstantData::Float { value: *f }),
ast::Number::Complex { real, imag } => Some(ConstantData::Complex {
value: num_complex::Complex64::new(*real, *imag),
}),
}
}
Some(ast::Expr::BooleanLiteral(ast::ExprBooleanLiteral { value, .. })) => {
Some(ConstantData::Boolean { value: *value })
}
// Only match Constant_kind nodes (no UnaryOp)
_ => None,
}
}

let (Some(start), Some(stop), Some(step_val)) =
(to_const(lower), to_const(upper), to_const(step))
else {
return false;
};

self.emit_load_const(ConstantData::Slice {
elements: Box::new([start, stop, step_val]),
});
true
}

fn emit_return_const(&mut self, constant: ConstantData) {
self.emit_load_const(constant);
emit!(self, Instruction::ReturnValue)
Expand Down
18 changes: 18 additions & 0 deletions crates/compiler-core/src/bytecode.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,7 @@ impl Constant for ConstantData {
Self::Bytes { value } => Bytes { value },
Self::Code { code } => Code { code },
Self::Tuple { elements } => Tuple { elements },
Self::Slice { elements } => Slice { elements },
Self::None => None,
Self::Ellipsis => Ellipsis,
}
Expand Down Expand Up @@ -857,6 +858,8 @@ pub enum ConstantData {
Str { value: Wtf8Buf },
Bytes { value: Vec<u8> },
Code { code: Box<CodeObject> },
/// Constant slice(start, stop, step)
Slice { elements: Box<[ConstantData; 3]> },
None,
Ellipsis,
}
Comment on lines +854 to 887

@coderabbitai coderabbitai Bot Mar 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if there are other match statements on ConstantData that may need updating
rg -n "match.*ConstantData" --type=rust -A10 | head -100

Repository: RustPython/RustPython

Length of output: 47

🏁 Script executed:

cat crates/jit/tests/common.rs

Repository: RustPython/RustPython

Length of output: 14312

🏁 Script executed:

rg "ConstantData::" --type=rust -B2 -A5 | head -150

Repository: RustPython/RustPython

Length of output: 9976

🏁 Script executed:

fd . crates/jit/tests --type f -name "*.rs"

Repository: RustPython/RustPython

Length of output: 295

🏁 Script executed:

rg "match.*\{" --type=rust -B3 -A15 | grep -A15 "ConstantData"

Repository: RustPython/RustPython

Length of output: 41022

🏁 Script executed:

rg "Slice\s*\{|Frozenset\s*\{" --type=rust -B2 -A2

Repository: RustPython/RustPython

Length of output: 16928

🏁 Script executed:

fd . crates/jit/tests --type f -name "*.rs" -exec grep -l "slice\|frozenset" {} \;

Repository: RustPython/RustPython

Length of output: 295

Update crates/jit/tests/common.rs to handle ConstantData::Slice and ConstantData::Frozenset variants.

The From<ConstantData> for StackValue implementation uses a wildcard match arm that will panic at runtime if these new constant types are encountered. Since the compiler actively generates ConstantData::Slice constants, any JIT test using slice expressions will fail with an unimplemented! panic. Add handling for both variants in the match statement.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@crates/compiler-core/src/bytecode.rs` around lines 854 - 887, The
From<ConstantData> for StackValue implementation currently uses a wildcard arm
and will panic for the new ConstantData::Slice and ConstantData::Frozenset
variants; update the match in From<ConstantData> for StackValue to explicitly
handle ConstantData::Slice (map it to an appropriate StackValue variant
representing a slice object, constructing start/stop/step from the elements
array) and ConstantData::Frozenset (convert elements Vec<ConstantData> into the
corresponding StackValue frozenset representation), ensuring you pattern-match
on ConstantData::Slice and ConstantData::Frozenset instead of relying on the
unimplemented wildcard arm.

Expand All @@ -878,6 +881,7 @@ impl PartialEq for ConstantData {
(Bytes { value: a }, Bytes { value: b }) => a == b,
(Code { code: a }, Code { code: b }) => core::ptr::eq(a.as_ref(), b.as_ref()),
(Tuple { elements: a }, Tuple { elements: b }) => a == b,
(Slice { elements: a }, Slice { elements: b }) => a == b,
(None, None) => true,
(Ellipsis, Ellipsis) => true,
_ => false,
Expand All @@ -904,6 +908,7 @@ impl hash::Hash for ConstantData {
Bytes { value } => value.hash(state),
Code { code } => core::ptr::hash(code.as_ref(), state),
Tuple { elements } => elements.hash(state),
Slice { elements } => elements.hash(state),
None => {}
Ellipsis => {}
}
Expand All @@ -920,6 +925,7 @@ pub enum BorrowedConstant<'a, C: Constant> {
Bytes { value: &'a [u8] },
Code { code: &'a CodeObject<C> },
Tuple { elements: &'a [C] },
Slice { elements: &'a [C; 3] },
None,
Ellipsis,
}
Expand Down Expand Up @@ -957,6 +963,15 @@ impl<C: Constant> BorrowedConstant<'_, C> {
}
write!(f, ")")
}
BorrowedConstant::Slice { elements } => {
write!(f, "slice(")?;
elements[0].borrow_constant().fmt_display(f)?;
write!(f, ", ")?;
elements[1].borrow_constant().fmt_display(f)?;
write!(f, ", ")?;
elements[2].borrow_constant().fmt_display(f)?;
write!(f, ")")
}
BorrowedConstant::None => write!(f, "None"),
BorrowedConstant::Ellipsis => write!(f, "..."),
}
Expand Down Expand Up @@ -987,6 +1002,9 @@ impl<C: Constant> BorrowedConstant<'_, C> {
.map(|c| c.borrow_constant().to_owned())
.collect(),
},
BorrowedConstant::Slice { elements } => Slice {
elements: Box::new(elements.each_ref().map(|c| c.borrow_constant().to_owned())),
},
BorrowedConstant::None => None,
BorrowedConstant::Ellipsis => Ellipsis,
}
Expand Down
13 changes: 13 additions & 0 deletions crates/compiler-core/src/marshal.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,16 @@ impl<Bag: ConstantBag> MarshalBag for Bag {
self.make_tuple(elements)
}

fn make_slice(
&self,
start: Self::Value,
stop: Self::Value,
step: Self::Value,
) -> Result<Self::Value> {
let elements = [start, stop, step];
Ok(self.make_constant::<Bag::Constant>(BorrowedConstant::Slice { elements: &elements }))
}

fn make_code(
&self,
code: CodeObject<<Self::ConstantBag as ConstantBag>::Constant>,
Expand Down Expand Up @@ -697,6 +707,9 @@ impl<'a, C: Constant> From<BorrowedConstant<'a, C>> for DumpableValue<'a, C> {
BorrowedConstant::Bytes { value } => Self::Bytes(value),
BorrowedConstant::Code { code } => Self::Code(code),
BorrowedConstant::Tuple { elements } => Self::Tuple(elements),
BorrowedConstant::Slice { elements } => {
Self::Slice(&elements[0], &elements[1], &elements[2])
}
BorrowedConstant::None => Self::None,
BorrowedConstant::Ellipsis => Self::Ellipsis,
}
Expand Down
26 changes: 26 additions & 0 deletions crates/vm/src/builtins/code.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,16 @@ fn borrow_obj_constant(obj: &PyObject) -> BorrowedConstant<'_, Literal> {
}
super::singletons::PyNone => BorrowedConstant::None,
super::slice::PyEllipsis => BorrowedConstant::Ellipsis,
ref s @ super::slice::PySlice => {
// Constant pool slices always store Some() for start/step (even for None).
// Box::leak the array so it outlives the borrow. Leak is acceptable since
// constant pool objects live for the program's lifetime.
let start = s.start.clone().unwrap();
let stop = s.stop.clone();
let step = s.step.clone().unwrap();
let arr = Box::leak(Box::new([Literal(start), Literal(stop), Literal(step)]));
BorrowedConstant::Slice { elements: arr }
}
_ => panic!("unexpected payload for constant python value"),
})
}
Expand Down Expand Up @@ -283,6 +293,22 @@ impl ConstantBag for PyObjBag<'_> {
.collect();
ctx.new_tuple(elements).into()
}
BorrowedConstant::Slice { elements } => {
let [start, stop, step] = elements;
let start_obj = self.make_constant(start.borrow_constant()).0;
let stop_obj = self.make_constant(stop.borrow_constant()).0;
let step_obj = self.make_constant(step.borrow_constant()).0;
// Store as PySlice with Some() for all fields (even None values)
// so borrow_obj_constant can reference them.
use crate::builtins::PySlice;
PySlice {
start: Some(start_obj),
stop: stop_obj,
step: Some(step_obj),
}
.into_ref(ctx)
.into()
}
BorrowedConstant::None => ctx.none(),
BorrowedConstant::Ellipsis => ctx.ellipsis.clone().into(),
};
Expand Down