Proposal: Taking Inspiration from V8’s JIT Architecture for RustPython

Summary

RustPython already has a strong foundation as a Python interpreter implemented in Rust. As the project evolves, there is an opportunity to explore a tiered execution architecture inspired by techniques used by modern JavaScript engines such as V8.

This proposal suggests investigating a feedback-driven, tiered JIT architecture for RustPython rather than attempting to build a large optimizing compiler all at once.

The central idea is:

Start with a lightweight interpreter, collect runtime feedback, identify hot code, and progressively compile frequently executed Python code into increasingly optimized machine code.

The goal is not to reproduce V8's implementation. Python and JavaScript have substantially different execution models and language semantics. Instead, the proposal is to adapt the architectural ideas that make V8 effective to RustPython's requirements.

Motivation

A traditional interpreter must repeatedly perform operations such as:

• opcode dispatch
• dynamic type checks
• attribute lookup
• method lookup
• binary-operation dispatch
• global-variable lookup
• function-call setup
• Python object manipulation

For example:

def sum_values(values):
    total = 0
    for value in values:
        total += value
    return total

If this function executes millions of times, repeatedly performing dynamic dispatch can become expensive.

A JIT could observe that:

• value is usually an integer
• total is usually an integer
• the addition operation repeatedly resolves to integer addition

and specialize the hot loop accordingly.

The important architectural question is therefore not simply:

"How do we compile Python?"

but:

"How can RustPython progressively specialize the parts of Python programs that actually matter at runtime?"

This is where V8 provides useful architectural inspiration.

1. Proposed Architecture

A possible RustPython execution pipeline could look like:

                  Python Source
                        │
                        ▼
                     Parser
                        │
                        ▼
                  Python Bytecode
                        │
                        ▼
                 ┌──────────────┐
                 │ Interpreter  │
                 │   / VM       │
                 └──────┬───────┘
                        │
                  Runtime Feedback
                        │
              ┌─────────┴─────────┐
              │                   │
          Cold Code           Hot Code
              │                   │
              ▼                   ▼
        Interpreter          Baseline JIT
                                  │
                                  ▼
                            Optimizing JIT
                                  │
                                  ▼
                           Native Machine Code

This introduces several execution tiers.

Tier 0 — Interpreter

All code initially executes through the existing RustPython VM.

This provides:

• fast startup
• compatibility
• simple fallback behavior
• a reliable execution path for uncommon code

Tier 1 — Baseline JIT

Frequently executed functions or loops are compiled using relatively inexpensive techniques.

The baseline compiler should prioritize:

• compilation speed
• predictable performance
• low memory overhead

It does not need aggressive optimization.

Tier 2 — Optimizing JIT

Code that remains hot can be compiled again using runtime feedback.

The optimizer could specialize:

• integer arithmetic
• floating-point arithmetic
• attribute access
• method calls
• global lookup
• builtin calls
• common container operations
• monomorphic or low-polymorphism call sites

This tier would be analogous conceptually to the role TurboFan plays in V8, while remaining specifically designed for Python.

2. Runtime Feedback

One of the most valuable ideas to borrow from V8 is feedback-directed optimization.

Rather than attempting to understand every possible behavior statically, RustPython can observe what actually happens during execution.

For example:

def add(a, b):
    return a + b

The interpreter could record observations such as:

Call site: add.a + add.b

Observed:
    int + int     → 95%
    float + float → 4%
    other         → 1%

The JIT could then generate a fast path for the common case.

Conceptually:

                 a + b
                   │
                   ▼
             Check operand types
               /           \
             int           other
              │              │
              ▼              ▼
       fast integer       generic
          addition         Python
                           dispatch

The generic path remains necessary because Python is dynamically typed.

3. Inline Caches

Python has many operations that could potentially benefit from inline caches.

For example:

user.name

A generic attribute lookup may involve several steps.

A specialized cache could remember information about the observed object layout.

Conceptually:

LOAD_ATTR name

        │
        ▼
   Inline Cache
        │
   ┌────┴────┐
   │         │
 known     unknown
 layout     layout
   │         │
   ▼         ▼
fast path  generic
           lookup

Potential candidates include:

• LOAD_ATTR
• STORE_ATTR
• global lookup
• method lookup
• binary operations
• subscription
• function calls

The cache should be invalidated when assumptions about the underlying Python objects cease to hold.

4. Object Shapes / Type Layouts

V8's hidden classes provide an interesting model for optimizing object property access.

Python's object model is different, so RustPython should not directly implement JavaScript-style hidden classes.

However, a related concept could be explored:

Track stable object layouts and use them as optimization information.

For example, objects repeatedly created with the same attribute structure could potentially share layout metadata.

Object A                    Object B

name ──► "Alice"            name ──► "Bob"
age  ──► 25                 age  ──► 30
city ──► "Paris"             city ──► "London"
       │                           │
       └──── compatible layout ────┘

The JIT could use this information to accelerate repeated attribute access.

This should be evaluated carefully because Python permits highly dynamic object mutation.

5. Specialization of Python Operations

A major opportunity is specializing common Python operations.

For example:

x + y

could have specialized implementations for common observed combinations:

int + int
float + float
str + str
list + list
generic Python objects

Similarly:

x < y

could specialize based on observed operand types.

The generated code could conceptually look like:

if operands are known integers:
    perform native integer comparison
else:
    invoke Python comparison protocol

This preserves Python semantics while allowing common cases to bypass expensive generic dispatch.

6. Function Calls

Function calls are particularly important for Python performance.

Consider:

def calculate(x):
    return helper(x)

If helper consistently refers to the same function, the JIT could specialize the call site.

Potential progression:

Generic Python call
        │
        ▼
Call feedback
        │
        ▼
Same target observed repeatedly
        │
        ▼
Specialized call
        │
        ▼
Potential inlining

Eventually, frequently executed small functions could potentially be inlined.

For example:

def square(x):
    return x * x

def compute(x):
    return square(x) + 1

could potentially become conceptually:

def compute(x):
    return (x * x) + 1

inside optimized machine code.

The observable Python semantics must remain unchanged.

7. Loop Optimization

Loops are likely to be among the highest-value JIT targets.

For example:

def calculate(values):
    result = 0

    for value in values:
        result += value

    return result

The interpreter could detect that this loop executes many times.

Runtime feedback might establish:

values → list
elements → int
result → int
addition → integer addition

The optimizing compiler could then produce a specialized loop.

Conceptually:

Generic Python loop
       │
       ▼
Profile
       │
       ▼
Hot loop detected
       │
       ▼
Type specialization
       │
       ▼
Optimized native loop

This is potentially much more valuable than attempting to JIT every function equally.

8. Deoptimization

A Python JIT needs a reliable escape mechanism.

Suppose the optimizer observes:

x = 10

and specializes subsequent operations assuming x is an integer.

Later, execution follows a path where the assumption is no longer valid.

The generated machine code must be able to transfer control back to a compatible interpreter or less-specialized execution tier.

Optimized code
      │
      │ assumption invalid
      ▼
Deoptimization
      │
      ▼
Reconstruct interpreter state
      │
      ▼
Continue execution

This is one of the most important architectural requirements.

A useful design principle would be:

Every optimization must have a clearly defined deoptimization or fallback path.

This makes aggressive specialization considerably safer.

9. Guard-Based Optimization

Instead of requiring static certainty, the JIT can insert guards.

For example:

             x + y
               │
               ▼
        ┌──────────────┐
        │ x is integer?│
        └──────┬───────┘
             yes
               │
               ▼
       ┌───────────────┐
       │ y is integer? │
       └──────┬────────┘
             yes
               │
               ▼
       Native integer add

If a guard fails:

Guard failure
      │
      ▼
Generic Python operation

This provides a practical bridge between dynamic Python semantics and native machine-code optimization.

10. Intermediate Representation

A dedicated IR would likely be required for an optimizing JIT.

A possible pipeline:

Python Bytecode
      │
      ▼
High-Level IR
      │
      ▼
Specialization
      │
      ▼
Optimization
      │
      ▼
Low-Level IR
      │
      ▼
Machine Code

The IR could eventually support optimizations such as:

• constant folding
• dead-code elimination
• common-subexpression elimination
• type specialization
• inline expansion
• bounds-check elimination where valid
• redundant guard elimination
• loop optimizations
• escape analysis

The initial implementation should probably support only a small subset.

11. Suggested Implementation Strategy

Rather than introducing a complete optimizing JIT immediately, the work could be divided into incremental stages.

Phase 1 — Profiling Infrastructure

Add lightweight runtime counters for:

• function invocation count
• loop execution count
• opcode frequency
• call-site targets
• observed operand types
• attribute access patterns

No native compilation is required yet.

The primary objective is to understand real workloads.

Phase 2 — Baseline JIT

Implement a simple JIT for a restricted subset of bytecode.

Potential first targets:

• integer arithmetic
• local variables
• comparisons
• simple branches
• simple loops
• function returns

The baseline JIT should prioritize correctness and compilation speed.

Phase 3 — Inline Caches

Introduce caches for common dynamic operations:

• LOAD_ATTR
• STORE_ATTR
• CALL
• BINARY_OP
• LOAD_GLOBAL

Measure their effectiveness independently from the JIT.

Phase 4 — Optimizing IR

Introduce a small SSA-based or equivalent IR.

Initially support only a limited set of operations.

For example:

LoadLocal
AddInt
CompareInt
Branch
Return

Additional Python semantics can be introduced incrementally.

Phase 5 — Specialization

Use runtime feedback to specialize hot code.

For example:

generic BINARY_OP
        ↓
observed int + int
        ↓
specialized AddInt

The optimizer should retain a generic fallback.

Phase 6 — Deoptimization

Add machinery to:

1. identify failed assumptions
2. reconstruct interpreter-visible state
3. return to the VM
4. continue execution correctly

This should be treated as a first-class component rather than an afterthought.

12. Compatibility as a Core Requirement

Python's dynamism makes compatibility particularly important.

The JIT must correctly handle features such as:

• monkey patching
• descriptors
• dynamic attribute changes
• custom __getattribute__
• custom __getattr__
• operator overloading
• metaclasses
• dynamic globals
• generators
• coroutines
• exceptions
• eval
• exec
• reflection
• tracing/debugging hooks

Therefore, optimization should never depend on an assumption that cannot be invalidated or otherwise proven safe.

The interpreter should remain the ultimate semantic fallback.

13. Architecture Proposal

A possible RustPython implementation could eventually look like:

                         RustPython
                             │
             ┌───────────────┴───────────────┐
             │                               │
         Frontend                         Runtime
             │                               │
          Parser                        Object Model
             │                               │
        Bytecode VM                  GC / Memory
             │                               │
             └───────────────┬───────────────┘
                             │
                         Profiler
                             │
                     Runtime Feedback
                             │
                             ▼
                       Tier Manager
                       /          \
                      /            \
                 Baseline        Optimizer
                    JIT              JIT
                      \              /
                       \            /
                        └────┬─────┘
                             │
                            IR
                             │
                     Native Codegen
                             │
                             ▼
                           CPU

This separates the concerns cleanly:

• VM — semantic correctness
• Profiler — runtime information
• Tier manager — decides when to compile
• Baseline JIT — inexpensive native execution
• Optimizer — aggressive specialization
• IR — compiler abstraction
• Code generator — machine code generation

14. What RustPython Should Borrow From V8

The proposal is specifically to borrow architectural principles, not implementation details.

Worth Investigating

• tiered execution
• runtime profiling
• feedback-directed optimization
• inline caches
• speculative specialization
• guard-based optimization
• deoptimization
• function inlining
• hot-loop optimization
• compact intermediate representations

Not Necessarily Appropriate to Copy Directly

• JavaScript-specific object shapes
• JavaScript-specific type assumptions
• V8's exact bytecode format
• V8's exact IR
• V8's compiler pipeline
• JavaScript-specific runtime optimizations

Python's object model and semantics should drive RustPython's implementation.

15. Success Metrics

The project should establish measurable benchmarks rather than judging the JIT only by peak synthetic performance.

Startup

• time to first execution
• time to first result
• JIT compilation overhead

Throughput

• interpreted execution
• baseline JIT execution
• optimized JIT execution

Memory

• bytecode memory
• JIT metadata
• generated machine code
• profiling information

Compilation

• compilation latency
• code-cache size
• optimization frequency
• deoptimization frequency

Compatibility

Run the JIT against RustPython's existing test suite and progressively expand coverage.

16. Proposed First Milestone

A realistic first milestone could be:

Build a profiling-driven prototype capable of identifying hot functions and loops and specializing a small subset of numeric Python operations.

For example:

def sum_range(n):
    total = 0

    for i in range(n):
        total += i

    return total

The prototype could initially demonstrate:

Interpreter
    ↓
Detect hot loop
    ↓
Collect type feedback
    ↓
Compile specialized loop
    ↓
Execute native code
    ↓
Fall back when assumptions fail

This would validate the fundamental architecture without requiring a complete Python JIT.

Conclusion

V8 demonstrates that a high-performance dynamic-language runtime does not necessarily need to choose between a pure interpreter and an all-or-nothing optimizing compiler.

Its architecture shows the value of progressively moving code through execution tiers based on actual runtime behavior.

RustPython could explore a similar philosophy:

          Start simple
               │
               ▼
          Interpret code
               │
               ▼
        Collect feedback
               │
               ▼
          Find hot code
               │
               ▼
       Generate baseline code
               │
               ▼
      Specialize hot operations
               │
               ▼
       Optimize aggressively
               │
               ▼
       Deoptimize when needed

The key opportunity is to make runtime feedback a central part of RustPython's execution architecture.

Rather than attempting to implement "a Python version of V8," the proposed direction is to develop a Rust-native, Python-aware tiered JIT that takes inspiration from the engineering principles behind modern dynamic-language engines while preserving Python's semantics.

This could provide a gradual path from today's interpreter toward a high-performance RustPython runtime without requiring the entire compiler stack to be built upfront.