Bug report
Bug description:
python -m cProfile -m MODULE (and python -m profile -m MODULE, and python -m profiling.tracing -m MODULE on main) runs the module with runpy.run_module(modname, run_name='__main__'), i.e. with the default alter_sys=False. While the module runs, sys.modules['__main__'] is still the profiler module and sys.argv[0] is the bare module name, unlike with python -m MODULE. Objects defined in the module therefore cannot be pickled, which also breaks ProcessPoolExecutor and multiprocessing with the spawn or forkserver start methods.
Profiling a script path was fixed for this in gh-132737 and gh-140729, but the -m branch was left as it was. The older gh-54123 describes the general problem for profile and trace; this is the remaining -m case.
# mod.py
import pickle
import sys
class Foo:
pass
print(sys.argv[0])
print(sys.modules['__main__'])
pickle.dumps(Foo())
print("pickled")
$ python -m mod
C:\demo\mod.py
<module 'mod' from 'C:\\demo\\mod.py'>
pickled
$ python -m cProfile -m mod
mod
<module 'cProfile' from 'C:\\cpython\\Lib\\cProfile.py'>
...
_pickle.PicklingError: Can't pickle <class '__main__.Foo'>: it's not found as __main__.Foo
when serializing Foo class
when serializing Foo object
python -m profile -m mod fails the same way. A module that calls ProcessPoolExecutor.map() on a function defined in it fails with PicklingError: Can't pickle <function double at 0x...>: it's not found as __main__.double, while python -m on the same module prints the expected result.
Expected: the same behavior as python -m mod: sys.argv[0] is the path of mod.py, sys.modules['__main__'] is the module being run, and pickle.dumps(Foo()) succeeds.
Passing alter_sys=True to run_module() in the -m branch of profile.main() and profiling.tracing.main() fixes it. multiprocessing.spawn and the sampling profiler's _sync_coordinator already do this. On 3.13 and 3.14 the same line is in Lib/cProfile.py, and the error there reads attribute lookup Foo on __main__ failed.
CPython versions tested on:
3.11, 3.12, 3.13, CPython main branch
Operating systems tested on:
Windows
Linked PRs
Bug report
Bug description:
python -m cProfile -m MODULE(andpython -m profile -m MODULE, andpython -m profiling.tracing -m MODULEon main) runs the module withrunpy.run_module(modname, run_name='__main__'), i.e. with the defaultalter_sys=False. While the module runs,sys.modules['__main__']is still the profiler module andsys.argv[0]is the bare module name, unlike withpython -m MODULE. Objects defined in the module therefore cannot be pickled, which also breaksProcessPoolExecutorandmultiprocessingwith thespawnorforkserverstart methods.Profiling a script path was fixed for this in gh-132737 and gh-140729, but the
-mbranch was left as it was. The older gh-54123 describes the general problem forprofileandtrace; this is the remaining-mcase.python -m profile -m modfails the same way. A module that callsProcessPoolExecutor.map()on a function defined in it fails withPicklingError: Can't pickle <function double at 0x...>: it's not found as __main__.double, whilepython -mon the same module prints the expected result.Expected: the same behavior as
python -m mod:sys.argv[0]is the path ofmod.py,sys.modules['__main__']is the module being run, andpickle.dumps(Foo())succeeds.Passing
alter_sys=Truetorun_module()in the-mbranch ofprofile.main()andprofiling.tracing.main()fixes it.multiprocessing.spawnand the sampling profiler's_sync_coordinatoralready do this. On 3.13 and 3.14 the same line is inLib/cProfile.py, and the error there readsattribute lookup Foo on __main__ failed.CPython versions tested on:
3.11, 3.12, 3.13, CPython main branch
Operating systems tested on:
Windows
Linked PRs