Visitar URL original
cProfile and profile: -m MODULE does not run MODULE as `__main__` · Issue #158985 · python/cpython · GitHub
Skip to content

cProfile and profile: -m MODULE does not run MODULE as __main__ #158985

Description

@sricursion

Bug report

Bug description:

python -m cProfile -m MODULE (and python -m profile -m MODULE, and python -m profiling.tracing -m MODULE on main) runs the module with runpy.run_module(modname, run_name='__main__'), i.e. with the default alter_sys=False. While the module runs, sys.modules['__main__'] is still the profiler module and sys.argv[0] is the bare module name, unlike with python -m MODULE. Objects defined in the module therefore cannot be pickled, which also breaks ProcessPoolExecutor and multiprocessing with the spawn or forkserver start methods.

Profiling a script path was fixed for this in gh-132737 and gh-140729, but the -m branch was left as it was. The older gh-54123 describes the general problem for profile and trace; this is the remaining -m case.

# mod.py
import pickle
import sys

class Foo:
    pass

print(sys.argv[0])
print(sys.modules['__main__'])
pickle.dumps(Foo())
print("pickled")
$ python -m mod
C:\demo\mod.py
<module 'mod' from 'C:\\demo\\mod.py'>
pickled

$ python -m cProfile -m mod
mod
<module 'cProfile' from 'C:\\cpython\\Lib\\cProfile.py'>
...
_pickle.PicklingError: Can't pickle <class '__main__.Foo'>: it's not found as __main__.Foo
when serializing Foo class
when serializing Foo object

python -m profile -m mod fails the same way. A module that calls ProcessPoolExecutor.map() on a function defined in it fails with PicklingError: Can't pickle <function double at 0x...>: it's not found as __main__.double, while python -m on the same module prints the expected result.

Expected: the same behavior as python -m mod: sys.argv[0] is the path of mod.py, sys.modules['__main__'] is the module being run, and pickle.dumps(Foo()) succeeds.

Passing alter_sys=True to run_module() in the -m branch of profile.main() and profiling.tracing.main() fixes it. multiprocessing.spawn and the sampling profiler's _sync_coordinator already do this. On 3.13 and 3.14 the same line is in Lib/cProfile.py, and the error there reads attribute lookup Foo on __main__ failed.

CPython versions tested on:

3.11, 3.12, 3.13, CPython main branch

Operating systems tested on:

Windows

Linked PRs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pendingThe issue will be closed if no feedback is providedstdlibStandard Library Python modules in the Lib/ directorytopic-profilingtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions