ujson crashes on import on Python 3.13.0b1 #629

Activity

1. hugovk commented on May 14, 2024

   

   hugovk

   on May 14, 2024

   Last edited by hugovk

   Member

   More actions

   Reproducible on macOS as well:

   ❯ /Users/hugo/github/python/cpython/3.13/python.exe -X dev -X tracemalloc=5 -c "import ujson"
   Assertion failed: (index > 0), function _get_module_index_from_def, file import.c, line 460.
   Fatal Python error: Aborted
   
   Current thread 0x00000001fc6f8c00 (most recent call first):
     File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
     File "<frozen importlib._bootstrap_external>", line 1315 in create_module
     File "<frozen importlib._bootstrap>", line 813 in module_from_spec
     File "<frozen importlib._bootstrap>", line 921 in _load_unlocked
     File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
     File "<string>", line 1 in <module>
   [1]    20525 abort      /Users/hugo/github/python/cpython/3.13/python.exe -X dev -X tracemalloc=5 -c

   Bisecting CPython:

   291cfa454b9c5b677c955aaf53fab91f0186b6fa is the first bad commit
   commit 291cfa454b9c5b677c955aaf53fab91f0186b6fa (HEAD)
   Author: Eric Snow <ericsnowcurrently@gmail.com>
   Date:   Sat May 4 15:24:02 2024 -0600
   
       gh-117953: Track Extra Details in Global Extensions Cache (gh-118532)
   
       We have only been tracking each module's PyModuleDef.  However, there are some problems with that.  For example, in some cases we load single-phase init extension modules from def->m_base.m_init or def->m_base.m_copy, but if multiple modules share a def then we can end up with unexpected behavior.
   
       With this change, we track the following:
   
       * PyModuleDef (same as before)
       * for some modules, its init function or a copy of its __dict__, but specific to that module
       * whether it is a builtin/core module or a "dynamic" extension
       * the interpreter (ID) that owns the cached __dict__ (only if cached)
   
       This also makes it easier to remember the module's kind (e.g. single-phase init) and if loading it previously failed, which I'm doing separately.
   
    Include/internal/pycore_importdl.h |  12 +-
    Python/import.c                    | 678 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------
    Python/importdl.c                  |  24 ++-
    3 files changed, 577 insertions(+), 137 deletions(-)
   

   • python/cpython@291cfa4
   • gh-117953: Track Extra Details in Global Extensions Cache python/cpython#118532
2. ericsnowcurrently commented on May 25, 2024

   

   ericsnowcurrently

   on May 25, 2024

   More actions

   The problem is that in the referenced commit I made the faulty assumption that the module def would have been initialized before being passed to PyState_FindModule(). However, what ujson is doing is totally valid. I'll have a fix up today, which will be part of the beta 2 release. Sorry for the inconvenience!

   🚀React with 🚀2Reacted by Hugo van Kemenade and Sam James
3. mgorny commented on May 26, 2024

   

   mgorny

   on May 26, 2024

   ContributorAuthor

   More actions

   I can confirm that the linked fix works. Thanks a lot!

   👍React with 👍1Reacted by Eric Snow

   🎉React with 🎉2Reacted by Hugo van Kemenade and Kirill Podoprigora
4. mgorny

   closed this as completedon May 26, 2024