Repository navigation
CLR not working on 32 bit Linux #1210
Description
Activity
I can also confirm this issue with pythonnet 2.5.0, 2.5.1 and the master branch (84e2735). The issue does not exist in pythonnet 2.4.0.
I am not using a VM nor a live USB stick. I have 32-bit ubuntu installed on a HDD. Below shows the crash report when using Python 3.5.2 from the OS. I also created Python 3.7.9 and 3.8.6 virtual environments and importing clr fails in these environments.
user@ubuntu:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.7 LTS Release: 16.04 Codename: xenial user@ubuntu:~$ mono --version Mono JIT compiler version 5.20.1.34 (tarball Tue Jul 16 22:57:14 UTC 2019) Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com TLS: __thread SIGSEGV: altstack Notifications: epoll Architecture: x86 Disabled: none Misc: softdebug Interpreter: yes LLVM: yes(600) Suspend: hybrid GC: sgen (concurrent by default) user@ubuntu:~$ python3 Python 3.5.2 (default, Oct 7 2020, 17:19:02) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import clr ================================================================= Native Crash Reporting ================================================================= Got a SIGSEGV while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= ...The complete crash report is available here
If I don't start the Python REPL to import clr but pass it in through the command line I get the following
user@ubuntu:~$ python3 -c "import clr" Traceback (most recent call last): File "<string>", line 1, in <module> SystemError: initialization of clr raised unreported exception Error in sys.excepthook: SystemError: ../Objects/listobject.c:199: bad argument to internal function During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 57, in apport_excepthook from cStringIO import StringIO File "<frozen importlib._bootstrap>", line 968, in _find_and_load SystemError: PyEval_EvalFrameEx returned a result with an error set Original exception was: Traceback (most recent call last): File "<string>", line 1, in <module> SystemError: initialization of clr raised unreported exception@filmor A summary of bisecting is available here
The criteria that I used to decide if a commit was good or bad was by running the following bash script (which I named issue1210.sh) after each bisect iteration. If pytest passed then that commit was considered good.
#!/bin/bash export LD_PRELOAD=/lib/i386-linux-gnu/libSegFault.so export SEGFAULT_SIGNALS=all export PYTHONUNBUFFERED=True export BUILD_OPTS="" export NUNIT_PATH="./packages/NUnit.ConsoleRunner.3.7.0/tools/nunit3-console.exe" export RUN_TESTS="mono $NUNIT_PATH" export EMBED_TESTS_PATH="" export PERF_TESTS_PATH="" source ~/py37env/bin/activate python --version pip --version PY_LIBDIR=$(python -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))') export LD_LIBRARY_PATH=$PY_LIBDIR:$LD_LIBRARY_PATH pip install --upgrade setuptools pip install --upgrade -r requirements.txt pip uninstall pythonnet -y coverage run setup.py install $BUILD_OPTS python -m pytestPlease let me know if you'd like further information.
Reacted by Benedikt ReinartzI've gone a little further to see if there are additional issues if the changes from PR #971 were temporarily removed.
Here's what I did:
- git checkout 4271e57
- deleted the lines that correspond to the
SetNoSiteFlag()function insrc/runtime/runtime.csandsrc/runtime/pythonengine.cs - ran my issue1210.sh script and pytest passed
- git checkout 770fc01
- deleted the lines that correspond to the
SetNoSiteFlag()function insrc/runtime/runtime.csandsrc/runtime/pythonengine.cs - ran my issue1210.sh script and got this error
770fc01 immediately follows 4271e57 so 770fc01 seems to also introduce errors on 32-bit linux.
770fc01 passed the tests on travis (ignoring that one pip issue) so it is okay on 64-bit linux.
I can also confirm that the issue does not exist in pythonnet 2.4.0.
However, in order to work on pythonnet 2.4.0, the version of mono MUST be 5.20.The current mono version (6.12) throws the following error when importing the clr module in pythonnet 2.4.0:
ImportError: /usr/lib/libmonosgen-2.0.so.1: undefined symbol: __cxa_begin_catch@daniol Sorry for the silent closure, I just skimmed the last message (was cleaning up old issues) and saw the
__cxa_begin_catchwhich has been solved quite a while ago. I can confirm that the issue exists in the current master as well. One additional problem is thatpythonnetuses thedotnetCLI for building now, which is not available for 32bit Linux. Installing from a wheel built on a 64bit machine works, though.I'm not sure that's worth the hassle (and I like the current clean
setup.py;)). Building on CI and distributing wheels should be enough, and on all platforms that are officially supported to run .NET >5,dotnet buildwill work.Also, to officially support 32 bit we need it in CI.
Since the build now depends on the
dotnetCLI (which is not available on 32-bit Linux) I built a wheel on 64-bit Linux (using commit 75f4398) and copied the wheel to 32-bit Linux. Unfortunately, the issue importingclrremains, although with a different error compared to the error that I got back in November 2020.$ mono --version Mono JIT compiler version 5.20.1.34 (tarball Tue Jul 16 22:57:14 UTC 2019) Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com TLS: __thread SIGSEGV: altstack Notifications: epoll Architecture: x86 Disabled: none Misc: softdebug Interpreter: yes LLVM: yes(600) Suspend: hybrid GC: sgen (concurrent by default) $ pip list Package Version ---------- ---------- cffi 1.15.0 clr-loader 0.1.7 pip 21.3.1 pycparser 2.21 pythonnet 3.0.0.dev1 setuptools 60.5.0 wheel 0.37.1 $ python Python 3.7.10 (default, Feb 20 2021, 21:21:24) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import clr Failed to initialize pythonnet: System.NullReferenceException: Object reference not set to an instance of an object. at Python.Runtime.NewReferenceExtensions.Borrow (Python.Runtime.NewReference& reference) [0x0000f] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Runtime.InitPyMembers () [0x0000b] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Runtime.Initialize (System.Boolean initSigs) [0x000a5] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.PythonEngine.Initialize (System.Collections.Generic.IEnumerable`1[T] args, System.Boolean setSysArgv, System.Boolean initSigs) [0x00012] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.PythonEngine.Initialize (System.Boolean setSysArgv, System.Boolean initSigs) [0x00005] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.PythonEngine.InitExt () [0x00032] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Loader.Initialize (System.IntPtr data, System.Int32 size) [0x0002f] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.NewReferenceExtensions.Borrow (Python.Runtime.NewReference& reference) [0x0000f] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Runtime.InitPyMembers () [0x0000b] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Runtime.Initialize (System.Boolean initSigs) [0x000a5] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.PythonEngine.Initialize (System.Collections.Generic.IEnumerable`1[T] args, System.Boolean setSysArgv, System.Boolean initSigs) [0x00012] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.PythonEngine.Initialize (System.Boolean setSysArgv, System.Boolean initSigs) [0x00005] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.PythonEngine.InitExt () [0x00032] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Loader.Initialize (System.IntPtr data, System.Int32 size) [0x0002f] in <64958489404d4e7ba32bf649886da953>:0 ValueError: Empty module name The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/user/venv/lib/python3.7/site-packages/clr.py", line 6, in <module> load() File "/home/user/venv/lib/python3.7/site-packages/pythonnet/__init__.py", line 43, in load if func(''.encode("utf8")) != 0: File "/home/user/venv/lib/python3.7/site-packages/clr_loader/wrappers.py", line 20, in __call__ return self._callable(ffi.cast("void*", buf_arr), len(buf_arr)) File "/home/user/venv/lib/python3.7/site-packages/clr_loader/mono.py", line 91, in __call__ res = _MONO.mono_runtime_invoke(self._ptr, ffi.NULL, params, exception) SystemError: <cdata 'MonoObject *(*)(MonoMethod *, void *, void * *, MonoObject * *)' 0xb6ec3610> returned a result with an error set >>> import clr Failed to initialize pythonnet: System.InvalidOperationException: This property must be set before runtime is initialized at Python.Runtime.Runtime.set_PythonDLL (System.String value) [0x00007] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Loader.Initialize (System.IntPtr data, System.Int32 size) [0x00023] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Runtime.set_PythonDLL (System.String value) [0x00007] in <64958489404d4e7ba32bf649886da953>:0 at Python.Runtime.Loader.Initialize (System.IntPtr data, System.Int32 size) [0x00023] in <64958489404d4e7ba32bf649886da953>:0 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/user/venv/lib/python3.7/site-packages/clr.py", line 6, in <module> load() File "/home/user/venv/lib/python3.7/site-packages/pythonnet/__init__.py", line 44, in load raise RuntimeError("Failed to initialize Python.Runtime.dll") RuntimeError: Failed to initialize Python.Runtime.dllPlease let me know if there is something else that you would like me to try or if there is additional information about my system that you would like know.
Yes, I see the same error. The failing borrow (due to a null pointer being returned) is from the
builtinsimport. I'll try to replace it byPyEval_GetBuiltins(which we should probably use anyhow).I have replaced the
PyModule_ImportbyPyEval_GetBuiltins, but this didn't improve things. In the debugger I can see that the address that we get for the Python functions is correct, but the return value is crap (nothing that I can immediately identify, but relatively close to our current code execution, so it could be that a value in one of the registers or on the stack is misinterpreted). It could be that this function call is the first one, where the return value matters and that's why we are seeing this problem. It could also be that this has to do with the unmanaged function pointers that we are using nowadays (we used to use P/Invoke). I'll try to create a minimal example to reproduce the issue, maybe this is a problem in Mono.I have tested this now with Python 3.9, here it already fails on
PyLong_FromLongLongwhich we call inRuntime.NewRunon a simple value. I have checked the stack and it looks wrong.What I would expect:
<return address (4 bytes)> <value as LE (8 bytes)>(this is also what you can see for "native" calls)
What I see:<some address (4 bytes)> <some other address (4 bytes)> <value as LE (8 bytes)>So the
callithat is generated for our unmanaged delegate call (which is correct, as said, I have verified the addresses) pushes another parameter (I think) onto the stack, which is then frankensteined with the lower half of the value into a wrong value that is returned. If it does that on all calls, it will quite quickly destroy the stack.Does anyone have more insight here or knows a bit more about how Mono translates
calli(or knows someone who does)?
Environment
Details
Pythonnet seems not to be working on a Linux 32-Bit environment (i686) at all. I tested it on a "Debian live" (USB) system in two different computers, using Python from the Debian repositories and also building python from source code, here are the results:
Using Python from the Debian repositories:
The python execution crashes just doing
import clrSee attachment: mono_crash.4383a3eb2.0.zip, trace.txt
Building Python from source code (with
--enable-shared):It doesn't crash, but clr throws an error when trying to load an assembly ("AttributeError: module 'clr' has no attribute 'AddReference'"). This was reported in some places around internet, but I don't have any
clrmodule already installed which could cause a name collision. See output:Notes