Visitar URL original
Some test modules fail when run as a script · Issue #104057 · python/cpython · GitHub
Skip to content

Some test modules fail when run as a script #104057

Closed
#104059
Closed
Some test modules fail when run as a script#104057
#104059
Labels
testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@Eclips4
Eclips4
opened on May 1, 2023
 C:\Users\KIRILL-1\CLionProjects\cpython> ./python Lib/test/test_module.py
Running Debug|x64 interpreter...
.....................F..............
======================================================================
FAIL: test_module_repr_with_full_loader (__main__.ModuleTests.test_module_repr_with_full_loader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\test\test_module.py", line 238, in test_module_repr_with_full_loader
    self.assertEqual(
AssertionError: "<module 'foo' (<class '__main__.FullLoader'>)>" != "<module 'foo' (<class 'test.test_module.FullLoader'>)>"      
- <module 'foo' (<class '__main__.FullLoader'>)>
?                        ^  ^^^^^
+ <module 'foo' (<class 'test.test_module.FullLoader'>)>
?                        ^^^^^^^^^  ^^^^^


----------------------------------------------------------------------
Ran 36 tests in 0.217s

FAILED (failures=1)

I'll soon send a PR to fix it.

Linked PRs

Activity

  1. Eclips4
    added
    type-bugAn unexpected behavior, bug, or error
    on May 1, 2023
  2. zware
    closed this as completedin #104059on May 1, 2023
  3. zware
    added a commit that references this issue on May 1, 2023

    gh-104057: Fix direct invocation of test_module (GH-104059)

    d448fcb
  4. zware
    reopened this on May 1, 2023
  5. zware
    changed the title [-]Running test_module file directly fails[/-] [+]Some test modules fail when run as a script[/+] on May 1, 2023

  6. terryjreedy commented on May 1, 2023

    @terryjreedy
    terryjreedy
    on May 1, 2023
    Member

    I grepped with IDLE for other candidates to fix.

    Searching "<class 'test." in F:\dev\3x\Lib\test\*.py ...
F:\dev\3x\Lib\test\test_descrtut.py: 42:     <class 'test.test_descrtut.defaultdict'>
F:\dev\3x\Lib\test\test_descrtut.py: 49:     <class 'test.test_descrtut.defaultdict'>
F:\dev\3x\Lib\test\test_descrtut.py: 51:     <class 'test.test_descrtut.defaultdict'>
F:\dev\3x\Lib\test\test_descrtut.py: 267:     classmethod <class 'test.test_descrtut.C'> 1
F:\dev\3x\Lib\test\test_descrtut.py: 270:     classmethod <class 'test.test_descrtut.C'> 1
F:\dev\3x\Lib\test\test_descrtut.py: 276:     classmethod <class 'test.test_descrtut.D'> 1
F:\dev\3x\Lib\test\test_descrtut.py: 279:     classmethod <class 'test.test_descrtut.D'> 1
F:\dev\3x\Lib\test\test_descrtut.py: 295:     classmethod <class 'test.test_descrtut.C'> 1
F:\dev\3x\Lib\test\test_descrtut.py: 299:     classmethod <class 'test.test_descrtut.C'> 1
F:\dev\3x\Lib\test\test_metaclass.py: 121:     Prepare called: ('C', (<class 'test.test_metaclass.B'>, <class 'object'>)) {'other': 'haha'}
F:\dev\3x\Lib\test\test_module.py: 239:             repr(m), "<module 'foo' (<class 'test.test_module.FullLoader'>)>")
F:\dev\3x\Lib\test\test_pdb.py: 582:     <class 'test.test_pdb.MyClass'>
F:\dev\3x\Lib\test\test_typing.py: 120:             "<class 'test.test_typing.AnyTests.test_repr.<locals>.Sub'>",
Hits found: 13
(Hint: right-click to open locations.)

    This caught the test_module line you just fixed. Some of these are in interactive output and I don't know if they will fail.

  7. Eclips4 commented on May 1, 2023

    @Eclips4
    Eclips4
    on May 1, 2023
    MemberAuthor

    Yeah, I saw these lines, and this should be discussed. Otherwise, I found some another scenarios with the same issue (easy to fix).

  8. Eclips4 commented on May 1, 2023

    @Eclips4
    Eclips4
    on May 1, 2023
    MemberAuthor

    However, I'll try to run file test_support on Windows and got interpreter crash. Can someone confirm that?
    ./python Lib/test/test_support.py
    Cannot confirm this on Linux.
    Please, use the current main branch if you try to reproduce it.

  9. Eclips4
    added a commit that references this issue on May 1, 2023

    pythongh-104057: Fix path

    b40f33c

  10. terryjreedy commented on May 1, 2023

    @terryjreedy
    terryjreedy
    on May 1, 2023
    Member

    main built yesterday

    F:\dev\3x>python -m test.test_support
Running Debug|x64 interpreter...
.....................s.F..

    Crash. Rebuild and try again. Same crash. "Debug assertion Failed."
    File minkernel/ctrs/ucrt/src/appcrt/lowio/write.cpp line 50, expression (_osfile(fh) & FOPEN)

  11. Eclips4 commented on May 1, 2023

    @Eclips4
    Eclips4
    on May 1, 2023
    MemberAuthor
    F:\dev\3x>python -m test.test_support
Running Debug|x64 interpreter...
.....................s.F..

    Crash. Rebuild and try again. Same crash. "Debug assertion Failed."
    File minkernel/ctrs/ucrt/src/appcrt/lowio/write.cpp line 50, expression (_osfile(fh) & FOPEN)

    Thanks!
    I think, it's a minimal reproducible example:

    import os


def make_bad_fd():
    file = open("test_file.txt", "wb")
    try:
        return file.fileno()
    finally:
        file.close()



fd = make_bad_fd()
os.write(fd, b"foo")

  12. terryjreedy commented on May 1, 2023

    @terryjreedy
    terryjreedy
    on May 1, 2023
    Member

    f:\dev\3x>python -m test -uall -v test_support runs fine.

    Running test_support from IDLE editor: get same Assertion Failure box. After Ignore, failure is

    .....................s.F.......s.........s...
======================================================================
FAIL: test_ignored_deprecations_are_silent (__main__.TestSupport.test_ignored_deprecations_are_silent)
Test support.ignore_deprecations_from() silences warnings
----------------------------------------------------------------------
Traceback (most recent call last):
  File "f:\dev\3x\Lib\test\test_support.py", line 52, in test_ignored_deprecations_are_silent
    self.assertEqual(len(messages), 0, messages)
AssertionError: 1 != 0 : ['You should NOT be seeing this.']

  13. Eclips4 commented on May 1, 2023

    @Eclips4
    Eclips4
    on May 1, 2023
    MemberAuthor

    f:\dev\3x>python -m test -uall -v test_support runs fine.

    Running test_support from IDLE editor: get same Assertion Failure box. After Ignore, failure is

    .....................s.F.......s.........s...
======================================================================
FAIL: test_ignored_deprecations_are_silent (__main__.TestSupport.test_ignored_deprecations_are_silent)
Test support.ignore_deprecations_from() silences warnings
----------------------------------------------------------------------
Traceback (most recent call last):
  File "f:\dev\3x\Lib\test\test_support.py", line 52, in test_ignored_deprecations_are_silent
    self.assertEqual(len(messages), 0, messages)
AssertionError: 1 != 0 : ['You should NOT be seeing this.']

    That's easy to fix. Just replace

            cls._test_support_token = support.ignore_deprecations_from(
            "test.test_support", like=".*You should NOT be seeing this.*"
        )

    to this:

            cls._test_support_token = support.ignore_deprecations_from(
            __main__, like=".*You should NOT be seeing this.*"
        )
  14. carljm
    added a commit that references this issue on May 1, 2023

    gh-104057: Fix direct invocation of test_super (#104064)

    605f878
  15. Eclips4
    added a commit that references this issue on May 1, 2023

    pythongh-104057: Fix path

    22c60b5

  16. 1 remaining item

  17. pablogsal
    added 2 commits that reference this issue on May 1, 2023

    pythongh-104057: Fix direct invocation of test_super (python#104064)

    ace23be

    pythongh-104057: Fix direct invocation of test_super (python#104064)

    cf9f75f

  18. chgnrdv commented on May 2, 2023

    @chgnrdv
    chgnrdv
    on May 2, 2023
    Contributor

    Same is for test_descrtut and test_extcall. test_peg_generator and test_importlib fail too, but for other reason. Actually, there are a lot of tests that fail when directly invoked. For example, all tests that import asyncio (or some other module that does it) fail with the same exception:

    $ grep -r "^import asyncio" Lib/test/test_*.py
Lib/test/test_builtin.py:import asyncio
Lib/test/test_contextlib_async.py:import asyncio
Lib/test/test_inspect.py:import asyncio
Lib/test/test_logging.py:import asyncio
Lib/test/test_os.py:import asyncio
Lib/test/test_sys_settrace.py:import asyncio
$ ./python Lib/test/test_pdb.py 
Traceback (most recent call last):
  File "/home/.../cpython/Lib/test/test_pdb.py", line 20, in <module>
    from unittest.mock import patch
  File "/home/.../cpython/Lib/unittest/mock.py", line 26, in <module>
    import asyncio
  File "/home/.../cpython/Lib/asyncio/__init__.py", line 8, in <module>
    from .base_events import *
  File "/home/.../cpython/Lib/asyncio/base_events.py", line 45, in <module>
    from . import staggered
  File "/home/.../cpython/Lib/asyncio/staggered.py", line 11, in <module>
    from . import tasks
  File "/home/.../cpython/Lib/asyncio/tasks.py", line 946, in <module>
    eager_task_factory = create_eager_task_factory(Task)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../cpython/Lib/asyncio/tasks.py", line 936, in create_eager_task_factory
    raise TypeError(
TypeError: Provided constructor does not support eager task execution

  19. Eclips4 commented on May 2, 2023

    @Eclips4
    Eclips4
    on May 2, 2023
    MemberAuthor

    For example, all tests that import asyncio (or some other module that does it) fail with the same exception:

    Please, read the #104070, I think you got the same problem

  20. chgnrdv commented on May 2, 2023

    @chgnrdv
    chgnrdv
    on May 2, 2023
    Contributor

    For example, all tests that import asyncio (or some other module that does it) fail with the same exception:

    Please, read the #104070, I think you got the same problem

    Yep, you're right. A lot of modules still fail for other reasons though.

  21. eryksun commented on May 2, 2023

    @eryksun
    eryksun
    on May 2, 2023
    Contributor

    main built yesterday

    F:\dev\3x>python -m test.test_support
Running Debug|x64 interpreter...
.....................s.F..

    Crash. Rebuild and try again. Same crash. "Debug assertion Failed." File minkernel/ctrs/ucrt/src/appcrt/lowio/write.cpp line 50, expression (_osfile(fh) & FOPEN)

    If you run a test directly, such as python_d -m test.test_support -vv -k test_make_bad_fd, then the reporting modes and reporting files for the debug C runtime's non-standard

    • assertions -- i.e. the macros _ASSERT, _ASSERTE, and _ASSERT_EXPR, or the macros _RPT0 and _RPTN used with the _CRT_ASSERT report type;
    • warnings -- i.e. the macros _RPT0 and _RPTN used with the _CRT_WARN report type; and
    • errors -- i.e. the macros _RPT0 and _RPTN used with the _CRT_ERROR report type --

    all remain set to their default values. The assertion and error report types default to showing a dialog box, while the warning report type defaults to outputting a message to an attached debugger, if any. For example:

    >>> from msvcrt import *
>>> CrtSetReportMode(CRT_ASSERT, CRTDBG_REPORT_MODE) == CRTDBG_MODE_WNDW
True
>>> CrtSetReportMode(CRT_ERROR, CRTDBG_REPORT_MODE) == CRTDBG_MODE_WNDW
True
>>> CrtSetReportMode(CRT_WARN, CRTDBG_REPORT_MODE) == CRTDBG_MODE_DEBUG
True

    On the other hand, if you run the main test suite normally, such as python_d -m test test_support -vv -m test_make_bad_fd, then _CrtSetReportMode() and _CrtSetReportFile() get called to configure the report mode for the _CRT_ASSERT, _CRT_WARN, and _CRT_ERROR report types. They either get configured to no reporting (i.e. mode 0) or, if verbose mode is enabled (i.e. -vv command-line option) to reporting to stderr (i.e. _CRTDBG_MODE_FILE mode with the file set to _CRTDBG_FILE_STDERR). Also, WinAPI SetErrorMode() is called with SEM_FAILCRITICALERRORS | SEM_NOALIGNMENTFAULTEXCEPT | SEM_NOGPFAULTERRORBOX | SEM_NOOPENFILEERRORBOX in order to suppress any message boxes due to system error reporting.

    One step that's missing is to explicitly configure the report mode for the standard assert macro in debug builds. This gets configured by the C runtime's _set_error_mode() function. For a console application (e.g. "python.exe"), the default behavior is to write to stderr (i.e. _OUT_TO_STDERR), which is what we want. On the other hand, for a GUI application (e.g. "pythonw.exe"), the default behavior is to show a message box (i.e. _OUT_TO_MSGBOX). If we add any tests that specifically use "pythonw.exe" instead of "python.exe", then we should explicitly configure _set_error_mode(_OUT_TO_STDERR), particularly in the SuppressCrashReport context manager, and ensure that the "pythonw.exe" process is spawned with a valid stderr file. Also, it appears that the msvcrt module wraps the set_error_mode() function, but it doesn't define the required constants OUT_TO_DEFAULT, OUT_TO_STDERR, OUT_TO_MSGBOX, and REPORT_ERRMODE. That oversight should be corrected.

  22. eryksun commented on May 2, 2023

    @eryksun
    eryksun
    on May 2, 2023
    Contributor

    You're running the tests incorrectly for how the debug build is supported on Windows. The debug CRT report modes are not set for each individual test, except in test cases that explicitly use the SuppressCrashReport() context manager. Generally you have to run the main test suite and supply the test you want to run as a command-line option, with an optional -m filter for test cases and -vv if you want the debug CRT assertion, warning and error messages written to stderr. For example: python_d -m test test_support -vv -m test_make_bad_fd.

  23. chgnrdv commented on May 2, 2023

    @chgnrdv
    chgnrdv
    on May 2, 2023
    Contributor

    @eryksun, are you replying to me? I use Linux, so I guess that none of errors that I mentioned above relate to how test output is handled by system, and most of them happen regardless of -m flag usage (but doesn't happen if libregrtest is involved, i.e. as in python -m test test_support example).

  24. carljm
    added a commit that references this issue on May 2, 2023

    Merge branch 'main' into cached_property_docs

    b9c39e3
  25. arhadthedev
    added
    testsTests in the Lib/test dir
    on May 3, 2023
  26. zware
    added a commit that references this issue on May 11, 2023

    gh-104057: Fix direct invocation of test_support (GH-104069)

    27419a7
  27. carljm
    added a commit that references this issue on May 11, 2023

    Merge branch 'main' into supermon

    fb47955

  28. hugovk commented on Nov 10, 2023

    @hugovk
    hugovk
    on Nov 10, 2023
    Member

    Thanks for the fixes! Anything left here, or okay to close?

  29. Eclips4 commented on Nov 10, 2023

    @Eclips4
    Eclips4
    on Nov 10, 2023
    MemberAuthor

    Thanks for the fixes! Anything left here, or okay to close?

    I think we did everything we had to do :)
    Thanks for the remind, Hugo!

  30. Eclips4
    closed this as completedon Nov 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions