Repository navigation
Add a function to detect if pytest is running #9502
Description
Activity
We currently just document how to do this, probably in a way similarly to what your plugin does.
Note you could also just check
if "pytest" in sys.modules:as long as your code under test does not import pytest.We do have a
PYTEST_CURRENT_TESTenvironment variable already. Personally I'd be fine with having aPYTEST_VERSIONor whatever which gets set very early, I supposePYTEST_CURRENT_TESTwill be set too late for it to be useful at import time.Reacted by Andrew Svetlov, Will Da Silva, Christian Decker and Miroslav MatějůAlso see pytest-dev/pytest-django#333 which has some additional considerations and use-cases around this.
Reacted by Andrew SvetlovPersonally I'd be fine with having a PYTEST_VERSION or whatever which gets set very early,
I'm OK with this solution. It would also remove that env var from
os.environat the end ofpytest.main(), I assume?A while ago I had an idle thought that something like this should be a python feature. I was thinking a builtin constant
__test__, similar to__debug__, that would normally be false, but that a test runner would set to true while discovering and running tests.In the Rust language for example it is very common to interleave unit tests with the code they're testing in the same file using an attribute. In Python this would be:
def fibonacci(n): ... if __test__: import pytest def test_fibonacci(n): assert fibonacci(10) == 55 with pytest.raises(ValueError): fibonacci(-1) def factorial(n): ... if __test__: def test_factorial(n): assert factorial(50) == 30414093201713378043612608166064768844377641568960512000000000000
Not for everyone, but I think there are cases I would have used this.
after the drop of python 3.6 we can use https://docs.python.org/3/library/contextvars.html
i would propose to use this for gathering whether pytest is running
i woudl strictly oppose exposing sometihng like config as a contextvar however unless we had a stack for nested pytest session
We currently just document how to do this, probably in a way similarly to what your plugin does.
Yes, I noted in the description why this doesn't work for the case of the django settings file 😊
I like the idea of the
PYTEST_VERSION(or whatever) environment variable. This is general and wouldn't requireimport pytestin production code. Yes it should be popped out ofos.environafter pytest completes its run.A
__test__builtin would be interesting, but it wouldn't be so easy to add. If it was only set by pytest, it wouldn't exist without importing pytest, unless it were actually added to Python, and placed in a backports package.contextvars seem like they'd also require
import pytestto use, since the var must be defined in a module—unless pytest shipped a side module to contain the var?One can use
sys.modules.getin such helpersPersonally im -1 on adding easy and obvious fast paths to make it easy to keep messy global configuration around
This stems from stuff going horrendously wrong every time a project uses that and then suddenly some tests need to run with different configurations or multiple configurations
One can use
sys.modules.getin such helpers🤔 Idk why I didn't think of that, thanks for the hint.
I think for now I will try
"pytest" in sys.modulesinstead of my plugin.Agree that it can go wrong as complexity creeps in.
I stopped my overengineering when I arrived at:
def we_are_running_via_pytest() -> bool: """Detect if we are running via pytest.""" # When we are running via pytest it is loaded to the sys modules dictionary. _we_are_running_via_pytest: bool = 'pytest' in sys.modules return _we_are_running_via_pytest
Source: https://docs.python.org/3/library/sys.html#sys.modules
Use it like so:
def get_configuration() -> Configuration: configuration: Configuration if we_are_running_via_pytest(): print('Creating test configuration.') configuration = _get_test_configuration() else: print('Creating runtime configuration.') configuration = _get_runtime_configuration() return configuration
Reacted by Bruno OliveiraThanks everyone!
I'm closing this for now, with the recommended solution being:
is_running_pytest = "pytest" in sys.modules
If somebody wants to contribute that to the documentation, it would be great!
But I think "pytest" in sys.modules returning True if theres a pytest import makes it kinda error prone
This might be over engineered but I think this works better and helps to avoid one of the edge case with
"pytest" in sys.modulesimport os import inspect import sys # List of pytest related files pytest_related_files = ( os.path.join('_pytest', 'python_path.py'), os.path.join('_pytest', 'logging.py'), os.path.join('_pytest', 'main.py'), os.path.join('_pytest', 'runner.py'), os.path.join('_pytest', 'config', '__init__.py'), os.path.join('_pytest', 'debugging.py') ) # Check if any of the pytest related files are in the current stack is_pytest_in_stack = any(frame.filename.endswith(pytest_related_files) for frame in inspect.stack()) # Check if a pytest is currently running is_pytest_running = os.environ.get("PYTEST_CURRENT_TEST") # Check if pytest module is imported # We probably dont need to use this as, if one of the other two condition is True, this should be True is_pytest_imported = "pytest" in sys.modules if is_pytest_imported and (is_pytest_in_stack or is_pytest_running): print(True) else: print(False)Do let me know if theres a case where this will not work
too many magic constants
inspect.stack()is also relatively slow as it makes Python build all the frame objects. Checking forPYTEST_CURRENT_TESTis a good idea if you want something a bit more robust than than thesys.modulescheck. But IMO thesys.modulescheck is good enough for most projects, it should be exceedingly rare to import pytest outside of the test process.I archived my pytest-is-running project btw.
What do you think about this
https://github.com/dheerajck/pytest/tree/devThis has been discussed above
contexvar will not help as mentioned in this comment
Also I think this will work with
_pytest/main.pyandconfig/__init__.py, I am just trying to cover the edge cases if theres any by adding othersI dont think it inspect.stack being slow is an issue in this case unless it is noticeably slow , because we should only use it in places like django settings(which is only run once per server start, in every other cases(like when a ut is running) we can use
os.environ.get("PYTEST_CURRENT_TEST")which works greatI am using
IS_PYTEST = sys.argv[0].endswith('pytest') or os.environ.get('PYTEST_XDIST_WORKER_COUNT')which works withpytestandpytest -n xI am aware of cases where this doesnt work like
python -m pytestbut this seems better than"pytest" in sys.modulesPYTEST_VERSIONenvironment variable is now defined when executing pytest (#12190). It will be available in the next minor release.Thanks @dheerajck for the PR. 👍
Reacted by dheeraj, Zach Newburgh, Serhii Tereshchenko, Gabriel LePoudre, Andrej730, Václav Dohnal and Aarni KoskelaReacted by Zach Newburgh- added a commit that references this issue
on Apr 18, 2024 Thanks for reviewing my code and letting me know about some good coding practices :)
Reacted by Bruno Oliveira- added a commit that references this issue
on Apr 28, 2024
What's the problem this feature will solve?
Django settings modules often load environment variables from a
.envfile:Loading from
.envis not desirable during test runs. Environment variables added for local debugging may affect the test run in unexpected ways, such as making remote API calls.The settings module should therefore only load the
.envfile if pytest isn't running:Changing behaviour during tests is usually undesirable, but here it's done to isolate the test run, which is a good thing.
pytest documents a pattern to change behaviour using fixtures here: https://docs.pytest.org/en/latest/example/simple.html#detect-if-running-from-within-a-pytest-run . Unfortunately this techinque is incompatible with this particular problem. pytest-django's fixtures load the settings module early, before fixtures from
conftest.pyfixtures run, so the pattern cannot be applied. And environment variable changes cannot be undone after the fact.To solve this I made pytest-is-running, which can be used like so:
pytest-is-running works because plugin fixtures can load early, and it uses hooks with
tryfirst=True.Describe the solution you'd like
Rather than have a documented workaround and plugin, I'd like to see a function in core pytest that provides the ability to check if pytest is running. It could simplify things for those using the both the documented pattern and the plugin.
It would also be good if the solution could somehow avoid the cost of doing
import pytest, so that non-test pathways do not import the heap of stuff in pytest only to ignore it because pytest isn't running. The plugin is deliberately structured to avoid this.Alternative Solutions
I wrote the plugin.
Additional context
none