Python SubTest
Is there any way to get PyCharm to understand subtest?
https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests
It's a great new functionality that is imo akin to test cases in other languages like C#. It appears PyCharm is unable to parse the output into distinct test cases, and rather flags the whole test failed on any failures.
I also have a suspicion that PyCharm seems to fail on the first failed subtest instead of letting it run through, but I'd have to confirm it first.
Please sign in to leave a comment.
+1000 to detecting the subtest and printing its message and kwargs as documented here: https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests
Not failing on the first subtest is less critical to me, but pycharm should definitely implement as its documented.
Hi,
PyCharm does detect subtests. Consider the example from https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests
Here's the output:
Is it different in your case? Or do you mean something else?
Hi Andrey! Sorry for the slow response, I lapse on checking these forums.
In my case I just see the first failure with no outline seen above. The test runner seems to be the difference: I am using Nosetests. I can get the above output to appear correctly only when using Unittests as a runner, but not pytest or Nosetests.
So on further testing I found that I can use the Unittests runner just fine, but I have to setup everything manually and PyCharm's auto-discovery does not pick up my tests. Probably why I went to nosetests originally because PyCharm sees the tests in that case, but not when Unittests is the default runner. What's the possible fix here?
It seems I didn't ask you what version you were using. Please try our new release 2019.1: https://www.jetbrains.com/pycharm/download
And let me know if the issue is still reproduced there.
I was on 2018.3 before. Just retested with the above 2019.1 point, and it's still true: Nosetests does not appear to generate the subTest output. Unittests still does, however PyCharm can't discover my tests when Unittests is the default for some reason. Manually creating a Unittests config for the particular file works fine.
Sorry for not responding earlier. Is the issue still relevant?
Can you provide a minimal project to reproduce it on our side?
Meant to update this earlier. The original request is no longer valid.
I figured out that the issue seems to be in our abstraction layer for unittest. Specifically, the way classes inherit from the abstraction is via a wrapper that does some dynamic stuff to wrap the Base class (deriving from unittest), thus making the inheritance opaque to any inspections. I was able to resolve it by making classes in question inherit from the Base directly (in addition to the wrapper). At that point PyCharm's static analysis was able to pick up the tests. The Unittests runner does not have problems with subTests.
Here are screens showing how are tests and subtests displayed in: PyCharm 2019.2.3
Notice that "Click to see difference" is super useful, but is completely missing in subtests.
I'm using Pycharm 2019.3.3 and was having issues where subtests were not showing up in test runner window. What I found was that at some point I changed my default runner to pytest. If your test runner is pytest it doesn't seem to display the nested subtests. If I change my test runner back to 'Unittests' it will properly display the nested test runners.
Would be nice if it could do this for pytest too because I use pytest for coverage but I'm using a feature of Unittests so I can see why it doesn't work.
I have similar issue using PyCharm 2021.3.1.
Moreover, if I set default test runner Unittests (Preferences->Tools->Python_Integrated_Tools->Testing->Default_test_runner->Unittests), the test runner is still _jb_pytest_runner.py
The only working workaround solutions are:
1. Uninstall pytest (which breaks my pytest specific testcases)
2. Using terminal with "python -m unittest module_name", which is lack of PyCharm advantages.
As Gmetzker pointed out, the issue dissappears if you switch to Unnittests runner:
Also, delete all existing run configurations (they might have been created before and use Pytest runner):