Fail TestSuite without failing TestCase Follow
Answered
I have implemented the test runner using SMTestRunner.
I have a requirement to fail the TestSuite without failing any TestCases. Something like the following structure
--- test suite [FAIL]
|--- test case 1 [PASS]
|--- test case 2 [PASS]
Using the following code
ServiceMessageBuilder.testSuiteStarted("test suite");
ServiceMessageBuilder.testStarted("test case 1");
ServiceMessageBuilder.testFinished("test case 1");
ServiceMessageBuilder.testStarted("test case 2");
ServiceMessageBuilder.testFinished("test case 2");
ServiceMessageBuilder.testSuiteFailed("test suite"); // Not Available
ServiceMessageBuilder.testFailed("test suite"); // Works, but increases the failed tests count
ServiceMessageBuilder.testSuiteFinished("test suite");
Let me know If there is any other way to achieve this. Any help will be appreciated.
Please sign in to leave a comment.
Hi,
What does it mean that suite fails but the failed tests count is 0?
Anna
We have Before/After Suite hooks. AfterSuite runs after all the testcases have been executed for cleanup. So, there can be a case when AfterSuite failed and all the testcases passed. I need to show error/stacktrace at suite level but the testcases count should show 0 testcases failed.
For junit in such situation I add special node for failed before/after suite method and this node is marked as failed. Never saw any problems with this
If the following is what you are suggesting then it will make the failed count 1 which is not correct because all the test cases passed.
Yes, this looks good to me. In your case there would be a failure when no actual failures are present in the tree, only in the output which sounds strange to me. IDEA doesn't support such cases, sorry
Thanks Anna for the information. Is there any way to hide the test counts?
No, I don't think so. com.intellij.execution.testframework.ui.TestStatusLine - the UI component which represent the status messages on top/right - you can remove it completely from UI (no api provided!) or have it as is.