1. IDEs Support (IntelliJ Platform) | JetBrains
  2. Community
  3. IntelliJ IDEA Open API and Plugin Development

Fail TestSuite without failing TestCase

Answered
Avatar
Kashish Munjal
Created

I have implemented the test runner using SMTestRunner. 

I have a requirement to fail the TestSuite without failing any TestCases. Something like the following structure

--- test suite [FAIL]
        |--- test case 1 [PASS]
        |--- test case 2 [PASS]

Using the following code

ServiceMessageBuilder.testSuiteStarted("test suite");
ServiceMessageBuilder.testStarted("test case 1");
ServiceMessageBuilder.testFinished("test case 1");
ServiceMessageBuilder.testStarted("test case 2");
ServiceMessageBuilder.testFinished("test case 2");
ServiceMessageBuilder.testSuiteFailed("test suite"); // Not Available
ServiceMessageBuilder.testFailed("test suite"); // Works, but increases the failed tests count
ServiceMessageBuilder.testSuiteFinished("test suite");

Let me know If there is any other way to achieve this. Any help will be appreciated.

7 comments
Avatar
Anna Kozlova
Created
Comment actions Permalink

Hi,

What does it mean that suite fails but the failed tests count is 0? 

Anna

0
Avatar
Kashish Munjal
Created
Comment actions Permalink

We have Before/After Suite hooks. AfterSuite runs after all the testcases have been executed for cleanup. So, there can be a case when AfterSuite failed and all the testcases passed. I need to show error/stacktrace at suite level but the testcases count should show 0 testcases failed.

0
Avatar
Anna Kozlova
Created
Comment actions Permalink

For junit in such situation I add special node for failed before/after suite method and this node is marked as failed. Never saw any problems with this

0
Avatar
Kashish Munjal
Created
Comment actions Permalink

If the following is what you are suggesting then it will make the failed count 1 which is not correct because all the test cases passed.

--- test suite [FAIL]
        |--- test case 1 [PASS]
        |--- test case 2 [PASS]
        |--- After hook  [FAIL]
0
Avatar
Anna Kozlova
Created
Comment actions Permalink

Yes, this looks good to me. In your case there would be a failure when no actual failures are present in the tree, only in the output which sounds strange to me. IDEA doesn't support such cases, sorry

0
Avatar
Kashish Munjal
Created
Comment actions Permalink

Thanks Anna for the information. Is there any way to hide the test counts?

0
Avatar
Anna Kozlova
Created
Comment actions Permalink

No, I don't think so. com.intellij.execution.testframework.ui.TestStatusLine - the UI component which represent the status messages on top/right - you can remove it completely from UI (no api provided!) or have it as is. 

0

Please sign in to leave a comment.