Reporting test results from a testing plugin


I'm writing a plugin that executes Javascript tests. My RunnableState is a JavaCommandLineState, and I execute my test runner as a Java main(), attaching it to SMTestRunnerConnectionUtil like other test runners do.

The test runner needs to report the results back so the UI can be updated. I followed an example from the ScalaTest integration in the Scala plugin, which just prints test results to stdout using the TeamCity format. However, the documentation for that
says that all tests must have their results printed between the start and end of their containing test suite.

In my case, I want a top-level test suite representing the browser where tests are being run, and I'm executing tests in parallel across browsers, so I want test results to be reported for the second browser while the first is still running tests, and for the UI to update in both subtrees. This means I can't use the TeamCity format and the simple approach of printing test results to stdout, it seems.

So now I'm looking at the bundled JUnit plugin, which uses a com.intellij.execution.testframework.AbstractTestProxy and related classes in the testframework package, and seems to update the test results tree. But, I can't figure out how the separate Java process started by the JavaCommandLineState talks back to the IDE process. Also, the JUnit plugin is hard to reverse-engineer, and may be total overkill for what I want to do.

Can you please advise?
- Are these the correct options for reporting test results? (Assuming I don't want to write my own UI like the TestNG plugin does)
- Is there a way to use the TeamCity format when suites are running concurrently?
- What API should I use to add test results and suites to the tree?
- How does the test runner process talk back to the IDE process if not via stdout?


Please sign in to leave a comment.