Reporting test results from a testing plugin

Hi,

I'm writing a plugin that executes Javascript tests. My RunnableState is a JavaCommandLineState, and I execute my test runner as a Java main(), attaching it to SMTestRunnerConnectionUtil like other test runners do.

The test runner needs to report the results back so the UI can be updated. I followed an example from the ScalaTest integration in the Scala plugin, which just prints test results to stdout using the TeamCity format. However, the documentation for that
http://www.jetbrains.net/confluence/display/TCD4/Build+Script+Interaction+with+TeamCity
says that all tests must have their results printed between the start and end of their containing test suite.

In my case, I want a top-level test suite representing the browser where tests are being run, and I'm executing tests in parallel across browsers, so I want test results to be reported for the second browser while the first is still running tests, and for the UI to update in both subtrees. This means I can't use the TeamCity format and the simple approach of printing test results to stdout, it seems.

So now I'm looking at the bundled JUnit plugin, which uses a com.intellij.execution.testframework.AbstractTestProxy and related classes in the testframework package, and seems to update the test results tree. But, I can't figure out how the separate Java process started by the JavaCommandLineState talks back to the IDE process. Also, the JUnit plugin is hard to reverse-engineer, and may be total overkill for what I want to do.

Can you please advise?
- Are these the correct options for reporting test results? (Assuming I don't want to write my own UI like the TestNG plugin does)
- Is there a way to use the TeamCity format when suites are running concurrently?
- What API should I use to add test results and suites to the tree?
- How does the test runner process talk back to the IDE process if not via stdout?

Thanks!
-Alex

Please sign in to leave a comment.