JUnit tests runner with milti-JVM support


This is most probably a feature request, but even if it is, I believe a bit of discussion is in order. :)
We use Idea and Teamcity, and we exploit their respective JUnit runner features to run CI tests. This works well, because we combine our own custom JUnit-bases harness with usability of bundled JUnit support: tests reports and responsibility engine on Teamcity, Idea tests GUI, remote run etc. The only complaint we have is that all tests are being run sequentially, so tests running time is long, while CPU utilization is fairly low. Yes, it is not a concern if all your tests can be executed in less than a second, but, sadly, that is not the case with us. And I failed to find a way of fixing this in our harness: org.junit.runner.Runner is in control of test methods and of how to invoke them, but class enumeration is a province of Idea JUnit tests runner. I was able to actually run test methods of a single test class in parallel (and, as far as I remember, JUnit also have some facility for this, though it failed to suit me), but this solution in not so good: all tests boil together in a single JVM cauldron, and unexpected side effects can be numerous, especially if this was not the case while tests were developed in the first place.

And here is a problem: surely, I can make multi-JVM runner of my own (or find someone else's) and call JUnut core directly, but it does not seem right, and I'm going to lose all Idea and Teamcity native support. So here is what I want, put as simply as possible: to set up JUnit runner so that is would split test classes list into N chunks, run each of those in separate JVM, collect all outputs and merge them in single report. Fancy Idea GUI support is an optional extra: if this feature would only work on Teamcity, it is fine for me. And, yet again, if the splitting mechanism could be customized, it would be fine, but pretty mush any naive way of doing so would still be much better than nothing.

Please sign in to leave a comment.