Code coverage for custom language Follow
The system for my custom language plugin provides its own unit testing framework and code coverage metrics. I'd like to integrate both of these into IDEA in a first-class manner. The unit testing aspect looks relatively straightforward (though I'm just starting the implementation now) through the runConfigurationProducer, configurationType, testFramework, testFinder, and testCreator extension points. Please let me know if I've missed any other critical EPs for this task!
I was hoping the same would be true for code coverage using the coverageEngine, coverageOptions, and coverageRunner EPs, but when I try to implement those I get "Cannot resolve symbol 'CoverageEngine/CoverageOptions/CoverageRunner'". Is this not a supported part of OpenAPI? Is there some other way to implement code coverage for a custom language? I found this old post about someone wanting to do similar and it sounds like there was going to be some kind of out-of-band conversation about the topic:
https://devnet.jetbrains.com/message/5286877
As always, thanks for any pointers!
Please sign in to leave a comment.
Once again I may have spoken too soon. I just added the jars from the coverage plugin to my plugin SDK and now I have access to those interfaces. I'll see if I can get things so they build properly for my plugin itself, but hopefully this is going in the right direction...
Hi Scott,
I'd be very interested in hearing how you get on with both of these - I'm planning to implement them soon. In case you haven't seen it, a good starting point for implementing the testing UI is here: https://devnet.jetbrains.com/message/5537291#5537291
I investigated implementing coverage for the language I'm working on (Clojure) which looked easy initially since it's JVM based, but turned out to be trickier than I'd hoped. I can't remember the details, but IIRC it was to do with the fact that the JVM coverage interfaces were class-based, and Clojure is not. I didn't investigate long enough to find a workaround, though.
Cheers,
Colin
Hi, Colin. I've started to implement the unit test integration today and am working on the test run configuration UI right now so I can capture the parameters for unit test execution. I'll let you know how that goes over the next few days.
Once I have integrated well, I'll look into integrating the code coverage info. So far it does seem that I can declare a dependency on the coverage plugin in IDEA 14 CE but not previous versions of IDEA CE (my plugin supports IDEA 12 and 13 as well). I imagine it should work with IDEA 12 and 13 UE, though. We'll see if that ends up being the case as I dig into it deeper. Again, I'll post my findings here. Hopefully I can get it all stitched together nicely!
Yes, my understanding is that the interfaces didn't change, but that it's only open source from v14 on. Hopefully the code should be sufficiently backward compatible, though.
Thanks for the info!
Colin, I've successfully implemented and released unit test integration for my custom language plugin:
http://www.illuminatedcloud.com/home/unittest
Moving on to code coverage now. Still trying to figure out the relevance of each piece and how it maps to the custom language's existing code coverage feature!
Let me know if you have any questions. I can't guarantee that I have perfect answers, but I've gotten it working successfully in IDEA 12, 13, and 14!
That's excellent, congrats Scott! Thanks for documenting all your progress, too. I'm wrapping up my current release hopefully this week (although I have a couple of days off in there), and I'll be on to the unit testing stuff after that. I'll let you know how I get on!