Any plans to support coverage-by-test in third-party plugins?
Quite a bit back I implemented code coverage support in my custom language plugin. At the time I received fantastic support from Anna Kozlova to assist with the implementations of both unit testing and coverage. However, at the time there was one key feature that was still Java-only and not exposed/supported via the plugin SDK, coverage-by-test. Here's the relevant portion of the original discussion thread:
https://intellij-support.jetbrains.com/hc/en-us/community/posts/206103879/comments/206086045
Is this still an internal-/Java-only feature, or is it now possible to add coverage-by-test support in third-party plugins? If it's not, are there any plans to add support for it to the plugin SDK? As I mention in the linked post, this level of fine-grained information is available in the coverage report I receive, so it'd be great if I could convey that to the user in a first-class manner via the coverage feature.
Thanks much in advance,
Scott
Please sign in to leave a comment.
Hi Scott,
do you request to change the api so the set `tests` in `com.intellij.coverage.actions.ShowCoveringTestsAction#actionPerformed` would be filled according to the runner used? Or do you mean something more complicated?
The small change to remove java specific code from common module looks doable and should not harm anyway.
Anna
Hi, Anna! Thanks for following up. Well, to be honest I'm having to dust off my brain cells a bit to put myself back where I was in July 2015 (the linked thread), but if memory does serve, when I tried to set up my coverage implementation to support coverage-by-test by overriding isCoverageByTestEnabled() and isCoverageByTestApplicable() to return true, things didn't work properly because the underlying implementation had some hard-coded, Java-specific behavior still. Your response at the time was:
"actually tests per line was implemented for java only and there is no real api for that yet."
To be fair, I haven't tried again to see if perhaps this has changed in the intervening years. Instead I posted here asking if this was in fact an expected/supported way for third-party plugins with coverage functionality.
In short, though, what I'd love to be able to do is tell the coverage framework that my coverage implementation does support coverage-by-test, then populate the coverage information data with those details so that the IDE displays it properly to the end user like it does with IntelliJ IDEA's internal coverage engine. The last time I tried to do that, it looked like that part wasn't ready for external use, though.
I'm not sure if that answers your specific question. Please let me know if it doesn't.
Scott
Hi Scott,
I hope that the changes I made today would help. Along with already existing method `com.intellij.coverage.CoverageEngine#canHavePerTestCoverage`, I've added `com.intellij.coverage.CoverageEngine#getTestsForLine` and `com.intellij.coverage.CoverageEngine#wasTestDataCollected` which should allow you to propagate data from your engine to IDEA's UI.
Please notify me if this is not enough.
Anna
Thanks, Anna! Two questions: 1) what's the best way for me to get my hands on a build with these changes early so I can give you feedback; 2) when might these be in an official build?
Scott, commit is on github; it would be included in the 2019.1 EAP2, should be released next week
Anna
Thanks, Anna. I'll let you know how it goes as soon as I get a chance to try it out.