Is IntelliJ IDEA code coverage runner available as command line?
Hi there,
I have an interesting observation in favour of IntelliJ IDEA's code coverage.
Context:
I have a Groovy project where I load dynamically other scripts.
The thing is that I have a library of these scripts and have covered them with Unit tests.
Due to legacy reasons (and ability to add new script) these scripts are not part of another more conventional loading / configuraiton mechanism, but are loaded dynamically. That way the Code Coverage plugin I've tried (JaCoCo) doesn't respect them and displays 0% coverage out of thousands of lines... which is expected..
But when I run them inside the IDEA using IDEA's integrated code coverage runner, it marks each line green/red and how many times it has been executed. When I export the report I see that even the scripts folder is covered... Which is great!
Questions:
- How does it work and JaCoCo doesn't? Does it take into account the source code together with the bytecode?
- Is the IDEA code coverage runner available as a command line tool? I want to be able to build a report with it using my Gradle builds scripts.
- If yes, how can upload such a report to SonarQube for historical analysis? Is it supported ? How can I accomplish this?
I didn't find answers to these questions nowhere, so that's why I'm asking here.
Thank you in advance.
Best regards,
Leni Kirilov
Please sign in to leave a comment.
Hi Leni,
IDEA's coverage is a java agent which instruments the code loaded to the VM, JaCoCo performs more or less the same, so to me it's strange that one works and another - doesn't. Where do you browse the JaCoCo results? There is a number of problems how IDEA displays JaCoCo results and may be that's the problem.
You may attach coverage-agent.jar as -javaagent to your java process and the coverage would be collected to the file, which is passed as argument. (The argument string can be found in the console when you run your tests with coverage.)
IDEA allows to export results in html but I think that SonarQube works with some internal format and I don't expect it to support our internal format
Anna
Hi, Anna
Thanks for the quick reply.
So both ways of instrumenting my code works, but shows different results and JaCoCo doesn't take into account the files which are compiled dynamically.
When I run the results inside IDEA both of them show low % (skipping the package with the dynamically loaded scripts)
The difference is with IDEA editor and exported results:
- when I open one of the so-called 0% covered scripts, all their lines are marked GREEN with exact numbers of times covered
- when I export the coverage results to a directory, it displays 90-95% coverage of the scripts as well (matching the green lines)
The JaCoCo one didn't mark the lines in GREEN nor it included them in the exported report.
Does this kind of explanation make it clearer?
Regarding your proposal:
- where can I find this coverage-agent.jar ? Is it part of the IDEA installation (I'm using 2016.2 on a Mac)
Anyway if you don't plan to integrate with SonarQube (which is my ultimate goal) , I think this would community post should be transformed to a Feature Request (unless something like this exists already
Thanks and have a great day!
Leni
Hi Leni,
Did you try to run JaCoCo in SonarQube?
http://docs.sonarqube.org/display/PLUG/Code+Coverage+by+Unit+Tests+for+Java+Project looks like they support only the known formats and I don't think that it's possible to save IDEA's coverage to one of the supported formats.
Anna
Hi Leni,
The coverage agent can be built from source (using Ant) using the project https://github.com/JetBrains/intellij-coverage but right now IntelliJ is probably the only program around which knows how to read its coverage files.
I have been working on a tool which will convert from intellij-coverage files to Cobertura XML files, as supported by Jenkins and Sonar, although I'm a bit stalled until I can Mavenise the build (not wanting to fork the project): https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000068130-Status-of-intellij-coverage-github-project
Sean.