Code Coverage: Excellent work

needs some interaction polishing, and some concept of a rollup, but otherwise it's looking very sweet.

Final quibbles:

1)The toolbar dropdown should toggle editor coverage highlighting. The "Analyze/Show coverage info" menu item serves no valuable purpose.

2)The project menu annotation needs to be done for packages/modules/source roots, not just classes. That would give me all the I need for rollups.

3)It's tougher than it should be to indicate via pattern which classes should be covered, since you can't select a package from the class browser.

Again, excellent work.


--Dave Griffith

23 comments
Comment actions Permalink

Thank you, Dave.

The first issue is next in the list. Coverage is already shown for packages in package view, it won't be difficult to extend this for separate source roots as well.

0
Comment actions Permalink

Is the API for this exposed yet? It shouldn't be tied to junit, it should be possible to turn it on for any run configuration in fact.

0
Comment actions Permalink

Yes, I'll open the API when I finish the implementation for junit.

0
Comment actions Permalink

+1

+1000 for also opening ModuleBasedRunConfiguration (or whatever its called, the exact class name escapes me off hand) at the same time ;)

0
Comment actions Permalink

Can I get a hint as to how to run/enable this? I've got a web module, with
some source files and some test classes. But I can't find any way to view
the code coverage from my tests?

Thanks,
Tobin


0
Comment actions Permalink

1)Create a JUnit run configuration for the tests in that module
2)Edit the configuration. There's a "Code Coverage" tab, with a toggle to enable coverage and a place to specify the classes you want covered.
3)Run your tests
4)Select your test runs name in the new dropdown menu in the toolbar. Code coverage numbers for all covered classes will show up in your project or package view.

There's also a way of getting left gutter highlights for covered and uncovered code, but I'm not going to describe it because it's 1) incredibly tedious and 2) going to change soon.

--Dave Griffith

0
Comment actions Permalink

1)Create a JUnit run configuration for the tests in that module

2)Edit the configuration. There's a "Code Coverage" tab, with a
toggle to enable coverage and a place to specify the classes you want
covered.


I had found this Code Coverage tab, but the "Enable code coverage" checkbox
is disabled, so I can't turn it on. Any reason why this checkbox would be
disabled?

Tobin


0
Comment actions Permalink

Why is this functionality bound to JUnit incidentally, instead of all run/debug configurations? I bet there are way more test cases using main() out in the wild than ones that use JUnit, and it'd be a great way to introduce the code coverage concept without forcing formal tests.

0
Comment actions Permalink

add emma.jar from IDEA_HOME/lib and open/save the run config a few times.. seems to be buggy

0
Comment actions Permalink

It is disabled since we support gathering coverage information with jre 1.5 only.

0
Comment actions Permalink

Ahh, I'm using 1.4. Thanks.

Tobin


0
Comment actions Permalink

If time permits we will implement coverage support for 1.4 jres. But these will come with usual custom class loader limitations as usual.

0
Comment actions Permalink

I've just tried out code coverage for the first time in build 5261 and I'm not seeing any numbers appear in the project/package view. Has the implementation changed in this build, or am I doing something wrong?

I do appear to be getting the left-hand gutter marks for the tested code, although they also appear in the JUnit test itself, which I'm assuming is an error? I wasn't planning on writing another JUnit test to test the JUnit test...

0
Comment actions Permalink

Not getting any information for production code might be a result of filtered packages misconfiguration. By default the package where test itself resides is chosen, but in case you have a different test/production code package structure you have to add those packages yourself.

As for seeing coverage for test themselves, there could be usecases where this is needed. For example we could have a "non trivial control flow in unit test" inspection, that could have a "simplify" quickfix making a choice based on coverage information. Until we do have a quickfix why not allow user to see this information and correct manually?:)

0
Comment actions Permalink

It's not that I don't have any information for Production code, as the left-hand gutter marks are being placed correctly in the Production as well as the test code. I'm just not getting any information in the Project/Package section, which Dave seems to suggest should be visible.

On another point, I changed the scope of the code coverage from package to specific classes, but it "remembers" the code coverage left-hand gutter marks from previous runs. They only disappear from the non-covered classes if you manually delete the .es files from the coverage directory - is this also expected?

I can see your point on the test classes, although it would be great if there was an option to turn this functionality off, especially if data would also be placed in the Project/Package panel for test classes.

Otherwise, I was really impressed with the new functionality, and I'll been showing it to people round the office today ;)

0
Comment actions Permalink

Good points, and the new setup in the 5261 build is better. I'd like to re-iterate, or second the motion for, some of the requests I've seen for this feature.

1. After running coverage again the display should be updated automatically. Right now I have to go to the drop-down and select the coverage data again to refresh.

2. We need a way inside the ide to clear coverage data. Deleting the files outside the ide isn't convienent, and doesn't always remove the coverage.

3. Colors for the gutter info need to be customizable. On my laptop the default colors are nearly invisible.

4. We need a shortcut available for selecting coverage, usual request to be able to do things without a mouse.

0
Comment actions Permalink

It worked once for me but then keeps failing now. When I open a file that's
supposed to have been covered, here's what I get

No message
java.lang.NullPointerException
at
com.intellij.coverage.CoverageSuite.getFilteredPackageNames(CoverageSuite.java:67)
at com.intellij.coverage.CoverageDataManager.b(CoverageDataManager.java:96)
at
com.intellij.coverage.CoverageDataManager.a(CoverageDataManager.java:105)
at
com.intellij.coverage.CoverageDataManager.access$500(CoverageDataManager.java:7)
at
com.intellij.coverage.CoverageDataManager$2$1.run(CoverageDataManager.java:2)
at
com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:0)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:598)
at
com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:79)
at com.intellij.ide.IdeEventQueue.a(IdeEventQueue.java:50)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:53)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:121)

(I'm having all kinds of issues with the error reporting tool, so back to a
good ole' copypaste)

I also tried killing the coverage file, with no success.

At the end of my tests, I get this line in the console:
raw session data merged into
{in 147 ms}
It's missing the first character of every path element. The file itself gets
created in the expected location though.


Vince.


0
Comment actions Permalink

Great!

Did you also see/fix the broken "session data merged" message I'm getting at
the end of a run?

Thanks :)

Vince.


0
Comment actions Permalink

Yep, merge should not happen, previous data should be removed upon next run.
I'm going to address this right now.

0
Comment actions Permalink

Cool, thanks.

Vince.


0
Comment actions Permalink

I have a problem with Coverage Suites. I use TeamCity that provides code coverage results as artifacts. But if I use it to show coverage of my classes nothing happens. Only error record appears in the idea.log:

java.io.IOException: file is corrupt or was not created by EMMA
at com.vladium.emma.data.DataFactory.mergeload(DataFactory.java:355)
at com.vladium.emma.data.DataFactory.load(DataFactory.java:56)
at jetbrains.buildServer.coverage.CoverageDataFileProvider.downloadCoverageData(CoverageDataFileProvider.java:123)
at jetbrains.buildServer.coverage.CoverageDataFileProvider$1.run(CoverageDataFileProvider.java:102)
at jetbrains.buildServer.TeamCityProcessManager$1.run(TeamCityProcessManager.java:62)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.runProcess(ProgressManagerImpl.java:38)
at com.intellij.openapi.application.impl.ApplicationImpl$1MyThread.run(ApplicationImpl.java:12)


When I have visited this folder I found that these files are empty. So IDEA can not download them and doesn't show error.

The second problem is if some suit doesn't exist it is shown in suits list and silently do nothing if applied.

0

Please sign in to leave a comment.