Code Coverage: Excellent work

Thank you, Dave.

The first issue is next in the list. Coverage is already shown for packages in package view, it won't be difficult to extend this for separate source roots as well.

Is the API for this exposed yet? It shouldn't be tied to junit, it should be possible to turn it on for any run configuration in fact.

Yes, I'll open the API when I finish the implementation for junit.

+1

+1000 for also opening ModuleBasedRunConfiguration (or whatever its called, the exact class name escapes me off hand) at the same time ;)

Can I get a hint as to how to run/enable this? I've got a web module, with
some source files and some test classes. But I can't find any way to view
the code coverage from my tests?

Thanks,
Tobin

1)Create a JUnit run configuration for the tests in that module
2)Edit the configuration. There's a "Code Coverage" tab, with a toggle to enable coverage and a place to specify the classes you want covered.
3)Run your tests
4)Select your test runs name in the new dropdown menu in the toolbar. Code coverage numbers for all covered classes will show up in your project or package view.

There's also a way of getting left gutter highlights for covered and uncovered code, but I'm not going to describe it because it's 1) incredibly tedious and 2) going to change soon.

--Dave Griffith

1)Create a JUnit run configuration for the tests in that module

2)Edit the configuration. There's a "Code Coverage" tab, with a
toggle to enable coverage and a place to specify the classes you want
covered.

I had found this Code Coverage tab, but the "Enable code coverage" checkbox
is disabled, so I can't turn it on. Any reason why this checkbox would be
disabled?

Tobin

Why is this functionality bound to JUnit incidentally, instead of all run/debug configurations? I bet there are way more test cases using main() out in the wild than ones that use JUnit, and it'd be a great way to introduce the code coverage concept without forcing formal tests.

add emma.jar from IDEA_HOME/lib and open/save the run config a few times.. seems to be buggy

It is disabled since we support gathering coverage information with jre 1.5 only.

Ahh, I'm using 1.4. Thanks.

Tobin

If time permits we will implement coverage support for 1.4 jres. But these will come with usual custom class loader limitations as usual.

I've just tried out code coverage for the first time in build 5261 and I'm not seeing any numbers appear in the project/package view. Has the implementation changed in this build, or am I doing something wrong?

I do appear to be getting the left-hand gutter marks for the tested code, although they also appear in the JUnit test itself, which I'm assuming is an error? I wasn't planning on writing another JUnit test to test the JUnit test...

Not getting any information for production code might be a result of filtered packages misconfiguration. By default the package where test itself resides is chosen, but in case you have a different test/production code package structure you have to add those packages yourself.

As for seeing coverage for test themselves, there could be usecases where this is needed. For example we could have a "non trivial control flow in unit test" inspection, that could have a "simplify" quickfix making a choice based on coverage information. Until we do have a quickfix why not allow user to see this information and correct manually?:)

It's not that I don't have any information for Production code, as the left-hand gutter marks are being placed correctly in the Production as well as the test code. I'm just not getting any information in the Project/Package section, which Dave seems to suggest should be visible.

On another point, I changed the scope of the code coverage from package to specific classes, but it "remembers" the code coverage left-hand gutter marks from previous runs. They only disappear from the non-covered classes if you manually delete the .es files from the coverage directory - is this also expected?

I can see your point on the test classes, although it would be great if there was an option to turn this functionality off, especially if data would also be placed in the Project/Package panel for test classes.

Otherwise, I was really impressed with the new functionality, and I'll been showing it to people round the office today ;)

Good points, and the new setup in the 5261 build is better. I'd like to re-iterate, or second the motion for, some of the requests I've seen for this feature.

1. After running coverage again the display should be updated automatically. Right now I have to go to the drop-down and select the coverage data again to refresh.

2. We need a way inside the ide to clear coverage data. Deleting the files outside the ide isn't convienent, and doesn't always remove the coverage.

3. Colors for the gutter info need to be customizable. On my laptop the default colors are nearly invisible.

4. We need a shortcut available for selecting coverage, usual request to be able to do things without a mouse.

+1

It worked once for me but then keeps failing now. When I open a file that's
supposed to have been covered, here's what I get

No message
java.lang.NullPointerException
at
com.intellij.coverage.CoverageSuite.getFilteredPackageNames(CoverageSuite.java:67)
at com.intellij.coverage.CoverageDataManager.b(CoverageDataManager.java:96)
at
com.intellij.coverage.CoverageDataManager.a(CoverageDataManager.java:105)
at
com.intellij.coverage.CoverageDataManager.access$500(CoverageDataManager.java:7)
at
com.intellij.coverage.CoverageDataManager$2$1.run(CoverageDataManager.java:2)
at
com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:0)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:598)
at
com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:79)
at com.intellij.ide.IdeEventQueue.a(IdeEventQueue.java:50)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:53)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:121)

(I'm having all kinds of issues with the error reporting tool, so back to a
good ole' copypaste)

I also tried killing the coverage file, with no success.

At the end of my tests, I get this line in the console:
raw session data merged into
{in 147 ms}
It's missing the first character of every path element. The file itself gets
created in the expected location though.


Vince.

Already fixed.

Great!

Did you also see/fix the broken "session data merged" message I'm getting at
the end of a run?

Thanks :)

Vince.

Yep, merge should not happen, previous data should be removed upon next run.
I'm going to address this right now.

Cool, thanks.

Vince.

I have a problem with Coverage Suites. I use TeamCity that provides code coverage results as artifacts. But if I use it to show coverage of my classes nothing happens. Only error record appears in the idea.log:

java.io.IOException: file is corrupt or was not created by EMMA
at com.vladium.emma.data.DataFactory.mergeload(DataFactory.java:355)
at com.vladium.emma.data.DataFactory.load(DataFactory.java:56)
at jetbrains.buildServer.coverage.CoverageDataFileProvider.downloadCoverageData(CoverageDataFileProvider.java:123)
at jetbrains.buildServer.coverage.CoverageDataFileProvider$1.run(CoverageDataFileProvider.java:102)
at jetbrains.buildServer.TeamCityProcessManager$1.run(TeamCityProcessManager.java:62)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.runProcess(ProgressManagerImpl.java:38)
at com.intellij.openapi.application.impl.ApplicationImpl$1MyThread.run(ApplicationImpl.java:12)


When I have visited this folder I found that these files are empty. So IDEA can not download them and doesn't show error.

The second problem is if some suit doesn't exist it is shown in suits list and silently do nothing if applied.