Code Coverage, Part Deux


Well, code coverage works now, and looks like a great first cut. Some quick notes, before I put all this in JIRA.

1) It looks like covered classes and paths are not being persisted with the run profile, although coverage toggling is. Check your serialization code.

2) It is probably adequate to toggle coverage viewing on a per-project basis. The current per-file basis toggling is far too clunky. A toggleable toolbar button would be sweet.

3) In this fallen world, it actually is important to have some coverage count reports, rather than just editor highlighting. A toolwindow with per-class percentages and rollups would be fine, although annotating the project window might be even nicer.

4) I've never had a use for the "how many times was this line executed" information that some tools give, but if it's easy to do, it might be worthwhile. Maybe in the editor gutter, like VCS annotations or line numbers.

5) The auto-merge and auto-synchronization with editing changes ROCK MIGHTILY!!!!! All on it's own, that technology is enough to differentiate your coverage tool from all competitors. That said, you do need to blow away the accumulated coverage information for a project. It's not so much that it is necessary to fix things if you get out of sync, but to allow coverage to be measured for different test suites.

6)It is more important to see uncovered code than covered code, so there should be gutter coloring for both.

7)You might wish to allow background coloring, rather than just gutter coloring.

8)Colors need to be configurable, but you knew that already.

--Dave Griffith

3 comments

I'm having trouble figuring out how to get code coverage to work.

I'm using JUnit, and have enabled code coverage with a pattern of org.* and com.mycompany.*

The thing that I'm testing (Eclipse runtime stuff, no UI) isn't showing anything when I choose Analyze->Show Coverage Info.

Admittedly, I don't know how to use Emma, but I can't find enough doc on it, or how to use it from IDEA.

Help?

0

Er, though UI does not tell it, the current implementation works with jre 1.5 only. Is it the case?

0

Partly the task of wiping out old information is solved by emma: when two versions of merged metadata associated with coverage data have different checksums, the first (that is the older) is thrown away.

I see your point in differentiating coverage information for different suites. I'm thinking of coverage suites abstraction that will include several test suites run: keeping separate coverage information seems sometimes inadequate, e.g. I want to run my refactoring tests separately, but keep unique coverage information.

Also I'm thinking about annotations to declare what implementation classes/packages this test class/package is intended to test. These should sometimes provide more sensible defaults for coverage filters than the current naive algorithm and possibly allow for "run tests affecting the change" action. These are just ideas though...

0

Please sign in to leave a comment.