Code coverage for test sources?

IDEA 8 still has code coverage available. In fact it added an additional code coverage runner. In IDEA 7 Emma was used as the code coverage runner. IDEA 8 added the IDEA Coverage Runner. (You can see some basic info on it form the wiki page when it was first introduced: http://www.jetbrains.net/confluence/display/IDEADEV/IDEA+Coverage+Runner).

To see the code coverage, you first need to turn it on. In the test's run configuration, click the code coverage tab, select the "record code coverage information", and then select a runner (Emma will be selected by default).

I know how to enable code coverage. The problem is that it shows the coverage only for classes in the production source folder. It does not show it for classes in the test source folder.

Here is how it looks:
http://img151.imageshack.us/img151/3692/idea7coveragesm4.png
http://img509.imageshack.us/img509/5868/idea8coveragesa7.png

Hello,

why you do not want to use unused declaration inspection in order to find
unused code and use code coverage to find code which is not covered by your
tests? i do not tnink that enabling coverage for test sources is good idea
... please convince me if i am wrong ;)

Thank you
-

Anna Kozlova
JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

I know how to enable code coverage. The problem is that it shows the
coverage only for classes in the production source folder. It does not
show it for classes in the test source folder.

---
Original message URL:
http://www.jetbrains.net/devnet/message/5227893#5227893

Here is the code which would have required code coverage. I'm right now making an object database garbage collector, and before I could write any production code, I needed a mock directed graph which I could use to test-drive the collector's code. Here is a version of the MockGraph just after I had done some big refactoring. (I moved the edges and statuses to the MockNode class - before that they were Maps in MockGraph. I also replaced a Set of root nodes with a special node called 'root' - now the root nodes are actually the edges of the 'root' node.) As a result the code became much simpler.

But in the code below there is one method which is not anymore needed. Can you figure out which one it is? Since the method is public, the inspection for an unused declaration will not find it (and Analyze | Inspect Code is too slow and reports too many false positives). Code coverage would have shown immediately that the createRootNode method is never used, but since this class is part of the test sources (in the src/test/java folder of a Maven module), IDEA 8 does not show code coverage for this class.

Luckily in this case I knew to expect that that method is not used, so I was able to do a Find Usages on it and then delete it. But if the code had been any more complicated, or the dead code would have been for example one execution path of a if-else construct, then detecting that dead code would not have been possible without code coverage.

Also, code coverage for test sources (as it was in IDEA 7) has also been useful for seeing where the tests fail. When a test failed in IDEA 7, that method would be colored red (fully or partially), and it was quick to see the pattern of failures in the test cases that you were working on.

It is also useful for seeing if there is some problem in the test setup, as a result of which the tests are not executed at all. When you see green all over the test code, then you know that the tests are working.

So I'm requesting you to enable recording code coverage for test sources, or at least making it optional. Test sources are not a second-class citizen which does not need all the support that production code gets. The test sources must be of highest quality, especially their readability, because that is what enables the maintainablity of the production code. I would even say that good tests are more important than the production code: If the production code is good but the test code is bad, then eventually the production code will also begin to rot [Clean Code, pages 123-124, "Keeping Tests Clean"]. However, if the production code is bad but the test code is good, then that will make it possible to refactor the production code to be better, or even discard the messy production code and rewrite better code using the same tests.

public class MockGraph implements Graph<String> {     private final Map<String, MockNode> nodes = new ConcurrentHashMap<String, MockNode>();     private final MockNode root = new MockNode();     public Iterable<String> getAllNodes() {         return Collections.unmodifiableCollection(nodes.keySet());     }     public Iterable<String> getRootNodes() {         return Collections.unmodifiableCollection(root.edges);     }     public Iterable<String> getConnectedNodesOf(String node) {         return Collections.unmodifiableCollection(getNode(node).edges);     }     public void createNode(String node) {         nodes.put(node, new MockNode());     }     public void createRootNode(String node) {         createNode(node);         root.edges.add(node);     }     public void removeNode(String node) {         nodes.remove(node);         root.edges.remove(node);     }     public void createDirectedEdge(@Nullable String from, String to) {         getNode(from).edges.add(to);     }     public void removeDirectedEdge(@Nullable String from, String to) {         getNode(from).edges.remove(to);     }     public long getStatus(String node) {         return getNode(node).status;     }     public void setStatus(String node, long status) {         getNode(node).status = status;     }     private MockNode getNode(@Nullable String node) {         if (node == null) {             return root;         }         MockNode n = nodes.get(node);         if (n == null) {             n = new MockNode();         }         return n;     }     private class MockNode {         public long status = NULL_STATUS;         public final List<String> edges = new CopyOnWriteArrayList<String>();     } }

EDIT: The forum software does not know how to format code or is too buggy on Chrome. Here it is better formatted: http://pastebin.com/f4ef7a4af

If can be useful in telling you where in the test source old and unused
cruft has accumulated.

It would be nice if test source coverage was at configurable.

-tt

ORFJackal wrote:

I know how to enable code coverage. The problem is that it shows the coverage only for classes in the production source folder. It does not show it for classes in the test source folder.

Here is how it looks:
http://img151.imageshack.us/img151/3692/idea7coveragesm4.png
http://img509.imageshack.us/img509/5868/idea8coveragesa7.png

Sorry - I read your original post too quickly and didn't pick up that your were talking about showing coverage in the source of the actual test.

Unfortunately I find the inline coverage to be flaky.

this is a regression in functionality IMO.  we should be able to see code coverage on any class, whether its a test class or not.

I'll chip in with another vote for support for test coverage in test classes.  We use it all the time to ensure that our test suites have run all of our tests, as we have a complex set of group dependencies.

I created a JIRA issue about this: http://www.jetbrains.net/jira/browse/IDEA-20981