Code Coverage with Sling

IDEA computes coverage passing coverage agent for the execution, coverage is calculated for all loaded classes which fqname matches coverage configuration 'include' pattern. I guess, the problem may come from the fact that coverage agent is not attached to the vm with tests or that the default include pattern is wrong for the case. It would help if you attach a screenshot of coverage test run & full output of the tests (selection on top level node, to get full classpath)

Anna

I hope that's the information you needed - I am using the default settings for tests and coverage so far.

Here is what I get when I execute Run '***' with coverage on the test-module:

And that's what I get when I start it on the project with the classes that needed to be tested:

BR,

Oliver

the second run was not performed as there were no tests found, is that expected? in the first run, were the classes in platform-core module loaded?

Anna

The result of the second run was expected. In the first run: yes, those classes were loaded, but not in the local vm.

About the second run: The tests are deployed to the configured AEM server and then executed there. But the test-module has a dependency to the platform-core module. And with the matching debugging configuration I can step through the tests (test classes and core classes).

So basically the data is not available because the sling junit testing framework does not provide it?

Oliver

IDEA treats the tests as simple junit and expect that listener would provide information about starting/finishing tests, etc. The listener is attached in the local vm though, so looks like the problems are similar to coverage problems. Remote vm doesn't have attached coverage engine and it doesn't propagate junit listener events. Additional VM start is performed by the framework? Is it possible to provide it with additional options?

The additional VM is configured as a remote app, so it is not started by the framework, but we can add parameters in the start script. At the moment we are using the following options:

-Xrunjdwp:transport=dt_socket,server=y,address=30303,suspend=n

Can copy -javaagent from the tests to the script (it's used arg file to pass arguments though it can be replaced with explicit args)? Then the coverage will be gathered and stored in the dedicated file. Then the file could be opened in the IDE. Is it a feasible solution?

I am not sure if I got that right. I do not seem to have -javaagent set:

Am I missing something?

In the first run, command line should contain -javaagent. It is included in command line only if you start with coverage but should not be there for simple run/debug

Hmm.. Now my JVM Options look like this:

-server -Xmx4096m -Djava.awt.headless=true -Djava.awt.headless=true -Xrunjdwp:transport=dt_socket,server=y,address=30303,suspend=n -javaagent:/Applications/IntelliJIDEA.app/Contents/lib/idea_rt.jar=55707:/Applications/IntelliJIDEA.app/Contents/bin -Dsling.run.modes=author,crx3,crx3tar

The application (AEM) starts, but when I run the tests with coverage in IntelliJ, nothing changes.

wrong agent, could you please attach output from run with coverage? should be -javaagent:coverageAgent.jar

Ah ok.. the parameter is there twice:

/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/bin/java -ea
-javaagent:/Users/user1/Library/Caches/IntelliJIdea2017.2/coverageJars/coverage-agent.jar=/private/var/folders/95/58w5lf657v596xzqd8n67bcr0000gn/T/coverage1273args
-Didea.test.cyclic.buffer.size=1048576
"-javaagent:/Applications/IntelliJIDEA.app/Contents/lib/idea_rt.jar=59008:/Applications/IntelliJIDEA.app/Contents/bin"

When I add the coverage agent, AEM does not want to start any more. I am getting several errors in the log:

16.11.2017 15:22:13.617 *ERROR* [FelixStartLevel] org.apache.sling.commons.metrics [org.apache.sling.commons.metrics.internal.MetricsServiceImpl(16)] Error during instantiation of the implementation object (java.lang.NoClassDefFoundError: com/intellij/rt/coverage/data/ProjectData)

java.lang.NoClassDefFoundError: com/intellij/rt/coverage/data/ProjectData

        at com.codahale.metrics.MetricRegistry.<init>(MetricRegistry.java)

        at org.apache.sling.commons.metrics.internal.MetricsServiceImpl.<init>(MetricsServiceImpl.java:55)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)

        at java.lang.Class.newInstance(Class.java:442)

        at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:237)

        at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:109)

        at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:906)

        at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:879)

        at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:749)

        at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:675)

        at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:430)

        at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:657)

        at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:341)

        at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:390)

        at org.apache.felix.scr.impl.Activator.access$200(Activator.java:54)

        at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:265)

        at org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)

        at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)

        at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)

        at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)

        at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)

        at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)

        at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:916)

        at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:835)

        at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:517)

        at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4542)

        at org.apache.felix.framework.Felix.startBundle(Felix.java:2173)

        at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)

        at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)

        at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.ClassNotFoundException: com.intellij.rt.coverage.data.ProjectData not found by com.adobe.granite.dropwizard.metrics [56]

        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1574)

        at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:79)

        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2018)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

        ... 33 common frames omitted

Because of this error some of the core bundles cannot start and so the whole thing is not coming up.

I saw that the folder names are constantly changing for the coverage data. This means I would have to merge everything manually, wouldn't it?

In this case I think we can abort the whole experiment. 

Thanks a lot for your time anyway!

As I mentioned, I suggest to store coverage information in one place and then reuse it inside IDE.

you need to pass after -javaagent:/Users/user1/Library/Caches/IntelliJIdea2017.2/coverageJars/coverage-agent.jar=

your_file.coverage false false false true .*

and you would need to ensure that directory ../coverageJars/ contains asm and trove (should be there but it's better to check)