Gradle JPS plugin broken: NoClassDefFoundError: org/apache/tools/ant/util/ReaderInputStream

Hi,

we wrote a plugin that allows users to create projects via the standard project wizard mechanism (custom module builder). All of that works great.

We also added support for Maven and Gradle. We essentially add a pom.xml or build.gradle, then use the ExternalSystems API to make IDEA refresh the projects and pick up the build files. See the applyBuildSystem method in our custom module builder.

This works, IDEA is picking up the build files and wires things up correctly (dependencies etc.). However, in case of Gradle there's a problem.

After creating a new project with our builder, we add a run configuration. That run configuration will recompile the project. However, compilation fails:


2015-06-17 15:20:15,910 [      0]   INFO - etbrains.jps.cmdline.BuildMain - Build process started. Classpath: /Applications/IntelliJ IDEA 14 CE.app/Contents/lib/jps-launcher.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/tools.jar:/Applications/IntelliJ IDEA 14 CE.app/Contents/lib/optimizedFileManager.jar:/Applications/IntelliJ IDEA 14 CE.app/Contents/lib/ecj-4.4.jar
2015-06-17 15:20:16,116 [    206]   INFO - etbrains.jps.cmdline.BuildMain - Connection to IDE established in 182 ms
2015-06-17 15:20:16,196 [    286]   INFO - jps.cmdline.JpsModelLoaderImpl - Loading model: project path = /Users/badlogic/workspaces/robovm-test/untitled21, global options path = /Users/badlogic/Library/Caches/IdeaIC14/plugins-sandbox/config/options
2015-06-17 15:20:16,431 [    521]   INFO - jps.cmdline.JpsModelLoaderImpl - Model loaded in 235 ms
2015-06-17 15:20:16,431 [    521]   INFO - jps.cmdline.JpsModelLoaderImpl - Project has 1 modules, 5 libraries
2015-06-17 15:20:16,537 [    627]   INFO - ellij.util.io.PagedFileStorage - lower=100; upper=200; buffer=10; max=631767040
2015-06-17 15:20:16,653 [    743]   INFO - etbrains.jps.cmdline.BuildMain - Pre-loaded process ready in 744 ms
2015-06-17 15:20:16,716 [    806]   INFO - .incremental.IncProjectBuilder - Building project; isRebuild:false; isMake:true parallel compilation:false
2015-06-17 15:20:16,722 [    812]   INFO - r.api.ClassFilesIndicesBuilder - class files data index disabled
2015-06-17 15:20:16,755 [    845]   INFO - .incremental.IncProjectBuilder - gradle-resources-test:untitled21: java.lang.NoClassDefFoundError: org/apache/tools/ant/util/ReaderInputStream
org.jetbrains.jps.incremental.ProjectBuildException: gradle-resources-test:untitled21: java.lang.NoClassDefFoundError: org/apache/tools/ant/util/ReaderInputStream
 at org.jetbrains.jps.incremental.IncProjectBuilder.buildTargetsChunk(IncProjectBuilder.java:971)
 at org.jetbrains.jps.incremental.IncProjectBuilder.buildChunkIfAffected(IncProjectBuilder.java:840)
 at org.jetbrains.jps.incremental.IncProjectBuilder.buildChunks(IncProjectBuilder.java:663)
 at org.jetbrains.jps.incremental.IncProjectBuilder.runBuild(IncProjectBuilder.java:370)
 at org.jetbrains.jps.incremental.IncProjectBuilder.build(IncProjectBuilder.java:191)
 at org.jetbrains.jps.cmdline.BuildRunner.runBuild(BuildRunner.java:137)
 at org.jetbrains.jps.cmdline.BuildSession.runBuild(BuildSession.java:293)
 at org.jetbrains.jps.cmdline.BuildSession.run(BuildSession.java:124)
 at org.jetbrains.jps.cmdline.BuildMain$MyMessageHandler$1.run(BuildMain.java:242)
 at org.jetbrains.jps.service.impl.SharedThreadPoolImpl$1.run(SharedThreadPoolImpl.java:41)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/apache/tools/ant/util/ReaderInputStream
 at org.jetbrains.jps.gradle.compiler.GradleResourcesBuilder.build(GradleResourcesBuilder.java:103)
 at org.jetbrains.jps.gradle.compiler.GradleResourcesBuilder.build(GradleResourcesBuilder.java:42)
 at org.jetbrains.jps.incremental.IncProjectBuilder.buildTarget(IncProjectBuilder.java:906)
 at org.jetbrains.jps.incremental.IncProjectBuilder.runBuildersForChunk(IncProjectBuilder.java:887)
 at org.jetbrains.jps.incremental.IncProjectBuilder.buildTargetsChunk(IncProjectBuilder.java:945)
 ... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.tools.ant.util.ReaderInputStream
 at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 19 more



It appears that an incremental build is performed, in an external process that's running BuildMain etc. That process doesn't seem to get the full classpath, including the Gradle and Ant JARs required for the build.

Closing and opening the project solves the issue, so my guess is that our hackish way of applying Gradle/Maven in the ModuleBuilder is the culprit. However, i'm at a loss at what i could do to fix this.

Any hints would be very appreciated,
Mario
10 comments
Comment actions Permalink

I rewrote our ModuleBuilder, basing it on the GradleModuleBuilder: https://github.com/robovm/robovm-idea/blob/2e05be9cec494c570a77d8a53fdc88da1f0b3025/src/main/java/org/robovm/idea/builder/RoboVmGradleModuleBuilder.java

The only change to the GradleModubleBuilder is that i add our templating engine, which will extract a few files into the content root, including the Gradle build file. This produces the same issue as the "old" ModuleBuilder. I guess i can rule out the builder as being the culprit.

Could it be an issue with the build.gradle file itself? Or with our custom CompileTask?

0
Comment actions Permalink

I can also rule out the build.gradle file, the error also happens in case of a plain build.gradle without any external Gradle plugins, e.g. RoboVM Gradle plugin.

The issue seems to be very similar to what's been reported here: https://youtrack.jetbrains.com/issue/KT-5676

The only thing i don't understand is: if the jps plugin doesn't have ant as a dependency, why does closing and reopening the project work?

0
Comment actions Permalink

I debugged the Gradle JPS plugin according to https://confluence.jetbrains.com/display/IDEADEV/External+Builder+API+and+Plugins

The class GradleResourceFileProcessor has a dependency on org.apache.tools.ant.util.ReaderInputStream. That class is instantiated and hence initialized in line 103 of GradleResourcesBuilder.

I then checked the URLClassLoader responsible for loading GradleResourcesBuilder, which is the same loader used to load GradleResourcesFileProcessor. Here's the list of JARs it uses for class loading: https://gist.github.com/badlogic/999e067d5a97cec069ac

No Ant tools to be found.

Next, i closed and reloaded the project, triggered a build and inspected the list of JARs of the URLClassLoader again: https://gist.github.com/badlogic/48434bc0197b80d62060

All of a sudden there's 5 more JARs available to the class loader. Including /Applications/IntelliJ%20IDEA%2014%20CE.app/Contents/lib/ant/lib/ant.jar, which contains the ReaderInputStream.

Now my question is: why aren't those JARs loaded by the external build process right after the project was created? Why do i need to close and reopen the project for those JARs to be loaded properly?

0
Comment actions Permalink

Hi Mario,

It seems the problem caused by using of auto-make option enabled (config.MAKE_PROJECT_ON_SAVE = true; at org/robovm/idea/components/RoboVmSdkUpdateComponent.java:87)
Using auto-make enabled IDEA forced to preload the JPS process earlier than your module can be recognized as gradle-based.
That's why the process classpath does not contain some entries specific only for gradle modules and this can be the reason why reopen the project helps.
I'd suggest not to use auto-make, at least for gradle projects.

0
Comment actions Permalink

I've tracked things down to BuildManager, which is responsible for spawning the external process.

In BuildManager#launchBuildProcess(), the following piece of code is responsible:

    final List cp = ClasspathBootstrap.getBuildProcessApplicationClasspath(true);
    cp.addAll(myClasspathManager.getBuildProcessPluginsClasspath(project));
    if (isProfilingMode) {
      cp.add(new File(workDirectory, "yjp-controller-api-redist.jar").getPath());
    }
    cmdLine.addParameter(classpathToString(cp));



ClasspathBootstrap#getBuildProcessApplicationClasspath adds 23 JARs to the classpath passed to the Launcher. That seems to be pretty much static.

Then BuildProcessClasspathManager#getBuildProcessPluginsClasspath() adds the rest based on the project. That adds a few static classpaths, and dynamic classpaths by walking through a bunch of BuildProcessParametersProviders. One of these is the GradleBuildProcessParametersProvider, which has this method:

  @Override
  @NotNull
  public List getClassPath() {
    if (myClasspath == null) {
      myClasspath = ContainerUtil.newArrayList();
      addGradleClassPath(myClasspath);
      final ModuleManager moduleManager = ModuleManager.getInstance(myProject);
      for (Module module : moduleManager.getModules()) {
        if (ExternalSystemApiUtil.isExternalSystemAwareModule(GradleConstants.SYSTEM_ID, module)) {
          addOtherClassPath(myClasspath);
          break;
        }
      }
    }
    return myClasspath;
  }

The Ant library is only added for modules that are external system aware. I guess i just need to make sure the module has this flag set? I guess i need to figure out how to set that flag :)

0
Comment actions Permalink

Hi,

wow, thanks a lot for the answer. Our plugin requires auto-make to be enabled (at least after project creation/import). We added enabling this automatically because otherwise users would forget about it and wonder why some functionality doesn't work. Would you know of any event i could subscribe to on which i can enable that option?

Thanks,
Mario

0
Comment actions Permalink

Unfortunately, there is no any events broadcasting after a project import finished.
The import is invocation of a set of usually independent ProjectDataService-s responsible for importing of different things, like module structure, dependencies, facets etc.
And if you run the import in async mode, there is no reliable place to hook when the module will be ready.
If synchronous creation/importing of your project is not an option, you can create a quickfix notification for a user (e.g. using editorNotificationProvider).

0
Comment actions Permalink

btw, you could create your own com.intellij.openapi.externalSystem.service.project.manage.ProjectDataService annotated with @Order(ExternalSystemConstants.BUILTIN_SERVICE_ORDER + 1)
see example in com.intellij.openapi.externalSystem.service.project.manage.ModuleDataService

0
Comment actions Permalink

That seems to be a workable idea. However, i also noticed that if i refresh the Gradle-based project via the Gradle tool window, i run into the same issue if auto-make is on.

I guess what i really need is an alternative to auto-make that's invoked when the user saves a source file and just compiles the .class file, putting it in the build output folder. I guess i could try doing "manually" by hooking into editor save events of virtual file system events.

Thanks for the help, much appreciated!

0
Comment actions Permalink

I fixed the issue by adding our own BuildProcessParametersProvider. It will add the optional JARs the GradleBuildProcessParametersProvider will only add if the module is Gradle aware. Works like a charm. Thanks again for your help!

0

Please sign in to leave a comment.