2019.1 EAP with gradle build delegation causes debug hot swap to reload many classes

With "delegate" option recompilation is performed by Gradle. What Gradle version do you use?

Does it help if you build via invoking Build action on this module instead of Recompile file (have the module in focus in Editor and select Build | Build Module <module name> action)?

I am using the latest gradle, 5.1.1.

Using build module executes the same gradle task "classes" and shows strange behavior. If I change a comment in my independent java class or an existing method and recompile using build module no classes get reloaded. If I cause a compile failure on the class changed then recompile using build 'module', then I see 592 classes reloaded.

>If I change a comment in my independent java class or an existing method and recompile using build module no classes get reloaded.

Is the class used anywhere in project? Could it be that it is just not loaded at runtime hence when you modify it it is not updated?

Also you may use Run | Reload Changed Classes action.

Note that this is the case when IDE just invokes Gradle task for Gradle to compile incrementally the project. The it updates the changed files (classes that were compiled).

Is it possible to have a sample project to check?

Thanks Andrey for the suggestions. I did some additional testing and include some of my results below. In all cases where a java file is modified, it is the same source file modified with a comment change. In all cases I have gradle daemon enabled. It is not clear to me exactly what determines how many classes get reloaded.

1. Application build with gradle, idea launch app, change source, IDEA build module - 1697 classes reloaded
2. Scenario #1, 2nd source change then IDEA build module - 2 classes reloaded
3. Scenario #1, stop app from IDEA, IDEA launch app, change source, then IDEA build module - 2 classes reloaded
4. Application build with gradle, idea launch app, then IDEA reload changed classes - 0 classes reloaded
5. Scenario #4, change source, gradle module build, then IDEA reload changed classes - 797 classes reloaded
6. Scenario #5, change source, gradle module build, then IDEA reload changed classes - 2 classes reloaded
7. Scenario #6, stop app from IDEA, IDEA launch app, change source file, gradle module build, IDEA reload changed classes - 2 classes reloaded
8. Source change (after start or after prior change) then IDEA build recompile - 1387 classes reloaded

Here are my observations  from the scenarios I tested.

1. Using "build -> recompile" always reloads more classes than necessary.
2. Any time the application is started after a full gradle build results in reloading of many classes.
3. Using "build -> module" works as expected except in the case of observation #2.
4. "run -> reload changed classes" works as expected except in the case of observation #2.
5. Switching between "build -> module" and "gradle build + run -> reload changed classes", and visa-versa, will cause more than expected class reloads.

Unfortunately I have been testing on our development project and I can't share that with you and I have not tried to create a test project that could reproduce this behavior as this could take a significant amount of time.

I am working on this because we want to streamline our development process. Building and starting the application take a long time and we want to limit these activities. Our build also includes a lot of generated and woven code so we can not only build from IDEA. A day in the developer will consist of some gradle builds and some IDEA builds. Prior to 2019.1 this looked something like the following (without gradle build delegation). Note in most cases day to day development does not modify sources that require weaving. Also note this procedure requires aligning the gradle and IDEA output directories (which is not recommended by IDEA). Without this alignment of output directories, compile from idea will not change classes loaded by the JVM at startup.

1. Update subversion workspace to pull in other developer changes (typically daily)
2. Build using gradle to generate sources and weave classes.
3. Build using IDEA to generate class files again, but now IDEA build cache and output directories are refreshed.
4. Weave using gradle (weaving was broken on step #3).
5. Launch application from IDEA
6. Debug application and change sources.
7. Compile changes from IDEA
8. Hot swap changes and go back to step #6.

This process involves some steps that could be eliminated (steps 3 and 4). Ideally the procedure would be something like the following if gradle can be used for all the building.

1. Update subversion workspace to pull in other developer changes (typically daily)
2. Build using gradle to generate sources and weave classes.
3. Launch application from IDEA
4. Debug application and change sources.
5. Compile changes from IDEA with gradle
6. Hot swap changes and go back to step #4.

I seem to be very close to being able to recommend a procedure based on the steps above but my observation #2 above creates a speed bump between steps 5 and 6. I suspect the problem is probably related to some build cache initialization when the application is launched being stale in some way.

>Here are my observations  from the scenarios I tested.

>Any time the application is started after a full gradle build results in reloading of many classes.

What Run/Debug Configuration do you use? Do you use get this issue even when you build only from IDE (when delegation to gradle option is used)? Do you compile only java classes?

For the testing I performed above I used a jar application launcher. The jar used for launching is constructed as part of the gradle build and contains a manifest classpath that has all the output directories of the gradle projects listed in it along with the project external jar dependencies. I think I saw similar behavior in another project that uses a Java Application launcher but I would need to verify that.

The startup behavior occurs for both compile from gradle command line + "run -> Reload changed classes" and IDEA build module + reload classes. Once the first large hot swap completes, subsequent compiles of the same file result in the expected number of class reloads (as long as they are the same type of compile as the first--gradle command line or IDEA build module).

>The startup behavior occurs for both compile from gradle command line + "run -> Reload changed classes" and IDEA build module + reload classes.

The point is to use only build action from IDE (since you have delegate to Gradle enabled IDE will use Gradle build in the end).

Could you clarify, do you build only Java? Thanks.

The application is a java application with a web application component in a jetty container.  All the code is java, however there are non-java compile steps to get to the final image. There is xml processing that generates java files that are in turn compiled into java classes. There are also java classes that are enhanced post compile by Eclipse Link weaver. The end result is a set of java classes that are launched in a single JVM.

In the tests I ran and reported hot swap reloading issues were on sources that are not generated and are not woven. The execution of the gradle targets used only performed 'compile' and did no source generation or weaving (for both command line and IDEA build testing). From what I can tell the gradle build is performing incremental compile and recompiling only my changes.

Note the only way I could get 'Run -> reload changed classes' to work was to compile from the command line. Changing a source file was not enough to get the classes to reload and building from IDEA triggered the reload automatically eliminating the need to 'reload changed classes'.

>Note the only way I could get 'Run -> reload changed classes' to work was to compile from the command line. Changing a source file was not enough to get the classes to reload and building from IDEA triggered the reload automatically eliminating the need to 'reload changed classes'.

Do you mean the Reload Changed Classes action does not work after you change the source code? Does it work when you issue build via Build | Build Module <module name> action?

Could you please attach screenshots of Settings (Preferences in macOS) | Build, Execution, Deployment | Build Tools | Gradle and File | Project Structure | Modules | <module of the source set you change classes in> | Paths tab. Do you have only one build.gradle file in project?

Could you please also try to re-import the project from scratch: File | new | Project from Existing Sources... action and select main build.gradle file to import from and check the issue? Thanks.

The answer to your first question is that typically when I build the module I choose the 'reload classes' option that pops up after the build completed. If this option is chosen then 'Reload changed classes' does not do anything since the changes were already reloaded. If I cancel the 'reload classes' after the module build then 'Reload changed classes' will do something. I guess I did not think about the case where someone would choose not to reload classes after building the module but wanted to refresh them in the running JVM later. Certainly building from the command line then using 'Reload changed classes' also works, but it behaves in the same way as building the module from IDEA in that the first reload triggers a lot of classes to be swapped and subsequent reloads behave as expected.

I will work on collecting the information you requested.  I had an idea last night about how we construct the artifacts in our build and how idea builds the project. When building from idea the 'classes' task is the top level task that is triggered across all gradle projects that should 'build' the system, but our gradle build scripts are structured more around the 'jar' task which is not used by idea. This makes the gradle 'project' build and incomplete build and not suitable for use after a subversion workspace update. I think it may be possible to restructure our build so that the 'classes' task will trigger these extra build steps so that an external gradle build is not required when working with IDEA. I will investigate this possibility and get back to you.

Andrey thanks for all your help. Based on our discussion I was able to get a scenario working. For others that have this issue, the key is to do all compilation within IDEA exclusively and also to avoid "Build -> recompile" to recompile a single file.

Here is what I did to get this working which keys off the fact that an IDEA 'build -> project' will delegate to the gradle tasks 'classes' and 'testClasses' for every project in the gradle build, including the root project. This means the gradle build files must be tweaked to do a bit of custom task execution triggered off these standard java plugin tasks.

1. Hooked in non-compile build tasks such as deploying war files to the root project's 'classes' task (i.e. classes.dependsOn('workspaceSetupTask')). 
2. Changed projects with weaving to output to in intermediate output directory other than the default (this helps with setup of the correct up-to-date task logic changes in gradle).
3. Hooked in code weaving task to the project's 'classes' task and configured the weave output to go to the standard gradle output directory. This is the directory the IDEA project is expecting to find the classes in.

By hooking in existing tasks to be triggered by the gradle 'classes' task I was able to get the IDEA 'build -> project' to correctly build the system that was then capable of running from the debugger.  I tested with the following scenario which worked as expected.

1. Cleaned project to remove all generated artifacts and files.
2. Imported the gradle project into IDEA.
3. Built project using 'build -> project' in IDEA.
4. Started debugging the project using a 'Jar Application' launcher.
5. Changed a file in the editor.
6. Used 'build -> module' to compile the changes (gradle does incremental compile and updated the appropriate classes).
7. Clicked the popup to reload classes (two classes were reloaded in this case).
8. Stepped through the changes in the debugger and verified the new code was loaded.

Thank you very much for the details! The "delegate" mode aims to solve exactly the case when however the complicated Gradle build logic is IDE delegates it all to Gradle so developer can customize the build (and hotswap) however it needs as far as Gradle provides such an ability.

I also want to thank you because due to your report we have found some cases that could be improved e.g. on compilation of a single file IDE will trigger module build. I also was able to reproduce the issue when first hotswap reloads many classes (in a Kotlin module though). The investigation will be performed in scope of https://youtrack.jetbrains.com/issue/IDEA-206742 issue.