How long should gradle project sync take for a project with hundreds of modules?
We have a gradle project with about 600 modules/sub-projects, all at a flat level (i.e. no projects-within-projects). We had been using the gradle idea plugin for years to generate ipr/iml type-intellij project, but have recently switched to using direct gradle-integration for a variety of reasons. Overall, it works well. One bit of a downside is that it takes roughly 3 minutes to re-sync when we "Import Gradle Project". There is no feedback to tell us what it is doing, so it's hard to know what we might tweak to see if we can get it faster. We are on the latest - 2019.2.x. Has same behavior under 2019.1.x. By comparison, when running a basic "gradle help" command even with no daemon, it takes at most about 30 seconds to initialize the project.
I realize there are likely many factors that go into how long a sync takes, but I'm looking for other real world examples of large projects so can know if 3 minutes is an average time or inordinately long.
请先登录再写评论。
I'd mention the case when antivirus/firewall could slow dow the build. If you have it please try with it disabled or make sure that IDE settings directories IDE installation home and project files are excluded from the scan.
Try also allocating more memory (heap size) to a Gradle daemon https://stackoverflow.com/a/47455056/2000323
What status messages/text output do you see in Build tool window during the Sync or Gradle task execution?
To see and profile what is really going on during the import or build process you could take several jstack thread dumps of the java process that IDE launches to run Gradle daemon for importing (project Sync) the project or Gradle task execution. You can find it in OS process tree by the
org.gradle.launcher.daemon.bootstrap.GradleDaemonJava main class that IDE launches.
We give 4G to the daemon already and it doesn't look to be having memory pressure during the sync.
I'm skeptical of anyvirus/firewall being the issue because other gradle tasks configure quickly or run as expected and are comparable between our dev machines and CI systems. I think it more has something to do with whatever steps/tasks Intellij is telling gradle to do in the ijinit.gradle that it sends to the tooling API (I haven't looked at that in detail yet).
The Build window doesn't report much. One strange bit is that it seems to configure/build buildSrc twice. This is somewhat new (i.e. don't think it did this when we first started experimenting with the gradle integration about 3 months ago). Here is an example output of a sync when there were no changes to any files (I did a sync/import and then did another). Total time was 2m 10s according to output in the other part of the Build window:
> Task :buildSrc:buildSrc:compileJava NO-SOURCE
> Task :buildSrc:buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:buildSrc:processResources NO-SOURCE
> Task :buildSrc:buildSrc:classes UP-TO-DATE
> Task :buildSrc:buildSrc:jar UP-TO-DATE
> Task :buildSrc:buildSrc:assemble UP-TO-DATE
> Task :buildSrc:buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:buildSrc:test NO-SOURCE
> Task :buildSrc:buildSrc:check UP-TO-DATE
> Task :buildSrc:buildSrc:build UP-TO-DATE
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.6.2/userguide/command_line_interface.html#sec:command_line_warnings
CONFIGURE SUCCESSFUL in 1m 11s
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:processResources NO-SOURCE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
CONFIGURE SUCCESSFUL in 347ms
> One strange bit is that it seems to configure/build buildSrc twice.
Gradle tooling doesn't provide anything about implicit Gradle `buildSrc` projects.
However IDEA import supports importing `buildSrc`, it queries the `buildSrc` project info separately, that's why you see the output related to the project twice.
It shouldn't have an impact on the performance since all required data is up-to-date and immediately ready for the second run (CONFIGURE SUCCESSFUL in 347ms).
> There is no feedback to tell us what it is doing, so it's hard to know what we might tweak to see if we can get it faster.
You're right, there are several usability issues related to lack of progress information during the long-running operations.
The most popular one is the progress of artifacts downloading. I hope we will fix it for the next release, you can vote/watch the issue at https://youtrack.jetbrains.com/issue/IDEA-212447
Also there are other places where the time can be spent in Gradle configuration phase, e.g. dependency resolution, script compilation, building of the custom tooling models etc. We are planning to add the progress information for such long-running operations.
Here is one of the related issues to track https://youtrack.jetbrains.com/issue/IDEA-197589
You can try 2019.3 eap, the import memory consumption was greatly reduced, see details at https://blog.jetbrains.com/idea/2019/10/preview-the-performance-improvements-in-intellij-idea-2019-3
Please, feel free to file an issue for import performance problems if you faced one. The import performance improvement is one of the main goals of the IDEA Gradle integration subsystem for 2019.3.