16 core workstation, 16GB RAM, multi module java/scala project, how to speed up compilation

I.e. when I do a clean build, 1-2 cores are used during compilation (I've enabled "compile independent projects in parallel"). Is there something I can do to use more processors and speed up compilation?

12 comments

You may try to disable Scala compile server (Settings | Scala | Run compile server).

The primary goal of the compile server is speed up incremental compilation, however (currently) it provides only a single instance of Scala compiler at a time (more info).

If you perform a clean build and your project contains many independent modules, disabling the compile server may improve compilation performance by allowing better parallelism.

Additionally, make sure that I/O isn't a bottleneck (are you using SSD?).

0

Thanks for the suggestion, I gave it a try. Unfortunatelly compilation times are about the same, around 5 mins 30 secs.

(I do have a very fast SSD)

0

The primal performance constraint in full project compilation is (probably) the Scala compiler itself, you may check an explanation from Martin Odersky.

0

yes the scala compiler is slow but independent modules could be compiled in parallel by i.e. multiple instances of the intellij scala server in a same fashion as mvn -T 2C

0

Please try the next Scala plugin nightly build (0.17.192 or 0.18.40), with or without the compile server.

It is very likely that parallel compilation speed will be improved with these builds.

0

The next nightlies are required - these builds will be available tomorrow.

0

ok,seems this improved build times by 2x!

CPU usage goes up to 60% (it uses more than 8 of my 16 cores)

That's a big improvement, thanks

0

When is this going to be officially released?

Cheers

0

That's good! By the way, don't forget to increase JVM maximum heap size apropriately (either in Project Settings | Compiler when compile server is disabled, or in Settings | Scala when compile server is enabled), because all the "parallel" compilers are instantiated within a single JVM.

0

Please sign in to leave a comment.