Memories of compiling large C++ systems...

Until recently I have been using FSC and defined ctrl-S to the compile-file action. This way when writing a single class and its test I could just compile either file by itself -- enough to run the unit test. This was quite fast. But recently FSC started failing to compile our system. Sometimes you would have compile twice; sometimes you had to restart FSC; and then finally it simply would never compile our system. So I switched the external compiler.

Wow, what a difference. My turnaround has gone in the dumps. The external compiler (nailgun?) does not honor compile-file and insists on searching all over the code base for dependents to update. I have gone from a 1-second to compile and run the test to 10s of seconds, sometimes much more.

It seems that the external compile does not compile what is needed for the file at hand, but does also does a reverse dependency compile. This is way worse than being slow, because if I have errors in classes that depend on the class I am working with (eg, change of signature), compling the class I am working with issue errors. Yes, the errors are real. But I don't need to know that /every/ time I compile the file. I need to know that when I make the project.

So, has anybody developed any strategies to get back to compiling with pleasure when using the external compiler?

-barry

0
6 comments

We know that FSC has some restrictions (including such things as you described), moreover Typesafe decided not to support FSC at all.
Our current approach is to use SBT build, but it have some restrictions too. Now it can't compile single file, and problem is from SBT, it can't do it too. I talked with Mark and Josh about such possibilities, they said that in some cases it's possible, so probably they will do it, but I can't say anything about when it will be implemented in SBT (actually we are using only part of Zinc).
What we can do is to implement non-FSC simple compiler in external system, but I'm not really sure, that somebody need it (may be we need it for such cases, when SBT make is much more slower, because sometimes it recompiles everything). Not sure about moving FSC approach to external system, main reason is that it's not supported by Typesafe.
Also people say that our SBT build is slower than Eclipse SBT build, so probably it's possible to fix it somehow to make it faster.
One more thing, which we will definetely do, we will replace our make by SBT build for SBT projects (so it will be SBT, not Zinc in such cases). It will be great for those who not using IDEA compiler at all.

I know that probably news are not promising, but we will do our the best for better compilation.

Best regards,
Aleksander Podkhaliuzin.

0
Avatar
Permanently deleted user

Thanks for the update Aleksander. I certainly hope you and the typesafe team come up with some way to make the unit-test process
efficient again. Simply supporting a compile-file that actualy compiles only a single file would be a vast improvement (if one can call
reverting a backward slide an improvement.)

-barry

0
Avatar
Permanently deleted user

Certainly one of my worries with scala as well.

Speeding up the compilation should really be the number one driver if they are interested in making the language more mainstream.

0
Avatar
Permanently deleted user

Other than very simple experiments with stock Scala (standard library only) in the REPL, I do everything via SBT. (And my favorite SBT plug-in is sbt-idea for generating IDEA project files from an SBT project definition). I run SBT independently of IDEA in a separate terminal window, though the same tactic can be used with the SBT plug-in for IDEA. I usually leave SBT in ~test:compile or ~test-only testname-pattern mode. Apart from the horrendously slow compilation of Specs2 tests in Scala 2.9.0-1, I've never been dissatisfied with compile times, and I work on moderately large systems.

The most important thing is to leave SBT running so the compiler code can be JIT-compiled to native code over the course of the first few compile cycles. I often see newcomers repeatedly entering sbt compile or sbt test in their shells. This is hugely wasteful.

I also disagree with the assertion that compile speed is of paramount importance. Tiny, rapid edit-compile-test cycles are not something I value. I think the trade-offs between compile speed (if compiling is inolved at all) and type safety when comparing languages leaves Scala far out front by comparison with the other languages of the day (possibly excepting Haskell, which I understand compiles quite quickly).


Randall Schulz

0
Avatar
Permanently deleted user

TDD is something I would never give up. It is without question the most reliable way I have found to produce high quality clean code.

Everyone is different but I would give up the language rather than stop practicing TDD.

This isn't mean to start a debate on coding practises, it's just to clarify why I find complitation speed of the upmost importance,.

Cheers

0

We are working on issues from this thread. Probably we will find solution by ourselves how to compile single file, also we have ideas how to help Zinc compiler to not recalculate set of changed files (IDEA already knows this set). So recompilation of small changes will be much faster, especially for big projects.
Also tomorrow plugin release will contain possibility to use Parallel compilation of independent modules, what can reduce time of full rebuild.

Best regards,
Aleksandr Podkhaliuzin.

0

Please sign in to leave a comment.