Compilation and editing (analysis) performance

This is bug for editing. There is shouldn't be option to turn off something. There is should be ability to interrupt code analyze when you start to type (to not freeze while typing). I'll find such places after some profiling.

Best regards,
Alexander Podkhalyuzin.

I too see the editing issue. On certain files I have, which for example contain lots of case classes in the same file, a single keypress can freeze my IDE for 30 seconds!


A while ago I also suggested an option to use the fsc but nothing came of it. I got round the issue by creating a symbolic link from scalac to fsc but since idea has changed the way it uses the scala facet, I can't for the life of me find the scalac it uses. Please can this option be added as it would really speed up developement...

Thanks, Paul.

Paul,

I'm guessing the plugin invokes the scala compiler directly, instead of via the shell script (as it shows phases etc while progressing).

At present I can't get access to the plugin source code. When I do, I'll try to see if I can hack it to use fsc internally and see what happens. :-)

Thanks for the info

Yes, we call scala.tools.nsc.Main.main() method. Can you suggest something for fsc like this?

Best regards,
Alexander Podkhalyuzin.

Yes, the main class for fsc is scala.tools.nsc.CompileClient.

Unfortunately, I don't think it's quite as simple as just switching the classes.

I think you need to start the compile server (scala.tools.nsc.CompileServer). Unfortunately, the compile server uses a messy mechansim for writing the port is selects and  the CompileClient needs to jump through hoops to retrieve it.

I've written an fsc plugin for a build tool I was working on, so it's all quite possible (the maven scala plugin does likewise).

Anyway, now that I've got the plugin to build (as per my other post), I can have a go at hacking it in. I doubt I'll be able to do much on it until the weekend however.

Are you happy to accept patches?

Cheers,
Andrew

I tried just switch class yesterday, but as you know it's not work.
We will be very happy to accept patch for fsc, because you know that mechanism better.
The best way to give us patch with something like FastScalacRunner.java near ScalacRunner.java, also uses reflection, but it's ok without reflection and it's ok in other module written in scala (so it will be just for scala 2.8, but as you know it's not only feature just for scala 2.8 in our plugin).

Best regards,
Alexander Podkhalyuzin.

Cool, I'll give it a go.

Just a quick update ... I've got the basics working.

I've added a "Use fsc" checkbox to the Scalac compiler configuration and can toggle between the compilers. The good news is that it reduces my compilation time from around 10 seconds to 2. Much better!

The bad news is that it's a bit messy. The root cause of the problems is that the fsc client attempts to launch the server using "scala" from the shell. That's why it fails (or launches the incorrect version of scala) depending on the particular user's installation. The scala implementation really needs fixing to launch the server using the same class path and jdk that the client was lauched with. (When I have time, I'll raise a ticket for the Scala guys about this issue).

Back to good news, the hack I'm using seems to be reliable enough. It works by starting the compile server itself prior to lauching the client.

Outstanding issues:

• What should happen if the fsc server is already running prior to IDEA's first compilation? There's no guarantee it's using the correct scala versions. At present, I'm thinking I should probably shut it down on the very first compilation to be safe
• Should the fsc server be shut down when IDEA exits?
• I'm not seeing the status messages when compiling ... however, it might just be that they update too quickly

Cheers,
Andrew

>The scala implementation really needs fixing to launch the server using the same class path and jdk that the client was lauched with.

I think you can obtain the path to the running bin/java and classpath using System.getProperty, I've used this previously to launch child JVMs with the same version and classpath.  Hopefully this helps, sure would be nice to obtain the benefits of fsc.

Hi Sam,

The problem is within the Scala compiler client itself, so there's nothing I can really do about it (other than encouraging the Scala devs to fix it, or supply a patch).

However, the work-around I'm using (which is similar to the one used by the maven plugin), appears good enough. I'm using it now in fact to use fsc within IntelliJ. :-)

Cheers,
Andrew

I've created a Scala ticket for the fsc issue here: http://lampsvn.epfl.ch/trac/scala/ticket/2510

Hi,

I dont think it is good idea to use FSC server. There is much better way. Instead of seconds it reduces compilation time to miliseconds!

Take look at scala.tools.nsc.interactive.RefinedBuildManager . It is incremental build manager developed for Eclipse Plugin. And there is way howto invoke it from command line:

Main method of object scala.tools.nsc.interactive.BuildManagerTest takes exactly same parameters as Scalac. It also have nearly same error output. But it does not exict after compilation, but waits for list of modified files to compile. It is pretty much like FSC, but does not require server.

All we need is list of files modifed from last compilation + list of file with errors and pass them to compiler.

Example:


>java -cp scala-compiler.jar;scala-library.jar scala.tools.nsc.interactive.BuildManagerTest
builder > J:\asterope\src\org\asterope\util\ParallelSeq.scala
compiling Set(J:\asterope\src\org\asterope\util\ParallelSeq.scala)
warning: there were deprecation warnings; re-run with -deprecation for details
Changes: Map()
builder >

I am currently working on patch which will use this.

Regards,
Jan

Hi Jan,

Sounds cool to me - I wasn't aware of it as an option.

The current fsc architecture/implementation has many wrinkles, so I'll put my patch on hold and await yours.

Any idea of the time frame?

I tryed to integrate it with Idea, but without much sucess. I have same problem as you with SFC, it needs to run continously and Idea is not supporting it very well.

I made simplyer change, which starts scalac each time, but just in incremental compilation mode on requested files. It should be still faster then normal scalac, since not all files are parsed. It is first shot, I would like to know your comments.

Fully integrate continous incremental compiler brings lot of problems, I will not have time to do it. But I would provide advice if someone would like to try.

Are you sure scala.tools.nsc.interactive.RefinedBuildManager will still be supported for 2.8? If I recall correctly, Miles said that they dropped this feature from the compiler as part of the 2.8 tooling because it was too complex and had numerious DRY violations.

Hi Jan,

I had a look at the RefinedBuildManager in detail. There's a few sticking points.

1. It does it's own dependency analysis. I'm pretty sure IDEA does it's own and only passes the necessary files for compilation anyway. However, there's probably a variation on it that just accepts files it needs to compile
2. It needs to run permanently. It could possibly run within IDEA itself. However, you'd need at a minimum isolated class loaders and probably one per module (and extra ones per test module). This would eat memory, but seems feasible.

In the mean time, I'll go back to finishing off the fsc implementation. Unfortunately, the current plugin trunk depends on unreleased core code.   Also, I'm away for a break this weekend, so it'll probably be a couple of weeks before I release it.

Sorry for broken compilation with 90.94 platform version, I hope it will not repeat again (I mean it will be synchronized with current IDEA releases, not with trunk).

IDEA has own error analyze, so best ideology to not use compiler (only to make/run your application) and fix all errors on a fly. Unfortunately, it's far from implementing this in IDEA, so you should use compiler more often. So It's good IDEA to have fast compiler, but I'm not sure that it's good idea to use experimental compiler features and run it permanently.

Best regards,
Alexander Podkhalyuzin.