Synchronize-performance observation

I've always been wondering why even with the native FileWatcher, the
synchronization (manual and on frame re-activation) is so slow. I learned to
live with it, but now I found out that if I select all root directories in the
project view and invoke "Synchronize selected files" action from the context
menu, the action finishes in no time (btw: the warning that this may take a long
time is simply wrong - at least compared to the time it takes when invoking the
Synchronize-action from the toolbar, which doesn't even show a warning).

So I had a look at the CPU snapshots and found that all the time is spent inside
java.io.File.lastModified() - isn't it the FileWatcher's job to avoid such
explicit up-to-date checks?

All those calls come from JarFileSystemImpl.a(JarFileInfo, boolean, boolean), so
this seems to be related to refreshing JAR files, not regular files. However, if
a JAR file itself doesn't change, I think it's safe to assume that the files it
contains haven't changed as well.

With that observation I think there's a lot of room for optimization - unless I
got something wrong. In that case I'd be interested in understanding what this
is: it would make it easier to accept the slowness ;)

Sascha

11 comments

Sascha, I would be very much interested in how many jar files are attached to your project?

0

Hello Eugene,

the project I tried this with has no jars attached except those used by an IDEA-JDK (+ idea.jar). The synchronization is actually slow with all my projects and I had to turn off synchronization on frame activation long ago.

I experience this on different Win XP SP2 machines, no network drives, and everything being pretty much standard.

Should I upload the CPU snapshots to your FTP server?

Sascha

0

To actually answer in numbers: The IDEA-JDK references about 60 jars and 4 source-zips. However, when changing to a J2SE JDK with about 10 jars and 1 zip, the synchronization is still as slow as before.

0

OK, I found out that is seems to be related to the number of jars that are known by IDEA in general, i.e. in all configured JDKs and libraries. This is indeed quite a large number and synchronization is significantly faster in "clean" installations.

Actually this makes sense in the way that IDEA also needs to synchronize those files, but I still don't understand why File.lastModified() is called at all. Maybe the FileWatcher doesn't watch non-project directories?

Anyway, I think it would be great to be able to use the automatic synchronization just for files under project roots. This could be significantly faster and catch about 95% the cases where files usually change: inside the project.

Sascha

0

Yes, please. Though I have some impression on what code could that actually be, getting some proof would not hurt.

0

And yes, the work to optimize it has already been started out.

0

I just uploaded 6068_sweinreuter_27.10.2006_12.20.25.zip to ftp.intellij.net/pub

I hope it's useful for you.

Sascha

0

No, I don't see it. Please upload to ftp.intellij.net/.uploads

0

Oops, I had the FTP client not configured properly and it went to a different server. ;)

Should be there now.

Sascha

0

Yep, it is. The slowdown is there where I thought.I've implemented a fix, and in case of successful remote run it'll get to 6.5 branch.

0

Cool, I'm looking forward to it ;)

0

Please sign in to leave a comment.