startup problem with big projects

I'm using IDEA 4.5.4 (build 2253) and struggle with serious startup problems. When starting IDEA it takes about 10' (or more) until the "loading project" window appears, thereby making any interaction with other programs impossible. The scanning of files then takes from a couple of minutes to (guess) 20' depending on whether the project was updated from CVS before or not.

When looking for possible problems I found that the idea_home/system/vcs directory consumed more than 100MB. I switched off local histroy, restarted idea and cleaned the vcs directory. But IDEA took extremely long to start up again, thereby again building up the vcs directory.

My questions now are:
1. Is the vcs directory related to the local history? If so, why does IDEA build it up when switched off; if not, what are these 100MB for and how can I disable it?
2. Is there any relationship at all between the slow start up time and these 100MB?
3. What else can I do to boost the startup time?

Thanks in advance
pat

BTW:
Some rough numbers on the project I'm working on: the project consists of about 300 components whereof 130 components are IDEA modules, most of the others just libraries. The 300 components correspond to about 45k files (including resource files, excluding CVS files), the 130 IDEA modules result in 20k class files.

10 comments
Comment actions Permalink

Hello Pat,

Lisser wrote:

1. Is the vcs directory related to the local history? If so, why does
IDEA build it up when switched off; if not, what are these 100MB for
and how can I disable it?


It is related to local history, but also needed for the inline change
markers you see when you have version control enabled in IDEA.

2. Is there any relationship at all between the slow start up time
and these 100MB?


I would say hardly. If you delete the vcs directory, IDEA starts up much
slower though, because the dir is reconstructed.

3. What else can I do to boost the startup time?


I usually exclude resource files from the IDEA project if I don't need
to access them from within IDEA. That way they also don't get stored in
the vcs directory.
I have also increased the -Xms setting because IDEA always needs more
memory after loading my projects than 32 MB and reallocating takes time.
You should probably also increase the -Xmx setting for IDEA with a
project that size, if you haven't already.
By the way: What platform are you on? How much memory does it have?

Bas

0
Comment actions Permalink

Please check that IDEA's system directory is on the local drive and that
you are not using network mapped drives for project and system paths.

In case you have Antiviral Monitor enabled, check that it's not monitoring
IDEA system and project folders.

NTFS hard links may also cause similar issues.

Could you please specify your hardware configuration?

--
Serge Baranov
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


0
Comment actions Permalink

Hello Bas

2. Is there any relationship at all between the
slow start up time and these 100MB?


I would say hardly. If you delete the vcs directory,
IDEA starts up much
slower though, because the dir is reconstructed.


yep, I thought IDEA wouldn't do that as I switched off local history - but I was wrong.

3. What else can I do to boost the startup time?


I usually exclude resource files from the IDEA
project if I don't need
to access them from within IDEA. That way they also
don't get stored in
the vcs directory.


Good point, I could do that (will me take some time to go through those 130 modules though)

I have also increased the -Xms setting because IDEA
always needs more
memory after loading my projects than 32 MB and
reallocating takes time.
You should probably also increase the -Xmx setting
for IDEA with a
project that size, if you haven't already.


I start IDEA with -Xms16m -Xmx256m. I can set -Xms higher, the -Xmx should be ok - at least I don't get OutOfMemoryExceptions.

By the way: What platform are you on? How much memory
does it have?


WindowsXP, 1GB RAM

Thanks for anwering so fast. I'll try the -Xms setting and the exclusion of resources.

Cheers
pat

0
Comment actions Permalink

Hello Serge

I'm using IDEA on WindowsXP, everything on the only (local) drive, all IDEA directories (as far as I know of them) are excluded from antiviral monitoring, and no hard links neither.

Only using "subst" to associate a drive letter with the directory where all the project files reside. But that shouldn't be a problem, should it?

Thanks
pat

0
Comment actions Permalink

Only using "subst" to associate a drive letter with the directory
where all the project files reside. But that shouldn't be a problem,
should it?


It actually can be the problem. IDEA's native FileWatcher library doesn't
work well in such cases. Please try whether it loads faster without substed
drives or try to move/delete IDEA_HOME\bin\FileWatcher2k.dll for a moment.

--
Serge Baranov
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


0
Comment actions Permalink

I tried these changes, to no avail:

- increasing -Xms option to 128m in idea.lax
- excluding all resources
- switch off antiviral monitor
- direct path instead of 'subst'-ituted

Actually it got even worse after the latest update of our project: IDEA didn't get to the "Loading Project" window within an hour. Maybe some recent change (in inter-module dependencies or something else) exceeded some threshold.

Now I'll try an EAP version (rumors tell that it starts better in huge project settings).

Thanks to Bas and Serge anyway
pat

0
Comment actions Permalink

Hi Pat,

Sorry to hear the performance didn't improve.

Lisser wrote:

Now I'll try an EAP version (rumors tell that it starts better in
huge project settings).


I'm curious if you will see any improvements. Please report back.

Do you use any plugins by the way? And it still could be IDEA doesn't
have enough memory. Perhaps you could try to run it with the bat file
and specify -verbosegc in the IDEA_JVM_ARGS. Then you can see if it is
garbage collecting all the time instead of doing something useful.

Good luck,
Bas

0
Comment actions Permalink

You are saying that -Xmx set to 256m should be OK. My experience, however, is that setting it higher may improve performance. I have 1GB RAM and -Xmx set to 512m - this works for me better than 256m.

0
Comment actions Permalink

Lisser wrote:

I tried these changes, to no avail:

- increasing -Xms option to 128m in idea.lax
- excluding all resources
- switch off antiviral monitor
- direct path instead of 'subst'-ituted


I don't know if it remains an issue, but in the past I've noticed that
module dependencies and libraries can significantly affect start up
time. What I try to do now is make sure all my modules have the absolute
minimum libraries and dependencies necessary. A common problem people
have is to have one big 'lib' folder with all their jars in it, and then
for each module, they just add all the jars to the classpath
(libraries). This caused excessive/unnecessary class parsing on start-up
(and resyncing, if I'm not mistaken). Especially, big jar files can be a
problem as well. If you have one big jar file, but you only depend on a
few interfaces in it (for example), then it's often a good idea to
pre-package a smaller jar with just those interfaces, and only add that
to the libraries. You can use Ant to do this kind of work once, whenever
libraries are updated.

--
Rob Harwood
Software Developer
JetBrains Inc.
http://www.jetbrains.com
"Develop with pleasure!"

0
Comment actions Permalink

Hi Rob

I assume the problem is exactly because of these dependencies. Unfortunately I have no control about those.

Anyway, using the EAP build was much of an improvement in that respect (not in others :)

thanks
pat

0

Please sign in to leave a comment.