Performance in build 5561

I know you guys are probably tired of hearing this, but there are still performance issues with Demetra, especially when opening/closing projects.

We have a very large project that is actually a suite of products. There are a lot of dependancies so we can't easily break them up. Roughly 10,000 classes, a couple of thousand jsp's and a thousand or so resource files.

In IDEA 5.1 ( after caching ) our project takes about 15 - 20 seconds to open.
In Demetra it takes 50 - 70 seconds.

Closing the same project in 5.1 takes about 3 seconds.
In Demetra it takes 35! ( It no longer throws an exception on close, but it still locks up the UI for the duration ).

21 comments
Comment actions Permalink

But it seems to work much faster than 5557, though.

0
Comment actions Permalink

Ditto for beta build 5594.

Is anyone else seeing these issues with opening/closing times being substantially longer than in IDEA 5.1? Anyone else working with really large projects? It would probably be more noticeable on huge projects. ( i.e. all my colleagues see this as well so it's not something with just my files )

0
Comment actions Permalink

Chris,
I've been working around closing performance issue, and hope to have fixed it. Please see the next build.

Eugene.

0
Comment actions Permalink

Thank you. I'll be eagerly anticipating the next build. We have multiple branches of our project in our source control and swapping back and forth is a bit of a pain because of the long load and close times. I look forward to checking out the next build and I'll let you know if I still see any problems. ;)

0
Comment actions Permalink

I just got the latest build ( 5622 ) and I see no improvement in opening or closing of our project. The times remain almost identical to those posted earlier.

0
Comment actions Permalink

Could you please upload profiler snapshots for both opening and closing the project?

0
Comment actions Permalink

Do you mean a before/after kind of thing? I'm assuming that the snapshot doesn't record a process such as open/close so I would have a pre-open snapshot, open snapshot, and after close snapshot, right?

0
Comment actions Permalink

Chris Hampton wrote:

Do you mean a before/after kind of thing? I'm assuming that the snapshot doesn't record a process such as open/close so I would have a pre-open snapshot, open snapshot, and after close snapshot, right?


That's true for memory snapshots but for CPU profiling you start the
profiler, do whatever you want to do, and then stop the profiler which
gives you a single snapshot of what happened when the profiler was active.

0
Comment actions Permalink

While this is certainly somewhat hackish, you can capture project loading snaphot by first loading a project, then unloading it, then starting profile session and loading it again.

0
Comment actions Permalink

To profile startup performance, add the line
-agentlib:yjpagent=cpu=times
to your idea.vmoptions and then press the CPU snapshot button twice when
the project has loaded (first time will give an error saying that CPU
recording is already started, second time will capture the snapshot).
R

0
Comment actions Permalink

Chris Hampton wrote:

I just got the latest build ( 5622 ) and I see no improvement in
opening or closing of our project. The times remain almost identical
to those posted earlier.


Here, on the other hand, projects now open with lightning quickness. In
previous EAP builds, project loading was definitely slower than with
IDEA 5.1.2, but in #5622 it is faster! Opening a project of about 3500
java files which took 23 seconds with 5.1.2, with #5622 only takes 13
seconds. Great work.

Bas

0
Comment actions Permalink

The CPU snapshots for opening and closing our project should now be uploaded ( if my ftp worked correctly ). The files are named:

5622_champton_open_17.08.2006_11.05.47.zip
5622_champton_close_17.08.2006_11.07.32.zip

0
Comment actions Permalink

Bas Leijdekkers wrote:

Here, on the other hand, projects now open with lightning quickness. In
previous EAP builds, project loading was definitely slower than with
IDEA 5.1.2, but in #5622 it is faster!


Same for me. I've made no measurements but my current project
definitely opens much more quickly with 5622. Great improvement indeed.

--
Mark Scott
mark@codebrewer.com

0
Comment actions Permalink

Great to hear someone noticing the optimizations we've been doing for the past two weeks. Thank you!

0
Comment actions Permalink

The snaphots show that more than 30 sec are spent in closing RandomAccessFile where IDEA stores its caches. Haven't you installed IDEA on a network drive that is likely to cause this and is highly discouraged?

0
Comment actions Permalink

No. I installed IDEA in the default install directory on my local drive. The installation for the beta is identical to my 5.1 install. The caches are in my Documents and Settings folder and IDEA is in Program Files\JetBrains.

0
Comment actions Permalink

Try disabling virus checking on your cache directory.

No. I installed IDEA in the default install directory on my local
drive. The installation for the beta is identical to my 5.1 install.
The caches are in my Documents and Settings folder and IDEA is in
Program Files\JetBrains.



0
Comment actions Permalink

Hey! Good call. That was it. Who'd a thunk it? What's weird is that my IDEA 5.1 doesn't suffer from this and it has a way bigger cache with more files...

Unfortunately I can only temporarily turn off my virus scanning. This is configured thru our domain and I have no ability to configure this and turn off scanning of individual directories. :(

Any ideas on why there's such a big difference between 6.0 and 5.1?

0
Comment actions Permalink

I'm glad that helped although doesn't sound like good news that you can't
permanently exclude that directory.

I've noticed that some virus checkers are a lot worse than others. Which
are you using? In my experience Symantec Antivirus is one of the better behaved
scanners in these sorts of situations, others I have tried over the years
have caused a much greater impact on performance.

No idea why 6.0 is worse than 5.1, but I assume it's because the cache access
patterns have changed significantly, and in a way that sends some virus checkers
into a scanning frenzy.

Chris

Hey! Good call. That was it. Who'd a thunk it? What's weird is that
my IDEA 5.1 doesn't suffer from this and it has a way bigger cache
with more files...

Unfortunately I can only temporarily turn off my virus scanning. This
is configured thru our domain and I have no ability to configure this
and turn off scanning of individual directories. :(

Any ideas on why there's such a big difference between 6.0 and 5.1?



0
Comment actions Permalink

We are using Symantec Antivirus. I don't know what the settings are but I wouldn't be surprised if they are cranked to the max. For some reason our IT department is paranoid about things like that... Even if I disable it, it re-enables after about 5 minutes or so. Must have a thread watching it.

I have to believe there are other companies where the developers are under similar restrictions. I can only hope that 6.0 gets some more optimization with virus scanners in mind...

0
Comment actions Permalink

Is there anywhere that IS excluded by default? If so you could move your
cache there by setting idea.system.path. Otherwise it sounds like you're
running low on options, it seems to me that the fault really lies with the
antivirus software and your IT department rather than JetBrains.

Regardless, I wish you luck in finding a solution. I've been in similar situations
before and I know it's not much fun :(

We are using Symantec Antivirus. I don't know what the settings are
but I wouldn't be surprised if they are cranked to the max. For some
reason our IT department is paranoid about things like that... Even
if I disable it, it re-enables after about 5 minutes or so. Must have
a thread watching it.

I have to believe there are other companies where the developers are
under similar restrictions. I can only hope that 6.0 gets some more
optimization with virus scanners in mind...



0

Please sign in to leave a comment.