Scanning project files is time consuming in Selena

In our team (where people use EAP builds) we see that scanning project files (after opening an *.ipr file) has become a much longer process. Do you want me to attach a CPU profiling data or logs ? Btw, the *.ipr file was generated by IDEA after opening once a pom.xml

18 comments
Comment actions Permalink

Oh, this gets better. Now when IDEA stucks with 95% CPU (happens from time to time) and I kill it - it rescans all libraries again when project is opened :( :(

0
Comment actions Permalink

Hello Evgeny,

Oh, this gets better. Now when IDEA stucks with 95% CPU (happens from
time to time) and I kill it - it rescans all libraries again when
project is opened :( :(


This is expected behavior. The alternative to a full rescan is leaving the
repository in an inconsistent state, with all kinds of unpredictable problems
happening later.

--
Dmitry Jemerov
Software Developer
JetBrains, Inc.
http://www.jetbrains.com/
"Develop with Pleasure!"


0
Comment actions Permalink

Ok, got it, thank you.
But, please, note that usual project scanning upon normal start-up has really become longer in the last couple of builds, as it seems to me.

0
Comment actions Permalink

On 2007-08-08 02:18:55 +0400, Evgeny Goldin <no_reply@jetbrains.com> said:

Ok, got it, thank you.
But, please, note that usual project scanning upon normal start-up has
really become longer in the last couple of builds, as it seems to me.


That's strange. We actually were working on speeding it up. Would you
please take a CPU snapshot of the startup?

0
Comment actions Permalink

Sure, will do. I've tried doing it a couple of times already - closing project makes idea.exe go 100% CPU right away as well. So I'll take snapshots for both opening and closing the project. May be it has something to do with ClearCase integration - last time I had performance problems during Search/Replace operation in the big textual file I was told it's due to CC.

Btw, I think the more exact way of saying what's happening on the start-up is that "IDEA does more than usual" which in result takes more time. I mean, I see the progress window appearing and disappearing a number of times (during which I can only think "Come on ! You've scanned it enough, aren't you ?" :) ), definitely more than I'm used to. Ok, I suppose the CPU snapshot will help - will take it tomorrow.

0
Comment actions Permalink

Two snapshots are uploaded to http://download.yousendit.com/4408B2D55CF91811 (couldn't attach them here) for opening and closing the project - both of them were taking 100% CPU.

0
Comment actions Permalink

On 2007-08-08 17:02:27 +0400, Evgeny Goldin <no_reply@jetbrains.com> said:

Two snapshots are uploaded to
http://download.yousendit.com/4408B2D55CF91811 (couldn't attach them
here) for opening and closing the project - both of them were taking
100% CPU.


Closing project spends all the time closing files. This is a very
definite sign of faulty antivirus software tracking IDEA's system
folder (Norton one, most probably).

0
Comment actions Permalink

On 2007-08-08 14:27:08 +0400, Maxim Shafirov <max@jetbrains.com> said:

On 2007-08-08 17:02:27 +0400, Evgeny Goldin <no_reply@jetbrains.com> said:

>> Two snapshots are uploaded to
>> http://download.yousendit.com/4408B2D55CF91811 (couldn't attach them
>> here) for opening and closing the project - both of them were taking
>> 100% CPU.


Closing project spends all the time closing files. This is a very
definite sign of faulty antivirus software tracking IDEA's system
folder (Norton one, most probably).


Same for project open

0
Comment actions Permalink

That's right, we have Symantec Antivirus v10.0.0.359 installed. I'll try excluding IDEA "system" folder from scanning. Thank you !

0
Comment actions Permalink

Beautiful ! After doing that I have project opening and closing in ~ 3 seconds now (down from ~ 30 seconds).

0
Comment actions Permalink

>> Closing project spends all the time closing files. This is a very
>> definite sign of faulty antivirus software tracking IDEA's system
>> folder (Norton one, most probably).
>
> Same for project open
>

These issues with the anti-virus software.... you'd rather place them in
the Tip Of The Day, always at first position , in the help
documentation on the top of each page and in some other places also
:):):) It is pretty often when people report slow IDEA and it happens
that the reason is AV sw :)

0
Comment actions Permalink

Good suggestion!

These issues with the anti-virus software.... you'd rather place them in
the Tip Of The Day, always at first position , in the help
documentation on the top of each page and in some other places also

0
Comment actions Permalink

On top of that suggestion (which is great, of course) I would also think of auto-detecting "hostile" software, like antivirus in general or Norton in particular - many products do so already ("Oh darling, you have ABC installed, would you like me to uninstall it for you ?"). Going further - IDEA may auto-detect them and exclude the "system" folder from scanning, if it's possible. If it's not - it may ask user to do so on the first start-up.

0
Comment actions Permalink

Yeah... It could time the project startup, and if the startup is too
long based on the size of the project it could ask you if you have your
virus software configured properly...

N.

Evgeny Goldin wrote:

On top of that suggestion (which is great, of course) I would also think of auto-detecting "hostile" software, like antivirus in general or Norton in particular - many products do so already ("Oh darling, you have ABC installed, would you like me to uninstall it for you ?"). Going further - IDEA may auto-detect them and exclude the "system" folder from scanning, if it's possible. If it's not - it may ask user to do so on the first start-up.

0
Comment actions Permalink

My "suggestion" for mentioning the AV software issues at the suggested
places was ironical :) and not really serious :)
I wanted to point out that it is pretty often case when the AV software
causes the IDEA to slow down and maybe it is good to think about some
place in the IDEA FAQs, helps and other documentation to point this out
to the users who, for example, don't read the newsgroups. Tip Of The Day
is also a possibility, of course :)

In my opinion, programming IDEA to detect this or that software is
expensive operation. One can never know what and why and where was
installed. Informing customers about the issue is cheaper.

Greetings to all!

0
Comment actions Permalink

But I still think it should be serious. I've run into this a couple of
times (usually when switching to the new EAP, as settings dir name
changes) and I know about the issue. It would be good for this
particular problem to be self-diagnosing because it is a particular
system configuration and not down to any problem with IDEA's performance.

Cheers,
N.

George Talev wrote:

My "suggestion" for mentioning the AV software issues at the suggested
places was ironical :) and not really serious :)
I wanted to point out that it is pretty often case when the AV software
causes the IDEA to slow down and maybe it is good to think about some
place in the IDEA FAQs, helps and other documentation to point this out
to the users who, for example, don't read the newsgroups. Tip Of The Day
is also a possibility, of course :)

In my opinion, programming IDEA to detect this or that software is
expensive operation. One can never know what and why and where was
installed. Informing customers about the issue is cheaper.

Greetings to all!

0
Comment actions Permalink

It may be expensive but, as I've said above, many products (firewalls/antiviruses mainly) do so already (to find out if any conflicting software is installed) and I suppose they have a good reason for it. So it's up to JetBrains to decide whether this situation of excluding the "systems" folder from antivirus scan is serious enough to spend the effort. Again, on my PC (Pentium 4, 2.93Ghz, 1.5Gb RAM) there was an enormous speedup ...

0
Comment actions Permalink

On 2007-08-08 21:40:08 +0400, Evgeny Goldin <no_reply@jetbrains.com> said:

On top of that suggestion (which is great, of course) I would also
think of auto-detecting "hostile" software, like antivirus in general
or Norton in particular - many products do so already ("Oh darling, you
have ABC installed, would you like me to uninstall it for you ?").
Going further - IDEA may auto-detect them and exclude the "system"
folder from scanning, if it's possible. If it's not - it may ask user
to do so on the first start-up.


If we'd only had a reliable way to diagnose precense of the
antivirus... The known issue is closing any files (random access, file
streams, channels, mapped files) slows down significantly, which we
just can measure and report to the user. But I'm afraid this is not the
only case antiviruses manifest themselves.

Max

0

Please sign in to leave a comment.