Why does Intellij 8 keep reparsing Java library class files ?

Whenever I restart my Vista Premium SP1 system, Intellij 8 will reparse all Java library class files whenever I open any project. This is time-consuming and unnecessary. I don't have this problem with Intellij 7 but I see this consistently happen in Intellij 8. Is this a bug or deliberate working of Intellij 8 ? The reparsing takes more than a minute if the Java libraries used are large.

On what circumstance other than new project does reparsing of Java library activated ?

19 comments
Comment actions Permalink

this should not happen

did IDEA shutdown unexpectedly? any entries in idea.log?

0
Comment actions Permalink

May I know where the log file is located ? I'm not able to find it in intellij directories or in intellij dir under User directory (Vista only).

Can someone explain what type of event will trigger reparsing of Java library files ? Intellij crash ? project crash ? Is it based on some status of log file ?

thank

0
Comment actions Permalink

In XP, the log file is at C:\Documents and Settings\YourUsername\.IntelliJIdea80\system\log\idea.log

0
Comment actions Permalink

I'm also experiencing this problem.

I've attached the log file but I can't see anything that might indicate why the rescanning takes place.



Attachment(s):
idea.log
0
Comment actions Permalink

Me too, but sometimes. This must be release bug.

0
Comment actions Permalink

I am also seeing this (under Ubuntu 8.04) - but only after IntelliJ had to be killed (it hung for some other reason) - as Yann suggests. I would, however, expect that the cache was not invalidated just because of that.

0
Comment actions Permalink

Everytime IDEA crashes (which is only 1-2 / day for me) it seems to clear some sort of cache which causes everything to get reindexed, including the JDKs. Seems a bit overkill to constantly reindex the JDKs but haven't figured out how to stop it.

0
Comment actions Permalink

Hello k,

Everytime IDEA crashes (which is only 1-2 / day for me) it seems to
clear some sort of cache which causes everything to get reindexed,
including the JDKs. Seems a bit overkill to constantly reindex the
JDKs but haven't figured out how to stop it.


There is no way to stop it, because after a crash the indexes can be in an
inconsistent state, and it's not possible to determine that only some of
the files should be reindexed.

Why does IDEA crash for you? Do you get HotSpot crashes?

--
Dmitry Jemerov
Development Lead
JetBrains, Inc.
http://www.jetbrains.com/
"Develop with Pleasure!"


0
Comment actions Permalink

There is no way to stop it, because after a crash the indexes can be in an
inconsistent state, and it's not possible to determine that only some of
the files should be reindexed.


I find also this behavior quite annoying. I have to kill IDEA sometimes (3/4 times a week), either because it hangs, either because system hangs. It's also painful when you debug plugins and you stop debugging process using "Stop" button because you are used to ;)

JDK indexing is quite long (5 minutes for me) and, as user, it's difficult to understand why it should be reindexed since you don't touch JDK files (not usually).

Isn't there a way to keep JDK index separated from project files index ?

Sylvain

0
Comment actions Permalink

Like Sylvain above, about half the time it is because I have to kill (mac "Force Quit") it from some sort of hang ("Not Responding"). The other times it just disappears. I've gotten rid of 3rd party plugins but no help. Sometimes it is during editing, sometimes during VCS syncs. I've tried to switch to eclipse but keep coming back. I've never lost data so it is just an annoyance.

0
Comment actions Permalink

Hello Sylvain,

>>

>> There is no way to stop it, because after a crash the indexes can be
>> in an
>> inconsistent state, and it's not possible to determine that only some
>> of
>> the files should be reindexed.

I find also this behavior quite annoying. I have to kill IDEA
sometimes (3/4 times a week), either because it hangs, either because
system hangs. It's also painful when you debug plugins and you stop
debugging process using "Stop" button because you are used to ;)

JDK indexing is quite long (5 minutes for me) and, as user, it's
difficult to understand why it should be reindexed since you don't
touch JDK files (not usually).

Isn't there a way to keep JDK index separated from project files index
?


No, there is no way, and we don't plan to work on separating them. We'd much
rather work on optimizing the indexing and fixing the hangs/crashes.

--
Dmitry Jemerov
Development Lead
JetBrains, Inc.
http://www.jetbrains.com/
"Develop with Pleasure!"


0
Comment actions Permalink

On the whole I like IntelliJ Idea. Were a head to head to be done with an Eclipse user versus an Idea user, I think the Idea user would demonstrate they are far more productive, and the quality of the code would be far better. But while v.8 adds some rather nice features, it has gotten noticibly more unstable; it frequently hangs, frequently hits 100% CPU, and there are other oddities that are new. The issue cited here in this thread is one such example...

Because some brilliant person had the idea to replace the multitude of cached information spread across several files in v.7 (and earlier) with one big honkin file in v.8. (see .IntelliJIdea80\system\caches\attrib.dat.storageData), now if anything "bad" should happen in the product the whole world has to be reparsed and reindexed. Brilliant, isn't it? Personally I get at least two hangs per day, and it reparses the world at least once every thirty minutes (which can take up to 5 minutes to reparse for more complex environments).

0
Comment actions Permalink

It would have been a smarter idea, if they wanted to have one big honkin file, to store flip-flops, timestamps, and crc's adjacent to the stored data. Then:

  • a timestamp check could verify whether or not the resource had been updated, and do NOT do a reparse if it had not
  • the flip-flops could determine whether or not the whole index or cache had been written out to disk
  • the crc's could be used on an individual record/segment basis which could allow two things:
    • verification that the record is intact and not corrupted
    • allows recovery should the record/segment be intact


This would not be too hard to implement. And incrementally loading and merging index data should be straight forward. On the whole it should make the user experience more satisfactory.

0
Comment actions Permalink

It happens to me on my Ubuntu box when I have to kill IntelliJ. However, unlike other people in this thread IntelliJ very rarely hangs on me (twice since 8.x has been released). Both times it has happened it was during a local debug session of a web app running in Weblogic.

Besides for those two occasions IntelliJ 8.x is quite stable for me.

I have actually had Ubuntu itself hang on me more than IntelliJ (desktop Linux is better than working in Windows but has nothing on Mac OS). Having to power down the system without being able to close IntelliJ properly also resulted in a reindex of the JDK.

0
Comment actions Permalink

This is still a problem and I see that there hasn't been any activity on this post for a couple of months. There were several suggestions, such as splitting up the cache files, using crc/timestamps indexes to know which part was corrupt, etc.  I wouldn't have a problem if you just did this reindexing in the background. Let me view/edit my files while it loads everything up. We all understand that IntelliSense won't be opporational until it's finished but at least let us work. It takes 5 minutes to reparse everything, this is too long! I understand that "crash" is often not related to IDEA; however, loss of productivity should be your concern. Let the user get back to work as fast as possible. Eclipse doesn't have this problem, IDEA 7 didn't have this problem. Are you telling me there isn't anything that you can do?

0
Comment actions Permalink

IntelliJ IDEA 9 (Maia) EAPs do have background indexing.

0
Comment actions Permalink

This is a huge issue. I was having hangs about once a day in 7.x so I thought I would try Intellij 8... Now the hangs happen about every 30-60 minutes (100% cpu). Then it takes forever to startup once I have killed it and restarted it. This is a huge kill for productivity and I need to justify the upgrade to version 8. I tried the milestone of version 9 and still had the hangs every 30-60 min.

-Ryan

0
Comment actions Permalink

I've been running over a day without issues and wanted to throw this out there for other people having issues with IntelliJ hanging at 100% CPU. I am running 64bit Ubuntu and Sun's 64bit VM because I sometimes run apps that need over 2G of RAM. I decided to enable the CPU/Mem snapshot buttons and to switch to a 32-bit VM and I have not had any issues. I think the issue is that a 64bit VM uses a lot more memory for object pointers (atleast without pointer compression) and also runs in -server mode by default. I was probably running out of heap even with -Xmx1G (and GC was spinning out of control). With the 32bit VM my heap usage is much lower.

Good luck!
Ryan

0
Comment actions Permalink

Similar experience to Ryan L's post:

I was running IntelliJ 8.1.3 on Ubuntu 8.04 32-bit quad core 4Gb with 32-bit JVM and it was very stable.
I installed IntelliJ 8.1.3 on Ubuntu 8.04 64-bit i7 6Gb with 64-bit JVM and it become very unstable (several times per day pinning one core to 100% and hanging).
Auto-thread dumps were empty.  Clearing the 1.2Gb disk cache had no improvement. idea.log showed no errors.

However running intelli from the console revealed the cause of the hang was insufficient PermGen space.  I increased it and IntelliJ became stable again.

0

Please sign in to leave a comment.