IntelliJ 8.0: more woes

All,

I really hate to have to do this, but I have to downgrade to 7.04 because 8.0 has been more of a hinderance than an improvement in productivity :-( Indexing / deindexing files is extraordinarily slow on OS X (haven't tried on another OS).

Just today:

  • Had to kill IntelliJ because, after changing a file type association, IJ started re-indexing lots of things. This seemed to complete, but after some point IntelliJ just started taking 100% of CPU while being entirely unresponsive. I had to force-quit it. Restarting it took probably 20 mn of re-indexing files while I could do nothing.
  • Just now, and what finally prompted my downgrade, I cmd-tabbed to IntelliJ, and a re-index seems to start again for no good reason. Given the progress bar indication, it seems that it's going to take 1/2 hour to go through the files affected.


Am I the only one encountering this kind of issues?

-Erik

21 comments
Comment actions Permalink

yes, i wish there were a way to exclude project directories from being indexed w/o having to exclude them from the project.

for example, i run an ant script that generates a lot of xml which shows up in the project -- after running the script it spends a minute or so indexing/reindexing the content.  i don't really care about indexing it for search purpose, but i still want to access it.

if i "exclude" the directory in the module properties it is fine, no indexing.  but i also can't get to those files from with IDEA either.

0
Comment actions Permalink

I see a lot of reindexing too, but the thing that really gets me is when I'm debugging and I click on another application running in Windows. When I click back onto IDEA, it starts taking over 50% CPU, and doesn't give me an edit cursor for a minute or so. When overall system memory is short (e.g. I'm running a VMWare database), IDEA will eventuall give me the 'Out of Memory' dialog when I click back to it (after about half a minute of lock-out). If I 'Ignore' this warning, and eventually its memory usage drops back again, and everything is fine until the next time I click on anpther application and back to IDEA.

Quite what it is doing when I click back to it, I don't know - IDEA gives no indication of background processes, but Task Manager or Process Explorer show lots of CPU and memory use.

0
Comment actions Permalink

Me too.  I have a really small j2ee project and idea just died.  Over 1gig of memory used and still clocking away.  Had to kill it.  This really sucks!

0
Comment actions Permalink

Please try the latest 8.0.1 EAP build: http://www.jetbrains.net/confluence/display/IDEADEV/Diana+EAP .
Check that IDEA heap is not too high. Otherwise the OS may be lacking free memory and move IDEA application to the swap space when you switch to another application. When you switch back, IDEA will be loaded back from swap being unresponsive during this time as described by Dave.

Dave, what are your IDEA heap settings? How much physical memory do you have? How much physical memory is free before you start IDEA?

0
Comment actions Permalink

Quite what it is doing when I click back to it, I don't know - IDEA gives no indication of background processes, but Task Manager or Process Explorer show lots of CPU and memory use.

I suppose that IntelliJ, upon getting focus again, is detecting what files have changed on disk, so it can update its indexes. This was no fast in 7.x, but it seems slower in 8.x on OS X.

The thing is, while developing, you constantly switch back and forth between windows: IntelliJ, browser, console, logs, you name it.

-Erik

0
Comment actions Permalink

Yes, it can be the case. You should avoid external changes in IDEA project roots. Directories with logs or output from the external tools should be set as Excluded in the module Paths configuration.

0
Comment actions Permalink

serge wrote:

Yes, it can be the case. You should avoid external changes in IDEA project roots. Directories with logs or output from the external tools should be set as Excluded in the module Paths configuration.

That's understood. But please note that the point of my original message is that these operations, in particular "de-indexing" (just making assumptions about how IntelliJ works here) seems to have become much slower than with version 7, to the point of being absolutely unbearable (hence my reverting back to version 7). In addition, there seems to be funny bugs that lock up the app entirely.

I can't rule out that there is something "wrong" or unusual with my particular setting, but at this point my opinion is that there is a regression in performance and stability here.

-Erik

0
Comment actions Permalink

My IDEA VM Options:

-Xms512m
-Xmx1024m
-XX:MaxPermSize=128m
-ea

If I set Xmx to less than 1024m, IDEA 8 runs out of memory loading the project, if I set it to more, IDEA simply won't run at all.

My machine has 4GB of physical memory, but AIUI under WinXP it can't use more than 3.5GB for applications.

Before I run IDEA, the machine shows about 1.47 GB available memory (Process Explorer). After the project is loaded, PE shows about 500MB available memory.

Your explanation about paging to swap space makes sense, as these problems seem particularly severe when I'm running my VMWare database, which does take a fair bit of memory.

But I'm stuck between a rock and a hard place - the machine & OS won't take more than 4GB RAM, I must have the VMWare database running, and IDEA will only work (albeit intermittently) with  1024MB.

I have disabled or removed all the plugins I don't need to use, so I'm open to suggestions for further memory reduction options.

Version 7 of IDEA seemed to use considerably less memory without problems, so unless I can find a way to get IDEA 8 working reliably, I will have to revert back to v7 or try another IDE, such as Eclipse :-(

0
Comment actions Permalink

Hello Erik,

If you see a lock-up of IntelliJ IDEA, you need to take a thread dump as
described in http://support.jetbrains.com/kb/entry!default.jspa?categoryID=4&externalID=260&fromSearchPage=true

This is the only way for us to see why the lock-up happened and what can
be done to fix it.

--
Dmitry Jemerov
Development Lead
JetBrains, Inc.
http://www.jetbrains.com/
"Develop with Pleasure!"


0
Comment actions Permalink

Do you use the latest 8.0.1 EAP build? If not, please try it. What's the size of the project? How many files/Mbytes? If it still runs out of memory with 512Mb heap, a profiler memory snapshot may help to diagnose the problem.

When running a lot of heavy applications on the development machine you should consider installing more RAM as it's not very expensive. Of course you will also have to install 64-bit OS.

0
Comment actions Permalink

yole wrote:

Hello Erik,


If you see a lock-up of IntelliJ IDEA, you need to take a thread dump as
described in http://support.jetbrains.com/kb/entry!default.jspa?categoryID=4&externalID=260&fromSearchPage=true


This is the only way for us to see why the lock-up happened and what can
be done to fix it.

Thanks will try to do this next time.

-Erik

0
Comment actions Permalink

I'm now using the latest 8.01 build.

The project is quite large, 2 modules with a total of about 4000 source files, about 25MB in size, about 20,000 of resource files about 30MB in size, and about 80MB of libraries. There's also test sources & resources.

I tried loading the project with 512MB heap, but as I expected, it threw up the 'Out Of Memory' dialog. I tried ignoring it, but after 15mins using 50% CPU, and no apparent progress, I killed it.

I'm now trying it with 768MB, and I'll let you know what happens.

It's not easy to install more RAM when you're working in a controlled environment - none of our machines has 64-bit Windows and it hasn't been 'approved' by our systems people. Our machines can't physically take more than 4GB memory, and they've recently been upgraded. So however cheap memory is, the only way I could get more is to buy a new machine and OS myself and work from home!

0
Comment actions Permalink

Nope, ran out of memory again. I'm going back to 1024MB at least that would load the project and work for a while...

0
Comment actions Permalink

IDEA should not take that much memory, please, upload zipped hprof
memory dump to our ftp per intructions in
http://jetbrains.net/devnet/docs/DOC-192 (you will need to connect to
running IDEA instance from YourKit profiler and set option of capturing
memory dump on low memory condition)

Dave Lorde wrote:

Nope, ran out of memory again. I'm going back to 1024MB at least that would load the project and work for a while...

---
Original message URL: http://www.jetbrains.net/devnet/message/5227754#5227754



--
Best regards,
Maxim Mossienko
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Comment actions Permalink

OK Maxim, but those instructions are for taking a CPU snapshot, they don't say anything about a memory dump or capturing on low memory - how do I do that?

0
Comment actions Permalink

Try running IDEA with this option:
-agentlib:yjpagent=usedmem=70

0
Comment actions Permalink

Dave Lorde wrote:

OK Maxim, but those instructions are for taking a CPU snapshot, they don't say anything about a memory dump or capturing on low memory - how do I do that?

---
Original message URL: http://www.jetbrains.net/devnet/message/5227784#5227784


Dave,

If you have a 1.6 JDK installed, you can use the jmap tool:
jmap -dump:file=idea.prof process_id_of_idea


-tt

0
Comment actions Permalink

If it runs out of memory, wouldn't adding the option -XX:+HeapDumpOnOutOfMemoryError to idea.vmoptions be enough to create a usable heap dump?

Bas

0
Comment actions Permalink

OK, I got a couple of memory dumps. I ran IDEA 8.01 with 512MB max heap and took a memory dump at the first 'Out Of Memory' dialog using jmap -dump. I then 'Ignore'd the dialog and continued loading. I 'Ignore'd a second  'Out Of Memory' dialog, and then 'OK'd an error message box that said "Could not save project. Java heap space" (odd when the project was actually loading).

After about 15 mins, IDEA had opened the project and I clicked on the 'Memory Dump' button and got a dump. Not long after, IDEA just vanished.

I am trying to FTP these two dump files to ftp://ftp.intellij.net/.uploads/.

0
Comment actions Permalink

Providing snapshot names would not hurt :)

Dave Lorde wrote:

OK, I got a couple of memory dumps. I ran IDEA 8.01 with 512MB max heap and took a memory dump at the first 'Out Of Memory' dialog using jmap -dump. I then 'Ignore'd the dialog and continued loading. I 'Ignore'd a second 'Out Of Memory' dialog, and then 'OK'd an error message box that said "Could
not save project. Java heap space" (odd when the project was actually loading).

After about 15 mins, IDEA had opened the project and I clicked on the 'Memory Dump' button and got a dump. Not long after, IDEA just vanished.

I am FTPing these two dump files to .


---
Original message URL: http://www.jetbrains.net/devnet/message/5227818#5227818



--
Best regards,
Maxim Mossienko
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Comment actions Permalink

I sent an email to support with the details, as instructed in the link...

The files are:

The dump taken at the first 'Out Of Memory' dialog using jmap -dump is called:     dlorde_idea_8.01 memory_dump.zip.
The dump taken with the 'Memory Dump' button after the project loaded is called:     9164_dlorde_28.11.2008_11.25.29.zip.

0

Please sign in to leave a comment.