Optimizing Dual/Quad Core Workstation for IDEA 7.0

Optimizing Dual/Quad Core Workstation for IDEA 7.0
-



During Selena EAP, I was interested that all long running tasks in IDEA
should maximize the use of dual core and quad core workstations since
dual and quad core machines are mainstream developer's desktop today.

I had questioned why certain operations like Find Usages of all methods of
the class didn't use 100% of both cores, and someone from Jetbrains said that
it was because of the disk access to IDEA's indices in system/caches directory.

This got me thinking if there was any system level solution to speed up IDEA,
and when I looked in my system/cache directory I found that it was only around
240 MB. Plus I have a system/js_caches and system/jsp_related_caches but those are tiny.

What I decided to try was create a RAMdisk to hold the system/cache directories,
and then link to it using a symbolic link (called Junction in Windows XP).

After I did this, when I do a big "Find Usages" search, IDEA no longer accesses my disk
drive. Almost any operation in IDEA, except compilation, is now CPU bound, so it should
maximize the use of your dual or quad cores.

I noticed a speedup in startup of IDEA and also when doing Find Usages.
I haven't had time to quantify the speedup, though. I only have a dual core
workstation. I would expect a quad core workstation would benefit more
from this system optimization.

The RAMDrive software I use supports image save/restore, so after you
set it up, you don't even know you are using a RAMDrive because the
RAMDrive image is preserved between reboots.

If you have a quad core and try this out, please let us know how big a
difference it makes.

-Alex

Here are the steps I followed
-


0. Prerequisites: Windows XP, NTFS filesystem. Plenty of RAM (4GB recommended) but
would work with less.

1. Check out your .IntelliJIDEA70 settings directory and find out how big
system/caches and system/js_caches (if present) and system/jsp_related_caches (if present),
etc.

2. Download RAMDisk "Enterprise (full)" Version 5.3.1.* for Windows XP.
trial software from http://www.ramdisk.tk/.

Install the RAMDisk software.

Download NTFS Link Shell extensions from
http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html

Install the NTFS Link Shell extensions. This adds some context menu commands
in the Windows Explorer to easily allow you to create junctions and hard links
in the filesystem.

Reboot.

3. Configure RAM Disk

To configure, goto Device Manager, and select RAMDrive Enterprise (x86)

a. Set Drive Letter
b. Set Disk Size larger than your estimated size of the caches, plus allow some room to grow.
My caches were about 220 MB, and I used a ram drive size of around 400 MB.
c. Press Configure File Processing to setup an image file to be used to intialize the contents of the ramdrive
at startup plus save the contents of the ramdrive on shutdown/reboot.
d. Reboot

4. You should now have a RAMDrive at a drive letter, e.g. G:
Copy .IntelliJIdea70/system/{cache,js_caches,jsp_related_caches,etc.} to G:


5. Delete the original .IntelliJIdea70/system/{cache,js_caches,jsp_related_caches,etc.}

6. For each subdirectory, e.g. cache, js_caches, etc., you moved, you need to

a. Right click on folder in the RAMDisk and select 'Pick Link Source'
b. Goto .IntelliJIdea70 and right-click on system directory and select Drop as... Junction


You should now have have symbolic links (junctions) setup from .IntellijIDEA70/system/
to the RAMDrive.

You can try rebooting your system. If you setup the Image File correctly for the RAMDrive,
then the filesystem on the RAMDrive will be saved to the image file before the machine shuts
down. Then, when you log back onto the machine, you should see the contents back in the RAMDrive.

7. You're done! IDEA caches are now all in memory.
Run IDEA and see (1) is startup any faster and (2) try a long "find usages" search and see if it is faster.



Attachment(s):
ramdrive_properties.png
ramdrive_filesystem.png
pick_link_source.png
dropas_junction.png
7 comments

Well, you didn't answered the main question - does Find Usages of all methods of the class use the 100% of all available cores now?

If not, then you did fixed something, but not the original problem.

0

I was experimenting with RAM disk and haven't noticed any major
performance improvements on XP and Mac. However I don't have big
projects. Also cache directories can be quite big. In my case - 878Mb.

Btw, quad core processor(Q6600) is not always faster than dual core(E6850).

-sergiy

0

Hi,

I'm sorry I don't have some quantitative results. I did this one day over Thanksgiving break, and right now I don't have any more time to work on it. I thought I'd go ahead and post what I did since it seemed to have speedup up IDEA and maybe someone else, esp. someone with a QUAD core workstation, could try it out for us.

The three main tasks I'm interested in are
1) syntax/inspections parse after loading a file into Editor pane.
2) Find usages of methods of a class
(Find Usages is used as part of many refactoring also so this one
is very important.)
3. Running Analyze->Inspect code.

Even before the RAMDrive, task #1 syntax and inspections parse appeared to use almost 100% of both cpus. This I would expect because it doesn't have to heavily access the indexes since syntax parse only concerns the file and the inspections parse within the Editor disables all global inspections.

For #2, with the RAMDrive I am seeing 70-100% cpu utilization on a couple long find usages I just did. I didn't want to mention that though, because I haven't done any
rigorous tasting, so take that with a very big grain of salt.

I have been seeing Find Usages not fully utilize both of my cpus, and so I was trying out this RAMdisk to try to eliminate Disk I/O as a cause. I have a suspiciion that there mayb e an inefficiency of how IDEA divides the subtasks may not keep both cpus busy.

I was hoping Jetbrains would do some of there own tests to show the performance gains while running IDEA on dual core or quad core, since I think that would be a powerful marketing message for professionals, but I haven't seen anything from Jetbrains on the matter. I just hope that Jetbrains engineers are all running QUAD CORE workstations; If you give them the horsepower, I'm sure they'll figure out a way to harness it!

-Alex

0

Sergiy> I was experimenting with RAM disk and haven't noticed any major
Sergiy> performance improvements on XP and Mac. However I don't have big
Sergiy> projects.

The only areas the RAMDrive made faster to me is startup and doing find usages. I suppose other tasks like Analyze Dependencies would be faster too, but I haven't run those yet.

I did notice a speed boost, but Your Mileage May Vary (YMMV) depending on your system. If you don't have lots of free memory, then reserving memory for the RAMDisk could cause your system to thrash the page file, which could end up slowing your system. In my case, I had 4 GB memory, and I wasn't using all of it.

Sergiy> Also cache directories can be quite big. In my case - 878Mb.

You might try wiping out your cache directory and letting it rebuild. My cache directory was originally 500 MB, but when I rebuilt it was only 224 MB. I think there were alot of unused indices from old projects or copies of my projects and/or due to EAP cruft.

Sergiy> Btw, quad core processor(Q6600) is not always faster than dual core(E6850).

Right, but that E6850 is at a higher clock speed. I agree if you are buying a new box, I would prefer higher clock speed dual core over slower quad cores, but for the same clock speed, I would expect QUAD core to run IDEA faster for these long tasks like find usages, analyze tasks, etc. as long as IDEA can split the work into 4 tasks. Since all of these tasks I'm talking about are read only, there shouldn't be much synchronization or locking problems, so I would think all threads should be able to run at full speed. I suppose memory contention could be a limiting factor.

Tom's hardware always has good CPU comparisons. I like to look at the Application Performance charts, e.g.
http://www.tomshardware.com/2007/11/19/the_spider_weaves_its_web/page30.html
WIith AMD introducing their Phenom QUAD core chips, I expect QUAD core chips to get much cheaper next year.

0

Instead of using a RAMDrive which eats into your system memory, a really nice solution might be to use a Flash based Hard Drive or for best performance an SDRAM Drive like this HyperDrive 4 reviewed here:

http://www.tomshardware.com/2007/11/07/hyperdrive_4_redefines_solid_state_storage/page12.html

I read the review of the Hyperdrive 4 SDRAM drive, and that thing is like greased lightning!

The SDRAM drives are still very pricey, but Flash based Solid State Hard Drives have dropped in price. You can get a 32 GB flash based hard drive for several hundred dolalrs.

Flash based hard drives are very low latency for random access reads, which seems like a perfect match for IDEA accessing it's index files. But the flash drives are slower than hard drives for write access. It would be interesting to have a RAID 0 array of 4 flash based hard drives. That would be lightning fast for read accesses
See this article for more on Flash based drives:
http://www.tomshardware.com/2007/08/13/flash_based_hard_drives_cometh/

I think a internal SATA Flash drive or SDRAM drive could be the ideal solution if your IDEA cachefiles are very large, i.e. more than 500 MB. Might be another year or two before the price gets cheap enough, though. If I built a new box today, I would still go with a SATA or SAS 10k drive.

0

I, for example do run Quad core, and have found that my quad core (Q6700)
scored 1.5-2.0 times better than my coworker's dual-core E6850 on Inspect
multiple files and Find Usages tasks.

--
regards,
Alexey Kudravtsev
Software Developer
JetBrains, Inc, http://www.jetbrains.com
"Develop with pleasure!"

"Alex" <no_reply@jetbrains.com> wrote in message
news:4397136.1196154701070.JavaMail.itn@is.intellij.net...

Hi,

>

I'm sorry I don't have some quantitative results. I did this one day over
Thanksgiving break, and right now I don't have any more time to work on
it. I thought I'd go ahead and post what I did since it seemed to have
speedup up IDEA and maybe someone else, esp. someone with a QUAD core
workstation, could try it out for us.

>

The three main tasks I'm interested in are
1) syntax/inspections parse after loading a file into Editor pane.
2) Find usages of methods of a class
(Find Usages is used as part of many refactoring also so this one
is very important.)
3. Running Analyze->Inspect code.

>

Even before the RAMDrive, task #1 syntax and inspections parse appeared to
use almost 100% of both cpus. This I would expect because it doesn't have
to heavily access the indexes since syntax parse only concerns the file
and the inspections parse within the Editor disables all global
inspections.

>

For #2, with the RAMDrive I am seeing 70-100% cpu utilization on a couple
long find usages I just did. I didn't want to mention that though,
because I haven't done any
rigorous tasting, so take that with a very big grain of salt.

>

I have been seeing Find Usages not fully utilize both of my cpus, and so I
was trying out this RAMdisk to try to eliminate Disk I/O as a cause. I
have a suspiciion that there mayb e an inefficiency of how IDEA divides
the subtasks may not keep both cpus busy.

>

I was hoping Jetbrains would do some of there own tests to show the
performance gains while running IDEA on dual core or quad core, since I
think that would be a powerful marketing message for professionals, but I
haven't seen anything from Jetbrains on the matter. I just hope that
Jetbrains engineers are all running QUAD CORE workstations; If you give
them the horsepower, I'm sure they'll figure out a way to harness it!

>

-Alex



0

You might also want to look at SMP Seesaw (http://www.mlin.net/SMPSeesaw.shtml). Info from the site:

SMP Seesaw is a utility for dual-CPU or dual-core computers that controls how Windows balances the compute load between the two processors. By default, Windows balances the processing load to both CPUs as evenly as possible among all programs. SMP Seesaw provides a convenient means to change the processor affinities for all running programs at once. In particular, the most common use is to dedicate one of your processors to a single program, thereby improving the performance or responsiveness of that program.

While the regular version is for dual-core systems, there is also a "Pro" version (also free) that supports up to 32 CPUs and has more options, and a SDK.

I only recently came across this utility, and have not had a chance to try it out, (and may not get a chance to for a while). But it might be useful to improve performance.
If anyone's tried it, please let us know your experience.

0

Please sign in to leave a comment.