Intellij Idea JVM options benchmark: defalut settings are worst

Hi, All.

I benchmarked VM options for Idea. Let me share my results.


Test: Startup + open big project > 100k classes

Environment: WinXp, jre 1.6.26, Xeon 2Gh

Tested VM options:

  1. Default (-Xms128m -Xmx512m)
  2. Default "enhanced" -Xms768m -Xmx768m
  3. UseParallelOldGC
  4. UseParNewGC
  5. UseConcMarkSweepGC UseParNewGC
  6. UseConcMarkSweepGC UseParNewGC NewSize=128m



All Parallel GCs share common part -Xms768m -Xmx768m -server -XX:ParallelGCThreads=8 (8 because of cpu cores)

      

Results:

Time to start + open project:

1)       Default => 36 secs

2)       Default+Xms768m => 37 seconds

3)       Any parallel config ~24 seconds


GC charts (less is better for all):
overhead.png
Max.png
Min.png
Note: ParNewGc and default show good min results because of low newSize on startup~40m, which is grows with time, so shouldn't be considered valid.

stats.PNG
Numbers are better than comments:
stats-compare.PNG

Conclusion and advices:

1) shipped settings are worst of all and should be refused for multicore processors.

2) Setting just large Xmx brings even larger pauses to application with small overhead win and impacts responsivness.

3) roughly, the best suited GC is CMS. Taking into account it could shorten major GC pauses, i suggest to use these settings (ParallelGCThreads depends on CPU):

-Xms768m

-Xmx768m

-XX:MaxPermSize=250m

-XX:ReservedCodeCacheSize=64m

-server

-XX:NewSize=128m

-XX:+UseParNewGC

-XX:ParallelGCThreads=8

-XX:+UseConcMarkSweepGC

-XX:CMSInitiatingOccupancyFraction=60

Any questions and suggestions ae very welcome. GC logs could be found in attachement.

regards, Alex



Attachment(s):
idea-gc-test.zip
6 comments

Hi Alexander,

Let me thank you for spotting the problem place and your effort in analysis. We should definitely look into that.

I've created corresponding ticket - IDEA-79172, expecting to close it for 11.1 release (2012 Q1/2).

Regards, Denis

0

Attached idea-gc-test.zip contains the data collected from unspecified IntelliJ Idea version, surely not IntelliJ IDEA 11 since the file is the same as the one you attached couple of monthes ago.
Also is not clear what plugins are enabled and whether your tests cater for cold (just after restart) and warm file system state.
E.g. I have 11s of startup / opening project with default gc / client VM settings for IDEA project and 13s of startup / opening project with your settings in warm file system state (I opened project three times) and
no other CPU activity on the system

Message was edited by: Maxim Mossienko

0

Hi, Maxim. I evaluated settings with idea 11 EAP.

0

Is not clear what plugins are enabled in your configuration and whether your tests  cater for cold (just after restart) and warm file system state.
E.g.  I have 11s of startup / opening project with default gc / client VM  settings for IDEA project and 13s of startup / opening project with your  settings in warm file system state (I opened project three times) and
no other CPU activity on the system.

0

Hi, Maxim.
Did you capture gc logs? If you could kindly share it, i'd like to take a look. Your results are strange for me. By general consideration, you should be faster having faster gc. and that's what i saw in my benchmarks. Probably yor machine has a less core processor, so 8 gc threads just create some contention overhead (optimal number is -XX:ParallelGCThreads=<#cpus < 8  ?  #cpus  :  3 + ((5 * #cpus) / 8) >) Meanwhile, i will try version 11 and provide my results.

The previous benchmarks were done on idea 11 eap in default configration. The system was up and running for a long time.

Thank you.

0

Hi, Maxim.
I checked my settings with version 11. I got:

GC Settings Start+Open project + finish background tasks Restart + rebuild
Custom 23-24 sec 3m28sec
Default 28-29 sec 4m47sec



Custom settings was:
-Xms768m
-Xmx768m
-XX:NewSize=128m
-XX:MaxNewSize=128m
-XX:+UseParNewGC
-XX:ParallelGCThreads=8
-XX:MaxTenuringThreshold=1
-XX:SurvivorRatio=8
-XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled
-XX:CMSInitiatingOccupancyFraction=65
-XX:+CMSScavengeBeforeRemark
-XX:+UseCMSInitiatingOccupancyOnly
-XX:MaxPermSize=250m
-XX:ReservedCodeCacheSize=64m
-ea

0

Please sign in to leave a comment.