Unexpected Performance Issue with Initial Load
I routinely clear the Aurora caches out between EAP's (after getting burned a few times from not doing so during the early ones).
While waiting for the necessary pre-parse of all the Java files and libraries in my project (it's big, but not absurdly so, mostly because of things like weblogic.jar), I started playing with the width of the progress window. Having randomly made it as wide as possible while it was parsing 4,000 or so "input" classes (generated XML wrappers), I noticed that it was taking a LONG time to parse the package, so I shrunk it to as narrow as possible. Having done so, the "outputs" package, which is pretty much equivalent, parsed significantly faster (at least 2x). So, I left the window as narrow as it would let me, and the whole reparse, which has consistently taken 20+ minutes for at least the last 2 months of EAP's, completed in something much nearer to 10 minutes.
Granted, to show that this actually makes a difference, I'd have to clear the caches and time it both ways, but would much rather play with the new EAP today. :) Has anyone noticed anything similar?
--Mike Werle
Please sign in to leave a comment.
Top tip Mike, it is definitely quicker with a narrow dialog.
"Michael Werle" <mkwerle@yahoo.com> wrote in message
news:23297357.1066683603647.JavaMail.itn@is.intellij.net...
burned a few times from not doing so during the early ones).
>
libraries in my project (it's big, but not absurdly so, mostly because of
things like weblogic.jar), I started playing with the width of the progress
window. Having randomly made it as wide as possible while it was parsing
4,000 or so "input" classes (generated XML wrappers), I noticed that it was
taking a LONG time to parse the package, so I shrunk it to as narrow as
possible. Having done so, the "outputs" package, which is pretty much
equivalent, parsed significantly faster (at least 2x). So, I left the
window as narrow as it would let me, and the whole reparse, which has
consistently taken 20+ minutes for at least the last 2 months of EAP's,
completed in something much nearer to 10 minutes.
>
the caches and time it both ways, but would much rather play with the new
EAP today. :) Has anyone noticed anything similar?
>
You are right indeed.
My guess is this is related to all the text updating that is not done (at least on the screen) when the dialog is narrow.
I wondered about this a while ago. Are they attempting to display the name of EVERY file? Wouldn't it be better (quicker) to adopt a different strategy; for instance updating the progress bar every half second or so with the file currently being parsed instead?
Or just showing the .jar file name.
I do not believe it can depend on the width of the dialog. If anyone wishes
to make an experiment (with exact timing!) it would be interesting to see
the results.
--
Valentin Kipiatkov
JetBrains, Inc
http://www.intellij.com
"Develop with pleasure!"
"Michael Werle" <mkwerle@yahoo.com> wrote in message
news:23297357.1066683603647.JavaMail.itn@is.intellij.net...
burned a few times from not doing so during the early ones).
>
libraries in my project (it's big, but not absurdly so, mostly because of
things like weblogic.jar), I started playing with the width of the progress
window. Having randomly made it as wide as possible while it was parsing
4,000 or so "input" classes (generated XML wrappers), I noticed that it was
taking a LONG time to parse the package, so I shrunk it to as narrow as
possible. Having done so, the "outputs" package, which is pretty much
equivalent, parsed significantly faster (at least 2x). So, I left the
window as narrow as it would let me, and the whole reparse, which has
consistently taken 20+ minutes for at least the last 2 months of EAP's,
completed in something much nearer to 10 minutes.
>
the caches and time it both ways, but would much rather play with the new
EAP today. :) Has anyone noticed anything similar?
>
OK. Attached is a jar with 1015 very simple classes in it. All of this form:
Complete the new project wizard to create a new project with no source, no libraries, just a default jdk.
Copy the attached jar file twice... When I did this I renamed it one.jar, two.jar, three.jar. These three jar files can all be in the same directory.
Go to Project Properties -> Main Module -> Libraries.
1) Add Named library and add one.jar... Click Apply and time the indexing (for me: 20 seconds)
2) Add Named library and add two.jar... click Apply and start timing, resize dialogue asap and time till finish (for me: 7 seconds)
3) (Control check) Add Named library and add three.jar... Click Apply and time the indexing (for me: 20 seconds)
So, it is significantly quicker with a smaller dialogue...
Attachment(s):
one.jar
Valentin Kipiatkov (JetBrains) wrote:
I really think it does. My project takes a long time to reparse fully.
With the dialog almost as wide as the screen so I could see the full
filenames, I counted 12 seconds for 1%. With the dialog as narrow as
possible, I counted 4 seconds for 1%. I have repeated this numerous
times within the same parsing with consistent results.
Ciao,
Gordon
--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919
Gordon Tyler wrote:
>> I do not believe it can depend on the width of the dialog. If anyone
>> wishes
>> to make an experiment (with exact timing!) it would be interesting to see
>> the results.
Think about it. When you have something that is 5 inches wide, 1% of that
looks larger than 1% of something that is only 1 inch wide. So you might
extrapolate that there are more files processed in the larger 1% than the
smaller one.
The only true test is the length of time to process ALL the files. If
all the files parse quicker with a smaller window than a larger one, then
there is something else to extrapolate. If that truly is the case, then
the Java Swing drawing code is smart enough to skip a redraw command if the
critical area has no change (you only see part of the path with a smaller
window). However all the drawing is supposed to be done in one thread,
which means we might be skipping some entries being displayed anyway. As
long as there is no synchronization code going on with the Swing progress
bar, there should be no appreciable difference.
Berin Loritsch wrote:
>> I really think it does. My project takes a long time to reparse fully.
>> With the dialog almost as wide as the screen so I could see the full
>> filenames, I counted 12 seconds for 1%. With the dialog as narrow as
>> possible, I counted 4 seconds for 1%. I have repeated this numerous
>> times within the same parsing with consistent results.
I counted the 1% as indicated by the numeric percentage indicator on the
right side of the progress bar. I very much doubt the scale of that
number changes with the width of the dialog.
Agreed, but it takes about 10 minutes to reparse fully and I have work
to do ;)
Ciao,
Gordon
--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919
In article <17708269.1066730855192.JavaMail.itn@is.intellij.net>,
Ash Searle <no_mail@jetbrains.com> wrote:
Why not omit the filename completely? I don't care what it's doing; I
only care when it will be done.
I wonder if compilation would speed up too if there was less chatter on the
screen....(are we supposed to be able to read those filenames as they whizz
by or what?) C'mon JetBrains, there's only one way to find out - make the
changes and let us do the testing!
Colin Smith
"Valentin Kipiatkov (JetBrains)" <valentin@intellij.com> wrote in message
news:bn36uj$5e5$1@is.intellij.net...
wishes
>
>
progress
was
clear
>
>
I didn't want to enter this debate but the filename information
is as important as the progress bar graphics.
The only reasonable suggestion I saw here was to update the filename
only every XXX fraction of a second (was said 1/2 sec, I'd prefer 1/4).
r.
Ok, I gave up an hour of my life and tested it (well, not really -- thank God for TiVo ;) ). Here's what I did:
--Reboot
--Shut down all apps except my virus checker (eTrust EZ Antivirus)
--Deleted all files in the following direcories:
o system\caches
o system\compiler
o system\jars
o system\log
o system\stat
--Started IDEA and marked the time.
--Resized the progress window.
--Time stopped when IDEA finished loading.
Note that I followed the whole process each time so that no file caching silliness would distort the results. In each case, about 92MB of files were created in the system\caches subdirectory of the Aurora preferences.
When I resized the window to nearly as wide as my screen goes (1400 pixels), the parse took just under 47.5 minutes. When I resized to as narrow as it would let me do so, the parse completed in just under 10 minutes, nearly 80% faster!!!
Granted, it "felt" much faster, but the results very much surprised me given that part of the file name was shown even when the window was as small as possible.
Note that the vast majority of the time was spent parsing libraries, not our application code, a great part of which was weblogic.jar and webservices.jar from Weblogic 8.1.
I created issue 19476[/url] in the tracker.
Richard Nemec wrote:
Human persistance of vision is 1/10 of a second. As the theory goes,
as long as the image changes ten times a second, it will appear to be
smooth animation. Some practicalities come into play from early movies.
When the image was displayed 24 times a second, we perceived a flicker.
What modern films do now is repeat each frame once so that the effective
refresh rate is 48 times a second but the image changes only 24 times
a second.
Point is that if we want to almost read the filenames, I would not display
anything faster than five times a second (2/10 of a second). Otherwise it
will look like a blur.
We've just found some problem in the progress dialog so it really slowed the
process when repainting. It's fixed now. Thanks for pointing to this
problem!
--
Valentin Kipiatkov
JetBrains, Inc
http://www.intellij.com
"Develop with pleasure!"
"Gordon Tyler" <gordon.tyler@quest.com> wrote in message
news:bn3ipn$20i$1@is.intellij.net...
wishes
see
>
>
>
>
Glad to help! Issue 19476[/url] probably should be closed in the build for which this was fixed. :)
Thanks for such a quick turn-around! Hopefully, this fix will help with perceptions of IDEA's performance, which on the whole, is really quite excellent. :)
--Mike