New JUnit Integration

Hi,

I like the new GUI version of the JUnit integration - it looks nice.

But, for quick testing I personally prefer text output which tells me ok or !ok. For quick testing, I dont need
execution times, memory deltas and nice looking icons.

Would it be possible to add an option to choose between
text and graphical output?

Also, would it be possible to export the statistics and test results to html (on suite basis, test case basis and test basis)?

Please, add a coloum to the statistics view to show if the test passed or not. Or make the columns configurable?

Anonther thing I found counter intuitive is the lack of navigating possibility to the test by double clicking the test name in the statistics view. Also, when I click on a test in the statistics view, I would like to see the test highlighted in the tree view of the test runner. Probably this could be done by a button "Autoscroll to Source".

What do you think about these suggestions or are they already planned for aurora?

/Bj?rn

4 comments


"bkueb" <jiveadmin@jetbrains.com> wrote in message
news:14017131.1047376475363.JavaMail.jrun@is.intellij.net...

Hi,

>

I like the new GUI version of the JUnit integration - it looks nice.

>

But, for quick testing I personally prefer text output which tells me ok

or !ok. For quick testing, I dont need

execution times, memory deltas and nice looking icons.

>

Memory details are essential when you look at it and wonder "What 1Mb was
used for!?"

There are some requests, see
http://www.intellij.net/tracker/idea/viewSCR?publicId=10161
http://www.intellij.net/tracker/idea/viewSCR?publicId=10150
http://www.intellij.net/tracker/idea/viewSCR?publicId=10127
is there any good replacement? If not than vote
http://www.intellij.net/tracker/idea/viewSCR?publicId=10124
It is not hard to put dots back, but it will make UI more complex, so I'm
looking for good altenative, and I hope it will server not only quick
testing.

Would it be possible to add an option to choose between
text and graphical output?

>
Do you want to see just dots, no tests tree?

Also, would it be possible to export the statistics and test results to

html (on suite basis, test case basis and test basis)?
>
Please, submit a request.

Please, add a coloum to the statistics view to show if the test passed or

not. Or make the columns configurable?
>
Yes, now statistics table in ugly rendered, I'm working on it.

Anonther thing I found counter intuitive is the lack of navigating

possibility to the test by double clicking the test name in the statistics
view. Also, when I click on a test in the statistics view, I would like to
see the test highlighted in the tree view of the test runner. Probably this
could be done by a button "Autoscroll to Source".
>
But, I sometimes want to double-click suite to get in and see its subtests
statistics... I'm thinking about it.

What do you think about these suggestions or are they already planned for

aurora?
>

/Bj?rn



0

As I have not yet installed 802, a question for those who have:

We have in our unit-test suite 4000+ tests. We tend to use the textui
for running since it can run all the tests in 6 seconds. With the
swingui, it takes well over 30 seconds.

- Is the new idea runner more like the swingui in performance?
- Can the textui still be used?

thanks

0

Ok, I'll anwer my own questions:

- You cannot seem to use the textui

(Correction: Our server-side unit tests, which have just over 1500 tests
runs in 6 seconds within idea using the textui. That is what I use for
the comparison below.)

My system is a dual P4 1.8MHz with 1.5G ram, with the idea.lax set
to 512M/512M.

The configuration for running our tests with the textui was imported
from 3.0.2, and had the following VM parameters: -Xms128m -Xmx512m.

Scenario 1:
- Idea is told to run a single test; our version of testall that
collects all tests in tedh classpath with a specified class name
prefix.
- When the tests completed, the idea runner indicated an elapsed
time of 8.935s. However:
- It took 6-7 seconds to populate the tree of tests
- While the tests are running, idea is completly unresponsive.
Can't even keep up to paint itself. (ie, Iconize, restore, and idea
stays grey for tens of seconds.)
- The wall time for running the test suite was well over 3 minutes.
(I didn't bother bringing up a stopwatch since the wall time is
orders of magnitude off the textui.)
- After idea did come back, one of the test cases and test methods
still had the yellow animating icon. (I assume this means the test
is being run, but other aspects of the runner indicate that all
is finished.)

Scenario 2:
- A test configuration for a package is defined (our top
level package). The VM parameters are left at 128/512.
- Responsiveness of idea is about the same; mostly a grey
screen. But sometimes you can click around, but its like:
click on menu, wait 3-4 seconds, get grey box, wait 2-3
seconds, get painted menu, ...)
- I couldn't get a timing from the idea runner, because I was
ever able to complete all the tests. After every attempt, at
least one test would appear to run forever. I could not find
a way to tell the idea runner to stop this test (ie, there is
no stop button like in the debugger, on an X button that kills
the panel.)
- Wall time was even worse letting idea discover the tests: well
over 7 minutes to run the all the tests.
- I had some failures for tests that are not selected by our
'testall' suite. It would be nice if the idea runner could
have the ability to specify some criteria for the class names
that are selected.

Senario 3:
- Identical setup to 2, expect I cleared all the VM parameters
- Nearly identical results.


The look of the runner is quite nice. Once it gets close to performance
of the textui it will be very nice. I'm not sure what we will do if the
performance never gets up there. Even with the the nice UI, we run tests
so often that we would rather just use the textui if its to bee so slow.
Will using the textui ever be an option again?


Barry Kaplan wrote:

As I have not yet installed 802, a question for those who have:

We have in our unit-test suite 4000+ tests. We tend to use the textui
for running since it can run all the tests in 6 seconds. With the
swingui, it takes well over 30 seconds.

- Is the new idea runner more like the swingui in performance?
- Can the textui still be used?

thanks


0

The look of the runner is quite nice. Once it gets close to performance
of the textui it will be very nice. I'm not sure what we will do if the
performance never gets up there. Even with the the nice UI, we run tests
so often that we would rather just use the textui if its to bee so slow.


We are working on this.

Will using the textui ever be an option again?


It's not decided yet. Definitely yes, if we won't get our runner's
performance near the same as one of the text one.

---
Valentin Kipiatkov
JetBrains, Inc
http://www.intellij.com
"Develop with pleasure!"


"Barry Kaplan" <bkaplan@abwg.com> wrote in message
news:3E6DFB09.4010505@abwg.com...

Ok, I'll anwer my own questions:

>

- You cannot seem to use the textui

>

(Correction: Our server-side unit tests, which have just over 1500 tests
runs in 6 seconds within idea using the textui. That is what I use for
the comparison below.)

>

My system is a dual P4 1.8MHz with 1.5G ram, with the idea.lax set
to 512M/512M.

>

The configuration for running our tests with the textui was imported
from 3.0.2, and had the following VM parameters: -Xms128m -Xmx512m.

>

Scenario 1:
- Idea is told to run a single test; our version of testall that
collects all tests in tedh classpath with a specified class name
prefix.
- When the tests completed, the idea runner indicated an elapsed
time of 8.935s. However:
- It took 6-7 seconds to populate the tree of tests
- While the tests are running, idea is completly unresponsive.
Can't even keep up to paint itself. (ie, Iconize, restore, and idea
stays grey for tens of seconds.)
- The wall time for running the test suite was well over 3 minutes.
(I didn't bother bringing up a stopwatch since the wall time is
orders of magnitude off the textui.)
- After idea did come back, one of the test cases and test methods
still had the yellow animating icon. (I assume this means the test
is being run, but other aspects of the runner indicate that all
is finished.)

>

Scenario 2:
- A test configuration for a package is defined (our top
level package). The VM parameters are left at 128/512.
- Responsiveness of idea is about the same; mostly a grey
screen. But sometimes you can click around, but its like:
click on menu, wait 3-4 seconds, get grey box, wait 2-3
seconds, get painted menu, ...)
- I couldn't get a timing from the idea runner, because I was
ever able to complete all the tests. After every attempt, at
least one test would appear to run forever. I could not find
a way to tell the idea runner to stop this test (ie, there is
no stop button like in the debugger, on an X button that kills
the panel.)
- Wall time was even worse letting idea discover the tests: well
over 7 minutes to run the all the tests.
- I had some failures for tests that are not selected by our
'testall' suite. It would be nice if the idea runner could
have the ability to specify some criteria for the class names
that are selected.

>

Senario 3:
- Identical setup to 2, expect I cleared all the VM parameters
- Nearly identical results.

>
>

The look of the runner is quite nice. Once it gets close to performance
of the textui it will be very nice. I'm not sure what we will do if the
performance never gets up there. Even with the the nice UI, we run tests
so often that we would rather just use the textui if its to bee so slow.
Will using the textui ever be an option again?

>
>

Barry Kaplan wrote:

As I have not yet installed 802, a question for those who have:

>

We have in our unit-test suite 4000+ tests. We tend to use the textui
for running since it can run all the tests in 6 seconds. With the
swingui, it takes well over 30 seconds.

>

- Is the new idea runner more like the swingui in performance?
- Can the textui still be used?

>

thanks

>

>


0

Please sign in to leave a comment.