The ultimate inspection .. just a dream ??
the Use Case :
-
You received a huge piece of code to maintain, that you know is
partially covered by tests.
CPD, and IDEA's inspections help you locate a very hot spot : a 600 line
method, full of duplication and poorly written code. You see crap, you
smell crap, you locate crap.
You are ready to jump in, with IDEA's scalpel, but you wonder :
- is this piece of code tested, so I can safely "fix" it,
and be told if I broke anything?
This is where the "ultimate feature" comes handy.
the "I had a dream" part:
-
With the cursor in the method you want to edit,
you push on ONE BUTTON, and IDEA does 3 things :
It
1/ creates a dynamic test suite with all the tests that exercise the
current method
2/ runs this test suite, watching the code usage, to build and ..
3/ displays the code coverage result of the method, by the tests
=> in green , all the code that will be used by the tests
=> in red , all the code that will NOT be used by the tests
=> in yellow, all the code that will MAYBE be used by the tests
This info would tell you
- where it's safe to refactor, and
- where you need to add some test before refactoring/fixing
the point 1/ has a related feature request :
"run only the tests that exercise the class/method .."
http://www.intellij.net/tracker/idea/viewSCR?publicId=8344
Just an idea, ..
Alain
Please sign in to leave a comment.
Look at the product JTest (the chunk I didn't shamelessly rip off for InspectionGadgets) from more on 1), and of course Clover for 2) and 3). Give it a few years of people getting used to the power of refactoring IDE's, and I don't doubt we'll be getting close to your vision.
Test induction, similarity checking, metrics, dependency analysis, code visualization, and even bug injection are all technologies that could really fly if built into a supple, syntax-aware IDE.
--Dave
How do you generate a test suite that knows what the code is supposed to do, and where problems might lie? I don't think you can.
Keith Lea wrote:
>
I never wrote that. Reread my original post :
>> 1/ creates a dynamic test suite with all the tests that exercise
the current method
Alain
Your test generator doesn't know what the code is supposed to do, merely what it actually does do. By analyzing the code to determine all possible execution paths, and reverse engineering input sets to exercise them all. If you're really slick, you'll include knowlege of Design-By-Contract stuff (preconditions, postconditions, invariants), and tailor your input sets to model those as well. JTest (www.parasoft.com) ships a commercial implementation of these ideas. It's pretty sweet. It wouldn't surprise me at all to see IDE's with that sort of technology integrated ship within the next three-to-five years.
In addition similarity analysis and inspections, I would add a good dependency analyzer/visualizer and metrics suite to your "find the crap" toolset. The US Army has a saying: "If you can see it, you can hit it. If you can hit it, you can kill it." IDEA's design-flaw killing toolset is about as good as can be. There's room for improvement, however, in integrating tools for sensing and identifying bugs and design flaws. My gut feel is that integrating those technologies with a refactoring IDE will have far greater payoff than having such tools stand-alone.
How do you generate a test suite that knows what the code is supposed to do, and where problems might lie? I don't think you can.
Yes, you're right there. But the idea is to test paths through the code. It doesn't really have to know whether the code is doing the right thing or not; as long as it comes up with a test suite (and suggestions for test data?) that really hammers it.
If you use reflection there is no way of knowing which tests executes the current method.
Wouldn’t it be a lot simpler and safer to just run all tests?
Sincerely
Jens Peter Grosen
Jens Peter Grosen wrote:
>> 1/ creates a dynamic test suite with all the tests
>> that exercise the current method
>>
>If you use reflection there is no way of knowing which
>tests executes the current method.
It's not about running the tests, its about the system telling us :
"I can guarantee that this fragment of code is exercised by tests"
I drive slower in the dark. Don't you?
> Wouldn't it be a lot simpler and safer to just run all tests?
What if the full test suite takes 10 minutes, and you just want to
tune/optimize one regexp routine, for which the whole test suite,
scattered over 4 classes, takes 2 seconds?
If it takes 2 seconds, I'll run the test every minute.
If it takes 10 minutes, I'll run the test every hour.
Don't you think it's important to test more often?
Alain
>If you use reflection there is no way of knowing which >tests executes the current method.
Sure there is. You're IDE runs the tests under a code coverage tool, which caches the test<->method mapping. Not the case yet, but certainly within the realm of the possible.
-Dave
Dave Griffith wrote:
Or JProbe Coverage[1], which doesn't require preinstrumentation of your
source code, whereas I believe Clover does.
Yes, I work for Quest, I gotta plug our products ;)
Ciao,
Gordon
1. http://www.quest.com/jprobe/
--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919
You guys have planned the sort of IDEA integration that Clover does? That'd be sweet, especially on the profiling side of your product.
Dave Griffith wrote:
JProbe 5.0 already integrates with IDEA 3.0 in the Run dialog. I don't
know how Clover integrates or how it compares to JProbe's integration.
Ciao,
Gordon
--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919