I have several libraries that are only needed for running my unit tests. For example, my MockObject jars.
Since IDEA goes out of its way to differentiate the portion of my project containing Test classes from Src classes, I think you should also be able to mark a library as for test use only. If there is a way to do this now, I can't find it.
The inability to make this distinction causes my libraries panel to be a bit cluttered and I end up with extraneous JARs in my build that shouldn't be there.
The 4.x version of NetBeans actually handles this quite well by defining build, run, and test classpaths or libraries.