Source Look-up Path for "Navigating from Stacktrace Sourcecode"

According to the documentation here: http://www.jetbrains.com/idea/webhelp/navigating-from-stacktrace-to-source-code.html
it's possible to jump directly from the stacktrace to the line in your source. This works perfectly fine for the majority of the time, but I've noticed an issue in a specific situation.

I have two modules in my project: my main module which I run 95% of the time, and a small module I just added to test one small aspect. Both modules have a Main class in the same package.
I have two different run configurations, each set to run one of the Main.main methods with the classpath correctly set.

The problem occurs when I try to click on a link to Main.java:xx which is in my test module. For some reason, it opens the Main.java class in my main module rather than the one in my test module.

How does IntelliJ find the .java file it's looking for? Can I change the order of the look-up paths somehow?