As of 2021.3, Tests hang when invoked from command line, but they run fine from IDE
I'm having trouble with some JUnit tests hanging forever (well over 24 hours) when I invoke them through the Gradle command line. If I run the same tests directly from IJ2021.3.2 in the IDE, they complete successfully in about 15 minutes.
NOTE: In IJ 2021.1.3, the tests ran fine under the command line and the IDE. So it looks like some change in 2021.2 or 2021.3 caused the problem.
I need these tests to run from a TeamCity job, automatically, upon commit, so I need to get them so they can run from the command line without hanging.
DIAGNOSIS, SO FAR
I attached a remote debug to the command line test run, and found the call in my code that never returns:
FindInProjectUtil.findUsages(model, getProject(), //<<HERE
usageInfo -> {
usages.add(usageInfo); // Never get here
return false;
},
new FindUsagesProcessPresentation(FindInProjectUtil.setupViewPresentation(true, model)));
By pressing pause on the remote debug session, I get a stack trace that differs in the last few calls each time, but is basically stuck in FindInProjectUtil#findUsages() which is stuck in the findUsages() at line 225 doing
ProgressManager.getInstance().runProcess(runnable, new EmptyProgressIndicator());
The stack trace typically looks like this:
sleep:-1, Thread (java.lang)
sleep:18, TimeoutUtil (com.intellij.util)
lambda$processDequeOnAllThreads$1:69, FilesScanExecutor (com.intellij.openapi.roots.impl)
run:-1, 242882974 (com.intellij.openapi.roots.impl.FilesScanExecutor$$Lambda$1384)
runOnAllThreads:42, FilesScanExecutor (com.intellij.openapi.roots.impl)
processDequeOnAllThreads:59, FilesScanExecutor (com.intellij.openapi.roots.impl)
processFilesInScope:397, FindInProjectTask (com.intellij.find.impl)
findUsages:161, FindInProjectTask (com.intellij.find.impl)
lambda$findUsages$0:223, FindInProjectUtil (com.intellij.find.impl)
run:-1, 1990610061 (com.intellij.find.impl.FindInProjectUtil$$Lambda$1369)
lambda$runProcess$2:188, CoreProgressManager (com.intellij.openapi.progress.impl)
run:-1, 1512775028 (com.intellij.openapi.progress.impl.CoreProgressManager$$Lambda$682)
lambda$executeProcessUnderProgress$12:624, CoreProgressManager (com.intellij.openapi.progress.impl)
compute:-1, 577605263 (com.intellij.openapi.progress.impl.CoreProgressManager$$Lambda$687)
registerIndicatorAndRun:698, CoreProgressManager (com.intellij.openapi.progress.impl)
computeUnderProgress:646, CoreProgressManager (com.intellij.openapi.progress.impl)
executeProcessUnderProgress:623, CoreProgressManager (com.intellij.openapi.progress.impl)
executeProcessUnderProgress:66, ProgressManagerImpl (com.intellij.openapi.progress.impl)
runProcess:175, CoreProgressManager (com.intellij.openapi.progress.impl)
findUsages:225, FindInProjectUtil (com.intellij.find.impl)
findUsages:215, FindInProjectUtil (com.intellij.find.impl)
findFileWithAnyMarker:249, GosuMultiFileTestCase (gw.gosu.ij.testframework) << My call, as shown above
findFileWithMarkers:224, GosuMultiFileTestCase (gw.gosu.ij.testframework)
configureByDirectory:184, GosuMultiFileTestCase (gw.gosu.ij.testframework)
doTest:70, GosuMultiFileTestCase (gw.gosu.ij.testframework)
doTest:52, GosuMultiFileTestCase (gw.gosu.ij.testframework)
doTest:108, GosuGenerateConstructorTest (gw.gosu.ij.codeInsight)
doTest:104, GosuGenerateConstructorTest (gw.gosu.ij.codeInsight)
testNoMoreConstructorsCanBeGenerated:64, GosuGenerateConstructorTest (gw.gosu.ij.codeInsight)
invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:566, Method (java.lang.reflect)
runTest:176, TestCase (junit.framework)
lambda$runBare$11:533, UsefulTestCase (com.intellij.testFramework)
run:-1, 1190617061 (com.intellij.testFramework.UsefulTestCase$$Lambda$52)
runTestRunnable:454, UsefulTestCase (com.intellij.testFramework)
lambda$runTestRunnable$27:695, HeavyPlatformTestCase (com.intellij.testFramework)
run:-1, 1103978440 (com.intellij.testFramework.HeavyPlatformTestCase$$Lambda$1301)
lambda$runAndWait$2:115, WriteAction (com.intellij.openapi.application)
compute:-1, 477956128 (com.intellij.openapi.application.WriteAction$$Lambda$569)
runWriteActionWithClass:935, ApplicationImpl (com.intellij.openapi.application.impl)
runWriteAction:961, ApplicationImpl (com.intellij.openapi.application.impl)
computeAndWait:134, WriteAction (com.intellij.openapi.application)
computeAndWait:128, WriteAction (com.intellij.openapi.application)
runAndWait:114, WriteAction (com.intellij.openapi.application)
runTestRunnable:695, HeavyPlatformTestCase (com.intellij.testFramework)
lambda$runBareImpl$24:618, HeavyPlatformTestCase (com.intellij.testFramework)
run:-1, 1809039832 (com.intellij.testFramework.HeavyPlatformTestCase$$Lambda$103)
lambda$runInEdtAndWait$1:40, EdtTestUtil (com.intellij.testFramework)
run:-1, 263199584 (com.intellij.testFramework.EdtTestUtil$$Lambda$104)
dispatch:303, InvocationEvent (java.awt.event)
dispatchEventImpl:770, EventQueue (java.awt)
run:721, EventQueue$4 (java.awt)
run:715, EventQueue$4 (java.awt)
doPrivileged:-1, AccessController (java.security)
doIntersectionPrivilege:85, ProtectionDomain$JavaSecurityAccessImpl (java.security)
dispatchEvent:740, EventQueue (java.awt)
dispatchEvent:407, IdeEventQueue (com.intellij.ide)
pumpOneEventForFilters:203, EventDispatchThread (java.awt)
pumpEventsForFilter:124, EventDispatchThread (java.awt)
pumpEventsForHierarchy:113, EventDispatchThread (java.awt)
pumpEvents:109, EventDispatchThread (java.awt)
pumpEvents:101, EventDispatchThread (java.awt)
run:90, EventDispatchThread (java.awt)
One interesting thing is the FileScanExecutor#runOnAllThreads() call, but that is just a wild guess.
That's as far as I have gotten. If I try to follow the code that is stuck by stepping through it, it just goes through a ridiculous number of calls, looking at every character of every file in the test, one by one.
How can I debug this?
请先登录再写评论。
In case it helps, here's what my method in GosuMultiFileTestCase is doing. It is looking for any of several marker characters.
Please try
1) pass rootDir.getPresentableUrl() for setDirectoryName()
2) or model.setCustomScope() with com.intellij.psi.search.GlobalSearchScopes#directoryScope(Project, rootDir, true)
1) rootDir.getPresentableUrl() and rootDir.getCanonicalPath() return identical strings, so that didn't help.
2) I am guessing you meant something like this:
That didn't avoid a hang, either.
I see a problem in FilesScanExecutor, in the loop starting at line 62, where idleCount.get() gets stuck at 11, and runnersCount.get() gets stuck at 12. This seems to be why it's hanging. I don't see where runnersCount is getting decremented anywhere. idleCount gets decremented if idle is true, but execution goes through this loop forever and idleCount is not decremented.
How are these method-local variables ever supposed to change, once they are not the same and the loop is entered?
Please try the latest IDEA 2022.1 EAP (EAP6 build 221.5080.9). The code in FilesScanExecutor#processDequeOnAllThreads is a bit different there. Let's check if it solves the problem.
I'm still trying to upgrade to IDEA 2022.1 EAP to try out your suggestion. It's not so easy. It seems I have to download IDEA 2022.1 EAP or else my IDE code won't even compile (under 2021.3.2). That's a bit strange. And then Gradle is having fits because it wants some stupid Gradle property set to something that tells it to fetch the EAP from the JetBrains website. For a true release, I just set it to the full 3-number release, but for an EA I have to set it to something different, because there's no third number. Is it 2022.1.EA? 2022.1-EA? Gradle gives no clear error messages that actually tell you where the problem is, or what it is.
Please try 221.5080-EAP-CANDIDATE-SNAPSHOT
Thanks.
I tried that, and under it my hanging tests still seem to hang. I still get stuck in that loop, with runnersCount == 12 and idleCount == 11.
I'd like to try and reproduce the problem. Is it possible?
It is possible, but I'm not sure how best to do it. I could zip up my workspace, but chances are that things would go wrong because of configuration problems and expected things not found. However, it is worth a try. Where should I put the zip file so you can get it?
Please upload the file at https://uploads.jetbrains.com/ and share the "upload id" in IDEA-291283 I've created for this case.
I did this yesterday. Is it what you wanted?
Yes, thanks! Let's proceed in the ticket.