How to avoid "Outdated stub in index" error?
Hello,
I have a custom EditorNotifications.Provider implementation which calls PsiClassOwner#getPackageName().
This can cause a "Outdated stub in index" error (see stack below from IDEA 2018.x, unfortunately I don't know the exact version).
I see that there are already several checks performed by the getPackageName() stack, eg:
- PsiFileImpl#getStubTree() checks: ApplicationManager.getApplication().assertReadAccessAllowed();
- StubTreeLoaderImpl#readFromVFile checks: DumbService.getInstance(project).isDumb()
So I assume the getPackageName() is relatively safe and it should return null if the file is not indexed for example.
I have also reviewed some other EditorNotifications.Provider implementations but I don't see what I am doing wrong or which check is missing in my implementation.
For example the com. intellij. psi. codeStyle. autodetect. DetectedIndentOptionsNotificationProvider is also accessing the PsiFile.
My code looks like this:
public EditorNotificationPanel createNotificationPanel( VirtualFile file, FileEditor fileEditor ) {
if ( !( fileEditor instanceof TextEditor ) ) {
return null;
}
Editor editor = ( (TextEditor) fileEditor ).getEditor();
Project project = editor.getProject();
if ( project == null ) {
return null;
}
PsiFile psiFile = PsiManager.getInstance( project ).findFile( file );
if ( psiFile == null ) {
return null;
}
if ( !( psiFile instanceof PsiClassOwner ) ) {
return null;
}
if ( psiFile instanceof PsiCompiledFile ) {
return null;
}
if ( !psiFile.isWritable() ) {
return null;
}
String pkg = ((PsiClassOwner) psiFile).getPackageName();
[...]
}
Stack:
Outdated stub in index:
file://c:/path/to/JavaSourceFile.java
indexed at 1522231326258 with document size 8808,
doc=DocumentImpl[file://c:/path/to/JavaSourceFile.java],
docSaved=true, wasIndexedAlready=true, queried at 1522238992830
indexed lengths={chars=8808, bytes=9031}
doc length=9122
file length=9354
cached PSI class com.intellij.psi.impl.source.PsiJavaFileImpl
projects with file: 1
java.lang.Exception
at
com.intellij.psi.stubs.StubTreeLoaderImpl.diagnoseLengthMismatch(StubTreeLoaderImpl.java:175)
at
com.intellij.psi.stubs.StubTreeLoaderImpl.checkLengthMatch(StubTreeLoaderImpl.java:145)
at
com.intellij.psi.stubs.StubTreeLoaderImpl.readFromVFile(StubTreeLoaderImpl.java:113)
at
com.intellij.psi.stubs.StubTreeLoaderImpl.readOrBuild(StubTreeLoaderImpl.java:53)
at
com.intellij.psi.impl.source.PsiFileImpl.getStubTree(PsiFileImpl.java:652)
at
com.intellij.psi.impl.source.PsiFileImpl.getGreenStubTree(PsiFileImpl.java:1022)
at
com.intellij.psi.impl.source.PsiFileImpl.getGreenStub(PsiFileImpl.java:629)
at
com.intellij.psi.impl.source.PsiJavaFileBaseImpl.getPackageName(PsiJavaFileBaseImpl.java:91)
at
myPlugin.MyNotifierProvider.createNotificationPanel(MyNotifierProvider.java)
Thanks
Reto
请先登录再写评论。
Does this problem steadily reproducible after running "cache invalidation" action ?
No the problem is not reproducible. I have also just one single report for this problem.
The developer says he has excluded the source/module which contains the file (the "JavaSourceFile.java" in the error message) but the tab of this file was open.
I am having a similar problem with
while running automated tests. The problem is non-deterministic in that it occurs only sometimes. There are many tests that might trigger it, and each time I run tests, I get a few of the tests randomly failing due to the error.
It is almost impossible to figure out what the underlying problem is, because it does not occur while running under the debugger and is non-deterministic otherwise. We are calling
which triggers the problem. What should we look for in our code? Is there an API we need to call before calling doHighlighting()?
Here is a full stack trace
@Bgoldberg Do the tests have clear caches every time? (If caches/indices are reused between test runs) This problem may happen if indices were not updated due to events about file changes: forced termination of tests, absence of explicit virtual file refresh on used files (our tests were used to skip this costly operation) or even bugs in indices itself that were fixed between IJ 2017.3 and IJ 2018.1
@Maxim Mossienko
Our tests use this logic:
where setDocumentText is
private void setDocumentText(final Document document, final String text) {if (!document.getText().equals(text)) {
runWriteCommandAction(getProject(), () -> {
document.setText(text);
FileDocumentManager.getInstance().saveDocument(document);
});
}
}
At what point should we clear caches? Would adding
in the finally clause above help?
Thanks.
The outdated stub is apparently in the index, so do I have to call some code to make sure the changes are indexed correctly? Perhaps the indexing thread is either not running or not keeping up with the test.
After changing document and saving it your code need to perform commit of document text to AST used by background error detection, indices, etc by calling PsiDocumentManager.getInstance(project).commitDocument(document);
I seem to get various spurious exceptions when debugging my test code. It looks like the test environment does not clean up properly. I am trying to work around this problem to get to the underlying issue.
We are also facing with the problem accidentally. Is there any recommendations how to avoid it?
It is annoying because it is non-deterministic. I imagine that some of our tests are somehow not following some contract correctly, but since it occurs most often during the shutdown after the test, we don't know where to look or what to look for. Can you provide some hints or documentation?
At this amount of details we can repeat only generic recommendations: first recommendation and second recommendation from this post.
Please clean test caches and attach all log files after encountering the problem.
Both of those links are to the same page.
Thanks for spotting. I fixed the links, just in case links point to my previous comments on this page.
FYI:
After two years, I have received a second report of that issue. Here is the stack (reporter has not mentioned the IDEA version):
{code}
Outdated stub in index: file://c:/path/to/JavaSourceFile.java indexed at 1586879831076 with document size 42241,
doc=DocumentImpl[file://c:/path/to/JavaSourceFile.java], docSaved=true, wasIndexedAlready=true, queried at 1591601035485
indexed lengths={chars=42241, bytes=43175}
doc length=42190
file length=43125
cached PSI class com.intellij.psi.impl.source.PsiJavaFileImpl
projects with file: 1
java.lang.Exception
at com.intellij.psi.stubs.StubTreeLoaderImpl.diagnoseLengthMismatch(StubTreeLoaderImpl.java:180)
at com.intellij.psi.stubs.StubTreeLoaderImpl.checkLengthMatch(StubTreeLoaderImpl.java:150)
at com.intellij.psi.stubs.StubTreeLoaderImpl.readFromVFile(StubTreeLoaderImpl.java:118)
at com.intellij.psi.stubs.StubTreeLoaderImpl.readOrBuild(StubTreeLoaderImpl.java:53)
at com.intellij.psi.impl.source.PsiFileImpl.getStubTree(PsiFileImpl.java:610)
at com.intellij.psi.impl.source.PsiFileImpl.getGreenStubTree(PsiFileImpl.java:933)
at com.intellij.psi.impl.source.PsiFileImpl.getGreenStub(PsiFileImpl.java:586)
at com.intellij.psi.impl.source.PsiJavaFileBaseImpl.getPackageName(PsiJavaFileBaseImpl.java:78)
at ch.abacus.idea.statics.ToBeReviewedCache.find(ToBeReviewedCache.java:84)
at ch.abacus.idea.statics.ToBeReviewedNotifierProvider.createNotificationPanel(ToBeReviewedNotifierProvider.java:97)
at ch.abacus.idea.statics.ToBeReviewedNotifierProvider.createNotificationPanel(ToBeReviewedNotifierProvider.java:48)
at com.intellij.ui.EditorNotifications$Provider.createNotificationPanel(EditorNotifications.java:37)
at com.intellij.ui.EditorNotificationsImpl.calcNotificationUpdates(EditorNotificationsImpl.java:117)
at com.intellij.ui.EditorNotificationsImpl.lambda$null$0(EditorNotificationsImpl.java:96)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:144)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$null$1(NonBlockingReadActionImpl.java:118)
at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1166)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:68)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:121)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:164)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:586)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:86)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:151)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:110)
at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:68)
at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$transferToBgThread$2(NonBlockingReadActionImpl.java:118)
at com.intellij.util.concurrency.BoundedTaskExecutor.doRun(BoundedTaskExecutor.java:220)
at com.intellij.util.concurrency.BoundedTaskExecutor.access$100(BoundedTaskExecutor.java:26)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.lambda$run$0(BoundedTaskExecutor.java:198)
at com.intellij.util.ConcurrencyUtil.runUnderThreadName(ConcurrencyUtil.java:224)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.run(BoundedTaskExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}