Document text change and PSIParser AssertionError in IDEA 14

Answered

Hello
I develop OSS plug-in for Intellij platform (https://github.com/raydac/netbeans-mmd-plugin/tree/master/mind-map/idea-mindmap) which supports Intellij IDEA 14+.

In the plug-in I change document text with code similar to the below code snippet

private void saveMindMapToDocument() { 
  final PsiDocumentManager psiManager = PsiDocumentManager.getInstance(getProject());
  final Document document = getDocument(); psiManager.doPostponedOperationsAndUnblockDocument(document);

  if (!this.mindMapPanel.isDisposed()) {
    final MindMap model = this.mindMapPanel.getModel();
    if (document != null && model != null) {
    Runnable runnable = new Runnable() {
      @Override public void run() {
        String packedMindMap = model.packToString(); document.setText(packedMindMap);
        VirtualFile vfile = FileDocumentManager.getInstance().getFile(document);
        if (vfile!=null){
          vfile.refresh(false,false);
        }
        psiManager.commitDocument(document);
      }
    };

    ApplicationManager.getApplication().runWriteAction(new Runnable() {
      @Override
      public void run() {
        final CommandProcessor processor = CommandProcessor.getInstance();
        processor.executeCommand(project, new Runnable() {
          @Override public void run() {
            action.run();
            if (document!=null) {
              processor.addAffectedDocuments(project,document);
            }
          }
        },null, null, document);
      }
    }
  }
 

after sequential several changes of the same document my PsiLexer starts generate bunch of errors in getTreeBuilt()

final ASTNode result = builder.getTreeBuilt();

since  IDEA 15+ I use mostly the same logic but through  com.intellij.openapi.application.TransactionGuard and all works well, may be I should update some inside PSI cache after document change to avoid such errors in IDEA 14?

the generated exception is:


java.lang.AssertionError: PsiElement(Topic:TOPIC)
    at com.intellij.psi.impl.source.text.DiffLog$InsertEntry.<init>(DiffLog.java:175)
    at com.intellij.psi.impl.source.text.DiffLog.nodeInserted(DiffLog.java:88)
    at com.intellij.psi.impl.source.text.DiffLog.nodeInserted(DiffLog.java:41)
    at com.intellij.lang.impl.PsiBuilderImpl$ConvertFromTokensToASTBuilder.nodeInserted(PsiBuilderImpl.java:1076)
    at com.intellij.lang.impl.PsiBuilderImpl$ConvertFromTokensToASTBuilder.nodeInserted(PsiBuilderImpl.java:1060)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:163)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:106)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:106)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:123)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.diff(DiffTree.java:51)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.diffTrees(BlockSupportImpl.java:276)
    at com.intellij.lang.impl.PsiBuilderImpl.merge(PsiBuilderImpl.java:1098)
    at com.intellij.lang.impl.PsiBuilderImpl.buildTree(PsiBuilderImpl.java:1025)
    at com.intellij.lang.impl.PsiBuilderImpl.getTreeBuilt(PsiBuilderImpl.java:1006)
    at com.igormaznitsa.ideamindmap.lang.MMPsiParser.parse(MMPsiParser.java:19)
    at com.intellij.psi.tree.ILazyParseableElementType.doParseContents(ILazyParseableElementType.java:64)
    at com.intellij.psi.tree.IFileElementType.parseContents(IFileElementType.java:43)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.ensureParsed(LazyParseableElement.java:172)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(LazyParseableElement.java:212)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(LazyParseableElement.java:36)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.isReplaceWholeNode(BlockSupportImpl.java:301)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.mergeTrees(BlockSupportImpl.java:246)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.makeFullParse(BlockSupportImpl.java:194)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.reparseRange(BlockSupportImpl.java:138)
    at com.intellij.psi.impl.DocumentCommitProcessor.doCommit(DocumentCommitProcessor.java:123)
    at com.intellij.psi.impl.DocumentCommitThread$4.run(DocumentCommitThread.java:474)
    at com.intellij.psi.impl.DocumentCommitThread.commitUnderProgress(DocumentCommitThread.java:484)
    at com.intellij.psi.impl.DocumentCommitThread.commitSynchronously(DocumentCommitThread.java:423)
    at com.intellij.psi.impl.PsiDocumentManagerBase$5.run(PsiDocumentManagerBase.java:377)
    at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:977)
    at com.intellij.psi.impl.PsiDocumentManagerBase.doCommit(PsiDocumentManagerBase.java:369)
    at com.intellij.psi.impl.PsiDocumentManagerBase.commitDocument(PsiDocumentManagerBase.java:282)
    at com.intellij.psi.impl.PsiDocumentManagerBase.commitAllDocuments(PsiDocumentManagerBase.java:216)
    at com.intellij.psi.impl.PsiDocumentManagerImpl$3.run(PsiDocumentManagerImpl.java:156)
    at com.intellij.util.ui.UIUtil.invokeLaterIfNeeded(UIUtil.java:2108)
    at com.intellij.psi.impl.PsiDocumentManagerImpl.save(PsiDocumentManagerImpl.java:152)
    at com.intellij.openapi.components.impl.stores.ComponentStoreImpl$SaveSessionImpl.save(ComponentStoreImpl.java:402)
    at com.intellij.openapi.components.impl.stores.ProjectStoreImpl$ProjectSaveSession.save(ProjectStoreImpl.java:484)
    at com.intellij.openapi.components.impl.stores.StoreUtil.doSave(StoreUtil.java:48)
    at com.intellij.openapi.project.impl.ProjectImpl.save(ProjectImpl.java:357)
    at com.intellij.ide.SaveAndSyncHandlerImpl.saveProjectsAndDocuments(SaveAndSyncHandlerImpl.java:149)
    at com.intellij.ide.SaveAndSyncHandlerImpl$3.onFrameDeactivated(SaveAndSyncHandlerImpl.java:103)
    at com.intellij.ide.FrameStateManagerImpl.fireDeactivationEvent(FrameStateManagerImpl.java:87)
    at com.intellij.ide.FrameStateManagerImpl.access$500(FrameStateManagerImpl.java:32)
    at com.intellij.ide.FrameStateManagerImpl$2$1.run(FrameStateManagerImpl.java:72)
    at com.intellij.util.concurrency.QueueProcessor.runSafely(QueueProcessor.java:238)
    at com.intellij.util.Alarm$Request$1.run(Alarm.java:351)
    at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:318)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:748)
    at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:577)
    at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:384)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
0
4 comments

Hi! I can't tell you yet what the issue is, but the code looks unnecessarily complicated. Let's try to simplify it first and see if it helps.

Why do you need doPostponedOperationsAndUnblockDocument? It only makes sense inside a write action, which doesn't seem to be the case here.

Why do you need vfile.refresh? After changing in-memory document text, this could only lead to memory-disk conflict dialog.

Why do you commit the document explicitly? It might be needed if you're working with PSI just after that, but that's not happening in this code sample. Changed documents are committed automatically in background.

executeCommand is normally outside runWriteAction, not inside. For this particular combination, using WriteCommandAction to combine the two might be beneficial.

Why do you need addAffectedDocuments? Document changes should be tracked automatically.

As for the exception, is it the very first one in idea.log (Help | Show log in...)? Is your PsiFile#isPhysical true?

 

0

I made such extra-coding in attempt to fix the error because decided that something was non-synchronized one on lower level of the platform

I have removed all extra code and now I make change through such piece of code, the document, which I change, has physical file on disk

if (!this.mindMapPanel.isDisposed()) {
final MindMap model = this.mindMapPanel.getModel();
final Document document = getDocument();
if (document != null && model != null) {
Runnable action = new Runnable() {
@Override
public void run() {
document.setText(model.packToString());
}
};

CommandProcessor.getInstance().executeCommand(project, new Runnable() {
@Override
public void run() {
ApplicationManager.getApplication().runWriteAction(action);
}
},"MMD>executeReadAction",null,document);
}
}

but anyway after two-three big changes in the document I have bunch of repeating errors in log

[  30061]  ERROR - .psi.impl.DocumentCommitThread - IntelliJ IDEA 14.0.5  Build #IC-139.1803.20
[  30061]  ERROR - .psi.impl.DocumentCommitThread - JDK: 1.8.0_102
[  30061]  ERROR - .psi.impl.DocumentCommitThread - VM: Java HotSpot(TM) 64-Bit Server VM
[  30061]  ERROR - .psi.impl.DocumentCommitThread - Vendor: Oracle Corporation
[  30061]  ERROR - .psi.impl.DocumentCommitThread - OS: Linux
[  30061]  ERROR - .psi.impl.DocumentCommitThread - Last Action:  
[  30081]  ERROR - impl.stores.ComponentStoreImpl - PsiElement(Topic:TOPIC)
java.lang.AssertionError: PsiElement(Topic:TOPIC)
    at com.intellij.psi.impl.source.text.DiffLog$InsertEntry.<init>(DiffLog.java:175)
    at com.intellij.psi.impl.source.text.DiffLog.nodeInserted(DiffLog.java:88)
    at com.intellij.psi.impl.source.text.DiffLog.nodeInserted(DiffLog.java:41)
    at com.intellij.lang.impl.PsiBuilderImpl$ConvertFromTokensToASTBuilder.nodeInserted(PsiBuilderImpl.java:1076)
    at com.intellij.lang.impl.PsiBuilderImpl$ConvertFromTokensToASTBuilder.nodeInserted(PsiBuilderImpl.java:1060)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:163)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:106)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:106)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:123)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.compareLevel(DiffTree.java:106)
    at com.intellij.util.diff.DiffTree.build(DiffTree.java:78)
    at com.intellij.util.diff.DiffTree.diff(DiffTree.java:51)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.diffTrees(BlockSupportImpl.java:276)
    at com.intellij.lang.impl.PsiBuilderImpl.merge(PsiBuilderImpl.java:1098)
    at com.intellij.lang.impl.PsiBuilderImpl.buildTree(PsiBuilderImpl.java:1025)
    at com.intellij.lang.impl.PsiBuilderImpl.getTreeBuilt(PsiBuilderImpl.java:1006)
    at com.igormaznitsa.ideamindmap.lang.MMPsiParser.parse(MMPsiParser.java:18)
    at com.intellij.psi.tree.ILazyParseableElementType.doParseContents(ILazyParseableElementType.java:64)
    at com.intellij.psi.tree.IFileElementType.parseContents(IFileElementType.java:43)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.ensureParsed(LazyParseableElement.java:172)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(LazyParseableElement.java:212)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(LazyParseableElement.java:36)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.isReplaceWholeNode(BlockSupportImpl.java:301)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.mergeTrees(BlockSupportImpl.java:246)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.makeFullParse(BlockSupportImpl.java:194)
    at com.intellij.psi.impl.source.text.BlockSupportImpl.reparseRange(BlockSupportImpl.java:138)
    at com.intellij.psi.impl.DocumentCommitProcessor.doCommit(DocumentCommitProcessor.java:123)
    at com.intellij.psi.impl.DocumentCommitThread$4.run(DocumentCommitThread.java:474)
    at com.intellij.psi.impl.DocumentCommitThread.commitUnderProgress(DocumentCommitThread.java:484)
    at com.intellij.psi.impl.DocumentCommitThread.commitSynchronously(DocumentCommitThread.java:423)
    at com.intellij.psi.impl.PsiDocumentManagerBase$5.run(PsiDocumentManagerBase.java:377)
    at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:977)
    at com.intellij.psi.impl.PsiDocumentManagerBase.doCommit(PsiDocumentManagerBase.java:369)
    at com.intellij.psi.impl.PsiDocumentManagerBase.commitDocument(PsiDocumentManagerBase.java:282)
    at com.intellij.psi.impl.PsiDocumentManagerBase.commitAllDocuments(PsiDocumentManagerBase.java:216)
    at com.intellij.psi.impl.PsiDocumentManagerImpl$3.run(PsiDocumentManagerImpl.java:156)
    at com.intellij.util.ui.UIUtil.invokeLaterIfNeeded(UIUtil.java:2108)
    at com.intellij.psi.impl.PsiDocumentManagerImpl.save(PsiDocumentManagerImpl.java:152)
    at com.intellij.openapi.components.impl.stores.ComponentStoreImpl$SaveSessionImpl.save(ComponentStoreImpl.java:402)
    at com.intellij.openapi.components.impl.stores.ProjectStoreImpl$ProjectSaveSession.save(ProjectStoreImpl.java:484)
    at com.intellij.openapi.components.impl.stores.StoreUtil.doSave(StoreUtil.java:48)
    at com.intellij.openapi.project.impl.ProjectImpl.save(ProjectImpl.java:357)
    at com.intellij.ide.SaveAndSyncHandlerImpl.saveProjectsAndDocuments(SaveAndSyncHandlerImpl.java:149)
    at com.intellij.ide.SaveAndSyncHandlerImpl$3.onFrameDeactivated(SaveAndSyncHandlerImpl.java:103)
    at com.intellij.ide.FrameStateManagerImpl.fireDeactivationEvent(FrameStateManagerImpl.java:87)
    at com.intellij.ide.FrameStateManagerImpl.access$500(FrameStateManagerImpl.java:32)
    at com.intellij.ide.FrameStateManagerImpl$2$1.run(FrameStateManagerImpl.java:72)
    at com.intellij.util.concurrency.QueueProcessor.runSafely(QueueProcessor.java:238)
    at com.intellij.util.Alarm$Request$1.run(Alarm.java:351)
    at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:318)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:748)
    at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:577)
    at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:384)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
0

I've skimmed through the plugin source, and one thing that looks suspicious is that you use some element types (e.g. TOPIC) both as lexer tokens and composite elements, in parser. This confuses incremental reparse that treats them as similar, yet the underlying types in the AST are very different, hence the assertion.

0

Thank you very much, I will investigate the situation and improve the lexer

0

Please sign in to leave a comment.