How do I debug com.intellij.psi.impl.source.tree.CompositeElement.java?
1.IntelliJ is giving me the following assertion error:
java.lang.Throwable
at com.intellij.openapi.diagnostic.Logger.assertTrue(Logger.java:168)
at com.intellij.psi.impl.source.tree.CompositeElement.getChildrenAsPsiElements(CompositeElement.java:473)
at com.intellij.psi.impl.source.PsiFileImpl.getChildren(PsiFileImpl.java:831)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at idea.plugin.psiviewer.util.IntrospectionUtil.getValue(IntrospectionUtil.java:64)
at idea.plugin.psiviewer.view.PropertySheetPanel.setTarget(PropertySheetPanel.java:61)
at idea.plugin.psiviewer.view.PsiViewerPanel.updatePropertySheet(PsiViewerPanel.java:256)
at idea.plugin.psiviewer.view.PsiViewerPanel.setSelectedElement(PsiViewerPanel.java:239)
at idea.plugin.psiviewer.view.PsiViewerPanel.access$600(PsiViewerPanel.java:59)
at idea.plugin.psiviewer.view.PsiViewerPanel$ViewerTreeSelectionListener.valueChanged(PsiViewerPanel.java:199)
at javax.swing.tree.DefaultTreeSelectionModel.fireValueChanged(DefaultTreeSelectionModel.java:635)
at javax.swing.tree.DefaultTreeSelectionModel.notifyPathChange(DefaultTreeSelectionModel.java:1093)
at javax.swing.tree.DefaultTreeSelectionModel.setSelectionPaths(DefaultTreeSelectionModel.java:294)
at javax.swing.tree.DefaultTreeSelectionModel.setSelectionPath(DefaultTreeSelectionModel.java:188)
at javax.swing.JTree.setSelectionPath(JTree.java:1634)
at javax.swing.plaf.basic.BasicTreeUI.selectPathForEvent(BasicTreeUI.java:2393)
at javax.swing.plaf.basic.BasicTreeUI$Handler.handleSelection(BasicTreeUI.java:3609)
at javax.swing.plaf.basic.BasicTreeUI$Handler.mousePressed(BasicTreeUI.java:3548)
at com.intellij.util.ui.MouseEventAdapter.mousePressed(MouseEventAdapter.java:43)
at java.awt.Component.processMouseEvent(Component.java:6532)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6300)
at java.awt.Container.processEvent(Container.java:2236)
at java.awt.Component.dispatchEventImpl(Component.java:4891)
at java.awt.Container.dispatchEventImpl(Container.java:2294)
at java.awt.Component.dispatchEvent(Component.java:4713)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4522)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
at java.awt.Container.dispatchEventImpl(Container.java:2280)
at java.awt.Window.dispatchEventImpl(Window.java:2750)
at java.awt.Component.dispatchEvent(Component.java:4713)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.awt.EventQueue$4.run(EventQueue.java:729)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at com.intellij.ide.IdeEventQueue.c(IdeEventQueue.java:843)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:675)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:391)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
java.lang.Throwable
at com.intellij.openapi.diagnostic.Logger.assertTrue(Logger.java:168)
at com.intellij.psi.impl.source.tree.CompositeElement.getChildrenAsPsiElements(CompositeElement.java:473)
at com.intellij.psi.impl.source.PsiFileImpl.getChildren(PsiFileImpl.java:831)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at idea.plugin.psiviewer.util.IntrospectionUtil.getValue(IntrospectionUtil.java:64)
at idea.plugin.psiviewer.view.PropertySheetPanel.setTarget(PropertySheetPanel.java:61)
at idea.plugin.psiviewer.view.PsiViewerPanel.updatePropertySheet(PsiViewerPanel.java:256)
at idea.plugin.psiviewer.view.PsiViewerPanel.setSelectedElement(PsiViewerPanel.java:239)
at idea.plugin.psiviewer.view.PsiViewerPanel.access$600(PsiViewerPanel.java:59)
at idea.plugin.psiviewer.view.PsiViewerPanel$ViewerTreeSelectionListener.valueChanged(PsiViewerPanel.java:199)
at javax.swing.tree.DefaultTreeSelectionModel.fireValueChanged(DefaultTreeSelectionModel.java:635)
at javax.swing.tree.DefaultTreeSelectionModel.notifyPathChange(DefaultTreeSelectionModel.java:1093)
at javax.swing.tree.DefaultTreeSelectionModel.setSelectionPaths(DefaultTreeSelectionModel.java:294)
at javax.swing.tree.DefaultTreeSelectionModel.setSelectionPath(DefaultTreeSelectionModel.java:188)
at javax.swing.JTree.setSelectionPath(JTree.java:1634)
at javax.swing.plaf.basic.BasicTreeUI.selectPathForEvent(BasicTreeUI.java:2393)
at javax.swing.plaf.basic.BasicTreeUI$Handler.handleSelection(BasicTreeUI.java:3609)
at javax.swing.plaf.basic.BasicTreeUI$Handler.mousePressed(BasicTreeUI.java:3548)
at com.intellij.util.ui.MouseEventAdapter.mousePressed(MouseEventAdapter.java:43)
at java.awt.Component.processMouseEvent(Component.java:6532)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6300)
at java.awt.Container.processEvent(Container.java:2236)
at java.awt.Component.dispatchEventImpl(Component.java:4891)
at java.awt.Container.dispatchEventImpl(Container.java:2294)
at java.awt.Component.dispatchEvent(Component.java:4713)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4522)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
at java.awt.Container.dispatchEventImpl(Container.java:2280)
at java.awt.Window.dispatchEventImpl(Window.java:2750)
at java.awt.Component.dispatchEvent(Component.java:4713)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.awt.EventQueue$4.run(EventQueue.java:729)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at com.intellij.ide.IdeEventQueue.c(IdeEventQueue.java:843)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:675)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:391)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
2. Trying to debug it. I would like to step through the methods of this class, but for some reason IntelliJ would not decompile it:
public int countChildren(@org.jetbrains.annotations.Nullable com.intellij.psi.tree.TokenSet tokenSet) { /* compiled code */ }
I am confused as to when IntelliJ attempts to decompile and when it just shows this /* compiled code */ message.
3. I found two occurrences of com/intellij/psi/impl/source/tree/CompositeElement.class in Idea Ultimate: ./lib/idea.jar and ./plugins/Kotlin/kotlinc/lib/kotlin-compiler.jar.
Curious about the reason behind the same fully-qualified class name occurring in two languages. Perhaps the reason it would not decompile for me is because it's picking up the Kotlin version?
4. Any way to decompile Kotlin classes?
5. I tried to add idea.jar to the classpath, it did not let me. Guess it's because it's on the classpath implicitly?
IntelliJ IDEA 2016.3
Build #IU-163.7743.44, built on November 17, 2016
Expiration date: December 22, 2016
JRE: 1.8.0_112-release-408-b2 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Please sign in to leave a comment.
The source is on github: https://github.com/JetBrains/intellij-community/blob/306d705e1829bd3c74afc2489bfb7ed59d686b84/platform/core-impl/src/com/intellij/psi/impl/source/tree/CompositeElement.java#L473
3. Kotlin compiler reuses code from IntelliJ IDEA to parse java files. kotlin-compiler.jar shouldn't be added to the classpath of IDE process or your application so it won't cause problems.
5. What do you mean by 'add idea.jar to the classpath'? Did you try to add the JAR to the list on 'Classpath' tab of the JDK editor in 'Project Structure'? And what do you mean by 'did not let me'? Was there any error message shown?
2. Due to historical reasons, the class is obfuscated and the decompiler cannot handle it.
Thank you Peter. I do consult my clone of Community Edition regularly, and saw this code. However, because I also work with Javascript and Typescript a lot, I tend to use Ultimate most of the time, and it would be very nice to be able to debug it, in addition to just seeing the code. (Any way to use Javascript plugin with Community?)
3. I have not added Kotlin to classpath, so all is good.
> What do you mean by 'add idea.jar to the classpath'? Did you try to add the JAR to the list on 'Classpath' tab of the JDK editor in 'Project Structure'?
> And what do you mean by 'did not let me'? Was there any error message shown?
No error messages, just silently ignored (search for "idea" on the pic above brings no result).
> 2. Due to historical reasons, the class is obfuscated and the decompiler cannot handle it.
Obfuscation for historical reasons can be a bit of a pain. Any plans to de-obfuscate it?
No immediate plans AFAIK. That would require a significant change to the build process while the benefits are unclear.
About the benefits. If the code was unobfuscated, an apparent IntelliJ bug that has caused this assertion error would have been fixed. I am pretty sure it's not my bug because I am not doing anything interesting there yet. Seems to be a bug in either the lexer or parser or IDEA core.
As it is, I have to keep too many projects/IDE instances open on my box at once: 1) Idea Ultimate with the plugin I am developing, 2) Community Edition for looking up comments and such, 3) The "Child" Idea being debugged, 4) whatever plugin I am using as an example. I do have enough RAM for that, but some people might not; and switching back and forth all the time can be taxing. Would be nice at least not to have to run Community all the time.
Also, in the absence of documentation, debugger is the next best thing. That would be a nice step to support your plugin ecosystem. Unless I am the only one who needs this.
Sorry for the inconveniences with debugging Ultimate classes. We'll see if we remove scrambling, at least from PSI-related packages.
In the meantime, we can try to help you finding out the issue. It's caused by some AST node not having a PSI equivalent, which is quite unexpected and abnormal. The assertion message should actually contain some information about the AST node, including its element type. Please check if there's anything unusual happening around that type. Is it composite or a leaf element? Does it have custom AST node class with overridden methods?
It would also help if you provided your plugin (in source or binary form) and steps to reproduce the issue.
Thank you Peter. It would be great if you de-obfuscate the PSI logic. While at it, maybe also deobfuscate Javascript? Would help those of us who are writing Javascript-transpiled languages and need to rely on debug configurations, breakpoins and such. Or is this part obfuscated on purpose?
As for my assertion error, it's related to my other post:
https://intellij-support.jetbrains.com/hc/en-us/community/posts/205451890-Trying-to-make-EOL-significant-in-my-grammar-
The offending node was LINE.
Irina
JavaScript is more complicated, because we have no free open-source IDE with that code, so I can't promise anything.
I cloned the repository, but wasn't able to reproduce the issue. What should I do for that?
On a Mac:
1. Pull the latest changes.
2. Run the plugin under IDEA Ultimate 2016.3
3. Open file testData/test.csv
IntelliJ IDEA 2016.3
Build #IU-163.7743.44, built on November 17, 2016
Licensed to IntelliJ IDEA Evaluator
Expiration date: December 22, 2016
JRE: 1.8.0_112-release-408-b2 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Never mind, have a different program now (which still needs help!) I've been switching back and forth for a while.
See the other thread:
https://intellij-support.jetbrains.com/hc/en-us/community/posts/205451890-Trying-to-make-EOL-significant-in-my-grammar-?page=1#community_comment_205122684
program=problem?
So NPE is solved, right? By adding elementTypeClass?
Yes and Yes. Thank you.
Do you think Grammar-Kit should give at least a warning in this case? Maybe IDEA, too.
Yes, there definitely should be more comprehensible diagnostics than just NPE. We'll try to add some.
BTW we've disabled scrambling for some core PSI parts, they'll be available in coming 163 & 171 releases.
Hi Irina,
Peter is absolutely right, our JS implementation is not open-source, so this code is obfuscated on purpose. Do you have any pointers to these JS-transpiled language plugins so I can see what we can do to help?
Not at the moment. I've already done custom debugger with breakpoints, configurations and all, based off Javascript debugger. I have not done "evaluate expression" yet.
I completely understand obfuscating Javascript. That stuff is valuable.
Also, I am considering embedding Javascript functions, statements, etc. into our language (right now, we are parsing everything from primitives). Any pointers on how to do that?
Sorry for the delayed reply.
It should be enough to provide token of com.intellij.lang.javascript.types.JSEmbeddedContentElementType class. You can use existing ones (com.intellij.lang.javascript.JSElementTypes#EMBEDDED_CONTENT, JSX_EMBEDDED_CONTENT or ES6_EMBEDDED_CONTENT) or provide your own implementation if you need some custom dialect (see https://github.com/JetBrains/intellij-plugins/blob/master/AngularJS/src/org/angularjs/lang/parser/AngularJSElementTypes.java#L24)
> BTW we've disabled scrambling for some core PSI parts, they'll be available in coming 163 & 171 releases.
Not in IntelliJ IDEA 2016.3.2 EAP, right?
PsiFileBase still looks scrambled for me.
Thank you Dennis, this is a good lead.
Irina, which build have you tried?
IntelliJ IDEA 2016.3.2 EAP
Build #IU-163.10154.17, built on December 14, 2016
Licensed to DreamBox Learning, Inc. / Irina Rapoport
Subscription is active until November 4, 2017
JRE: 1.8.0_112-release-408-b6 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Maybe it has something to do with the "Statement cannot be decomposed although reducible" error I've been getting? That seems to come from a decompiler:
https://android.googlesource.com/platform/tools/idea/+/e782c57d74000722f9db4c9426317410520670c6/plugins/java-decompiler/engine/src/org/jetbrains/java/decompiler/modules/decompiler/DomHelper.java
Although ever since it rebuilt caches, it no longer gives me that message.
The situation is quite sad. We've told the scrambler to exclude PSI files. It works, but unfortunately only partially. The scrambler still changes some class files for some mysterious reason, and our decompiler can't process them afterwards. It seems that it's not very easy to fix that right now, but I've filed a couple of requests which you can vote/watch:
https://youtrack.jetbrains.com/issue/IDEA-165757
https://youtrack.jetbrains.com/issue/IDEA-165756
Now field names and line numbers are not scrambled (at least supposed to). So attaching Community source to Ultimate jars should work for PSI infrastructure. I can't guarantee the unintended bytecode changes don't spoil everything, but at least in some (hopefully, the majority) cases this should make debugging easier.