PsiParser - complex element Follow
how should custom PsiParser implementation parse complex elements?
let's say, bnf looks like this:
xFile ::= line*
line ::= KEY
there is
XLineImpl extends ASTWrapperPsiElement
yet there is no wrapper for KEY element
auto generated XTypes.Factory can createElement for XLine but not for KEY
so if parser does PsiBuilder.mark() twice (for LINE and KEY), createElement throws an error when trying to create KEY
if parser only mark() once and possibly remapCurrentToken, we only see either LINE or KEY in the resulting tree.
how do we parse a valid line to
LINE
KEY
?
update:
if auto generated XTypes.Factory can createElement for both XLine and for KEY like this:
return new ASTWrapperPsiElement(node);
we get past create element error
however adding LINE marker around KEY
with Marker.precede(), Marker.done()
does not add LINE node to the tree
Please sign in to leave a comment.
Alternatively, element type can implement ICompositeElementType and return proper composite node in createCompositeNode
Thank you Maxim!
well it is auto generated factory which throws an error.
createElement is "patched" now. But I am still unable to wrap KEY in LINE by calling precede() and done() on PsiBuilder.
1) when looking at this diagram, it appears that calling mark() and done() twice would create a complex element:
http://www.jetbrains.org/intellij/sdk/docs/reference_guide/custom_language_support/implementing_parser_and_psi.html
2) this other example:
https://upsource.jetbrains.com/idea-community/file/1731d054af4ca27aa827c03929e27eeb0e6a8366/plugins%2Fproperties%2Fproperties-psi-impl%2Fsrc%2Fcom%2Fintellij%2Flang%2Fproperties%2Fparsing%2FPropertiesParser.java
too calls builder.mark() twice
I must be missing something.
I think your should be more clear in your problem / goal description.
If are going to stay on bnf level then lets look for existing project (e.g. https://github.com/JetBrains/intellij-plugins/tree/master/Dart) or ask GrammarKit specific question in https://github.com/JetBrains/Grammar-Kit/issues. Btw, it is not wise to fix the parser / types in such case.
Another (but more complex) option is go hand written parser way and my previous comment on your api question was based on assumption that you are going this way.
each Marker.mark / done will produce distinct composite ast node with type as given in argument.
There are two done calls in properties parser as AST structure should have two composite elements: inner, with
you are right. Let's narrow this down.
I am trying to implement PsiParser from 0.
However I am using GrammarKit generated files as a guide. I assume that some GrammarKit autogenerated files (not the parser of course) may still be reused with this new custom parser, no?
I may need to write my own XLineImpl, XKeyImpl etc but I am walking one step at a time (walk before I run ;))
my problem at this stage is:
is this possible to create composite element e.g.
LINE
KEY
using nothing but PsiBuilder
?
> each Marker.mark / done will produce distinct composite ast node with type as given in argument.
This is what I expect, too. This does not happen. ?:|
Outer marker has no effect.
IElementType type passed to Marker.done(type) is simply ignored.
Maxim, my apologies.
It turns out that PsiBuilder produces a correct (as expected) ASTNode.
However by the time the tree is displayed in PsiViewer, it appears differently from PsiBuilder output.
The problem lies elsewhere.
Thank you very much for your time, Maxim! I appreciate this.
It is ok to reuse generated code until you decide to generate it again :)
Yes, custom node implementation is possible, as I described per fixing factory or via implementing
PsiViewer shows PSI, not ast nodes
> It is ok to reuse generated code until you decide to generate it again :)
well autogen code was tweaked to test parser.
I will implement ICompositeElementType. Actually, will write custom factory for
> PsiViewer shows PSI, not ast nodes
a. However GrammarKit somehow succeeds to keep Psi tree very similar to AST tree. At least, every leaf (e.g. KEY) from bnf structure appears in PsiViewer.
LINE
KEY
- just what I am trying to achieve.
AstWrapperPsiElement skips noncomposites when PsiElement.getChildren() is called. However custom code visiting all PSI tree structure will see tokens. So depending on how viewer is implemented it will see slightly different structure.
As reference I'd always use com.intellij.psi.impl.DebugUtil.treeToString - we use it for parsing tests
com.intellij.psi.impl.DebugUtil.treeToString
cheers! this certainly helps.
found the source of my woes.
I omitted these two lines:
from this example:
https://upsource.jetbrains.com/idea-community/file/1731d054af4ca27aa827c03929e27eeb0e6a8366/plugins%2Fproperties%2Fproperties-psi-impl%2Fsrc%2Fcom%2Fintellij%2Flang%2Fproperties%2Fparsing%2FPropertiesParser.java
instead of
sorry about the confusion. :8}