How to wrap composite PsiElement on top of Lexer recognized elements in Grammar-Kit ? Follow
Hi All,
I am new to intellij plugin writing. I started writing a intellij plugin for one of our custom language. I am following the tutorial given in intellij official site. Also I downloaded their Grammar-Kit repo from github to understand the code base.
While writing my plugin I am using jflex for Lexer and bnf grammar for parser. I find difficulty to implement a lexer to send proper tokens to the parser.
What I see in the Grammar-Kit repo that Lexer (for bnf) is pretty simple and it is only recognizing String, Number, Id, white space, comments and special characters('(', '*' etc.). It does not recognize bnf keywords like 'private', 'external', 'meta' etc.
Now when I am seeing the PSI tree of a example bnf file say for the line,
private myRule ::= '(' myExpression ')' ';'
is as below,
BnfFile:Dummy.bnf(0,43) BNF_RULE:myRule(0,43) BNF_MODIFIER(0,7) PsiElement(id)('private')(0,7) PsiWhiteSpace(' ')(7,8) PsiElement(id)('myRule')(8,14) PsiWhiteSpace(' ')(14,15) PsiElement(::=)('::=')(15,18) PsiWhiteSpace(' ')(18,19) BNF_SEQUENCE: '(' myExpression ')' ';'(19,43) BNF_STRING_LITERAL_EXPRESSION: '('(19,22) PsiElement(string)(''('')(19,22) PsiWhiteSpace(' ')(22,23) BNF_REFERENCE_OR_TOKEN: myExpression(23,35) PsiElement(id)('myExpression')(23,35) PsiWhiteSpace(' ')(35,36) BNF_STRING_LITERAL_EXPRESSION: ')'(36,39) PsiElement(string)('')'')(36,39) PsiWhiteSpace(' ')(39,40) BNF_STRING_LITERAL_EXPRESSION: ';'(40,43) PsiElement(string)('';'')(40,43)
What i am seeing here is 'private' is recognized as PsiElement.id
by the Lexer, after that some code wrap it with a BnfModifier object which is declared the bnf file The same for BNF_SEQUENCE, BNF_REFERENCE_OR_TOKEN, BNF_STRING_LITERAL_EXPRESSION etc. They are all not recognized by Lexer, however some code wrap them with bnf file declared objects.
I want to understand how this wrapping is done on top of Lexer recognized tokens. It will help me to recognize keyword in our DSL and highlight them differently, auto-complete them etc.
Thanks,
Subhojit
Please sign in to leave a comment.
This is done by PsiParser https://github.com/JetBrains/Grammar-Kit/blob/master/gen/org/intellij/grammar/parser/GrammarParser.java#L469 on top of output from Lexer.
GrammarKit generates all necessary PSI (if wanted) and corresponding parser code from BNF declaration.
Thanks, this helps.