Grammar-Kit BNF complete syntax

已回答

Ross Goldberg

创建于 2023年08月07日 02:07

Is a complete syntax for the Grammar-Kit BNF file format available anywhere?

The existing information seems incomplete & disjoint.

6 条评论

Karol Lewandowski

创建于 2023年08月07日 10:44

Hi Ross,

Yes, see: https://github.com/JetBrains/Grammar-Kit/blob/master/grammars/Grammar.bnf

Could you also please point to what is incomplete and what problem you have? It'll help us improve the docs.

Ross Goldberg

创建于 2023年08月07日 13:42

Sorry, I should have said complete language spec, because I would like more than just the syntax.

I have problems because the existing overview at https://github.com/JetBrains/Grammar-Kit/tree/master is confusing. Simple bnf file attempts produce odd errors in Live Preview. I could indicate where the documentation is lacking / confusing if I learn why the simple bnf attempts are failing in Live Preview. e.g.,

x ::= a | 'b' | c

Gives a BAD_CHARACTER error on b and a BAD_CHARACTER error on the trailing newline in:

a b c

The docs seem to indicate that there shouldn't be an error for b, so what's wrong with the above bnf?

How do I fix the newline error? I have seen bnf files that don't seem to explicitly deal with newlines; I imagine that they don't have errors on every newline, so I'm not sure why this is occurring.

Also, since the newline causes an error, why aren't the spaces causing errors?

Ross Goldberg

创建于 2023年08月07日 16:19

Every single bnf file I've tried has errors in the Live Preview. Grammar.bnf, Simple.bnf from the simple_language_plugin example, etc.

Something seems to be incredibly wrong with the Live Preview in 2023.2.

Karol Lewandowski

创建于 2023年08月08日 06:31

Hi Ross,

All the available resources are in the Grammar-Kit repository. Thank you for reporting the lack of the BNF language reference. We planned to improve Grammar-Kit documentation, and reporting problems helps us to prioritize the work.

I just checked your grammar example, and it works, but with slight changes.
'b' is a string literal and needs to be defined in tokens:

{
  tokens=[
    b='b'
  ]
}
x ::= a | 'b' | c

Also, the rule is a choice, so only a, b, or c can be parsed correctly.

I also checked sample.bnf from https://github.com/JetBrains/Grammar-Kit/blob/master/TUTORIAL.md, and it correctly parses the example code.

Ross Goldberg

创建于 2023年08月08日 08:04

From https://github.com/JetBrains/Grammar-Kit/tree/master#tokens:

Implicit tokens are tokens not specified via tokens attribute. Unquoted implicit tokens (aka keyword tokens) have names equals to their values. Quoted implicit tokens (aka text-matched tokens) are slow because they are matched by text and not by an IElementType constant returned by a lexer. Text-matched tokens can span more than one real token returned by the lexer.

How can you have "quoted implicit tokens" if quoted tokens that aren't in the global `tokens` list don't work?

(also, the 's' from the end of "equals" should be deleted in: "have names equals to their values". i.e. it should read "have names equal to their values").

I knew that `a | 'b' | c` is a choice, but that doesn't explain why Live Preview gives an error about the newline but not about the spaces. Maybe I missed something in the docs, but, otherwise, whitespace handling needs to be better documented, and Live Preview possibly needs to be fixed.

I didn't refer to `sample.bnf` from the link you provided; I referenced `Simple.bnf` from https://github.com/JetBrains/intellij-sdk-code-samples/blob/main/simple_language_plugin/src/main/java/org/intellij/sdk/language/Simple.bnf

Ross Goldberg

创建于 2023年08月14日 16:50

Any answers for the above issues? Thanks.

请先登录再写评论。