Grammar-Kit BNF complete syntax

Answered
Avatar
Ross Goldberg
Created

Is a complete syntax for the Grammar-Kit BNF file format available anywhere?

The existing information seems incomplete & disjoint.

Votes

0

Share

6 comments
Avatar
Karol Lewandowski
Created

Hi Ross,

Yes, see: https://github.com/JetBrains/Grammar-Kit/blob/master/grammars/Grammar.bnf

Could you also please point to what is incomplete and what problem you have? It'll help us improve the docs.

0
Avatar
Ross Goldberg
Created

Sorry, I should have said complete language spec, because I would like more than just the syntax.

I have problems because the existing overview at https://github.com/JetBrains/Grammar-Kit/tree/master is confusing. Simple bnf file attempts produce odd errors in Live Preview. I could indicate where the documentation is lacking / confusing if I learn why the simple bnf attempts are failing in Live Preview. e.g.,

x ::= a | 'b' | c

Gives a BAD_CHARACTER error on b and a BAD_CHARACTER error on the trailing newline in:

a b c

The docs seem to indicate that there shouldn't be an error for b, so what's wrong with the above bnf?

How do I fix the newline error? I have seen bnf files that don't seem to explicitly deal with newlines; I imagine that they don't have errors on every newline, so I'm not sure why this is occurring.

Also, since the newline causes an error, why aren't the spaces causing errors?

0
Avatar
Ross Goldberg
Created

Every single bnf file I've tried has errors in the Live Preview. Grammar.bnf, Simple.bnf from the simple_language_plugin example, etc.

Something seems to be incredibly wrong with the Live Preview in 2023.2.

0
Avatar
Karol Lewandowski
Created

Hi Ross,

All the available resources are in the Grammar-Kit repository. Thank you for reporting the lack of the BNF language reference. We planned to improve Grammar-Kit documentation, and reporting problems helps us to prioritize the work.

I just checked your grammar example, and it works, but with slight changes.
'b' is a string literal and needs to be defined in tokens:

{
  tokens=[
    b='b'
  ]
}
x ::= a | 'b' | c

Also, the rule is a choice, so only a, b, or c can be parsed correctly.

I also checked sample.bnf from https://github.com/JetBrains/Grammar-Kit/blob/master/TUTORIAL.md, and it correctly parses the example code.

​
0
Avatar
Ross Goldberg
Created

From https://github.com/JetBrains/Grammar-Kit/tree/master#tokens:

Implicit tokens are tokens not specified via tokens attribute. Unquoted implicit tokens (aka keyword tokens) have names equals to their values. Quoted implicit tokens (aka text-matched tokens) are slow because they are matched by text and not by an IElementType constant returned by a lexer. Text-matched tokens can span more than one real token returned by the lexer.

How can you have "quoted implicit tokens" if quoted tokens that aren't in the global `tokens` list don't work?

(also, the 's' from the end of "equals" should be deleted in: "have names equals to their values". i.e. it should read "have names equal to their values").

I knew that `a | 'b' | c` is a choice, but that doesn't explain why Live Preview gives an error about the newline but not about the spaces. Maybe I missed something in the docs, but, otherwise, whitespace handling needs to be better documented, and Live Preview possibly needs to be fixed.

I didn't refer to `sample.bnf` from the link you provided; I referenced `Simple.bnf` from https://github.com/JetBrains/intellij-sdk-code-samples/blob/main/simple_language_plugin/src/main/java/org/intellij/sdk/language/Simple.bnf

1
Avatar
Ross Goldberg
Created

Any answers for the above issues? Thanks.

0

Please sign in to leave a comment.