Processing newlines

Hi,

In my custom language (using Grammar-Kit, awesome plugin :)), I have statements that can be finished either by a semicolon or by a newline.

The problem is, I can't use the newline token in my grammar as I never receive that token. I just receive

PsiWhitespace
because in my
ParserDefinition.getWhitespaceTokens()
the newline is part of the returned set.

Is there a way of being able to expect a newline in some situations, other than removing the newline token from
getWhitespaceTokens()
and adding newlines everywhere in my grammar?

Thank you in advance!
3 comments

I can't answer for Grammar-Kit, but it is possible if you have a PsiBuilder.

There is a WhitespaceSkippedCallback you can set, but I would say that if newlines are significant in your language they should not be in the whitespace set. The whitespace set is purely for tokens that have no semantic meaning in the language. That is why they are removed during parsing.

0

Yeah, I agree with that, but the problem is I only care at one specific point. The alternative will be adding an optional newline check after every single token or rule, but that seems to be a bit too much... In the end, if nobody has a solution for either Grammar Kit or some IDEA API itself, I'll do that :)

Thank you!

0

I do it in my Lua plugn because in one case a newline is used by the Lua 5.1 parser to resolve an ambigutity.

I let newlines be part of the whitepsace set, but used the PsiBuilder interface to register a listener for skipped whitespace tokens, and then I counted the newlines in that listener.

So. You can do it using PsiBuilder. If grammer-kit generates a PsiBuilder and parsing code you can modify then you can do what I did.

0

Please sign in to leave a comment.