Processing newlines


In my custom language (using Grammar-Kit, awesome plugin :)), I have statements that can be finished either by a semicolon or by a newline.

The problem is, I can't use the newline token in my grammar as I never receive that token. I just receive

because in my
the newline is part of the returned set.

Is there a way of being able to expect a newline in some situations, other than removing the newline token from
and adding newlines everywhere in my grammar?

Thank you in advance!
Comment actions Permalink

I can't answer for Grammar-Kit, but it is possible if you have a PsiBuilder.

There is a WhitespaceSkippedCallback you can set, but I would say that if newlines are significant in your language they should not be in the whitespace set. The whitespace set is purely for tokens that have no semantic meaning in the language. That is why they are removed during parsing.

Comment actions Permalink

Yeah, I agree with that, but the problem is I only care at one specific point. The alternative will be adding an optional newline check after every single token or rule, but that seems to be a bit too much... In the end, if nobody has a solution for either Grammar Kit or some IDEA API itself, I'll do that :)

Thank you!

Comment actions Permalink

I do it in my Lua plugn because in one case a newline is used by the Lua 5.1 parser to resolve an ambigutity.

I let newlines be part of the whitepsace set, but used the PsiBuilder interface to register a listener for skipped whitespace tokens, and then I counted the newlines in that listener.

So. You can do it using PsiBuilder. If grammer-kit generates a PsiBuilder and parsing code you can modify then you can do what I did.


Please sign in to leave a comment.