Grammar-Kit: contextual id precedence over keyword


The HowTo/Tutorial documents suggest that keywords are in some sense hoisted above declared tokens in the grammar. There are some situations where I do not want this behavior (i.e. the language allows keywords to be used in certain contexts as literal values or identifiers).

See the screenshot for an example of this scenario. Both 'method' occurrences are determined to be the keyword (determined via the PSI Viewer, but I made it obvious with highlighting options). In this case, I want the 'method' occurrence inside the square brackets (after http) to be lexed as an id token, not the method keyword used elsewhere.

One solution I tried earlier was an alternation in that position in the grammar that began with id and used the keyword as a fall-through. That doesn't appear to do the trick, though.

Does this requirement mean I have to leave PEG and handwrite the logic for contextual disambiguation.

Here's my grammar so far. Critique is welcome.

Comment actions Permalink

one way to disambiguate "token" vs "text" is to use different lexical states in JFlex, depending on current context correct tokens can be returned accordingly.

Comment actions Permalink

I read up on lexical states in lex (I'm fairly new to the tool). I've got everything working now.

Thanks for the pointer.


Please sign in to leave a comment.