JFlex avoid bad character

Permanently deleted user

创建于 2014年06月25日 10:27

Hallo,
I guess my question is pretty basic:
I am using JFlex for generating my lexer:

I'm defining the following rule:

SETTING =  ("blocked"|"multiple"|"more_is_better"|"less_is_better")

<SETTINGS>{SETTING}
{
     yybegin(NEXT_STATE); return MyTypes.SETTING;
}

. {
     return TokenType.BAD_CHARACTER;
}

this definition leads to the problem that the tokenizer reads a bad character if the first char is typed. This again leads to bad auto completion (see screenshot). You can see that the first typed char 'm' is not regarded by code completion and I guess it's due to the BAD_CHARACTER.

Can you please tell me how to modify the lexer to not return BAD_CHARACTER if the typed token is the start of SETTING? Especially, which TokenType can I return then? I tried it with TokenType.DUMMY_HOLDER and TokenType.CODE_FRAGMENT.
However I think that is a pretty basic problem and I was hoping for a solution that helps me to understand the Lexer-Algorithms better.
Thanks for any help,
Sebastian

1 条评论

Jon Akhtar

创建于 2014年06月28日 03:31

You only have 1 rule in the defaullt state, the SETTING rule will never execute because you never yybegin(SETTINGS) so every character will be consumed by the default rule "."

For fun place the SETTING rule in the default state:

{SETTING} return MyTypes.SETTING;

请先登录再写评论。