Custom language syntax highlighter glitch?

I'm developing a custom language plugin for LiveScript. I've created basic lexer and grammar parsing and then added syntax highlighting for strings, comments and numbers. LiveScript supports string interpolation, which means you can do this:

"Interpolated string contains #{a / 2 + 2 * 2} expression."

The strings parses fine, and when I first open the file or use Ctrl+V to paste the text, it highlights correctly (see ok.png). But if I change any text after the closing "}", highlighter seems to lose track of where the string begins and ends and messes up the colors (see nok.png). If close and re-open the file, or cut-and-paste the same text, it highlights correctly again.

Is it a problem with my code or something else?

Here's my syntax highlighter class:

 
public class LiveScriptSyntaxHighlighter extends SyntaxHighlighterBase {
   // Strings
   public static final
TextAttributesKey STRING =
         createTextAttributesKey("LS_STRING", DefaultLanguageHighlighterColors.STRING);
   public static final
TextAttributesKey NUMBER =
         createTextAttributesKey("LS_NUMBER", DefaultLanguageHighlighterColors.NUMBER);
   public static final
TextAttributesKey COMMENT_LINE =
         createTextAttributesKey("LS_COMMENT_LINE", DefaultLanguageHighlighterColors.LINE_COMMENT);
   public static final
TextAttributesKey COMMENT_BLOCK =
         createTextAttributesKey("LS_COMMENT_BLOCK", DefaultLanguageHighlighterColors.BLOCK_COMMENT);


   @NotNull
   @Override
   public
Lexer getHighlightingLexer() {
      return new FlexAdapter(new LiveScriptLexer((Reader) null));
   
}

   private TextAttributesKey[] getKeySetFor(TextAttributesKey taKey) {
      return new TextAttributesKey[] {taKey};
   
}

   @NotNull
   @Override
   public
TextAttributesKey[] getTokenHighlights(IElementType tokenType) {
      IElementType[] strings = {
            LiveScriptTypes.STRING,
            
LiveScriptTypes.STRING_START,
            
LiveScriptTypes.STRING_END,
            
LiveScriptTypes.STRING_INTER_START,
            
LiveScriptTypes.STRING_INTER_END,
            
LiveScriptTypes.HEREDOC,
            
LiveScriptTypes.BACKSTRING,
      
};

      for
(IElementType i : strings)
         if (tokenType.equals(i))
            return getKeySetFor(STRING);

      
Map<IElementType, TextAttributesKey> types = new HashMap<IElementType, TextAttributesKey>();
      
types.put(LiveScriptTypes.NUMBER, NUMBER);
      
types.put(LiveScriptTypes.COMMENT_LINE, COMMENT_LINE);
      
types.put(LiveScriptTypes.COMMENT_BLOCK, COMMENT_BLOCK);

      for
(IElementType e : types.keySet())
         if (tokenType.equals(e))
            return getKeySetFor(types.get(e));


      return new
TextAttributesKey[0];
   
}
}


Attachment(s):
nok.PNG
ok.PNG
3 comments
Comment actions Permalink

Does your lexer use multiple states?

0
Comment actions Permalink

Yes, it does. I have project up on GitHub so you can easily see any file: https://github.com/modo-lv/LiveScriptBrains/tree/0363ea144aecffa75d1da3747bbcb2dd06dfeb40

0
Comment actions Permalink

I still don't know what exactly was the problem, but I managed to solve it by rewriting the lexer configuration in a more simplified manner (using less states and avoiding yypushback() calls).

0

Please sign in to leave a comment.