Syntax highlight lexer "Error Shifting Segments"

I've almost got a fully working syntax highlighting lexer. However, I've run into a problem I don't know how to solve. Sometimes, when editing the middle of a file (that is being highlighted by my lexer) I get:

[ 273853]  ERROR - pi.editor.ex.util.SegmentArray - Error shifting segments: myStarts[3173] = -1, myEnds[3173] = 4

java.lang.Throwable

 at com.intellij.openapi.diagnostic.Logger.error(Logger.java:113)

 at com.intellij.openapi.editor.ex.util.SegmentArray.shiftSegments(SegmentArray.java:149)

 at com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.documentChanged(LexerEditorHighlighter.java:260)

 at com.intellij.openapi.editor.impl.DocumentImpl.a(DocumentImpl.java:773)

 at com.intellij.openapi.editor.impl.DocumentImpl.a(DocumentImpl.java:716)

 at com.intellij.openapi.editor.impl.DocumentImpl.deleteString(DocumentImpl.java:519)

...

Per: (https://github.com/ceylon/ceylon-ide-intellij/issues/6) this seems to be an issue with incremental invocation.

In particular I've noticed this happens when changing an item from a keyword to not a keyword. That is, if I edit a 'for' statement in the middle of a file, to say 'fo' or 'fora', the change from a keyword, to not a keyword seems to irritate the intelliJ syntax highlight engine. If I change a quotation, this doesn't happen, I presume because editing the middle of a quotation doesn't change the underlying token type (since it's still a quotation).

How do I force the syntax highlighter to recognize that the token type has been changed, and it shouldn't just try to 'shift' it?

Please sign in to leave a comment.