Syntax highlighing stops after 5 symbols Follow

Could you please link to your full repository?

Here is the repository at the time of writing the Stack Overflow question

Minecraft.flex: missing required %eof definition

%class MinecraftLexer
%implements FlexLexer
%unicode
%function advance
%type IElementType
%eof{  return;
%eof}

I'd suggest continuing with Lexer/Parser-tests before continuing to build BNF/PSI to stabilize development continuously. See https://plugins.jetbrains.com/docs/intellij/parsing-test.html

ParserDefinition#createElement will need to return this after making `item` rule public to generate _any_ PSI

return MinecraftTypes.Factory.createElement(node)

I did what you said, which you can see in the updated GitHub repo, but the issue is not resolved.

additional notes, TBC

1) You cannot use COMMENT in your Grammar public rule 'item' AND use it in org.intellij.sdk.language.MinecraftParserDefinition#getCommentTokens simultaneously.

2) public rules in BNF are simple written as 'ruleName ::= <content>', not pre-fixed with 'public' keyword

try starting with simplified

minecraftFile ::= (item|comment)*

private comment ::= COMMENT

item ::= (COMMAND|ARGUMENT|SELECTOR|NUMBER|STRING)

Basic lexer tests can look like

import com.intellij.lexer.Lexer;
import com.intellij.testFramework.LexerTestCase;
import org.intellij.sdk.language.MinecraftLexerAdapter;

public class MinecraftLexerTest extends LexerTestCase {

    public void testOne() {
        doTest("function as @e",
                "MinecraftTokenType.COMMAND ('function')\n" +
                        "WHITE_SPACE (' ')\n" +
                        "MinecraftTokenType.ARGUMENT ('as')\n" +
                        "WHITE_SPACE (' ')\n" +
                        "MinecraftTokenType.SELECTOR ('@e')");
    }

    @Override
    protected Lexer createLexer() {
        return new MinecraftLexerAdapter();
    }

    @Override
    protected String getDirPath() {
        return null;
    }
}

I have again implemented what you are saying, but the problem still persists. (GitHub repo if you care)

To reiterate, the PSI tree works 100% as expected, but for phrases longer than 4 characters I need to reload the file for them to get highlighted in their color. They still show up in the PsiViewer while writing, but doesn't get highlighted before reloaded.

We (hajdam and myself during a twitch live stream) looked over your code and provide a pull request that fixes the issue. I'm not entirely sure if it was completely your mistake but here is what we discovered by debugging the code:

It seems that providing single bad characters does not work well with the highlighting engine. When you inspect your original implementation and you type, e.g.

func

what you would expect is that the whole word "func" is highlighted in red, but this is not the case since you return bad characters one at a time. So when you type the final "n" of "function", IntelliJ has a chain of 7 wrong characters and it appears that the highlighting engine does not work correctly in that situation (meaning, it doesn't update properly).

The solution we found was that you should return several bad characters as one BAD_CHARACTER token. At this point, I'm not sure if this is a problem of the IntelliJ Platform or if it's always been like that. We'll discuss this internally.