[fr,br] %debug is useless in idea flex lexer

here is simple flex lexer for "my language" plugin

package org.intellij.lang.my;

import com.intellij.lexer.FlexLexer;
import com.intellij.psi.TokenType;
import com.intellij.psi.tree.IElementType;

%%

%class _My
%implements FlexLexer
%function advance
%type IElementType

%debug

%state HELLO, WORLD

%%

<YYINITIAL> {
    [ \t]+          { yybegin(YYINITIAL); return TokenType.WHITE_SPACE; }
    "Hello"         { yybegin(YYINITIAL); return TokenType.BAD_CHARACTER; }
    "World"         { yybegin(YYINITIAL); return TokenType.CODE_FRAGMENT; }
}

after flex compilation we get

  private static java.io.Reader zzReader = null; // Fake


_My(java.io.Reader in) {
    this.zzReader = in;
  }


  /**
   * Runs the scanner on input files.
   *
   * This main method is the debugging routine for the scanner.
   * It prints debugging information about each returned token to
   * System.out until the end of file is reached, or an error occured.
   *
   * @param argv   the command line, contains the filenames to run
   *               the scanner on.
   */
  public static void main(String argv[]) {
...
        _My scanner = null;
        try {
          scanner = new _My( new java.io.FileReader(argv[i]) );
          do {
            System.out.println(scanner.advance());
          } while (!scanner.zzAtEOF);
        }
...
  }

this variable zzReader is never used, so method main() is useless

as workaround i use custom code in flex file

%debug

%{
    _My(java.io.FileReader fileReader) throws java.io.IOException {
        String text = com.intellij.openapi.util.io.FileUtil.loadTextAndClose(fileReader);
        java.nio.CharBuffer chb = java.nio.CharBuffer.wrap(text);
        reset(chb, 0, chb.length(), YYINITIAL);
    }
%}


it works, but it's a bit cumbersome

so, the suggestion is: idea should generate working constructor itself

Please sign in to leave a comment.