How to test an IntelliJ-JFlex Lexer without a parser?

I first want to say that the tutorial for creating a custom language plugin has been pretty helpful, but I'm now stuck trying to figure out how to use the Lexer. I first tried doing this with my own custom language, then reproduced the problem by following the 'Simple' example nearly exactly. The only exception is that I am not generating a parser, as I do not want to worry about the complexity of a parser for the language I'm making a plugin for at the moment.


Here is my test code:

@Test
public void testAdvance() throws Exception {
   IElementType token;
   int i = 0;
   myReader = new java.io.StringReader("# You are reading the \".properties\" entry.\n" +
           "! The exclamation mark can also mark text as comments.\n" +
           "website = http://en.wikipedia.org/");
   myLexer = new SimpleLexer(myReader);
   System.out.println(myLexer.yylength());
   token = myLexer.advance();
   System.out.println(myLexer.yylength());
   System.out.println(token.toString());

   // Temp test:
   char[] mycbuf = new char[100];
   myReader.read(mycbuf,0,99);
   System.out.println(mycbuf);

}



The result of running this JUnit test is no token output:


0
0


java.lang.NullPointerException
 at test.com.simpleplugin.SimpleLexerTest.testAdvance(SimpleLexerTest.java:145)


Process finished with exit code -1

Here is what the project looks like:

SimplePlugin.PNG

For easy reproducability, here is the entire repository as it stands:
https://github.com/bbarker/SimpleIntelliJPlugin

2 comments
Comment actions Permalink

The API that you're trying to test is not the one that IntelliJ uses for working with lexers. Instead of passing the text as Reader to the constructor, you need to create a SimpleLexerAdapter in your test, and to call the start() method passing the text to lex as a parameter.

0
Comment actions Permalink

Thanks for the very quick response!

I was able to add a new test as you suggested, which worked. I'm excited to move back to my real plugin.

0

Please sign in to leave a comment.