Some methods in parser definition are not invoked


My Syntax highlighting is wrking perfectly. And now I am implementing the parser.
The document says to use grammer kit plugin. Is there any simple example to learn how the plugin works?

And I found out that

in my parser definition below methods are invoked.

  • createLexer
  • createFile
  • getFileNodeType
  • getWhitespaceTokens
  • getCommentTokens


  • createParser
  • getStringLiteralElements
  • createElement

are never invoked.(I am saying that because print statement put at the beggining of the 3 methods does'nt print anything).

Instead I get the following error..

[  35095]  ERROR - currency.PrioritizedFutureTask - java.lang.NullPointerException
java.util.concurrent.ExecutionException: java.lang.NullPointerException
    at java.util.concurrent.FutureTask$Sync.innerGet(
    at java.util.concurrent.FutureTask.get(
    at com.intellij.concurrency.PrioritizedFutureTask.access$301(
    at com.intellij.concurrency.PrioritizedFutureTask$
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
    at java.util.concurrent.ThreadPoolExecutor$
Caused by: java.lang.NullPointerException
    at com.intellij.lexer.FlexAdapter.locateToken(
    at com.intellij.lexer.FlexAdapter.getTokenType(
    at com.intellij.lang.impl.PsiBuilderImpl.cacheLexemes(
    at com.intellij.lang.impl.PsiBuilderImpl.<init>(
    at com.intellij.lang.impl.PsiBuilderImpl.<init>(
    at com.intellij.lang.impl.PsiBuilderImpl.<init>(
    at com.intellij.lang.impl.PsiBuilderFactoryImpl.createBuilder(
    at com.intellij.psi.tree.ILazyParseableElementType.doParseContents(
    at com.intellij.psi.tree.IFileElementType.parseContents(
    at com.intellij.psi.impl.source.tree.LazyParseableElement.ensureParsed(
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(
    at com.intellij.psi.impl.source.tree.SharedImplUtil.getFirstChild(
    at com.intellij.psi.impl.source.PsiFileImpl.getFirstChild(
    at com.intellij.psi.SingleRootFileViewProvider.findElementAt(
    at com.intellij.psi.SingleRootFileViewProvider.findElementAt(
    at com.intellij.psi.impl.source.PsiFileImpl.findElementAt(
    at com.intellij.psi.util.PsiUtilCore.getElementAtOffset(
    at com.intellij.psi.util.PsiUtilBase.getLanguageInEditor(
    at com.intellij.psi.util.PsiUtilBase.getPsiFileInEditor(
    at com.intellij.codeInsight.highlighting.BraceHighlightingHandler$1$1.compute(
    at com.intellij.codeInsight.highlighting.BraceHighlightingHandler$1$1.compute(
    at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(
    at com.intellij.codeInsight.highlighting.BraceHighlightingHandler$
    at com.intellij.concurrency.JobUtil$
    at com.intellij.concurrency.JobUtil$
    at java.util.concurrent.FutureTask$Sync.innerRun(
    at com.intellij.concurrency.PrioritizedFutureTask.access$101(
    at com.intellij.concurrency.PrioritizedFutureTask$
    ... 4 more
[  35098]  ERROR - currency.PrioritizedFutureTask - IntelliJ IDEA 11.1.2  Build #IC-117.418
[  35098]  ERROR - currency.PrioritizedFutureTask - JDK: 1.6.0_32
[  35099]  ERROR - currency.PrioritizedFutureTask - VM: Java HotSpot(TM) 64-Bit Server VM
[  35099]  ERROR - currency.PrioritizedFutureTask - Vendor: Sun Microsystems Inc.
[  35099]  ERROR - currency.PrioritizedFutureTask - OS: Linux
[  35099]  ERROR - currency.PrioritizedFutureTask - Last Action:  
[  35099]  ERROR - currency.PrioritizedFutureTask - Original exception:  

Looking forward for any idea what is going on.

Thank You..!!

Comment actions Permalink

The Grammar-Kit plugin itself is an example of how to write a grammar and generate a working parser. I am also writing a plugin based on Grammar-Kit, and I found some useful code in a plugin for the R language ( I also found another plugin based on Grammar-Kit which helped me understand how it works:

There's a lot of trial and error before understanding how it works exactly, I had quite a lot of problems in the past two weeks, but in the end I'm very proud of what I have achieved so far :).

I guess you have an error in your Flex file, which is causing the NPE. From what I understood, it should be very "dumb", and not implement any rule (the BNF/parser will do that for you). You should only have statements like this:

"your_token" { return YourLanguage.THE_TOKEN_TYPE; }

If you decide to use Grammar-Kit, you will have to adapt your Flex file anyway to use the TokenTypes generated by Grammar-Kit.

Good luck!
Comment actions Permalink

Hi Bastien,

Thanks a lot for the example given, This error happens when I invoke the parserdefinition.

Below is my lexer



import com.intellij.lexer.FlexLexer;
import com.intellij.psi.tree.IElementType;


%class _jaggery2
%implements FlexLexer
%function advance
%type IElementType
%eof{ return;

/*line terminators */

   LineTerminator = \r|\n|\r\n
   InputCharacter = [^\r\n]

      WhiteSpace = {LineTerminator} | [ \t\f]+

    /* comments */

     TraditionalComment = "/*" [^*] ~"*/" | "/*" "*"+ "/"
     LineComment = "//" {InputCharacter}* {LineTerminator}

    /* identifiers */

     DecIntegerLiteral = 0 | [1-9][0-9]*

%state STRING



      <YYINITIAL> {

     "request"  |
      "response" |
      "session"  |
      "application"  {return JaggeryTokenTypes.VARIABLE;}



      <YYINITIAL> {

     "print"  |
      "Log"   {return JaggeryTokenTypes.OUTPUT;}


      /*Http Client*/

           <YYINITIAL> {

         "get"  |
          "post" |
          "put"  |
          "del"  |
          "XMLHttpRequest" {return JaggeryTokenTypes.CLIENT;}



        <YYINITIAL> {

         "URIMatcher"  |
          "include"       {return JaggeryTokenTypes.UTILS;}



         <YYINITIAL> {

         "Feed"  |
          "Entry"       {return JaggeryTokenTypes.FEED;}


          /*File System*/

         <YYINITIAL> {

          "File"       {return JaggeryTokenTypes.FILESYS;}


          /*Data Storage*/

          <YYINITIAL> {

         "Database"  |
          "MetadataStore"       {return JaggeryTokenTypes.STORAGE;}



            <YYINITIAL> {

            "parse"  |
             "XML"        {return JaggeryTokenTypes.DATAFORMAT;}



            <YYINITIAL> {

              "Sender"     |
              "WSRequest"  |
              "WSStub"     {return JaggeryTokenTypes.ADDONS;}


       {LineTerminator}     {return JaggeryTokenTypes.LINE_TERMINATOR;}
     {WhiteSpace} {return JaggeryTokenTypes.WHITE_SPACE;}
     {TraditionalComment}     {return JaggeryTokenTypes.BLOCKCOMMENT;}
     {LineComment}            {return JaggeryTokenTypes.LINECOMMENT;}

     {DecIntegerLiteral} {return JaggeryTokenTypes.INTEGER;}

    \"                   { yybegin(STRING); }


   <STRING> {
   \"      { yybegin(YYINITIAL);

            return JaggeryTokenTypes.STRING;


.    {
         return JaggeryTokenTypes.BAD_CHARACTER; }


What am I doing wrong here. And one thing I noticed in your is that there is zzBufferArray which is not created in mine.
Your kind help is highly appreciated


Comment actions Permalink


I think I found the mistake. I had put wrong skeleton file. It should be idea-flex.skeleton.flex. And after replacing zzBuffer[zzCurrentPosL++] by zzBufferL.charAt(zzCurrentPosL++)
issues solved.



Please sign in to leave a comment.