Some methods in parser definition are not invoked

Hi,


My Syntax highlighting is wrking perfectly. And now I am implementing the parser.
The document says to use grammer kit plugin. Is there any simple example to learn how the plugin works?

And I found out that

in my parser definition below methods are invoked.

  • createLexer
  • createFile
  • getFileNodeType
  • getWhitespaceTokens
  • getCommentTokens


methods

  • createParser
  • getStringLiteralElements
  • createElement


are never invoked.(I am saying that because print statement put at the beggining of the 3 methods does'nt print anything).

Instead I get the following error..

[  35095]  ERROR - currency.PrioritizedFutureTask - java.lang.NullPointerException
java.util.concurrent.ExecutionException: java.lang.NullPointerException
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
    at java.util.concurrent.FutureTask.get(FutureTask.java:83)
    at com.intellij.concurrency.PrioritizedFutureTask.access$301(PrioritizedFutureTask.java:31)
    at com.intellij.concurrency.PrioritizedFutureTask$1.run(PrioritizedFutureTask.java:77)
    at com.intellij.concurrency.PrioritizedFutureTask.run(PrioritizedFutureTask.java:113)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
    at newair.org._jaggery2.advance(_jaggery2.java:587)
    at com.intellij.lexer.FlexAdapter.locateToken(FlexAdapter.java:93)
    at com.intellij.lexer.FlexAdapter.getTokenType(FlexAdapter.java:57)
    at com.intellij.lang.impl.PsiBuilderImpl.cacheLexemes(PsiBuilderImpl.java:208)
    at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:176)
    at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:150)
    at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:184)
    at com.intellij.lang.impl.PsiBuilderFactoryImpl.createBuilder(PsiBuilderFactoryImpl.java:52)
    at com.intellij.psi.tree.ILazyParseableElementType.doParseContents(ILazyParseableElementType.java:62)
    at com.intellij.psi.tree.IFileElementType.parseContents(IFileElementType.java:39)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.ensureParsed(LazyParseableElement.java:160)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(LazyParseableElement.java:204)
    at com.intellij.psi.impl.source.tree.LazyParseableElement.getFirstChildNode(LazyParseableElement.java:32)
    at com.intellij.psi.impl.source.tree.SharedImplUtil.getFirstChild(SharedImplUtil.java:44)
    at com.intellij.psi.impl.source.PsiFileImpl.getFirstChild(PsiFileImpl.java:757)
    at com.intellij.psi.SingleRootFileViewProvider.findElementAt(SingleRootFileViewProvider.java:414)
    at com.intellij.psi.SingleRootFileViewProvider.findElementAt(SingleRootFileViewProvider.java:400)
    at com.intellij.psi.impl.source.PsiFileImpl.findElementAt(PsiFileImpl.java:558)
    at com.intellij.psi.util.PsiUtilCore.getElementAtOffset(PsiUtilCore.java:389)
    at com.intellij.psi.util.PsiUtilBase.getLanguageInEditor(PsiUtilBase.java:148)
    at com.intellij.psi.util.PsiUtilBase.getPsiFileInEditor(PsiUtilBase.java:183)
    at com.intellij.codeInsight.highlighting.BraceHighlightingHandler$1$1.compute(BraceHighlightingHandler.java:119)
    at com.intellij.codeInsight.highlighting.BraceHighlightingHandler$1$1.compute(BraceHighlightingHandler.java:116)
    at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:892)
    at com.intellij.codeInsight.highlighting.BraceHighlightingHandler$1.run(BraceHighlightingHandler.java:116)
    at com.intellij.concurrency.JobUtil$3.call(JobUtil.java:134)
    at com.intellij.concurrency.JobUtil$3.call(JobUtil.java:131)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at com.intellij.concurrency.PrioritizedFutureTask.access$101(PrioritizedFutureTask.java:31)
    at com.intellij.concurrency.PrioritizedFutureTask$1.run(PrioritizedFutureTask.java:70)
    ... 4 more
[  35098]  ERROR - currency.PrioritizedFutureTask - IntelliJ IDEA 11.1.2  Build #IC-117.418
[  35098]  ERROR - currency.PrioritizedFutureTask - JDK: 1.6.0_32
[  35099]  ERROR - currency.PrioritizedFutureTask - VM: Java HotSpot(TM) 64-Bit Server VM
[  35099]  ERROR - currency.PrioritizedFutureTask - Vendor: Sun Microsystems Inc.
[  35099]  ERROR - currency.PrioritizedFutureTask - OS: Linux
[  35099]  ERROR - currency.PrioritizedFutureTask - Last Action:  
[  35099]  ERROR - currency.PrioritizedFutureTask - Original exception:  




Looking forward for any idea what is going on.

Thank You..!!

3 comments
Comment actions Permalink

The Grammar-Kit plugin itself is an example of how to write a grammar and generate a working parser. I am also writing a plugin based on Grammar-Kit, and I found some useful code in a plugin for the R language (https://code.google.com/p/r4intellij/). I also found another plugin based on Grammar-Kit which helped me understand how it works: http://code.google.com/p/ice-framework-idea-plugin/

There's a lot of trial and error before understanding how it works exactly, I had quite a lot of problems in the past two weeks, but in the end I'm very proud of what I have achieved so far :).

I guess you have an error in your Flex file, which is causing the NPE. From what I understood, it should be very "dumb", and not implement any rule (the BNF/parser will do that for you). You should only have statements like this:

"your_token" { return YourLanguage.THE_TOKEN_TYPE; }



If you decide to use Grammar-Kit, you will have to adapt your Flex file anyway to use the TokenTypes generated by Grammar-Kit.

Good luck!
0
Comment actions Permalink

Hi Bastien,

Thanks a lot for the example given, This error happens when I invoke the parserdefinition.

Below is my lexer




////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


package newair.org;

import com.intellij.lexer.FlexLexer;
import com.intellij.psi.tree.IElementType;


%%



%public
%class _jaggery2
%implements FlexLexer
%unicode
%function advance
%type IElementType
%eof{ return;
%eof}


/*line terminators */


   LineTerminator = \r|\n|\r\n
   InputCharacter = [^\r\n]

      WhiteSpace = {LineTerminator} | [ \t\f]+

    /* comments */


     TraditionalComment = "/*" [^*] ~"*/" | "/*" "*"+ "/"
     LineComment = "//" {InputCharacter}* {LineTerminator}

    /* identifiers */


     DecIntegerLiteral = 0 | [1-9][0-9]*



%state STRING

%%

     /*variables*/


      <YYINITIAL> {

     "request"  |
      "response" |
      "session"  |
      "application"  {return JaggeryTokenTypes.VARIABLE;}

     }



    /*output*/


      <YYINITIAL> {

     "print"  |
      "Log"   {return JaggeryTokenTypes.OUTPUT;}

     }

      /*Http Client*/


           <YYINITIAL> {

         "get"  |
          "post" |
          "put"  |
          "del"  |
          "XMLHttpRequest" {return JaggeryTokenTypes.CLIENT;}


           }


        /*utils*/


        <YYINITIAL> {

         "URIMatcher"  |
          "include"       {return JaggeryTokenTypes.UTILS;}


         }



        /*Feed*/


         <YYINITIAL> {

         "Feed"  |
          "Entry"       {return JaggeryTokenTypes.FEED;}


          }



          /*File System*/


         <YYINITIAL> {


          "File"       {return JaggeryTokenTypes.FILESYS;}

        }






          /*Data Storage*/


          <YYINITIAL> {

         "Database"  |
          "MetadataStore"       {return JaggeryTokenTypes.STORAGE;}


          }



           /*Dataformats*/


            <YYINITIAL> {

            "parse"  |
             "stringify"|
             "XML"        {return JaggeryTokenTypes.DATAFORMAT;}


          }



          /*Add-ons*/


            <YYINITIAL> {


              "Sender"     |
              "WSRequest"  |
              "WSStub"     {return JaggeryTokenTypes.ADDONS;}



         }


<YYINITIAL>{
       {LineTerminator}     {return JaggeryTokenTypes.LINE_TERMINATOR;}
     {WhiteSpace} {return JaggeryTokenTypes.WHITE_SPACE;}
     {TraditionalComment}     {return JaggeryTokenTypes.BLOCKCOMMENT;}
     {LineComment}            {return JaggeryTokenTypes.LINECOMMENT;}


     {DecIntegerLiteral} {return JaggeryTokenTypes.INTEGER;}

    \"                   { yybegin(STRING); }

}



   <STRING> {
   \"      { yybegin(YYINITIAL);

            return JaggeryTokenTypes.STRING;


           }
    }



.    {
         yybegin(YYINITIAL);
         return JaggeryTokenTypes.BAD_CHARACTER; }






//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


What am I doing wrong here. And one thing I noticed in your lexer.java is that there is zzBufferArray which is not created in mine.
Your kind help is highly appreciated




Thanks..!!
















0
Comment actions Permalink

Hi,

I think I found the mistake. I had put wrong skeleton file. It should be idea-flex.skeleton.flex. And after replacing zzBuffer[zzCurrentPosL++] by zzBufferL.charAt(zzCurrentPosL++)
issues solved.

Thanks.!!

0

Please sign in to leave a comment.