Confucion of ILazyParseableElementType

Hi,

Can somebody please explain How to embed languages using ILazyParseableElementType.
Tutorial says,

"An important feature which can be implemented at lexer level is mixing  languages within a file (for example, embedding fragments of Java code  in some template language). If a language supports embedding its  fragments in another language, it needs to define the chameleon token  types for different types of fragments which can be embedded, and these  token types need to implement the ILazyParseableElementType interface.  The lexer of the enclosing language needs to return the entire fragment  of the embedded language as a single chameleon token, of the type  defined by the embedded language. To parse the contents of the chameleon  token, IDEA will call the parser of the embedded language through a  call to ILazyParseableElementType.parseContents()."


But I don't understand what this chameleon token  types are. And also I don't find any com.intellij.psi.tree.IChameleonElementType class available. And also I don not find ILazyParseableElementType interface. Instead I found ILazyParseableElementType class.


Thanks..!!

5 comments
Comment actions Permalink

You will need to do some leg work.

Look at the Groovy plugin. It uses Chameleon tokens to handle GroovyDoc parsing.

You use ILazyParseableElementType, and basically subclass it or make an anonymous class with it. You make 1 token that represents the entire element (The whole groovy doc comment). And you use that in your main parser.

Later you parse the text of that element using a separate parser for groovydoc.

The groovy example is what I learned from. So just keep at it.

1
Comment actions Permalink

Thank you very much for your help. I looked into groovy plugin. Now I have a general
understanding about what happens.

My lexer returns some html content of HTML_TEXT

    public static final IElementType HTML_TEXT =  new HTMLelementType("HTML_ELEMENT",StdFileTypes.HTML.getLanguage());



HTMLelementType is a class that extends ILazyParseableElementType.
As Far as I understand the whole html content should be parsed. Is'nt that?

But I get the following exception

[  32930]  ERROR - aemon.impl.PassExecutorService - NULL_PSI_ELEMENT
java.lang.IllegalAccessError: NULL_PSI_ELEMENT
     at com.intellij.psi.util.PsiUtilCore$1.getLanguage(PsiUtilCore.java:52)
     at com.intellij.codeInsight.daemon.impl.DefaultHighlightVisitor.a(DefaultHighlightVisitor.java:144)
     at com.intellij.codeInsight.daemon.impl.DefaultHighlightVisitor.visit(DefaultHighlightVisitor.java:100)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass$8.run(GeneralHighlightingPass.java:595)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.a(GeneralHighlightingPass.java:773)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.access$3600(GeneralHighlightingPass.java:80)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass$10.run(GeneralHighlightingPass.java:779)
     at com.intellij.codeInsight.daemon.impl.DefaultHighlightVisitor.analyze(DefaultHighlightVisitor.java:84)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.a(GeneralHighlightingPass.java:776)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.access$3600(GeneralHighlightingPass.java:80)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass$10.run(GeneralHighlightingPass.java:779)
     at com.intellij.codeInsight.daemon.impl.RefCountHolder.analyze(RefCountHolder.java:250)
     at com.intellij.codeInsight.daemon.impl.analysis.HighlightVisitorImpl.analyze(HighlightVisitorImpl.java:135)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.a(GeneralHighlightingPass.java:776)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.a(GeneralHighlightingPass.java:641)
     at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.collectInformationWithProgress(GeneralHighlightingPass.java:203)
     at com.intellij.codeInsight.daemon.impl.ProgressableTextEditorHighlightingPass.doCollectInformation(ProgressableTextEditorHighlightingPass.java:58)
     at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:61)
     at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass$1$1.run(PassExecutorService.java:350)
     at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1053)
     at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass$1.run(PassExecutorService.java:342)
     at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:218)
     at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.a(PassExecutorService.java:340)
     at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:316)
     at com.intellij.concurrency.JobUtil$3.call(JobUtil.java:134)
     at com.intellij.concurrency.JobUtil$3.call(JobUtil.java:131)
     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
     at com.intellij.concurrency.PrioritizedFutureTask.access$101(PrioritizedFutureTask.java:31)
     at com.intellij.concurrency.PrioritizedFutureTask$1.run(PrioritizedFutureTask.java:70)
     at com.intellij.concurrency.PrioritizedFutureTask.run(PrioritizedFutureTask.java:113)
     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
     at java.lang.Thread.run(Thread.java:662)




Where  have I gone wrong?  Your reply is highly appreciated..

Thanks..!!

0
Comment actions Permalink

You need to describe your problem better I think.

The error probably comes from the fact that com.intellij.lang.html.HTMLParserDefinition doesn't create PSI elements.

0
Comment actions Permalink

Hi,

Thanks for replying. My scenario is I need to get syntax highlighting and code formating for a language called jaggery.
It consists of html and javascript.
Javascript is within the <% and %> brackets.

html <%javascript %> html

So I used MultiplePsiFilesPerDocumentFileViewProvider and TemplateLanguageFileViewProvider as described in tutorial
Below is my implementation according to that


///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


public class JaggeryFileViewProvider extends  MultiplePsiFilesPerDocumentFileViewProvider implements  TemplateLanguageFileViewProvider {

    private Language myTemplateDataLanguage1;

    private Language language;

    private Language dataLang;


    public JaggeryFileViewProvider(PsiManager manager, VirtualFile virtualFile, boolean physical) {

        super(manager, virtualFile, physical);

        dataLang = TemplateDataLanguageMappings.getInstance(manager.getProject()).getMapping(virtualFile);

        if (dataLang == null)

        {

            dataLang = StdFileTypes.JS.getLanguage();

        }


        if (dataLang instanceof TemplateLanguage) {

            myTemplateDataLanguage1 = StdFileTypes.HTML.getLanguage();

        } else {


            myTemplateDataLanguage1 =  LanguageSubstitutors.INSTANCE.substituteLanguage(dataLang, virtualFile,  manager.getProject());

        }


    }


    public JaggeryFileViewProvider(PsiManager manager, VirtualFile virtualFile, boolean physical, Language language) {

        super(manager, virtualFile, physical);

        this.myTemplateDataLanguage1 = language;


    }

    @NotNull

    @Override

    public Language getBaseLanguage() {

        return JaggeryLanguage.INSTANCE;  //To change body of implemented methods use File | Settings | File Templates.

    }

    @NotNull

    @Override

    public Language getTemplateDataLanguage() {

        return myTemplateDataLanguage1;  //To change body of implemented methods use File | Settings | File Templates.

    }

    @Override

    protected MultiplePsiFilesPerDocumentFileViewProvider cloneInner(VirtualFile virtualFile) {

        return new JaggeryFileViewProvider(getManager(), virtualFile,  false, myTemplateDataLanguage1);

    }


    @NotNull

    @Override

    public Set<Language> getLanguages() {

        return new THashSet<Language>(Arrays.asList(new Language[]{JaggeryLanguage.INSTANCE, myTemplateDataLanguage1}));

    }

    @Override

    protected PsiFile createFile(Language lang) {

        language = lang;

        if (lang == myTemplateDataLanguage1) {

            PsiFileImpl file = (PsiFileImpl)  LanguageParserDefinitions.INSTANCE.forLanguage(lang).createFile(this);  

            file.setContentElementType(
JaggeryTokenTypes.TEMPLATE_DATA);

            return file;


        } else if (lang == JaggeryLanguage.INSTANCE)

            return LanguageParserDefinitions.INSTANCE.forLanguage(JaggeryLanguage.INSTANCE).createFile(this);

        else return null;


    }

}

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////



Then in JaggeryTokenTypes class




public static final IElementType TEMPLATE_JAVASCRIPT_TEXT = new JaggeryElementType("TEMPLATE_JAVASCRIPT_TEXT");
public static final IElementType OUTER_ELEMENT_TYPE = new JaggeryElementType("OUTER_ELEMENT_TYPE");





public static  TemplateDataElementType TEMPLATE_DATA =

    new TemplateDataElementType("JAGGERY_TEMPLATE_DATA", JaggeryLanguage.INSTANCE, TEMPLATE_JAVASCRIPT_TEXT,OUTER_ELEMENT_TYPE );



    public static final IElementType JAVASCRIPT_TEXT = TEMPLATE_JAVASCRIPT_TEXT;
    public static final IElementType HTML_TEXT =  new HTMLelementType("HTML_OUTER_ELEMENT",StdFileTypes.HTML.getLanguage());
     public static final IElementType OUTERBRACKETS = new JaggeryElementType("OUTERBRACKETS");

//please note that syntax highlighting for html is done extending LayeredLexerEditorHighlighter


///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


HTMLelementType would be as follows.


public class HTMLelementType extends ILazyParseableElementType {
    public HTMLelementType(@org.jetbrains.annotations.NotNull @org.jetbrains.annotations.NonNls String debugName, @org.jetbrains.annotations.Nullable Language language) {
        super(debugName, language);

}

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////



And my lexer returns  JaggeryTokenTypes.HTML_TEXT    for HTML content
And                          JaggeryTokenTypes.JAVASCRIPT_TEXT   for Javascript content
And <,%,>          JaggeryTokenTypes.OUTERBRACKETS          for alien outerbrackets

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Hope it is clear so. So can't we use ILazyParseableElementType to parse the content of html
like that.
Or else is there another way to parse within the JaggeryFileViewProvider.
  using
        LanguageParserDefinitions.INSTANCE.forLanguage(lang).createFile(this)



Please clarify me.

Thanks for the help..:)  !!!!!

0

Please sign in to leave a comment.