Perl5 plugin for Intellij IDEA

Helped, fixed the problem. Thanks

Helped. Fixed the problem. Thanks.

Ok, new problem: is there a way to distinct XML token from HTML one?
My hilighter dont' know which hilighter it should call (XML or HTML). It's always xml one:


Currently, my hilighter checking token's language and calls appropriate hilighter. But it doesn't work for html/xml.

Token types are the same for XML and HTML. The language is usually determined by their parents, which are language-specific. For highlighter purposes, I'm afraid, you have to specify the language somewhere in the highlighter code and guess the language there by the context.

One more question about this task. Currently, to implement adaptive highlighting in multiline blocks I have several token types: multiline, multiline_html, multiline_xml, etc. Because LayeredLexer should be set up by token type.
Is there other way? I'd like to have one multiline token and be able to choose what lexer i should use for this particular block.

I'm afraid not with LayeredLexerEditorHighlighter. Even in bare LexerEditorHighlighter there seems to be too little context for you to implement a non-trivial lexer selection algorithm.

What is the best way to make and use my own LexerEditorHighlighter?

Ok, i've solved this by implementing my own LayeredLexerAdaptive with extended logic. Was not possible to inherit from LayeredLexer so i've cloned it and extended a bit.

Delete. Figured out myself.

How to make my own project type with specific settings?
I need to make Perl project, installation path, additional library paths and so on.

Ok, some concept question:

Perl is a really context-dependent:

word word can be:

• method class

• function function
• function handle

Depedning on what happened before. So i need to implement a lot of analysis into the parser. Will I be able to reuse this analysis in annotator, for example or will i need to re-analyse tree again?

I did it and it works fine, but...
When i'm starting to type new statement, all annotations turns off and back on only after i'm completing valid statement. So code is blinkning all the time :)
What can i do with this?

Which PSI elements are the annotations attached to? Please try to attach them to leaf elements whenever possible (e.g. name identifiers). Otherwise the highlighting might notice that the underlying element has changed (which by itself may indicate an issue of a too large incremental reparse) and remove all associated highlights immediately, before newer ones are available.

There's no such thing as project type in IDEA. When creating a project, you can provide your own ModuleBuilder that allows you to give users a convenient UI to set up a project. When the project is ready, the configuration will probably be saved in a Facet (and its FacetConfiguration) attached to one of the modules. Or you can have project- or IDE-wide setting page for Perl by having a Project/ApplicationService that saves the settings and a Configurable page in the settings UI.

I don't know much about Perl, so I'll answer on a very abstract level.

If you need to implement lots of analysis in the parser, that's fine. It probably means that you'll reflect the results of the analysis in the PSI tree structure: in element types, how nodes are nested etc. This is trivially reusable in the annotator. AFAIK there's no good way of reusing more information, e.g. some additional structures you've built in the process of parsing. OTOH complicated parsing likely means that it'll be slow and not very incremental, which is obviously a problem.

You could also think about having a very simple PSI structure that's easy to be produced in the parser, and defer all the complicated analysis until it's needed in annotators, completion etc. Then you can cache and share it between all these clients quite easily. If it's possible to do this, I'd go this way.

Another way: parse the file twice. The first time you extract some important information (if you do this via FileBasedIndexExtension, IDEA will cache it on disk for you), and the second part, in your actual PSI parser, will be guided by this important information and make decisions based on it. That might work, but there's so many things that could go wrong here, so I'd only advise to do this as a last resort.

Ok, lexer and parser are in beta-version and i am using plugin in my daily work. It's already better than NP++ but still worse than other IDE's :)

Question:
I need to implement PHP-style perl code, when perl is mixed with HTML with same format:

<html> <? some perl here ?> </html>

Perl code in one file should be parsed as one file, not separately, like independent blocks.
How should i do this?

it seems that i need to use fileViewProvider, but not sure about it and not sure how.

Would be nice to get an advice.
Thanks.

That's quite a lot to do, actually. You need to have several PSI trees in one file. You should have a custom view provider (extends MultiplePsiFilesPerDocumentFileViewProvider implements TemplateLanguageFileViewProvider) for that, have PhpStylePerl as a base language and HTML as the template data language. Both files should build PSI on the same text, but skip the parts that belong to another language by dedicating special token types to them, skipping them in the parser and creating OuterLanguageElement PSI for these tokens.

The best start would be to check out CFML plugin (https://github.com/JetBrains/intellij-plugins/blob/master/CFML/src/com/intellij/coldFusion/model/files/CfmlFileViewProviderFactory.java) and get inspiration from its source.

Ok, i've digged a bit.

Am I right, that my Perl lexer should understand and lex HTML and Perl and than, we are building two trees from mixed tokens? What mean's that I can't reuse my existing Perl lexer and HTML lexer and make them work together. Actually, i need to write lexer for perl + html.

I thought that it works like some lexer separates html from perl and than, perl's and html's lexers do their jobs from pure sources (kinda like Chameleon tokens, but as one source).

Your base language (be it Perl or not, I don't know if you want to have Perl without HTML) should understand Perl and have a notion of HTML fragments. It should lex these fragments as tokens of a special type (as in CfmlElementTypes#TEMPLATE_TEXT). But it shouldn't be able to fully lex HTML, it can just understand where a fragment starts and where it ends. There are probably some markers of it in the text.

The template data file will have a special content element type (as in CfmlElementTypes#TEMPLATE_DATA) that'll do the lexing for you. You'll still need to build a highlighting lexer yourself, but that's quite easy (see CfmlHighlighter).

Thank you.
Yes, they can be together and separately and i need them both and i guess i figured out how to handle them in DRY.

Works like a charm, thanks :)

First question:

In the beginning i've asked about highlighting HTML, SQL and so on. It seems to me now, that i can do it using the same way as PHP-style perl and have html/sql autocompletion and in SQL case will be able to work with data sources. Am I right?
If I am, is it possible to make optional (lazy) support of features. For example, activate sql support if there is SQL Support plugin and treat queries like strings if there is no SQL support.

Second question:

I'd like to make templates for new Perl files. Some of them should be dumb (static templates), others should be smart, like class templates, which name should depend on filename and it's position if files ierarchy. Where start to dig?

Yes, you can have other language support with multiple-PsiFile approach. But if you only need self-contained fragments that don't depend on other fragments in the same file (e.g. for resolve), you can also use language injection (PsiLanguageInjectionHost) or just embed them into your main tree with ILazyParseableElementType. And you can define an optional plugin dependency on SQL support plugin, as described at https://confluence.jetbrains.com/display/IDEADEV/IntelliJ+IDEA+Plugin+Structure.

Second: please see Settings | File and Code Templates. There are many examples there, some relatively "smart". Templates should be packaged in "fileTemplates" directory of your plugin jar, e.g. https://github.com/JetBrains/intellij-community/tree/3f7e93e20b7e79ba389adf593b3b59e46a3e01d1/plugins/groovy/groovy-psi/resources/fileTemplates

Are you saying that in fileViewProvider solution template blocks being parsed as a whole, not like independent blocks?

It depends on your code, but mostly it's used for templating languages, where data language (e.g. HTML) fragments are interspersed with templating directives (e.g. PHP). But nevertheless these data language fragments are parts of a whole, so they should be treated as such and be parts of a single PSI tree. Language injection or lazy-parseable elements can have independent contents, e.g. a Java file may contain several SQL statements in its string literals, and they are completely different and not related at all. That's one of the reasons we use injection for them, and not multi-root view provider.

Choosing between injection and lazy-parseable elements is simple: if you can determine the areas of another language when lexing/parsing, choose the latter. Otherwise - injection.

Question about smart auto-completion. Tutorial's example contributors as based on token type from lexer. What if i need more, for example, to check siblings or parent in the PSI tree?

Sorry, I don't understand the question. In general, you should implement CompletionContributor where you have access to the whole PSI tree. This class also has an extensive javadoc about completion.

Is there trick with code folding on multi-tree documents?

What trick? There's nothing special about folding in such files.

Sorry, my bad. Forgot that it's a different language and i need to specify FoldingBuilder for it.

In last month i've got so much new information, sometimes feels like my head going to blow up. And sometimes i become very stupid, sorry :)