Perl5 plugin for Intellij IDEA

Hi everyone.

Recently i've decided to try to make perl5 plugin for InterlliJ IDEA. I've seen feew attempts to start, but they've failed after creating four base classes :)

The problem with perl is his too free syntax, which requires a very custom lexer and parser. I've tried to port perl's original lexer, but it's too big, got lot of legacy stuff, so i've droped the idea.

Currently, i'm making lexer with JFlex and it works, but need tunings for different perlish situations. Anyway, some syntax is already highighted:
46fdd3eb65.jpg
The second problem is that language plugins development documentation looks outdated (of course, not so many readers), but examples helps in such cases. Sometimes...

But is there some gurus in language plugins who could help with tricky questions or just to save some time in useless tries?

Also, if you would like to participate in perl5 plugin development, you are welcome: https://github.com/hurricup/Perl5-IDEA

1
236 条评论
Avatar
Permanently deleted user

Helped, fixed the problem. Thanks

0
Avatar
Permanently deleted user

Helped. Fixed the problem. Thanks.

0
Avatar
Permanently deleted user

Ok, new problem: is there a way to distinct XML token from HTML one?
My hilighter dont' know which hilighter it should call (XML or HTML). It's always xml one:

http://joxi.ru/KAxexZ3FQpyzr8.png
Currently, my hilighter checking token's language and calls appropriate hilighter. But it doesn't work for html/xml.

0

Token types are the same for XML and HTML. The language is usually determined by their parents, which are language-specific. For highlighter purposes, I'm afraid, you have to specify the language somewhere in the highlighter code and guess the language there by the context.

0
Avatar
Permanently deleted user

One more question about this task. Currently, to implement adaptive highlighting in multiline blocks I have several token types: multiline, multiline_html, multiline_xml, etc. Because LayeredLexer should be set up by token type.
Is there other way? I'd like to have one multiline token and be able to choose what lexer i should use for this particular block.

0

I'm afraid not with LayeredLexerEditorHighlighter. Even in bare LexerEditorHighlighter there seems to be too little context for you to implement a non-trivial lexer selection algorithm.

0
Avatar
Permanently deleted user

What is the best way to make and use my own LexerEditorHighlighter?

0
Avatar
Permanently deleted user

Ok, i've solved this by implementing my own LayeredLexerAdaptive with extended logic. Was not possible to inherit from LayeredLexer so i've cloned it and extended a bit.

0
Avatar
Permanently deleted user

Delete. Figured out myself.

0
Avatar
Permanently deleted user

How to make my own project type with specific settings?
I need to make Perl project, installation path, additional library paths and so on.

0
Avatar
Permanently deleted user

Ok, some concept question:

Perl is a really context-dependent:

word word can be:

  • method class

  • function function
  • function handle


Depedning on what happened before. So i need to implement a lot of analysis into the parser. Will I be able to reuse this analysis in annotator, for example or will i need to re-analyse tree again?

0
Avatar
Permanently deleted user

I did it and it works fine, but...
When i'm starting to type new statement, all annotations turns off and back on only after i'm completing valid statement. So code is blinkning all the time :)
What can i do with this?

0

Which PSI elements are the annotations attached to? Please try to attach them to leaf elements whenever possible (e.g. name identifiers). Otherwise the highlighting might notice that the underlying element has changed (which by itself may indicate an issue of a too large incremental reparse) and remove all associated highlights immediately, before newer ones are available.

0

There's no such thing as project type in IDEA. When creating a project, you can provide your own ModuleBuilder that allows you to give users a convenient UI to set up a project. When the project is ready, the configuration will probably be saved in a Facet (and its FacetConfiguration) attached to one of the modules. Or you can have project- or IDE-wide setting page for Perl by having a Project/ApplicationService that saves the settings and a Configurable page in the settings UI.

0

I don't know much about Perl, so I'll answer on a very abstract level.

If you need to implement lots of analysis in the parser, that's fine. It probably means that you'll reflect the results of the analysis in the PSI tree structure: in element types, how nodes are nested etc. This is trivially reusable in the annotator. AFAIK there's no good way of reusing more information, e.g. some additional structures you've built in the process of parsing. OTOH complicated parsing likely means that it'll be slow and not very incremental, which is obviously a problem.

You could also think about having a very simple PSI structure that's easy to be produced in the parser, and defer all the complicated analysis until it's needed in annotators, completion etc. Then you can cache and share it between all these clients quite easily. If it's possible to do this, I'd go this way.

Another way: parse the file twice. The first time you extract some important information (if you do this via FileBasedIndexExtension, IDEA will cache it on disk for you), and the second part, in your actual PSI parser, will be guided by this important information and make decisions based on it. That might work, but there's so many things that could go wrong here, so I'd only advise to do this as a last resort.

1
Avatar
Permanently deleted user

Ok, lexer and parser are in beta-version and i am using plugin in my daily work. It's already better than NP++ but still worse than other IDE's :)

Question:
I need to implement PHP-style perl code, when perl is mixed with HTML with same format:

<html> <? some perl here ?> </html>

Perl code in one file should be parsed as one file, not separately, like independent blocks.
How should i do this?

it seems that i need to use fileViewProvider, but not sure about it and not sure how.

Would be nice to get an advice.
Thanks.

0

That's quite a lot to do, actually. You need to have several PSI trees in one file. You should have a custom view provider (extends MultiplePsiFilesPerDocumentFileViewProvider implements TemplateLanguageFileViewProvider) for that, have PhpStylePerl as a base language and HTML as the template data language. Both files should build PSI on the same text, but skip the parts that belong to another language by dedicating special token types to them, skipping them in the parser and creating OuterLanguageElement PSI for these tokens.

The best start would be to check out CFML plugin (https://github.com/JetBrains/intellij-plugins/blob/master/CFML/src/com/intellij/coldFusion/model/files/CfmlFileViewProviderFactory.java) and get inspiration from its source.

1
Avatar
Permanently deleted user

Ok, i've digged a bit.

Am I right, that my Perl lexer should understand and lex HTML and Perl and than, we are building two trees from mixed tokens? What mean's that I can't reuse my existing Perl lexer and HTML lexer and make them work together. Actually, i need to write lexer for perl + html.

I thought that it works like some lexer separates html from perl and than, perl's and html's lexers do their jobs from pure sources (kinda like Chameleon tokens, but as one source).

0

Your base language (be it Perl or not, I don't know if you want to have Perl without HTML) should understand Perl and have a notion of HTML fragments. It should lex these fragments as tokens of a special type (as in CfmlElementTypes#TEMPLATE_TEXT). But it shouldn't be able to fully lex HTML, it can just understand where a fragment starts and where it ends. There are probably some markers of it in the text.

The template data file will have a special content element type (as in CfmlElementTypes#TEMPLATE_DATA) that'll do the lexing for you. You'll still need to build a highlighting lexer yourself, but that's quite easy (see CfmlHighlighter).

1
Avatar
Permanently deleted user

Thank you.
Yes, they can be together and separately and i need them both and i guess i figured out how to handle them in DRY.

0
Avatar
Permanently deleted user

Works like a charm, thanks :)

0
Avatar
Permanently deleted user

First question:

In the beginning i've asked about highlighting HTML, SQL and so on. It seems to me now, that i can do it using the same way as PHP-style perl and have html/sql autocompletion and in SQL case will be able to work with data sources. Am I right?
If I am, is it possible to make optional (lazy) support of features. For example, activate sql support if there is SQL Support plugin and treat queries like strings if there is no SQL support.

Second question:

I'd like to make templates for new Perl files. Some of them should be dumb (static templates), others should be smart, like class templates, which name should depend on filename and it's position if files ierarchy. Where start to dig?

0

Yes, you can have other language support with multiple-PsiFile approach. But if you only need self-contained fragments that don't depend on other fragments in the same file (e.g. for resolve), you can also use language injection (PsiLanguageInjectionHost) or just embed them into your main tree with ILazyParseableElementType. And you can define an optional plugin dependency on SQL support plugin, as described at https://confluence.jetbrains.com/display/IDEADEV/IntelliJ+IDEA+Plugin+Structure.

Second: please see Settings | File and Code Templates. There are many examples there, some relatively "smart". Templates should be packaged in "fileTemplates" directory of your plugin jar, e.g. https://github.com/JetBrains/intellij-community/tree/3f7e93e20b7e79ba389adf593b3b59e46a3e01d1/plugins/groovy/groovy-psi/resources/fileTemplates

1
Avatar
Permanently deleted user

Are you saying that in fileViewProvider solution template blocks being parsed as a whole, not like independent blocks?

0

It depends on your code, but mostly it's used for templating languages, where data language (e.g. HTML) fragments are interspersed with templating directives (e.g. PHP). But nevertheless these data language fragments are parts of a whole, so they should be treated as such and be parts of a single PSI tree. Language injection or lazy-parseable elements can have independent contents, e.g. a Java file may contain several SQL statements in its string literals, and they are completely different and not related at all. That's one of the reasons we use injection for them, and not multi-root view provider.

Choosing between injection and lazy-parseable elements is simple: if you can determine the areas of another language when lexing/parsing, choose the latter. Otherwise - injection.

1
Avatar
Permanently deleted user

Question about smart auto-completion. Tutorial's example contributors as based on token type from lexer. What if i need more, for example, to check siblings or parent in the PSI tree?

0

Sorry, I don't understand the question. In general, you should implement CompletionContributor where you have access to the whole PSI tree. This class also has an extensive javadoc about completion.

0
Avatar
Permanently deleted user

Is there trick with code folding on multi-tree documents?

0

What trick? There's nothing special about folding in such files.

0
Avatar
Permanently deleted user

Sorry, my bad. Forgot that it's a different language and i need to specify FoldingBuilder for it.

In last month i've got so much new information, sometimes feels like my head going to blow up. And sometimes i become very stupid, sorry :)

0

请先登录再写评论。