ERB like template language

Answered

Like how Ruby has ERB (Embedded RuBy) files, Elixir has EEX (Embedded EliXir) files (http://elixir-lang.org/docs/stable/eex/EEx.html).  They even use similar tags, <% %> and <%= %>, for embedding the language (Ruby for ERB and Elixir for EEX) inside another language.  They also share the common pattern that parts of the target language, like HTML, can be wrapped by a control structure like:

<%= if true do %>

It is obviously true

<% else %>

This will never appear

<% end %>

So, how is this done in Rubymine and/or which APIs can I use in my intellij-elixir to implement EEX?  Do I need to write a completely new grammar that replicates all of the Elixir grammar rule(from https://github.com/KronicDeth/intellij-elixir/blob/master/src/org/elixir_lang/Elixir.bnf), but allows for the <% %> tags in certain places, or is there some way to reuse it but split inside the <%= %> and <% %> tags?  I guess this comes down to whether the GrammarKit grammars are composable when embedding.  If you are aware of an open-source plugin for a language that has a similar templating language that would also be helpful for something for me to look at.

5 comments
Comment actions Permalink

Hey Luke,

Generated and handwritten parsers can be composed and mixed in any way.

Check out this thread https://intellij-support.jetbrains.com/hc/en-us/community/posts/207645985-language-composition-with-Grammar-Kit-

1
Comment actions Permalink

Hi Luke,

here's what ERb lexer does, it produces 3 types of tokens:

  1. ERb specific tokens (such as <%, <%=, etc)
  2. Ruby code characters (everything in between <% and %>). This does not need to be fully lexed, just one token for this content
  3. Template language characters (everything outside of <% and %>). This does non need to be fully lexed too, just one token

ERb editor highlighter is extending LayeredLexerEditorHighlighter, you can call registerLayer on token types 2 and 3 to get highlighting for corresponding languages.

You'll also need file view provide to extend MultiplePsiFilesPerDocumentFileViewProvider and implement TemplateLanguageFileViewProvider to indicate that there are multiple languages in this file and that it's actually a template language. In case of ERB it automatically detects template language based on file extension

1
Comment actions Permalink

Thanks Dennis, I was wondering about how the lexer would work, so your post helps with that.

The other open question I have is, how do I handle that Elixir token will represent only a partially valid segment?  i.e. there will be Elixir tokens with just `if foo do` or `else` in them.  Do all the Elixir tokens get merged together somehow before passing the normal language parser?  If that's the case, what will the template language tokens show up as?  Do I need to add those tokens to the rules in my Elixir.bnf?

0
Comment actions Permalink

In case of Ruby, there's an additional lexer that can eat both ERb stuff and template data and provides tokens that are considered "outer" language tokens and Ruby parser just skips them. That is achieved by creating special Ruby file in file view provider when asked for a file with language Ruby. That file has special content element type that implements custom parseContents method.

Here's the open source example of the template language: https://github.com/JetBrains/intellij-plugins/tree/master/handlebars

 

1
Comment actions Permalink

Finally got around to doing my EEx implementation: https://github.com/KronicDeth/intellij-elixir/pull/923.  Anyone stumbling on this later: comment if you need more specifics than the PR diff.

0

Please sign in to leave a comment.