How Can I prevent FragmentElementType from affect native HTML parsing?

Hi, I'm develop a plugin for supporting Regularjs. Which is a template engine basing on HTML. I have already finish lexer and parser.

But there is some problem.

 

Picture 1 shows that,when not using my plugin, native html parser deals with `on-mouseleave={this.toggleSubmenu($event,false)} ` well, it treat the content after `=` as PsiElement(XML_ATTRIBUTE_VALUE)。

But once I apply my plugin to this code, as picture 2 shows, it will treat the content after `=` as PsiElement(RegularFragmentElementType), so that the parser will not work well.



And what is worse, in the code `class="menu-item menu-item-withsub {item.className||''}"`, since the {item.className||''} will be lexer as PsiElement(RegularFragmentElementType), this will broken the parser rule(string) of native html.

Could anyone tell me how to fix this problem?

19 comments
Comment actions Permalink

You're using TemplateDataElementType and PsiElement(RegularFragmentElementType) is an OuterLanguageElement, right?

If not, it should be so. If yes, then you have an opportunity to customize the text that HTML lexer/parser see (TemplateDataElementType#createTemplateText, TemplateDataElementType#appendCurrentTemplateToken). There, you can analyze the template data tokens you have and possibly insert additional characters in place of template language fragments, to make the text well-formed enough.

It seems that in your case it should be enough to insert some identifier after "=" or a space.

1
Comment actions Permalink

Thanks for reply. Your solution seems fantastic.

But how can I use TemplateDataElementType?  The only way I can figure out is the extents a new class base on  TemplateDataElementType, and override the code of TemplateDataElementType#appendCurrentTemplateToken, change the code as below:

protected void appendCurrentTemplateToken(StringBuilder result, CharSequence buf, Lexer lexer) {
result.append("");
}
 

Is this right?

0
Comment actions Permalink

I don't see the code below, but from your description it seems to be the approach I had in mind.

0
Comment actions Permalink

Hi, Peter

I have updated my code above. Could you see it?

 

0
Comment actions Permalink

Yes, now I see it. With this implementation, you'll get empty HTML text. Please call super.appendCurrentTemplateToken to ensure the text in the file is appended, and after that you can also append something else, to make HTML parser happier.

0
Comment actions Permalink

Hi, Peter

Thanks a lot. Your idea really works! 

There are still a problem, though it may not a big deal. 

As the picture shows below, the code `class="menu-item {item.className || '''}"` should be parse as

"menu-item" , "RegularFragmentElementType", but as you can see, here it's parsed as "menu-i", "RegularFragmentElementType", "term".  In here, I just replace {item.className || '''} as space as you said. Is this a bug of PsiViewer or I have do something wrong?

Thanks again for your help!

0
Comment actions Permalink

Now I have found a critical problem.

Here is my code.  

And now I have found that I couldn't type in '=', and when I type in other word, it will be block. I thought result is just for creating a virtual psifile for native html parser, without affecting the original file, am I wrong?

0
Comment actions Permalink

It seems that I was wrong, and it's much more complicated. Changing the text is not enough, it also requires some quite complicated PSI patching, for which there's no API yet. Most other template language supports of ours have the same issue, unfortunately. With attributes, the only viable strategy so far seems to add quotes to the template text. For cases like the one with CSS classes, the highlighting errors are just suppressed.

I'll investigate and see if we can extract some API.

0
Comment actions Permalink

Do you means that I should change the grammar of template language, changing

`on-mouseenter={this.toggleSubmenu($event,true)}`

to

`on-mouseenter="{this.toggleSubmenu($event,true)}"`。

If so, I'm afraid that I couldn't do that, since we have a lot code writing in this way. 

Thanks anyway.

And please tell us the result of investigating, that's will be helpful for someone who has the same problem.

0
Comment actions Permalink

Hi, Peter.

Is there any progress?

0
Comment actions Permalink

Not in this release, unfortunately. I've filed an issue that you can watch: https://youtrack.jetbrains.com/issue/IDEA-169107

0
Comment actions Permalink

Sad. Does it means this feature will not work until next year?

0
Comment actions Permalink

We plan to release 2017.2 in summer. Note that while we'll strive to fix this request until then, I can't absolutely guarantee that.

0
Comment actions Permalink

Hi, Peter,

It seems that it would be too long to wait this feature. I thought whether I could do this in another way.  Is there any specific way  to achieve this rather than a general api?

0
Comment actions Permalink

There's only a very hacky way so far:

1. remember the inserted strings and their offsets in a thread-local field in overridden "createTemplateText" or "appendCurrentTemplateToken",

2. use this information in overridden "prepareParsedTemplateFile" and remove the inserted strings from template data file leaves, so that its text corresponds to the original one. Removing is done by replacing leaves with their copies with a new text (LeafElement#replaceWithText).

0
Comment actions Permalink

Thanks a lot! I will try it soon.

0
Comment actions Permalink

Hi Peter,

This is my implementation of  "prepareParsedTemplateFile": 

@Override
protected void prepareParsedTemplateFile(@NotNull FileElement root) {
super.prepareParsedTemplateFile(root);
ArrayList<Pair<Integer, String>> pairs = hackyStringMapThreadLocal.get();
int size = pairs.size();
for(int i = size -1; i >= 0; i--){
Pair<Integer, String> entry = pairs.get(i);
LeafElement oldLeaf = root.findLeafElementAt(entry.getFirst());
LeafElement newLeaf = oldLeaf.replaceWithText(oldLeaf.getText().replace(entry.getSecond(), ""));
root.replaceChild(oldLeaf, newLeaf);
}
}

As you said, I need remove all my inserted strings in "appendCurrentTemplateToken", but I tried PsiElement#replaceChild, this method will throw some Exception, breaking out the while loop, so I wander if there another way to replace leaf node. Could you have any suggestion to me?

 

 

0
Comment actions Permalink

You don't need replaceChild, because rawReplaceWithText/replaceWithText already does that. Please use rawReplaceWithText. If the exception persists, please post it here.

0
Comment actions Permalink

It seems works very well!  Peter, thank you very much!

0

Please sign in to leave a comment.