Language injection into strings and string concatenation?

Answered

My plugin's custom language supports an integrated query language that can be used both statically and dynamically. Dynamic usage takes place via strings, typically a concatenation of string literals and other string expressions to parameterize the query, e.g.:

List<User> users = Database.query('SELECT Id, Username FROM User WHERE Username LIKE \'' + usernamePattern + '\'');

I've managed to implement PsiLanguageInjectionHost for my language's string literals and have registered a LanguageInjector implementation to inject the query language when found in the correct context. Now I get the desired behavior when the argument to Database.query() is a single string literal, but it doesn't work properly when the argument is a concatenation.

When I inject SQL into concatenations of Java string literals it treats the full concatenation as a single expression in the injected language, so there must be some way to make this work. Any tips on how that happens? I can see a notion of Shred in PsiLanguageInjectionHost but don't know if that's potentially useful.

Thanks in advance for any insights!

11 comments
Comment actions Permalink

Please see com.intellij.lang.injection.ConcatenationAwareInjector, Java implementation to separate elements and operands is com.intellij.psi.impl.source.tree.injected.JavaConcatenationToInjectorAdapter

1
Comment actions Permalink

Perfect. Thanks, Yann!

0
Comment actions Permalink

Yann, with your pointer I have this all working for the most part. I do have three follow-up questions/issues:

Invalid Concatenated Strings

First, there's one corner case that's not working properly, and I'm not sure if it's going to be possible to make it work properly...but no harm in asking.

Basically if the concatenation of string literals and expressions results in a total string literal that's not syntactically correct, it looks like there are parser errors. For example:

List<SObject> results = Database.query('SELECT Id, Name FROM ' + sobjectType + ' WHERE LastModifiedDate = TODAY()');

where sobjectType is passed into the method that runs this query. That effectively yields the following injected fragment:

SELECT Id, Name FROM  WHERE LastModifiedDate = TODAY()

which is invalid because FROM must specify an object/table name.

Is there some way that I can "stub" something for each these non-string literal expressions that will help ensure that the total query will seem valid to the parser?

Language Injections Config

Second question: how can one register an entry in Settings>Editor>Language Injections? Right now I have much of that config encapsulated in my implementation, but it'd be great if it could be externalized in that config screen.

Language Injection Comments

This one may well fall out of the answer to the one immediately above, but since I'm detecting contexts in which injection should happen automatically now based on invocation signature, I also need to determine when an injection is happening explicitly because of a // Language=<language> comment. I'm forced to look for that comment at present, but my guess is that I shouldn't have to do that, especially if the config concept above helps to externalize invocation signature detection. If not, though, is there something in the SDK that helps to find these comments vs. the PSI inspection I'm having to do now?

Thanks again in advance for any additional info you can provide!

0
Comment actions Permalink

Quick update on this...I managed to make things work a bit more seamlessly, albeit not perfectly. If the injected concatenation doesn't parse properly, I suppress some of my code inspections that would otherwise display errors such as unresolvable references, and I also registered a highlightErrorFilter implementation that suppresses parse errors if found in an injected fragment based on a concatenation of string literals and other expressions, or in a single string literal if found specifically at the end (assuming further concatenation will occur).

This approach (mostly?) eliminates the noise in the situation described previously, and unless it's a catastrophic failure to parse the document, (most) reference injections, code completions, etc., still work. Even in the catastrophic case, you still get proper syntax highlighting and such.

Hopefully Yann or someone will weigh in with an even better option, but for now I feel like I have an implementation that I can ship to my end users and provide them the benefits of language injection here with only a few snags in cases of complex concatenation.

0
Comment actions Permalink

re Language injection Config/Comments: please see org.intellij.plugins.intelliLang.inject.LanguageInjectionSupport (org.intellij.plugins.intelliLang.inject.AbstractLanguageInjectionSupport) which will provide all the features

1
Comment actions Permalink

Thanks again, Yann. I'll take a look today and see if I can get that implemented as well before the next release.

0
Comment actions Permalink

Yann, I've run an interesting issue with ConcatenationAwareInjector. Just like JavaConcatenationToInjectorAdapter, I've implemented my own ApexConcatenationToInjectorAdapter and registered it as a multiHostInjector. That works great in both editions of IntelliJ IDEA, but when I loaded the plugin in WebStorm, I received an unsatisfied dependency error:

2019-07-11 14:33:59,567 [  40203]  ERROR - nsions.impl.ExtensionPointImpl - com.illuminatedcloud.intellij.inject.ApexConcatenationToInjectorAdapter has unsatisfied dependency: class com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager among unsatisfiable dependencies: [[class com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager]] where DefaultPicoContainer (parent=DefaultPicoContainer (root)) was the leaf container being asked for dependencies. 
com.intellij.openapi.extensions.impl.PicoPluginExtensionInitializationException: com.illuminatedcloud.intellij.inject.ApexConcatenationToInjectorAdapter has unsatisfied dependency: class com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager among unsatisfiable dependencies: [[class com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager]] where DefaultPicoContainer (parent=DefaultPicoContainer (root)) was the leaf container being asked for dependencies.

I found that ConcatenationInjectionService is only registered in JavaPlugin.xml. I was able to resolve this by expressing an optional dependency for WebStorm in plugin.xml:

<depends optional="true" config-file="plugin-lightweight-ides.xml">com.intellij.modules.webstorm</depends>

where plugin-lightweight-ides.xml is:

<idea-plugin>

<extensions defaultExtensionNs="com.intellij">
<!-- Required for lightweight IDEs where this component isn't injected -->
<projectService serviceInterface="com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager"
serviceImplementation="com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager"/>
</extensions>

</idea-plugin>

That solved the problem for WebStorm with no negative impact on IntelliJ IDEA. I assumed the same issue would be required with other lightweight IDEs, so I tested next in PyCharm. Unsurprisingly I saw the same error and had to express a similar optional dependency for it to work properly:

<depends optional="true" config-file="plugin-lightweight-ides.xml">com.intellij.modules.python</depends>

However, that one concerned me because IntelliJ IDEA Ultimate Edition also includes the Python module, so I tested it with that plugin installed and enabled. Sure enough, I then received an error about duplicate component registration:

2019-07-11 14:28:39,768 [  26593]   INFO - roject.impl.ProjectManagerImpl - Fatal error initializing 'com.intellij.openapi.components.impl.ServiceManagerImpl' 
com.intellij.ide.plugins.PluginManager$StartupAbortedException: Fatal error initializing 'com.intellij.openapi.components.impl.ServiceManagerImpl'
at com.intellij.ide.plugins.PluginManager.handleComponentError(PluginManager.java:256)
at com.intellij.openapi.components.impl.PlatformComponentManagerImpl.handleInitComponentError(PlatformComponentManagerImpl.java:43)
at com.intellij.openapi.components.impl.ComponentManagerImpl$ComponentConfigComponentAdapter.getComponentInstance(ComponentManagerImpl.java:500)
...
Caused by: org.picocontainer.defaults.DuplicateComponentKeyRegistrationException: Key com.intellij.psi.impl.source.tree.injected.ConcatenationInjectorManager duplicated

Now I'm not sure how to make a multiHostInjection implementation work in all IDEs. This seems to introduce a bit of a catch-22, no? Or is there some other way to register all of this via plugin.xml that would solve this issue? It seems like the simplest solution would be to ensure that ConcatenationInjectionManager is registered as a project service in all IDEs instead of just IntelliJ IDEA, but of course that would require you guys to rev all IDEs and would only work on that newer version and later. I also don't know if there's a legitimate reason that it's just registered for Java and not other languages.

Thoughts? This certainly seems like an oversight if not an actual bug. Thanks in advance for any additional insight you can provide.

0
Comment actions Permalink

It appears that this same exists exists with PhpStorm and RubyMine. Because all three of these plugins are available for installation into IntelliJ IDEA Ultimate Edition, you can't reliably build a dependency on their respective plugin module names to initialize ConcatenationInjectionManager. As a result, as far as I can tell concatenation injection isn't really possible in the lightweight IDEs in a way that also works properly with IntelliJ IDEA Ultimate Edition.

0
Comment actions Permalink

I agree that the proper solution would be registering ConcatenationInjectionManager in the platform, I've created an issue about that so you can track progress on that. I hope we'll fix it soon.

0
Comment actions Permalink

Thanks, Nikolay!

0
Comment actions Permalink

For anyone coming to this thread hoping to implement language injection into concatenated strings in a custom plugin, note that the resolution of this issue:

https://youtrack.jetbrains.com/issue/IDEA-218478

is that it's not possible in a supported manner at this point. A new enhancement request issue has been created for a sanctioned way to do this in the plugin SDK:

https://youtrack.jetbrains.com/issue/IDEA-220176

 

0

Please sign in to leave a comment.