How to implement PsiNamedElement with complex identifiers?
Hi, I am currently trying to implement reference support for the Nix Expression Language. The language supports to condense the attribute definitions of nested attribute sets. For example, the set
{
x.a = 1;
x.b = 2;
}
is equal to the following set:
{
x = {
a = 1;
b = 2;
};
}
The keys in the set are not represented with simple tokens, because they can also contain strings and interpolations:
{
id = "some id";
elements.${id}."Name" = "some name";
}
I tried to implement PsiNamedElement
for the assignments, but this doesn't work correctly. IntelliJ seems to assume that only the first token is part of the identifier. For x.a
in my first example, I can only rename the element if my cursor touches the x
, not the a
. IntelliJ will also only highlight the x
as the identifier, never the a
. The only alternative idea I have is to implement PsiNamedElement
for each individual attribute. In this case, I probably still have problems with string attributes, but at least simple identifier may work reckonable well. Anyway, if I cite the documentation:
The referencing element at the point of usage (e.g., the
x
in the expressionx + 1
) should not implementPsiNamedElement
since it does not have a name.
The same PSI tree classes which are used for the attribute path in front of the assignment are also used at other places to access variables and attributes. I would rather avoid duplicate all the grammar rules (I use Grammar-Kit).
Are there any recommendations?
请先登录再写评论。
Colin Fleming to specify Symbol declaration on a PSI element use
PsiSymbolDeclarationProvider
or implementPsiSymbolDeclaration
directly on the element (as stated here). Similarly for the references. It all depends on whether you are implementing your own language or contributing on top of an existing language. I can try to help you if you provide some more details about the language or framework for which you create support.Small update.
This works now. My
getName
method did not return exactly the same asgetNameIdentifier().getText()
. The rest of my text is still valid after fixing thegetName
method. I can only rename the element when the cursor touches the first token of the name (but if the cursor touches the first token, the whole name is highlighted now).You'll probably have to use
com.intellij.psi.PsiNamedElement
orcom.intellij.psi.PsiNameIdentifierOwner
for each individual attribute. And provide custom Rename refactoring https://plugins.jetbrains.com/docs/intellij/rename-refactoring.html#custom-rename-ui-and-workflowYann Cebron, thanks. I will look into the reference you provided. Regarding my citation from the documentation:
Should I duplicate the PSI classes for the attributes, so that one version can implement
PsiNamedElement
, or is it fine if all attributes implement this interface? I guess I may returnnull
ingetName()
and other methods if the node is used outside of assignments?Johannes Spangenberg I think it's difficulat to use
PsiNamedElement
in this situation, as none of these seems to be “declaration” of the identifier. It more looks like a reference and I think it should be treated like such. Now if you're lacking declaration in the source file, I think you should try the experimentalSymbol
API. Each segment of the reference can reference aSymbol
, which does not need to “exist” in the source code, yet you can rename it, search its usages, etc. For a reference please have a look here: https://plugins.jetbrains.com/docs/intellij/symbols.html . Let me know if you have any questions.Piotr Tomiak I played around with the
Symbol
API, and it seems to be a better fit, partially because it allows having multiple declarations for one symbol, and also allows having one declaration for multiple symbols. Anyway, I am now wondering how to implement Find Usages. The Find Usages documentation seems to assume that I am usingPsiNamedElement
. At first, I thought that maybe during indexing, IDEA just stores some kind of index or map fromPointer<Symbol>
to elements which use the symbol, so that it might just work out of the box. However, I have now implemented my first prototype using symbols, and Find Usages does not seem to work yet. What do I have to do, to make it work? Go to Declaration seems to work by implementingNavigatableSymbol
.There are also topics where I am not quite sure if I am missing something. I think the features
all more or less work out of the box when using
PsiNamedElement
. With the symbol API, it looks like I have to implement them separately. For the renaming, I guess I have to implementRenamePsiElementProcessor
, not yet sure about the other two features. Is it right that I have to implement these features separately, or did I miss something important?I just realized later today that there are more optional interfaces for symbols:
NavigatableSymbol
(which I already mentioned above)DocumentationSymbol
(which seems to provide quick documentation support)SearchTargetSymbol
(which seems to provide support for Find Usages and highlighting)RenameableSymbol
(which probably provides support for renaming)There is also
PresentableSymbol
, but I am not sure about its purpose, as the presentation is already defined by all the other interfaces for their individual use cases.Is there any example I could look at regarding
SearchTargetSymbol
andRenameableSymbol
? I am especially wondering how to implementUsageHandler.buildSearchQuery
for cross file references. TheRenameableSymbol
interface also looks non-trivial, but I haven't really looked into it yet.Johannes Spangenberg I think those 4 interfaces is exactly what you need. The Symbol API might be more scattered over several interfaces, but thanks to that you have a better flexibility. Anyway, a pretty complex example of the usage of Symbol API is WebSymbols framework: https://github.com/JetBrains/intellij-community/tree/master/platform/webSymbols/src/com/intellij/webSymbols . You can find examples for documentation, search for usages and for rename support in
documentation
,search
andrefactoring
packages. The main interface isWebSymbol
: https://github.com/JetBrains/intellij-community/blob/master/platform/webSymbols/src/com/intellij/webSymbols/WebSymbol.kt . Note, that it is not implementingDocumentationSymbol
,SearchTargetSymbol
norRenameableSymbol
, since this is handled by respective factories:DocumentationTargetProvider
,SymbolSearchTargetFactory
andSymbolRenameTargetFactory
, which reach out toWebSymbol
properties and allow for a bit more flexibility. A lot of the code in search and refactoring is there to support interoperability of Web Symbols with PSI elements, you won't be needing that of course.If you're interested in some more info on Web Symbols you can find it here: https://plugins.jetbrains.com/docs/intellij/websymbols.html . In short - the framework is aimed at reducing boilerplate around Symbol API and is heavily used in Web framework support for Vue, Angular and Astro. If the Nix language can use symbols from other languages (especially if that language is using PSI declared things) then it might be worth looking into basing your support on Web Symbols.
This API looks like it would also be extremely useful for me. There's one thing that I'm unclear on, the doc states: “The platform obtains the target symbol from a declaration or by resolving a reference and then uses it to perform an action”. It's not clear how this should work, since PsiReference only allows references to PsiElements. Should these be provided by getOwnDeclarations() and getOwnReferences() in PsiElement? Are they the main entry points to obtaining Symbol instances?
Colin Fleming to specify Symbol declaration on a PSI element use
PsiSymbolDeclarationProvider
or implementPsiSymbolDeclaration
directly on the element (as stated here). Similarly for the references. It all depends on whether you are implementing your own language or contributing on top of an existing language. I can try to help you if you provide some more details about the language or framework for which you create support.Piotr Tomiak I see, thanks - I hadn't read the next doc section. This would be for my own language, my Clojure support currently relies on PomTarget, PomDeclarationSearcher etc, because that was the only thing available at the time I implemented it. Since Clojure is extensible using macros, it can't be statically parsed into a standard PSI, so I use these mechanisms for the language support. It's pretty horrible, I create a lot of implementations of LightElement to point to etc. The Symbol API looks much nicer.
If a particular element could refer to both a Symbol or a PsiElement, would the best way be to just use the Symbol API for references, and return a Psi2Symbol instance when referring to a PsiElement?
Colin Fleming If you need to have cooperation between PSI and Symbols, I think it might be better to check out Web Symbols. There is a lot of boilerplate to create to support PSI → Symbol and Symbol → PSI refactorings, find usages, etc.
Web Symbols API is relatively complex, but very powerful. There are 2 main areas of interest:
- references - you need to create a
PsiSymbolReferenceProvider
extendingWebSymbolReferenceProvider
+ you need to register completion provider, preferably extendingWebSymbolsCompletionProviderBase
- declarations - you need to create a
WebSymbolDeclarationProvider
At a later stage you may want to decide to use the
WebSymbolQueryExecutor
to resolve symbols on the reference. This is a preferred way, as it allows for high level of extensibility, but you can also use your own code to figure out the list of symbols. The idea of query executor is that there are various scopes, which can provide symbols. A scope can be a code block with defined variables, a global variable scope, or a very local thing, like a particular tag, which is a scope with attributes, or an object type, which has properties and methods. WebSymbols are distinguished from each other through namespace, kind and name, which allows one scope to contain symbols of different kinds. The additional benefit is that in many frameworks there are meta syntaxes and for instance an attribute name is actually built from various other symbols (example). For this you can define patterns, through which symbols of one kind and built from symbols of other kind. The additional benefit of using query executor is that you can have global statically defined symbols through JSON.When instantiating a query executor on a particular context,
WebSymbolQueryConfigurator
s are run with the context as a parameter. Based on the context each configurator can return set of tailoredWebSymbolsScope
s (e.g.:Angular2WebSymbolsQueryConfigurator
). These scope are then queried for particular kind and name of symbol and pattern evaluation happens. Usually a scope would extendWebSymbolsScopeWithCache
and get populated with available symbols in theinitialize
method (e.g.NgContentSelectorsScope
).Unfortunately I don't have a complete example of how to use these classes, so you would need to experiment a bit. I am constantly working on improving documentation for Web Symbols though, so at some point an example implementation will be present.
Here are examples of
WebSymbolReferenceProvider
andWebSymbolsCompletionProviderBase
, which work with CSS functions in CSS files:Sorry for the delay in replying Piotr, and thanks for the very detailed post. Unfortunately it looks like WebSymbols are pretty new and not well supported in all IDE versions I support, although I'm going to drop some older versions soon once 2023.3 is done. Are there any examples of how to get PSI interop with the raw Symbol API I could look at? I'd be interested in seeing what would be involved in making that work.
Colin Fleming You can look at the WebSymbols source code. There's all you need to provide cooperation. Please note that Symbols API is also experimental and have undergone major changes throughout releases, so the older versions of IDEs might also have some issues with Symbols API.
Some time has passed, but I just found some time and energy to work on this topic again. I implemented variable resolution (and some basics for attribute resolution) using the Symbols API at NixOS/nix-idea#79. It mostly works, but I noticed one issue so far. My
NixUsageSearcher
uses the full-text search, just as I have seen it for other languages in the source code of IntelliJ. I was wondering whether it is possible and makes sense to create a separate index for my references, instead of using the text search? I have currently the problem that theUsageSearcher
is not working for references that use string notation with escape sequences. Due to escape sequences, there are a lot of ways to write a reference to the same symbol.It would be the most convenient if I could just provide some index-keys from the references and symbols. All reference locations could automatically be added to the index, and some generic searcher could call
MySymbol.getSearchKeys()
and find all references which were indexed under one of these keys. But I am also open for other solutions.I also noticed a few other strange things, but they are currently not causing any problems for me. Anyway, here is a list. Just in case you are interested in some feedback.
PsiSymbolReferenceSearcher
andPsiSymbolDeclarationSearchParameters
. They seem to powerSearchService.searchPsiSymbolReferences
andSearchService.searchPsiSymbolDeclarations
. However, all of this seems to be completely unused. Am I supposed to implement the interfaces? Otherwise, it might make sense to deprecate the interfaces and the two methods, as people may assume that this is how IntelliJ finds the references, while IDEA actually usesUsageSearcher
instead.Usage.getDeclaration()
is to filter declarations from the result of the usage search. All usages which return true from this method are removed from the results. Maybe it makes sense to document this better. I actually added an option to let the user decide if they want to see declarations as part of the result.CodeInsightTestFixture.testFindUsagesUsingAction()
works with the symbols API, but generally fails if there is only one or no reference.CodeInsightTestFixture.testFindUsagesUsingAction()
does filter out all declarations, which means you cannot use it to test the declaration search, which is still relevant for highlighting. (FYI, I opted to just implementUsageSearchParameters
and trigger my own usage search. See SymbolNavigationTest.java:312.)SearchTarget.getPresentation()
was renamed toSearchTarget.presentation()
, which makes it inconsistent withgetUsageHandler()
and other methods.