How to implement PsiNamedElement with complex identifiers?

Answered

Hi, I am currently trying to implement reference support for the Nix Expression Language. The language supports to condense the attribute definitions of nested attribute sets. For example, the set

{
  x.a = 1;
  x.b = 2;
}

is equal to the following set:

{
  x = {
    a = 1;
    b = 2;
  };
}

The keys in the set are not represented with simple tokens, because they can also contain strings and interpolations:

{
  id = "some id";
  elements.${id}."Name" = "some name";
}

I tried to implement PsiNamedElement for the assignments, but this doesn't work correctly. IntelliJ seems to assume that only the first token is part of the identifier. For x.a in my first example, I can only rename the element if my cursor touches the x, not the a. IntelliJ will also only highlight the x as the identifier, never the a. The only alternative idea I have is to implement PsiNamedElement for each individual attribute. In this case, I probably still have problems with string attributes, but at least simple identifier may work reckonable well. Anyway, if I cite the documentation:

The referencing element at the point of usage (e.g., the x in the expression x + 1) should not implement PsiNamedElement since it does not have a name.

The same PSI tree classes which are used for the attribute path in front of the assignment are also used at other places to access variables and attributes. I would rather avoid duplicate all the grammar rules (I use Grammar-Kit).

Are there any recommendations?

0
14 comments
Official comment

Colin Fleming to specify Symbol declaration on a PSI element use PsiSymbolDeclarationProvider or implement PsiSymbolDeclaration directly on the element (as stated here). Similarly for the references. It all depends on whether you are implementing your own language or contributing on top of an existing language. I can try to help you if you provide some more details about the language or framework for which you create support.

You'll probably have to use com.intellij.psi.PsiNamedElement or com.intellij.psi.PsiNameIdentifierOwner for each individual attribute. And provide custom Rename refactoring https://plugins.jetbrains.com/docs/intellij/rename-refactoring.html#custom-rename-ui-and-workflow

1

Johannes Spangenberg I think it's difficulat to use PsiNamedElement in this situation, as none of these seems to be “declaration” of the identifier. It more looks like a reference and I think it should be treated like such. Now if you're lacking declaration in the source file, I think you should try the experimental Symbol API. Each segment of the reference can reference a Symbol, which does not need to “exist” in the source code, yet you can rename it, search its usages, etc. For a reference please have a look here: https://plugins.jetbrains.com/docs/intellij/symbols.html . Let me know if you have any questions.

1

Johannes Spangenberg I think those 4 interfaces is exactly what you need. The Symbol API might be more scattered over several interfaces, but thanks to that you have a better flexibility. Anyway, a pretty complex example of the usage of Symbol API is WebSymbols framework: https://github.com/JetBrains/intellij-community/tree/master/platform/webSymbols/src/com/intellij/webSymbols . You can find examples for documentation, search for usages and for rename support in documentation, search and refactoring packages. The main interface is WebSymbol: https://github.com/JetBrains/intellij-community/blob/master/platform/webSymbols/src/com/intellij/webSymbols/WebSymbol.kt . Note, that it is not implementing DocumentationSymbol, SearchTargetSymbol nor RenameableSymbol, since this is handled by respective factories: DocumentationTargetProvider, SymbolSearchTargetFactory and SymbolRenameTargetFactory, which reach out to WebSymbol properties and allow for a bit more flexibility. A lot of the code in search and refactoring is there to support interoperability of Web Symbols with PSI elements, you won't be needing that of course.

If you're interested in some more info on Web Symbols you can find it here: https://plugins.jetbrains.com/docs/intellij/websymbols.html . In short - the framework is aimed at reducing boilerplate around Symbol API and is heavily used in Web framework support for Vue, Angular and Astro. If the Nix language can use symbols from other languages (especially if that language is using PSI declared things) then it might be worth looking into basing your support on Web Symbols.

1

Small update.

IntelliJ will also only highlight the x as the identifier, never the a.

This works now. My getName method did not return exactly the same as getNameIdentifier().getText(). The rest of my text is still valid after fixing the getName method. I can only rename the element when the cursor touches the first token of the name (but if the cursor touches the first token, the whole name is highlighted now).

0

Yann Cebron, thanks. I will look into the reference you provided. Regarding my citation from the documentation:

The referencing element at the point of usage (e.g., the x in the expression x + 1) should not implement PsiNamedElement since it does not have a name.

Should I duplicate the PSI classes for the attributes, so that one version can implement PsiNamedElement, or is it fine if all attributes implement this interface? I guess I may return null in getName() and other methods if the node is used outside of assignments?

0

Piotr Tomiak I played around with the Symbol API, and it seems to be a better fit, partially because it allows having multiple declarations for one symbol, and also allows having one declaration for multiple symbols. Anyway, I am now wondering how to implement Find Usages. The Find Usages documentation seems to assume that I am using PsiNamedElement. At first, I thought that maybe during indexing, IDEA just stores some kind of index or map from Pointer<Symbol> to elements which use the symbol, so that it might just work out of the box. However, I have now implemented my first prototype using symbols, and Find Usages does not seem to work yet. What do I have to do, to make it work? Go to Declaration seems to work by implementing NavigatableSymbol

There are also topics where I am not quite sure if I am missing something. I think the features

  • renaming,
  • quick documentation, and
  • highlighting of declarations and usages when putting the cursor on a reference

all more or less work out of the box when using PsiNamedElement. With the symbol API, it looks like I have to implement them separately. For the renaming, I guess I have to implement RenamePsiElementProcessor, not yet sure about the other two features. Is it right that I have to implement these features separately, or did I miss something important?

0

I just realized later today that there are more optional interfaces for symbols:

  • NavigatableSymbol (which I already mentioned above)
  • DocumentationSymbol (which seems to provide quick documentation support)
  • SearchTargetSymbol (which seems to provide support for Find Usages and highlighting)
  • RenameableSymbol (which probably provides support for renaming)

There is also PresentableSymbol, but I am not sure about its purpose, as the presentation is already defined by all the other interfaces for their individual use cases.

Is there any example I could look at regarding SearchTargetSymbol and RenameableSymbol? I am especially wondering how to implement UsageHandler.buildSearchQuery for cross file references. The RenameableSymbol interface also looks non-trivial, but I haven't really looked into it yet.

0

This API looks like it would also be extremely useful for me. There's one thing that I'm unclear on, the doc states: “The platform obtains the target symbol from a declaration or by resolving a reference and then uses it to perform an action”. It's not clear how this should work, since PsiReference only allows references to PsiElements. Should these be provided by getOwnDeclarations() and getOwnReferences() in PsiElement? Are they the main entry points to obtaining Symbol instances?

0

Colin Fleming to specify Symbol declaration on a PSI element use PsiSymbolDeclarationProvider or implement PsiSymbolDeclaration directly on the element (as stated here). Similarly for the references. It all depends on whether you are implementing your own language or contributing on top of an existing language. I can try to help you if you provide some more details about the language or framework for which you create support.

0

Piotr Tomiak I see, thanks - I hadn't read the next doc section. This would be for my own language, my  Clojure support currently relies on PomTarget, PomDeclarationSearcher etc, because that was the only thing available at the time I implemented it. Since Clojure is extensible using macros, it can't be statically parsed into a standard PSI, so I use these mechanisms for the language support. It's pretty horrible, I create a lot of implementations of LightElement to point to etc. The Symbol API looks much nicer.

If a particular element could refer to both a Symbol or a PsiElement, would the best way be to just use the Symbol API for references, and return a Psi2Symbol instance when referring to a PsiElement?

0

Colin Fleming If you need to have cooperation between PSI and Symbols, I think it might be better to check out Web Symbols. There is a lot of boilerplate to create to support PSI → Symbol and Symbol → PSI refactorings, find usages, etc. 

Web Symbols API is relatively complex, but very powerful. There are 2 main areas of interest:
- references - you need to create a PsiSymbolReferenceProvider extending WebSymbolReferenceProvider + you need to register completion provider, preferably extending WebSymbolsCompletionProviderBase

- declarations - you need to create a WebSymbolDeclarationProvider

At a later stage you may want to decide to use the WebSymbolQueryExecutor to resolve symbols on the reference. This is a preferred way, as it allows for high level of extensibility, but you can also use your own code to figure out the list of symbols. The idea of query executor is that there are various scopes, which can provide symbols. A scope can be a code block with defined variables, a global variable scope, or a very local thing, like a particular tag, which is a scope with attributes, or an object type, which has properties and methods. WebSymbols are distinguished from each other through namespace, kind and name, which allows one scope to contain symbols of different kinds. The additional benefit is that in many frameworks there are meta syntaxes and for instance an attribute name is actually built from various other symbols (example). For this you can define patterns, through which symbols of one kind and built from symbols of other kind. The additional benefit of using query executor is that you can have global statically defined symbols through JSON.

When instantiating a query executor on a particular context, WebSymbolQueryConfigurators are run with the context as a parameter. Based on the context each configurator can return set of tailored WebSymbolsScopes (e.g.: Angular2WebSymbolsQueryConfigurator). These scope are then queried for particular kind and name of symbol and pattern evaluation happens. Usually a scope would extend WebSymbolsScopeWithCache and get populated with available symbols in the initialize method (e.g. NgContentSelectorsScope).

Unfortunately I don't have a complete example of how to use these classes, so you would need to experiment a bit. I am constantly working on improving documentation for Web Symbols though, so at some point an example implementation will be present.

Here are examples of WebSymbolReferenceProvider and WebSymbolsCompletionProviderBase, which work with CSS functions in CSS files:

class WebSymbolCssFunctionReferenceProvider : WebSymbolReferenceProvider<CssFunction>() {
  override fun getSymbol(psiElement: CssFunction): WebSymbol? {
    if (psiElement.parent !is CssTerm 
        || psiElement.parent.parent !is CssTermList 
        || psiElement.parent.parent.parent !is CssDeclaration) {
      return null
    }

    val queryExecutor = WebSymbolsQueryExecutorFactory.create(psiElement)
    val queryScope = getCssFunctionQueryScope(queryExecutor, psiElement)
    return queryExecutor
      .runNameMatchQuery(WebSymbol.NAMESPACE_CSS, 
                         WebSymbol.KIND_CSS_FUNCTIONS,
                         psiElement.name.takeIf { it.isNotEmpty() } 
                         ?: return null, scope = queryScope)
      .takeIf {
        it.isNotEmpty()
        && !it.hasOnlyExtensions()
        && !it.hasOnlyStandardCssSymbols()
      }
      ?.asSingleSymbol()
  }
}
class WebSymbolCssFunctionCompletionProvider 
: WebSymbolsCompletionProviderBase<CssTerm>() {

  override fun getContext(position: PsiElement): CssTerm? = 
    position.parentOfType()

  override fun addCompletions(parameters: CompletionParameters, 
                              result: CompletionResultSet, 
                              position: Int,
                              name: String, 
                              queryExecutor: WebSymbolsQueryExecutor, 
                              context: CssTerm) {
    val patchedResultSet = result.withPrefixMatcher(result.prefixMatcher.cloneWithPrefix(name))
    val queryScope = getCssFunctionQueryScope(queryExecutor, context)
    processCompletionQueryResults(queryExecutor, patchedResultSet, 
                                  NAMESPACE_CSS, KIND_CSS_FUNCTIONS, name,
                                  position, context, queryScope) {
      it.withDisplayName((it.displayName ?: it.name) + "()")
        .withInsertHandlerAdded(ParenthesesInsertHandler.WITH_PARAMETERS)
        .addToResult(parameters, patchedResultSet, 
                     CssCompletionUtil.CSS_PROPERTY_VALUE_PRIORITY.toDouble())
    }
  }
}

fun getCssFunctionQueryScope(registry: WebSymbolsQueryExecutor, context: PsiElement): List<WebSymbolsScope> =
  context.contextOfType<CssDeclaration>()
    ?.propertyName
    ?.let { 
       registry.runNameMatchQuery(WebSymbol.NAMESPACE_CSS, 
                                  WebSymbol.KIND_CSS_PROPERTIES, it) 
    }
  ?: emptyList()

 

 

0

Sorry for the delay in replying Piotr, and thanks for the very detailed post. Unfortunately it looks like WebSymbols are pretty new and not well supported in all IDE versions I support, although I'm going to drop some older versions soon once 2023.3 is done. Are there any examples of how to get PSI interop with the raw Symbol API I could look at? I'd be interested in seeing what would be involved in making that work.

0

Colin Fleming You can look at the WebSymbols source code. There's all you need to provide cooperation. Please note that Symbols API is also experimental and have undergone major changes throughout releases, so the older versions of IDEs might also have some issues with Symbols API.

0

Please sign in to leave a comment.