Resolving Reference over the whole PSI tree (speed optimization)

The bad thing with Mathematica (nowadays the Wolfram Language) is that global definitions can appear everywhere. Therefore, to ensure I can resolve all references in my Mathematica plugin, I have (when everything else fails) to go through the whole PSI tree to find the point of definition. For this purpose I implemented a PsiElementProcessor and use PsiTreeUtil.processElements where I start at the FILE node and process all elements. So basically it is only

MathematicaGlobalSymbolProcessor globalProcessor = new MathematicaGlobalSymbolProcessor(myVariable);
PsiTreeUtil.processElements(myVariable.getContainingFile(), globalProcessor);


where the processor implementation looks like

  @Override
  public boolean execute(@NotNull PsiElement element) {
    if (element instanceof Set || element instanceof SetDelayed) {
      final List<Symbol> assignmentSymbols = MathematicaPsiUtililities.getAssignmentSymbols(element);
      if (assignmentSymbols != null) {
        for (Symbol symbol : assignmentSymbols) {
          if (symbol.getSymbolName().equals(myStartElement.getSymbolName())) {
            myReferringSymbol = symbol;
            return false;
          }
        }
      }
    }
    return true;
  }


With PsiTreeUtil there are some alternatives to this approach: (1) First collectElements to find all places of definition and extract the reference element after that (2) collectElementsOfType which is basically the same as (1) but it doesn't use a filter.

Which is the fastest method if you have to inspect the whole Psi tree to resolve a reference?

Cheers
Patrick

PS: I should mention, that I'm stunned by the permormance which IDEA already has with this implementation. In a large Mathematica file with about 9000 lines of deeply nested code, it takes only some seconds until everything is re-checked and the green spot indicates that everything is up-to-date. You guys are awsome!

5 comments
Comment actions Permalink

In addition to the stubs indicated by Yann, you might consider caching. I'm caching the symbols from an entire file in a map on the PsiFile itself using the CachedValuesManager, since many files don't change (libs, files in the background) and even for a file being actively edited, after a change there will be many resolves accessing the same info. That might be something to consider, although my use case is a little unusual.

0
Comment actions Permalink

Hi Yann, hi Colin,

@Colin: Currently, I'm not providing resolve operations over the boundary of a file, but I use caching myself. I'm caching the resolve results in the PsiElements itself. So if I have an x which is defined at some other place, say x = 3, I store both: the x knows the place where it is defined and the x in x=3 knows all its usages in the code. I clear the cache with the help of "subtreeChanged()", which works pretty well.

@Yann: I have already thought about using Indexing, but as I understood this correctly, this is mainly used to find quickly the externally visible functions of files and not the functions which are local to the file context, right? Because in my question, I'm not talking about *externally visible definitions*. I need to go through the whole file in order to find global definitions that are local to the file. Exported function are somewhat easier to determine. Therefore, everytime I type something in the currently opened file, the file gets re-parsed and the PsiTree is rebuild/updated. This is what the little green rectangle in the editor top indicates, right? As a consequence, it wouldn't make any difference whether or not I'm building an Index, because even this Index is rebuilt when I edit the file, correct? What would help me is, when the Indexer would not traverse the whole PsiTree on an edit action but only the subtree which has changed.

Question: Is creating a File Based Index with all defined functions faster even when I currently edit this file or is the same as traversing manually through the PsiTree as I do it now?

Cheers
Patrick

0
Comment actions Permalink

Hi Colin, see my message above. I didn't know whether you are pinged when I reply to Yann.

0
Comment actions Permalink

There is already an resolve cache implementation. Have you considered just using it?

ResolveCache


You just use it to implement your resolve() methods in your reference PSI classes by supplying a class that implements the ResolveCache interface

ResolveResult[] results = ResolveCache.getInstance(project).resolveWithCaching(this, RESOLVER, true, false);


Also, I resolve all the references in my PsiFile from my annotator which will refresh the resolve caches.


I use a different method to collect global symbols for the navigation tree. But you could use it for a global symbol map too.

In my PsiFile implementation I extend AtomicNotNullLazyValue and use a visitor to walk my Psi tree. I set the value to null in my override of ClearCaches.


For stubs, the whole point of them it to prevent the PsiTree from loading. So they may work for a file being edited, but that isnt their primary use case.

0

Please sign in to leave a comment.