Mark element at caret and FindUsage missing a resolved element

Permanently deleted user

创建于 2014年07月05日 03:03

I am out of ideas here. My reference resolving is working correctly but the FindUsage highlighting is missing to mark an element, although I did FindUsages on this element. Here is a screenshot:

In the first line is the definition of head and it is used in the following two lines. You cannot see the cursor, but I started FindUsage on the second head which is not highlighted. When I do a "go to declaration" on this element, I correctly jump to the first "head=1". I have checked the reference connections between the elements and they seem fine. When I change the second line to "sym_?head", which is correct Mathematica syntax too, the element is highlighted as it should be. I have checked the implementations for the PsiElements in question and I cannot find a difference compared to the "?" operator.

Question: Is it possible that the underscore prevents the correct finding of the last part in sym_head because in Java it is a valid identifier part? Is there something else I can check?

Best regards
Patrick

7 条评论

Jon Akhtar

创建于 2014年07月05日 08:47

Do you use the DefaultWordScanner in your implementaiton of FindUsages?

It breaks words using JavaIdentifierCharacters. That would explain Find Usages.

Since it would not turn sym_head into <sym> <head>

I am assuming you have set your lexer up to produce NAME _ NAME tokens for sym_head,

Permanently deleted user

创建于 2014年07月05日 12:25

Hi Jon,

yes, currently I'm using the standard-implementation of FindUsage as it works correctly otherwise. Yes, the lexer scans the string "symbol_head" into {IDENTIFIER, UNDERSCORE, IDENTIFIER} because the underscore is an operator in Mathematica.
Do you know what extension point I have to implement to overwrite the default behavior, because there are several:

lang.findUsagesProvider
highlightUsagesHandlerFactory
usageTargetProvider
findUsagesHandlerFactory
customUsageSearcher

I guess it is customUsageSearcher?

Cheers
Patrick

Jon Akhtar

创建于 2014年07月05日 19:18

 <lang.findUsagesProvider language="Lua"
                         implementationClass="com.sylvanaar.idea.Lua.findUsages.LuaFindUsagesProvider"/>

Here is the one I implement in my plugin.

Did you use DefaultWordsScanner to implement getWordsScanner? If you used SimpleWordScanner then it won't work for you.

Did you provide the proper tokens to the DefaultWordsScanner? So that it knows that the _ is not part of an identifier?
 @NotNull
public WordsScanner getWordsScanner() {
    return  new DefaultWordsScanner(new LuaLexer(),
        LuaTokenTypes.IDENTIFIERS_SET, LuaTokenTypes.COMMENT_SET, LuaTokenTypes.LITERALS_SET) {{
        setMayHaveFileRefsInLiterals(true);
    }};
}

Can you show some code, maybe that would help.

Permanently deleted user

创建于 2014年07月06日 03:50

Hi Jon,

I looked into your Lua plugin and implemented a basic FindUsageProvider myself but that doesn't fix the problem. If you want to look at the code, please find at the bottom of this post links to the lexer Token and the FindUsageProvider implementation.
I did some further debugging and here are more information: First of all, I have turned on "mark element at caret" and the weird thing is, when I don't have a valid definition, the "head" gets highlighted: See that it is correctly underlined

when I insert a valid definition for "head", then it is no longer highlighted. Not when I move the caret over it and not when I use FindUsage. Additionally, so that you don't need to dig through my code, here is the Psi-tree for the above "sym_head" code

The marked blue Symbol node is "head". The Blank node is the underscore operator. Here is the Psi-Tree for the code "sym?head" where the correct highlighting of usages works and which only differs in the ? operator

Additionally, I did some further debugging. The problematic code seems to be in com.intellij.codeInsight.daemon.impl.IdentifierHighlighterPass#highlightTargetUsages and there in the call findUsagesHandler.findReferencesToHighlight(target, scope). This method calls ReferencesSearch.search(target, searchScope, false).findAll() of the
class com.intellij.psi.search.searches.ReferencesSearch. From there it is a bit hard to follow,
but I believe that in com.intellij.util.UniqueResultsQuery#findAll the list of processor is empty for the example where it doesn't work.

final CommonProcessors.CollectProcessor<T> processor = new CommonProcessors.CollectProcessor<T>(Collections.synchronizedList(new ArrayList<T>()));

Do you have any ideas?

Cheers

Patrick

https://github.com/halirutan/Mathematica-IntelliJ-Plugin/blob/feat_findUsageProvider/src/de/halirutan/mathematica/parsing/MathematicaElementTypes.java

https://github.com/halirutan/Mathematica-IntelliJ-Plugin/blob/feat_findUsageProvider/src/de/halirutan/mathematica/codeinsight/findusage/MathematicaFindUsageProvider.java

Jon Akhtar

创建于 2014年07月06日 06:49

I want to say that your:sym_head is not really a reference to what you think it is.

You should add a property that shows the resolved element so you can see it in PsiViewer.

I have this on my references:

 public PsiElement getResolvedElement() {
    return resolve();
}

In PsiViewer I can see what is going on.

What is the enclosing PSI element that contains the 2 Symbols?

I think that your second identifier is a reference to itself.

Also, what is the enclosing PsiElement that holds the entire symbol expression?

Aslo, why is your name identifier the symbol, and not the name identifier?

One thing that I see is that  you don't have much semantic meanting in your Psi for what the combined symbol means. Woudn't it be a reference also?

I have run into this problem, when I could not resolve references to the correct defintion, this would prevent them from being highligted.

I had this problem a lot with fields

a.b = 1
a.b.c = 2

pur cursor nn b, both b's should highlight.

Do you perhaps need a symbol type that is essentially a "qualified" symbol? 

At the endo of the day you could also implement ReferencesSearch, but

Nothing really jumps out at me. I would need to spend some time looking at your Psi in PsiViewer in order to understand it better.

Permanently deleted user

创建于 2014年07月07日 04:32

Hi Jon,

first of all, thanks for your patience. I think I solved the problem. To prove you wrong and to show that my references work, I wrote a small Action which highlights all usages by doing the work that FindUsages does manually. Indeed, this worked the way I expected it. The main part consists of simply finding the definition of an identifier (in Mathematica this is called a Symbol) and then track down all Symbols that refer to that element:

final List<PsiElement> usages = new ArrayList<PsiElement>();
if (element instanceof Symbol) {
  final PsiReference ref = element.getReference();
  if (ref != null) {
    final PsiElement resolve = ref.resolve();
    if (resolve != null && resolve instanceof Symbol) {
      if(!resolve.equals(element))
        usages.add(resolve);
      final Collection<Symbol> symbolsInFile = PsiTreeUtil.findChildrenOfType(psiFile, Symbol.class);
      for (Symbol symbol : symbolsInFile) {
        final PsiReference reference = symbol.getReference();
        if (reference != null) {
          final PsiElement resolve1 = reference.resolve();
          if (resolve1 != null && !symbol.equals(resolve1) && resolve.equals(resolve1)) {
            usages.add(symbol);
          }
        }
      }
    }
  }
}

Since this worked, I debugged the FindUsages (which uses the same approach than "highlight element at caret") and the problematic part is indeed the call findUsagesHandler.findReferencesToHighlight(target, scope)
in the file com.intellij.codeInsight.daemon.impl.IdentifierHighlighterPass#highlightTargetUsages. To circumvent this, I implemented my own FindUsageHandlerFactory and FindUsageHandler and the result is, that it
works. When I have time, I need to carefully track down, where exactly the search goes wrong, because in this current, crude implementation, my FindUsageHandler is far too slow for large files.
Let me answer your questions anyway, since you took the time to look into this issue with me:

> What is the enclosing PSI element that contains the 2 Symbols?
In the case of sym_head it is "Blank" which is the name of the underscore operator.

> I think that your second identifier is a reference to itself.
No, because if it would reference to itself, the "Goto declaration" wouldn't work correctly which it does.

> Aslo, why is your name identifier the symbol, and not the name identifier?
From the lexer I get IDENTIFIER token, but since in Mathematica, every name is called Symbol, my PsiNode for identifier is Symbol. Also, in Mathematica there don't exists any keywords. Everything, even If, Do, Return, are normal Symbols which
can be used and mis-used like any other thing. Additionally, I started this plugin without having any idea what I'm doing. I just looked at other plugins and tried to understand why they chose the design they chose and mimiced it.

> combined symbol means
No, there are no combined symbols. sym_head is really similar to sym+head or sym/head. The _ is just an operation between sym and head.

> At the endo of the day you could also implement ReferencesSearch
Unfortunately, this is what I will need to do since exactly there lies the problem.

Jon, thank you very very much for your help. I really appreciate it.

Cheers
Patrick

Permanently deleted user

创建于 2014年07月07日 04:44

As a sidenote: I cloned your Lua repository to check what you did. I didn't know Lua (only the name) until tonight and it seems since the language looks easy, your plugin is a good reference when one wants to see how other features can be implemented.

请先登录再写评论。