StubIndices getting mixed up

I have multiple different StubIndex implementations, derived from "StringStubIndexExtension".  I seem to be getting instances of one index returning PsiElements from another index, and I'm not quite sure how to track down how/why this might be happening.  All of the index classes are listed in my plugin.xml with the <index> tag.  I've checked/double checked all of my serialize/deserialze functions to make sure data is being written/read in the proper order with the proper types.

What can cause this, and what can I do to track down the issue?

Additional possibly relevant information:
One case that seems to trigger this frequently is when using FileContentUtil.reparseFiles().  More specifically:
1) The custom language I am supporting has a form of macro (#if SOMEDEFINE ... #endif)
2) I have an UI to toggle these in Intellij; doing so builds a collection of files that use said macro by querying a StubIndex for elements, and then building the list of their containing VirtualFiles
3) I pass this collection of VirtualFiles through FileContentUtil.reparseFiles() because the state of the macro value changes the way the file is parsed.  Code inside of an #if/#endif block might disappear (gets turned into PsiComment during parsing), and code from a coresponding #else/#elseif block might suddenly get parsed when it previously wasn't.

My expectation is that each file should get reparsed, and a correct stub tree will be rebuilt.  The issue tends to happen after the second or third reparse.

7 comments
Comment actions Permalink

(assuming the problem reproduces with clean caches)
Does your StringStubIndexExtension's have different StubIndexKey's / names ?

0
Comment actions Permalink

They do I believe:

 
public static final StubIndexKey<String, ScriptFunctionDeclaration> FUNCTION_DECLARATION = StubIndexKey.createIndexKey( "script.functiondeclaration" );

 
public static final StubIndexKey<String, ScriptGlobalFunctionDeclaration> GLOBAL_FUNCTION = StubIndexKey.createIndexKey( "script.globalfunction" );


Those are two examples.  There are instances where they both might contain an element with the same string/name, but pointing to different Psi elements.  BTW, this issue also happens occassionally even without my reparsing code.

0
Comment actions Permalink

Do StubElementType's for ScriptFunctionDeclaration and ScriptGlobalFunctionDeclaration have different externalIds ?
(If you have some source code shared I'd give much more specific advice)

0
Comment actions Permalink

Unfortunately I can't share full source code, but they do have different external ID's.  I setup them up to use the same string as the StubIndexKey.  On another note, I was using a CachedValueManager in my PsiFileBase implementation and I noticed someone having an issue similar to mine in the following thread:
https://devnet.jetbrains.com/message/5566195#5566195

I've since disabled the usage of the CachedValueManager and I'm not seeing this issue at the moment.  Maybe there some extra consideration that needs to be taken when using this setup?  Maybe invalidating the cache?

Here's the source from the relevant files re the Index and Type implementations:

public class GlobalFunctionElementType extends IStubElementType<GlobalFunctionDeclarationStub, ScriptGlobalFunctionDeclaration>
{
   public GlobalFunctionElementType() {
      super( "GLOBAL_FUNCTION_DECLARATION", ScriptLanguage.INSTANCE );
   }

   @NotNull
   @Override
   public String getExternalId() {
      return "script.globalfunction";
   }

   @Override
   public void indexStub( @NotNull GlobalFunctionDeclarationStub stub, @NotNull IndexSink indexSink ) {
      indexSink.occurrence( GlobalFunctionKeyIndex.GLOBAL_FUNCTION, stub.getName() );
   }

   ...
}

public class GlobalFunctionKeyIndex extends StringStubIndexExtension<ScriptGlobalFunctionDeclaration> {

   public static final StubIndexKey<String, ScriptGlobalFunctionDeclaration> GLOBAL_FUNCTION = StubIndexKey.createIndexKey( "script.globalfunction" );

   private static final GlobalFunctionKeyIndex ourInstance = new GlobalFunctionKeyIndex();

   public static GlobalFunctionKeyIndex getInstance() {
      return ourInstance;
   }

   @NotNull
   public StubIndexKey<String, ScriptGlobalFunctionDeclaration> getKey() {
      return GLOBAL_FUNCTION;
   }
}



public class FunctionDeclarationElementType extends IStubElementType<FunctionDeclarationStub, ScriptFunctionDeclaration> {

   public FunctionDeclarationElementType() {
      super( "FUNCTION_DECLARATION", ScriptLanguage.INSTANCE );
   }

   @NotNull
   @Override
   public String getExternalId() {
      return "script.functiondeclaration";
   }

   @Override
   public void indexStub( @NotNull FunctionDeclarationStub stub, @NotNull IndexSink indexSink ) {
      indexSink.occurrence( FunctionDeclarationKeyIndex.FUNCTION_DECLARATION, stub.getName() );
   }

   ...
}

public class FunctionDeclarationKeyIndex extends StringStubIndexExtension<ScriptFunctionDeclaration> {

   public static final StubIndexKey<String, ScriptFunctionDeclaration> FUNCTION_DECLARATION = StubIndexKey.createIndexKey( "script.functiondeclaration" );

   public static final FunctionDeclarationKeyIndex ourInstance = new FunctionDeclarationKeyIndex();

   public static FunctionDeclarationKeyIndex getInstance() {
      return ourInstance;
   }

   @NotNull
   public StubIndexKey<String, ScriptFunctionDeclaration> getKey() {
      return FUNCTION_DECLARATION;
   }
}

0
Comment actions Permalink

The issue might be indeed related to caching in your PsiFile implementation, proper caching is quite complex for unsaved documents in presence of async psi document commit.

0
Comment actions Permalink

Is there a good strategy for handling this?  I haven't seen the problem today since disabled the cache, but I'd obviously like to use it, for performance reasons.

0
Comment actions Permalink

Let's create a test demonstrating the problem, then some solution can be proposed. My understanding is that problem can be in plugin assumptions on psi tree / document / virtual file as well as some issue with nonuniqueness of psi you have linked.

0

Please sign in to leave a comment.