Shared indexes usage

Answered

Permanently deleted user

Created January 12, 2021 08:26

Hi,

I'm trying to use new shared indexes feature (Intellij 2020.3 - 203.5981.155)

I did the flow as described here:
https://www.jetbrains.com/help/idea/shared-indexes.html#shared-jdk-indexes

I've created the intellij.yaml in project root as described.

I'm trying to understand if it's working properly or not - but I don't find any indication that it downloaded the indexes from my storage (I've checked the storage and there are no downloads from it)

I've checked the idea.log as well and I don't see any indication about downloading shared indexes.

This is my storage hierarchy:

In the yaml I put the following:

sharedIndex:
    project:
        - url: <storage_root>/intellij-index/project/products/

Am I doing something wrong ?

How can I test and validate that it's working ?

Thanks!

Kobi

p.s I'm not sure it's related but I see this error in the idea.log constantly:

2021-01-12 10:23:05,014 [64684991] ERROR - n.impl.GeneralHighlightingPass - In file: file:///Users/kobi.c/projects/products2/intellij.yaml 
java.lang.ClassCastException: class com.intellij.execution.ui.FragmentedSettings$Option cannot be cast to class com.intellij.openapi.components.BaseState (com.intellij.execution.ui.FragmentedSettings$Option and com.intellij.openapi.components.BaseState are in unnamed module of loader com.intellij.util.lang.UrlClassLoader @3dd3bcd)
at com.intellij.configurationScript.schemaGenerators.OptionClassJsonSchemaGenerator$describe$2$1.invoke(OptionClassJsonSchemaGenerator.kt:43)
at com.intellij.configurationScript.schemaGenerators.OptionClassJsonSchemaGenerator$describe$2$1.invoke(OptionClassJsonSchemaGenerator.kt:16)
at org.jetbrains.io.JsonObjectBuilder.map(JsonBuilder.kt:66)
at com.intellij.configurationScript.schemaGenerators.OptionClassJsonSchemaGenerator$describe$2.invoke(OptionClassJsonSchemaGenerator.kt:42)
at com.intellij.configurationScript.schemaGenerators.OptionClassJsonSchemaGenerator$describe$2.invoke(OptionClassJsonSchemaGenerator.kt:16)
at org.jetbrains.io.JsonObjectBuilder.map(JsonBuilder.kt:66)
at com.intellij.configurationScript.schemaGenerators.OptionClassJsonSchemaGenerator.describe(OptionClassJsonSchemaGenerator.kt:40)
at com.intellij.configurationScript.schemaGenerators.RunConfigurationJsonSchemaGenerator.generateDefinitions(RunConfigurationJsonSchemaGenerator.kt:97)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactoryKt.doGenerateConfigurationSchema(IntellijConfigurationJsonSchemaProviderFactory.kt:122)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactoryKt.generateConfigurationSchema(IntellijConfigurationJsonSchemaProviderFactory.kt:86)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactoryKt.access$generateConfigurationSchema(IntellijConfigurationJsonSchemaProviderFactory.kt:1)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactory$schemeContent$2.invoke(IntellijConfigurationJsonSchemaProviderFactory.kt:32)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactory$schemeContent$2.invoke(IntellijConfigurationJsonSchemaProviderFactory.kt:30)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactory.getSchemeContent(IntellijConfigurationJsonSchemaProviderFactory.kt)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactory.access$getSchemeContent$p(IntellijConfigurationJsonSchemaProviderFactory.kt:30)
at com.intellij.configurationScript.IntellijConfigurationJsonSchemaProviderFactory$MyJsonSchemaFileProvider$schemeFile$1$1.getContent(IntellijConfigurationJsonSchemaProviderFactory.kt:54)
at com.intellij.psi.AbstractFileViewProvider$VirtualFileContent.getText(AbstractFileViewProvider.java:465)
at com.intellij.psi.AbstractFileViewProvider.getContents(AbstractFileViewProvider.java:160)
at com.intellij.psi.impl.source.PsiFileImpl.loadTreeElement(PsiFileImpl.java:211)
at com.intellij.psi.impl.source.PsiFileImpl.getTreeElement(PsiFileImpl.java:114)
at com.intellij.psi.impl.source.PsiFileImpl.calcTreeElement(PsiFileImpl.java:733)
at com.intellij.psi.impl.source.PsiFileImpl.getNode(PsiFileImpl.java:937)
at com.intellij.psi.impl.source.PsiFileImpl.getFirstChild(PsiFileImpl.java:744)
at com.intellij.psi.util.PsiTreeUtil.getChildOfType(PsiTreeUtil.java:340)
at com.intellij.json.psi.impl.JsonFileImpl.getTopLevelValue(JsonFileImpl.java:30)
at com.jetbrains.jsonSchema.impl.JsonOriginalPsiWalker.getRoots(JsonOriginalPsiWalker.java:253)
at com.jetbrains.jsonSchema.impl.JsonSchemaReader.lambda$read$0(JsonSchemaReader.java:101)
at com.intellij.util.AstLoadingFilter.forceAllowTreeLoading(AstLoadingFilter.java:156)
at com.intellij.util.AstLoadingFilter.forceAllowTreeLoading(AstLoadingFilter.java:148)
at com.jetbrains.jsonSchema.impl.JsonSchemaReader.read(JsonSchemaReader.java:101)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.computeSchemaObject(JsonCachedValues.java:44)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.lambda$getOrCompute$5(JsonCachedValues.java:214)
at com.intellij.psi.util.CachedValuesManager$1.compute(CachedValuesManager.java:153)
at com.intellij.psi.impl.PsiCachedValueImpl.doCompute(PsiCachedValueImpl.java:54)
at com.intellij.util.CachedValueBase.lambda$getValueWithLock$1(CachedValueBase.java:235)
at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:111)
at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:236)
at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:43)
at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:78)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:150)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.getOrCompute(JsonCachedValues.java:214)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.computeForFile(JsonCachedValues.java:100)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.getSchemaObject(JsonCachedValues.java:39)
at com.jetbrains.jsonSchema.impl.JsonSchemaServiceImpl.getSchemaObject(JsonSchemaServiceImpl.java:319)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.lambda$computeSchemaForFile$6(JsonCachedValues.java:225)
at com.intellij.psi.util.CachedValuesManager$1.compute(CachedValuesManager.java:153)
at com.intellij.psi.impl.PsiCachedValueImpl.doCompute(PsiCachedValueImpl.java:54)
at com.intellij.util.CachedValueBase.lambda$getValueWithLock$1(CachedValueBase.java:235)
at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:111)
at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:236)
at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:43)
at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:78)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:150)
at com.jetbrains.jsonSchema.impl.JsonCachedValues.computeSchemaForFile(JsonCachedValues.java:222)
at com.jetbrains.jsonSchema.impl.JsonSchemaServiceImpl.getSchemaObject(JsonSchemaServiceImpl.java:326)
at com.jetbrains.jsonSchema.impl.JsonSchemaBasedLanguageInjector.getLanguageToInject(JsonSchemaBasedLanguageInjector.java:32)
at org.jetbrains.yaml.schema.YamlJsonSchemaLanguageInjector.getLanguagesToInject(YamlJsonSchemaLanguageInjector.java:22)
at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.processInPlaceInjectorsFor(InjectedLanguageManagerImpl.java:418)
at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.probeElementsUp(InjectedLanguageUtilBase.java:207)
at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.enumerate(InjectedLanguageUtilBase.java:137)
at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.enumerateEx(InjectedLanguageManagerImpl.java:318)
at org.intellij.plugins.intelliLang.references.InjectedReferencesContributor.doGetReferencesByElement(InjectedReferencesContributor.java:101)
at org.intellij.plugins.intelliLang.references.InjectedReferencesContributor.lambda$getInjectionInfo$0(InjectedReferencesContributor.java:62)
at com.intellij.psi.util.CachedValuesManager$1.compute(CachedValuesManager.java:153)
at com.intellij.psi.impl.PsiCachedValueImpl.doCompute(PsiCachedValueImpl.java:54)
at com.intellij.util.CachedValueBase.lambda$getValueWithLock$1(CachedValueBase.java:235)
at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:111)
at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:236)
at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:43)
at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:78)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:150)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:120)
at org.intellij.plugins.intelliLang.references.InjectedReferencesContributor.getInjectionInfo(InjectedReferencesContributor.java:60)
at org.intellij.plugins.intelliLang.references.InjectedReferencesContributor$1.getReferencesByElement(InjectedReferencesContributor.java:54)
at com.intellij.psi.impl.source.resolve.reference.ReferenceProvidersRegistryImpl.getReferences(ReferenceProvidersRegistryImpl.java:204)
at com.intellij.psi.impl.source.resolve.reference.ReferenceProvidersRegistryImpl.mapNotEmptyReferencesFromProviders(ReferenceProvidersRegistryImpl.java:167)
at com.intellij.psi.impl.source.resolve.reference.ReferenceProvidersRegistryImpl.doGetReferencesFromProviders(ReferenceProvidersRegistryImpl.java:146)
at com.intellij.psi.impl.source.resolve.reference.ReferenceProvidersRegistry.lambda$getReferencesFromProviders$0(ReferenceProvidersRegistry.java:39)
at com.intellij.psi.util.CachedValuesManager$1.compute(CachedValuesManager.java:153)
at com.intellij.psi.impl.PsiCachedValueImpl.doCompute(PsiCachedValueImpl.java:54)
at com.intellij.util.CachedValueBase.lambda$getValueWithLock$1(CachedValueBase.java:235)
at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:111)
at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:236)
at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:43)
at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:78)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:150)
at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:120)
at com.intellij.psi.impl.source.resolve.reference.ReferenceProvidersRegistry.getReferencesFromProviders(ReferenceProvidersRegistry.java:38)
at com.intellij.psi.impl.source.resolve.reference.ReferenceProvidersRegistry.getReferencesFromProviders(ReferenceProvidersRegistry.java:31)
at org.jetbrains.yaml.psi.impl.YAMLScalarImpl.getReferences(YAMLScalarImpl.java:73)
at com.intellij.codeInsight.highlighting.HyperlinkAnnotator.annotate(HyperlinkAnnotator.java:45)
at com.intellij.codeInsight.daemon.impl.DefaultHighlightVisitor.runAnnotators(DefaultHighlightVisitor.java:136)
at com.intellij.codeInsight.daemon.impl.DefaultHighlightVisitor.visit(DefaultHighlightVisitor.java:116)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.runVisitors(GeneralHighlightingPass.java:336)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.lambda$collectHighlights$5(GeneralHighlightingPass.java:269)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.analyzeByVisitors(GeneralHighlightingPass.java:295)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.lambda$analyzeByVisitors$6(GeneralHighlightingPass.java:298)
at com.intellij.codeInsight.daemon.impl.DefaultHighlightVisitor.analyze(DefaultHighlightVisitor.java:96)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.analyzeByVisitors(GeneralHighlightingPass.java:298)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.collectHighlights(GeneralHighlightingPass.java:266)
at com.intellij.codeInsight.daemon.impl.GeneralHighlightingPass.collectInformationWithProgress(GeneralHighlightingPass.java:212)
at com.intellij.codeInsight.daemon.impl.ProgressableTextEditorHighlightingPass.doCollectInformation(ProgressableTextEditorHighlightingPass.java:84)
at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:56)
at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$1(PassExecutorService.java:400)
at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1137)
at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$2(PassExecutorService.java:393)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:658)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:610)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:65)
at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.doRun(PassExecutorService.java:392)
at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$0(PassExecutorService.java:368)
at com.intellij.openapi.application.impl.ReadMostlyRWLock.executeByImpatientReader(ReadMostlyRWLock.java:172)
at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:183)
at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:366)
at com.intellij.concurrency.JobLauncherImpl$VoidForkJoinTask$1.exec(JobLauncherImpl.java:188)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
2021-01-12 10:23:05,015 [64684992] ERROR - n.impl.GeneralHighlightingPass - IntelliJ IDEA 2020.3 Build #IU-203.5981.155 
2021-01-12 10:23:05,016 [64684993] ERROR - n.impl.GeneralHighlightingPass - JDK: 11.0.9; VM: OpenJDK 64-Bit Server VM; Vendor: JetBrains s.r.o. 
2021-01-12 10:23:05,016 [64684993] ERROR - n.impl.GeneralHighlightingPass - OS: Mac OS X

7 comments

Arina Efremova

Created January 12, 2021 11:11

Hi Kobi,

Are there any entries in the idea.log referring to a 'SharedIndexDownloadService'? Can you upload the whole log to https://uploads.jetbrains.com/ after restarting the IDE and attempting to download indexes again, and provide the upload id for us to check?

Arina Efremova

Created January 12, 2021 13:09

A bug report was also created for the error that you posted: IDEA-259302.

Developers are looking into whether it could be the reason for the behavior that you observed.

Permanently deleted user

Created January 12, 2021 13:44

Thanks @...!

I've uploaded the log, Upload id: 2021_01_12_2DMeovH4mNaVpbaH

I do see SharedIndexDownloadService in the log but only for JDK and not for project indexes download.

2021-01-12 15:38:33,086 [83613063] INFO - oad.SharedIndexDownloadService - There is no need to download shared indexes for JdkSharedIndexSuggestion(SharedIndexId(kind=jdk, url=https://index-cdn.jetbrains.com/v2/jdk, indexId=145ce5a5c334a005b63ed5b02eadfbf174e2cad13dd5a896f3cfaca55887a569#java version "1.8.0_232"), lazy). Similar compatible shared indexes are available locally.

Thanks again for the quick response!

Arina Efremova

Created January 12, 2021 14:28

No problem Kobi.

I forwarded the logs to the Dev team. Let's wait for what they have to say.

I'll keep you posted.

Permanently deleted user

Created February 09, 2021 14:58

Hi @...

Any update on this ?

Thanks

Arina Efremova

Created February 09, 2021 15:21

Hi Kobi,

Not at the moment, sorry. I'll ping the responsible developer, he will post an update to the YouTrack ticket.

Gerardnico

Created October 05, 2023 12:27

I got the same problem. Thanks to your tips in the direction of the log, I found my mistake.

For those of you that stumbled on this page, here the mistake that I made.

The error is that they ask you to put in the yaml file the URL with the project

https://url/project/java-mono/

but the data should not be uploaded in this directory but in the root. ie

https://url/

If you dig in the log, you see that it tries to download the list:

https://url/vcs/list.json.xz

The indexes are generated in a directory that is good for all projects, not for one project only.

All the best

Please sign in to leave a comment.