Paralellize RelatedItemLineMarkerProviders in one plugin
My current situation is...
I am developing a plugin that adds different types of RelatedItemLineMarkerInfo - Objects to different Psi Elements. Most of them can get calculated fastly, but some of these line markers take a big amount of time to calculate, sometimes around 20 seconds * (which should not be topic of this question).
These slow line markers slow down the entire plugin, and prevent the fast line markers from getting displayed.
I, as a user of this plugin, would like to...
...already see all fastly generatable line markers as fast as possible, while the few slow ones continue to load in the background, in order to start working with the plugin faster.
I, the developer, have already tried to...
...split the single RelatedItemLineMarkerProvider into two separate classes, one for the fast and one for the slow ones. This does sadly not result in both categories of line markers getting collected in parallel.
My observations so far
The IDE uses multiple threads for collecting line markers, called "JobScheduler FJ Pool */3". To make sure that the long calculation is only executed once for all the line markers that need this data, I introduced a synchronization mechanism (using synchronized blocks, wait and notifyAll). It easily happens, that these three threads all wait for this calculation. This might be the cause, why the fast line markers are not getting collected meanwhile.
Is such a parallelization possible? If yes, how?
* Unimportant info about the time-consuming task: The task of my plugin is to jump to referenced files, where the references are given in our companies internal XML-based DSL. The fast line markers only go forward in this graph of referenced files (which is easy, because the file references can get transformed to file paths pretty easily). The few slow line markers need to do backward file jumps: Jumping to a file that references the current file. To get this information, all XML files in the currently opened project have to get analyzed and parsed to build up the reference graph data structure (it is then kept up to date by a BulkFileListener, only the initial setup takes a lot of time). My testing project creates a graph with about 2.000 nodes, an average vertex degree of 6.
Please sign in to leave a comment.
Have you considered using com.intellij.codeInsight.daemon.LineMarkerProvider#collectSlowLineMarkers instead?
That is sadly not helping much. I tried to directly implement LineMarkerProvider for the calculation-expensive line markers (and slowed them further down by Thread.sleep(60000) to better investigate the IDE behavior) and use its collectSlowLineMarkers - method.
Observations:
When the IDE starts, the fast line markers for the current viewport of the currently opened file get displayed instantly. By scrolling away immediately I can see, that the rest of the file has no line markers (neither the slow nor the fast ones). After roughly 3 seconds, this happens a second time: The current viewport (now different because I scrolled) gets its fast line markers. The reminder of the 60 seconds artificial loading delay happens nothing in the IDE ui: no fast line markers are generated, not for other parts of this file and not for other files. When the slow line markers have finished calculating, suddenly all markers (fast and slow) appear in every file.
When not using LineMarkerProvider#collectSlowLineMarkers, but RelatedItemLineMarkerProvider#collectNavigationMarkers instead, only the initial viewport lines get their line markers during startup, and only if it does not contain any of the slow line markers.
Where does this behavior come from, and how can it get optimized?
The "collectSlowLineMarkers" differs from the fast line marker only by the ability to batch line marker calculation for multiple PsiElements at once.
Unfortunately, we still need to wait until all markers are calculated in order to correctly remove/add/merge all line markers in the file.
As a workaround in your situation I'd recommend to just not show any line markers if it's known to be too slow - that is, do not show line markers until that huge graph of yours is calculated.