Prevent frequent files reindexing when creating lots of them programatically (> 1k) without UI freeze

Hi,

I'm creating plugin which, among others, pulls large number of files (up to ~1300) from remote server and saves them in the project structure. Whole the operation takes above 10 seconds. I don't want it to freeze UI, so I'm running it in the background like this:

ProgressManager.getInstance().run(new Task.Backgroundable())

Background mode works fine, but it causes problems when I'm creating files in filesystem (using File#createNewFile). It frequently reindexes groups of refreshed files (~20 files), what idea.log file proves:

...
2017-09-07 13:39:47,367 [ 50565] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 17 to update, calculated in 1ms
2017-09-07 13:39:47,444 [ 50642] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 77ms; general responsiveness: ok; EDT responsiveness: ok
2017-09-07 13:39:47,559 [ 50757] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 20 to update, calculated in 0ms
2017-09-07 13:39:47,770 [ 50968] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 210ms; general responsiveness: ok; EDT responsiveness: ok
2017-09-07 13:39:47,782 [ 50980] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 21 to update, calculated in 1ms
2017-09-07 13:39:47,865 [ 51063] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 83ms; general responsiveness: ok; EDT responsiveness: ok
2017-09-07 13:39:47,879 [ 51077] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 14 to update, calculated in 1ms
2017-09-07 13:39:47,939 [ 51137] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 60ms; general responsiveness: ok; EDT responsiveness: ok
2017-09-07 13:39:48,107 [ 51305] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 17 to update, calculated in 0ms
2017-09-07 13:39:48,263 [ 51461] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 156ms; general responsiveness: ok; EDT responsiveness: ok
2017-09-07 13:39:48,273 [ 51471] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 13 to update, calculated in 1ms
2017-09-07 13:39:48,357 [ 51555] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 84ms; general responsiveness: ok; EDT responsiveness: ok
...

In the result, I end up with the couple of confusing progress indicators at the bottom of IDE and around half of files empty (not sure why now, but it shouldn't affect my question) due to this frequent reindexing. When using:

DumbService#runWhenSmart()

creation works perfectly, all the files are created with the content and the reindexing is done once, at the very end, for all the files:

2017-09-07 13:42:35,531 [  57499]   INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 1278 to update, calculated in 16ms 
2017-09-07 13:42:36,520 [ 58488] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 988ms; general responsiveness: ok; EDT responsiveness: ok

but the case here is that the UI is frozen for that time (including progress indicator), even having it run in the background. Without the background mode, it also worked as it should, but with freezes as well.

What I want to achieve is creation of all those files without reindexing meanwhile, having responsive UI during creation (action in background) and reindexing done at the end of the operation. Is it possible to do this? It there any way to delay reindexing or wrap the files creation in some code that does not reindex at the time it is performed?

1 comment

Ok, it was my fault, as I was implementing VirtualFileListener. It was overriding created files and causing reindexing. Now I'm bypassing it by checking the requestor and I'm able to successfully create large amount of files programatically with indexing done at the end using DumbModeTask.

0

Please sign in to leave a comment.