Create task in background with modal progress bar on editor while processing content in background
My plugin (source code here) is about highlighting ansi sequences in log files under IntelliJ editor like the console. Everything works fine until a large log file with thousands of ansi sequences is loaded which causes the UI to freeze for quite a while due to executing everything under EDT. I'm trying to fix this by improving my highlighter algorithm and moving it to a background task as follows:
public void highlightANSISequences(Editor editor) {
ProgressManager.getInstance().run(new Task.Backgroundable(project, "Highlighting ANSI Sequences...", false) {
@Override
public void run(@NotNull ProgressIndicator indicator) {
//process editor large content with thousands of ansi sequences
}
});
}
However the problem with the code above is that it leaves the editor exposed to user change as well as external change while expensive ansi highlighting is running on the background. Is there any way I can place a modal progress indicator on top of the editor and lock the file to prevent external changes while the highlighter is doing its job?
Please sign in to leave a comment.
What would happen if some code (in EDT) desires to change the editor's document? Would the UI freeze?
The usual way to solve similar problems is to run a cancellable background process, and restart it if anything happens. Please see "Preventing UI Freezes" section in http://www.jetbrains.org/intellij/sdk/docs/basics/architectural_overview/general_threading_rules.html.
Thanks for the hint.
"What would happen if some code (in EDT) desires to change the editor's document? Would the UI freeze?"
I'm clearing all the highlights / fold regions, and reworking them from scratch on fileContentReloaded, which yes, when the file is large causes the UI to freeze.
I had a look at the referred "Preventing UI Freezes" section, it suggests using ProgressIndicatorUtils.schedule* when on EDT (my case) but I couldn't find a good reason to prefer it over ProgressManager.run(Task.Backgroundable). In fact, I'm preferring the latter because it allows updating a progress indicator with a custom message more intuitively, both approaches give the option to make the task cancellable.
That said, here's more context about my problem: I'm testing on a file with 10k lines each tagged with a random ansi sequence, to render those I need 10k calls to both MarkupModel.addRangeHighlighter() and FoldingModel.addFoldRegion() (one per highlight / fold region), both methods must be invoked in the UI thread and that's what causes the freeze, just calling both methods 10k times in the UI thread without doing anything else freezes the UI for several seconds.
I ended up splitting the large number of EDT calls into smaller chunks (100 calls per chunk), and made the background task sleep for a few milliseconds in between UI calls so to give the EDT a chance to "breath" after processing each small chunk through Application.invokeAndWait(), more dirty work was then needed to handle simultaneous highlighting of multiple editors under the same background task and make sure only one thread is submitting UI tasks to EDT. Things seem to work smoothly now, but I wanted to share the idea and hope for advice if I'm missing something, code is here.
There's one more related problem left: the editor seems to save and reload its FoldingModel state internally every time it gets reopened. This also causes the UI to freeze for a few seconds as all fold regions get restored before the editor opens. Is there a way to prevent a particular file type from saving/reloading its folding state?
Note that I'm using folding to hide ANSI sequences.
Task.Backgroundable has a possibility to make the task cancellable by user pressing on "Cancel", while ReadTask supports automatic cancel and restart when a UI freeze is about to occur.
From your message it seems you don't need a read action (which is really strange, but possible), so Task.Backgroundable approach should be fine, although you need to be prepared to document changes. Even if you block the editor, the document might be changed by some other activity: VFS refresh, another plugin, another editor, etc.
Breaking up the UI changes into small chunks seems to be a good idea. I don't think you need sleeping here, UI thread should be able to breathe without it just as well. Applcation#invokeAndWait should be enough. On versions prior to 2017.2, you might get "Too many events posted" messages in the log, which you can fix by using TransferToEDTQueue, since 2017.2 that won't be necessary.
For handling several editors, I'd use BoundedTaskExecutor, but unfortunately it's not very easy to combine it with Task.Backdroundable.
I'm not aware of any ways to turn off folding saving/reloading. Could you please file a YouTrack issue about that being slow, and attach a CPU snapshot?
I ended up keping Task.Backgroundable approach, as for document changes I'm handling them through a fileContentReloaded listener. Note that I'm blocking user interaction with the editor in preview mode just to prevent inconsistent behavior with the folded ansi sequences.
I tried removing Thread#sleep calls, and yes Application#invokeAndWait alone was enough to keep the UI responsive. Unfortunately TransferToEDTQueue doesn't seem to support invokeAndWait so it's not of much help here. I haven't spotted a "Too many events posted" in the logs, but if it's just a warning message and is fixed in new versions, then I guess it shouldn't be a problem.
I had a look at BoundedTaskExecutor, and although it was very tempting to use it's actually not a good fit for what I'm trying to achieve. Here's the scenario I'm addressing: I have 3 large log files opened then some external tool updates all 3 of them, in this case I'd rather have the highlighter process all 3 files simultaneously than process file 1, then 2, then 3. To achieve this I used a circular queue where each item/node refers to a file and carries context about how many small "chunks" have been processed, as soon as a file gets fully processed it gets removed from the circular queue, and the background task stops when the queue becomes empty. A few more consideration had to be handled, but that's the main idea. Note that I made sure all write operations to the queue (all have constant time) happen in the UI thread so to avoid any synchronization overhead.
That said, I finally found a workaround to disable folding saving/reloading, and will be releasing it soon. If anyone is interested, the workaround can be found here under #applyWorkaroundToDisableFoldingStateRestoration().
I just filed a performance issue here, unfortunately the CPU snapshot instructions did not work (sorry didn't have time to investigate), but I did attach a console thread dump where the UI freeze was happening.
Thanks for the thorough answer and sorry for the delay, I had to find a workaround for folding state restoration freeze before I follow up.