dumbService.runReadActionInSmartMode() blocks the UI?

runReadActionInSmartMode() will block the pooled thread from which you're calling the method but will not block the UI thread, so the problem is somewhere else. How exactly are you handling the Future instance returned from executeOnPooledThread()?

Thanks!

I'm just calling future.get(); afterwards. Below is the full method. Is there anything that I can do to debug this? Maybe I'll try to fork a new thread from the call site of this method.

**Update:** It still blocks the UI even I create a new thread from the call site of the method below.

public Collection<File> query(final Collection<File> javaFiles) {
    Future<Collection<File>> future = ApplicationManager.getApplication().executeOnPooledThread(new Callable<Collection<File>>() {
        @Override
        public Collection<File> call() throws Exception {
            DumbService dumbService = DumbService.getInstance(mModule.getProject());
            return dumbService.runReadActionInSmartMode(new Computable<Collection<File>>() {
                @Override
                public Collection<File> compute() {
                    return long_running_read_action_task_that_queries_PsiReferences(javaFiles);
                }
            });
        }
    });
    try {
        return future.get();
    } catch (InterruptedException e) {
        e.printStackTrace();
        return Collections.emptySet();
    } catch (ExecutionException e) {
        e.printStackTrace();
        throw new IllegalStateException(e);
    }
}

probably a stupid question, but is there a chance your long_running_read_action_task_that_queries_PsiReferences(javaFiles); would be blocking the UI thread itself ? synchronization maybe ?

If you debug your plugin, and reproduce the blocking scenario you  should be able to take a thread dump while it's blocked and see what's the UI thread is doing

future.get() is a synchronous call that will block the current thread and wait for the future execution to complete. If you're calling it from the UI thread, it will block the UI thread.

Thanks for the answers Dmitry and Thibaut. However, it looks like runReadActionInSmartMode indeed blocks the UI thread. The code below blocks the UI the same 

way as my old code.

    @Override
    public Collection<File> someMethod() {
        DumbService dumbService = DumbService.getInstance(mModule.getProject());
        return dumbService.runReadActionInSmartMode(new Computable<Collection<File>>() {
            @Override
            public Collection<File> compute() {
                try {
                    Thread.sleep(10000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                return Collections.emptySet();
            }
        });

    }

And this code does not:

    @Override
    public Collection<File> someMethod() {
        try {
            Thread.sleep(10000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return Collections.emptySet();
    }

Could you please just take a thread dump and see what the AWT thread is waiting for, and why it's blocked? If it's not obvious, you can attach the dump here and we'll take a look at it.

I did the thread dump and found the cause. (should have done that earlier but there were some issues that prevented me from running the plugin from IDEA).

Basically there is a write action on the main thread waiting for the read action to complete. Since AFAIK write actions can only be executed on the UI thread, hence the UI is blocked until the read action finishes.

What's the best approach to solve this though? The write action is initiated by the user. How to make sure that I won't launch the long-running read action until the write action starts? Or maybe I should have broken down that long-running read action?

I was able to improve the responsiveness of the UI by putting the thread to sleep for a while, but that's obviously hacky.

        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        DumbService dumbService = DumbService.getInstance(mModule.getProject());
        return dumbService.runReadActionInSmartMode(new Computable<Collection<File>>() {
            @Override
            public Collection<File> compute() {
                try {
                    Thread.sleep(10000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                return Collections.emptySet();
            }
        });

Blocking of AWT thread when waiting for a write action can only happen if some thread already has taken a read action (runReadActionInSmartMode doesn't do this while waiting). Please check that another thread (e.g. the one waiting for Future) doesn't have the read action already.

runReadActionInSmartMode doesn't do this while waiting

Did you mean it doesn't aquire the read token if it's still in dumb mode?

There is indeed a read action in the thread before the write action is invoked. But the write action is initiated by the user and I don't have control of that.

Is it generally a bad idea to have long-running read actions (since it'll block the UI)?

No, runReadActionInSmartMode doesn't acquire long read actions while in dumb mode. And it's a bad idea, because it blocks the UI. Since you have no control over the user, the background read actions have to be short, or they have to be canceled and restarted whenever a write action is about to occur. The latter can be achieved by the help of ProgressIndicatorUtils class, but that's complicated, error-prone and if you can avoid it, do so. So is it your situation, that there's another thread doing a long read action? If you attach a thread dump, I might be able to understand better.

Thanks. Yes, in my code, there's a monolithic long-running read action in the thread.
I'm trying to break it down into smaller pieces. It's something like the following (Apparently the actual code is more complex):

Before:
dumbService.runReadActionInSmartMode( ()-> {
  for (int i=1; i<10; i++) {
     long_running_method_reading_Psi_data();
  }
});

After:
for (int i=1; i<10; i++) {
  dumbService.runReadActionInSmartMode( ()-> {
     long_running_method_reading_Psi_data();
  });
  Thread.sleep(100); // give write actions an opportunity to run
}

This seems to improve the UI responsiveness a bit but forcing the background to sleep in the loop is wasteful. Is there a better way?

Please note that between read actions everything can change, so you might read inconsistent data the PSI data you read in different read actions might be inconsistent. PSI can also become invalid once you leave a read action. Please think if it's OK for you.

I'd just remove the sleep, there's no need for it. Write actions have a priority, so a read action won't start if there's a write action waiting to begin.

Thanks Peter! Yup that makes sense.

Another direction is to make those long running methods faster. Not sure if it's possible though. The most time-consuming part in the code is using ReferenceSearch.
The code goes like this:

for(PsiElement e:allElements) {
  refs = ReferenceSearch.search(e, scope).findAll();
}

Assuming the scope is already trimmed down to minimal, is there a way to optimize this?
This has to be wrapped in a read action, right?

Actually, ReferencesSearch is one of the things that don't need to be wrapped in a read action. It runs concurrently on multiple threads inside and is supposed to take all the needed read actions itself. So please try to remove it.

There's little you can do to optimize the search itself. The only thing I can think of is reducing the search scope if you know precisely where you want to search. You can also profile (e.g. with YourKit), see what takes long during the search and tell us about it so we can think how to optimize it :)

That's great to know! I'll give it a try and report back. Thanks for the quick response Peter!

It's working much better after I moved ReferenceSearch out of the read action. Thanks again Peter!