Unable to terminate sub-process from which I'm draining stdout/stderr


I realize this is more of a general Java question than it is a plugin SDK question, but I figured I'd start here because there's a decent likelihood that others have worked through this already, and it may also be that I can use something already in the plugin SDK to help out with it as well.

Basically my plugin runs an external process to accomplish some tasks. The external process writes JSON-formatted response information to stdout and free-form information to stderr. I need to consume both completely to handle and present the results of running the external process.

I've written my own simple class to take care of this that creates two identical threads for draining the external process' stdout and stderr streams respectively (due to the blocking nature of the stream reads). As long as the process completes, this has worked perfectly for some time.

Now I have a situation where the external process may not complete, so I've given the user the option to cancel that operation via a progress indicator that's displayed while the external process runs. I have the external process running on a monitored sub-thread, and when the user clicks cancel, I cancel the sub-thread (it's a Future created from a single-thread executor service) with mayInterruptIfRunning=true. That, in turn, causes the watchdog for the running external process to receive an InterruptedException at which point it calls ProcessCloseUtil.close() on the external process. In a perfect world, that would kill the external process, but--and this is why I'm posting this question--that's not happening. From the parent process' perspective it seems that the child is gone, but the two stream consumer threads are both stuck in blocking reads against the child process' stdout and stderr streams respectively, and ProcessCloseUtil.close()'s scheduled call to closeStreams() is blocked waiting for those streams to close. If you kill the sub-process from outside the IDE, all of that obviously stops blocking and cleans up properly.

I debugged this a quite a bit last night and ultimately the issue seems to be that reads against these sub-process output streams are non-interruptible. I found a few threads online on the topic, but none of the posted solutions worked for me because, as far as I can tell, the only way to detect EOF on a stream is to try to read from it which takes you back into a non-interruptible blocking state. I did see that one person had gone so far as to create a library that used JNI to communicate with sub-processes to get around this, but my suspicion(/hope) is that I shouldn't have to do that. For one thing obviously IntelliJ IDEA itself is constantly running external sub-processes and piping their two output streams into console views, but is also allowing those sub-processes to be killed cleanly.

Am I missing something, either obvious or perhaps not-so-obvious? I really don't want to leave zombie processes and associated IDE threads hanging around until/unless the sub-process is killed explicitly by the end user.

Thanks in advance for any insights you can provide!

Comment actions Permalink

I"m not sure whether this will solve your problem or not, but this is how I handle it in the Haxe plugin...https://github.com/HaxeFoundation/intellij-haxe/blob/c2bf69e75d6947bc4ad3c4f5e1211e150d9a3692/src/common/com/intellij/plugins/haxe/util/HaxeProcessUtil.java#L51

Basically, this starts the process, sets up the stream readers, and runs a loop which sleeps for a bit, checks whether the stream(s) has(have) any output before making a read call, and checks for cancellation.  No blocking reads.

Comment actions Permalink

Thanks, Eric. So the good news is that by trying out a similar approach, I was able to close the Java streams and allow the threads consuming those streams to complete. The bad news is that the child process was still running...or rather the GRANDchild process was still running! I noticed that the remaining process was not actually the one I executed, but rather one that was executed by the process I executed directly. Basically the one I'm executing is a wrapper for a node process.

Evidently calling Process.destroy() (which is called by ProcessCloseUtil.close()) does not destroy the entire process tree, only that process, so I truly was ending up with an orphaned process. In this case that was important because it had bound a TCP port that's needed across invocations, so once the orphan is there it prevents further such processes from running successfully.

I then found OSProcessUtil.killProcessTree() which did in fact resolve all of this for me. I suppose the jury is still out on whether I should be calling that directly, but since it's in the com.intellij.execution.process package, I'm going to take the chance so that things do clean up properly.

Thanks again for the response!

Comment actions Permalink

And thanks for the update!  Now I can go tweak my process runner to do the same.  (I didn't have any issues, but it's better to get it right.)  :smile:


Please sign in to leave a comment.