JDI ClassNotLoadedException
Answered
I'm investigating an issue where expression evaluation fails when using the latest version of Clojure with my plugin. In the latest version, they upgraded the version of ASM used to generate bytecode and Clojure now generates Java 8-compatible bytecode. Previously it was compatible with Java 6.
When evaluating an expression, I always receive a com.sun.jdi.ClassNotLoadedException: Class java.lang.Object not loaded. The evaluation essentially hangs, since it seems to retry indefinitely but always receives this exception. I really have no idea how to diagnose or fix this, can anyone help?
Please sign in to leave a comment.
Could you provide a sample project to reproduce it? That would help a lot. Thanks!
Sure, I've created one at https://github.com/cursive-ide/cursive-2099. To reproduce:
Let me know if you need any more information.
I was able to reproduce the issue, thanks!
For some reason java.lang.Object is not reported as visible from the DynamicClassLoader used in evaluation, so jdi method args types checks fail. In 1.09 it does not happen. Definitely some changes in 1.10 trigger that.
I can fix the hang by fixing the endless Object class loading, but evaluation would still fail. We need to find the reason why Object is not reported as visible.
Thanks Colin Fleming for sharing this valuable information over here.:-)
Hi Egor, when you say Object is not visible, what do you mean by that? Which method in DynamicClassLoader is not behaving as you would expect? I had a quick look, and I can't see why the recent changes to that class would break anything.
Ok, I've investigated this some more, with some help from the Clojure team. As far as I can tell, DynamicClassLoader didn't change in 1.10 - there was one change to it that was later reverted. However using git bisect, I found the offending commit, it's this one:
https://github.com/clojure/clojure/commit/38705b49fd3dbae11e94c576ef49ff3eb1c47395
There's some details in the commit message, but that commit changed two things: it bumped the bytecode opcode version that Clojure generates from 1.5 to 1.8, and made a change to the generation of the stack map frames, specifically the getCommonSuperClass method which always returns java/lang/Object. Unless there's something about the opcode version that causes JDI to behave differently, I'm suspicious of the getCommonSuperClass change.
Having debugged this further I see what you mean by visibility now. However I'm no closer to understanding why Object isn't visible. Here's a list of the visible classes before and after the change:
So, many of the base JDK classes are visible, but Object is not. I have no idea why, though.
It seems that only the generated bytecode version matters, if I change it back to 49 and leave the rest as is (COMPUTE_FRAMES etc.) it starts working.
Starting with version 50 it stops working... Will try to investigate further.
My be connected with changed bytecode verification in java 6, but not sure.
I think I moved a little further:
Compile.eval changes the classloader for the generated class, so when we evaluate it in the debugger jdi checks arguments in one (new) classloader but then tries to load the Object class into another (context) classloader. Not sure how it worked before :) But will try to find a way to make it work.
The workaround that seems to be working:
add
context.setClassLoader(invoke.declaringType().classLoader());
into ClojureCodeFragmentFactory right before the main invokeMethod (invoke invocation :)
Fantastic, thank you Egor! I haven't had a chance to try this today but I'll try it tomorrow and let you know.
After testing this today, I can confirm that the workaround fixes the issue - thank you again! I would never have worked that out on my own.
Are there likely to be any side effects of this? Is the EvaluationContext created for each evaluation, or might there be weirdness later if the context is later used with a different classloader?
ClassLoader in EvaluationContext is mostly used for loading not yet loaded classes and searching for classes (for example to invoke a static method). I'm not sure about possible side effects but if you call a method of the class that is loaded in a separate class loader, it seems correct to set it as the evaluation context class loader.
Unfortunatelly I was not able to find out the exact change in java 6 class loading/verification that caused this difference in the behavior :(