JDI ClassNotLoadedException

Answered

I'm investigating an issue where expression evaluation fails when using the latest version of Clojure with my plugin. In the latest version, they upgraded the version of ASM used to generate bytecode and Clojure now generates Java 8-compatible bytecode. Previously it was compatible with Java 6.

When evaluating an expression, I always receive a com.sun.jdi.ClassNotLoadedException: Class java.lang.Object not loaded. The evaluation essentially hangs, since it seems to retry indefinitely but always receives this exception. I really have no idea how to diagnose or fix this, can anyone help?

14 comments
Comment actions Permalink

Could you provide a sample project to reproduce it? That would help a lot. Thanks!

0
Comment actions Permalink

Sure, I've created one at https://github.com/cursive-ide/cursive-2099. To reproduce:

  1. Install Cursive (https://plugins.jetbrains.com/plugin/8090).
  2. Clone the repo and open the project.
  3. Open src/cursive_2099/core.clj
  4. Put a breakpoint on line 5: (+ a b)
  5. Debug the -main method using the green gutter run icon.
  6. When execution stops at the breakpoint, try to evaluate expressions 'a' or 'b' (just the variable names, without the quotes).
  7. The evaluation should hang. Sometimes I can see in the status bar that the evaluation message is flickering, this is because it's retrying repeatedly after receiving the exception.

Let me know if you need any more information.

0
Comment actions Permalink

I was able to reproduce the issue, thanks!

For some reason java.lang.Object is not reported as visible from the DynamicClassLoader used in evaluation, so jdi method args types checks fail. In 1.09 it does not happen. Definitely some changes in 1.10 trigger that.

0
Comment actions Permalink

I can fix the hang by fixing the endless Object class loading, but evaluation would still fail. We need to find the reason why Object is not reported as visible.

0
Comment actions Permalink

Thanks Colin Fleming for sharing this valuable information over here.:-)

0
Comment actions Permalink

Hi Egor, when you say Object is not visible, what do you mean by that? Which method in DynamicClassLoader is not behaving as you would expect? I had a quick look, and I can't see why the recent changes to that class would break anything.

0
Comment actions Permalink

Ok, I've investigated this some more, with some help from the Clojure team. As far as I can tell, DynamicClassLoader didn't change in 1.10 - there was one change to it that was later reverted. However using git bisect, I found the offending commit, it's this one:

https://github.com/clojure/clojure/commit/38705b49fd3dbae11e94c576ef49ff3eb1c47395

There's some details in the commit message, but that commit changed two things: it bumped the bytecode opcode version that Clojure generates from 1.5 to 1.8, and made a change to the generation of the stack map frames, specifically the getCommonSuperClass method which always returns java/lang/Object. Unless there's something about the opcode version that causes JDI to behave differently, I'm suspicious of the getCommonSuperClass change.

0
Comment actions Permalink

Having debugged this further I see what you mean by visibility now. However I'm no closer to understanding why Object isn't visible. Here's a list of the visible classes before and after the change:

0 = {com.sun.tools.jdi.ClassTypeImpl@39806} "class user$eval147$fn__148 (loaded by instance of clojure.lang.DynamicClassLoader(id=2791))"
1 = {com.sun.tools.jdi.ClassTypeImpl@39826} "class java.lang.Object (no class loader)"
2 = {com.sun.tools.jdi.ArrayTypeImpl@39827} "array class java.lang.Object[] (no class loader)"
3 = {com.sun.tools.jdi.ArrayTypeImpl@39828} "array class java.lang.Object[][] (no class loader)"
4 = {com.sun.tools.jdi.ClassTypeImpl@39829} "class clojure.lang.AFunction (loaded by instance of sun.misc.Launcher$AppClassLoader(id=2675))"
5 = {com.sun.tools.jdi.ClassTypeImpl@39830} "class user$eval147 (loaded by instance of clojure.lang.DynamicClassLoader(id=2791))"
6 = {com.sun.tools.jdi.ArrayTypeImpl@39831} "array class boolean[] (no class loader)"
7 = {com.sun.tools.jdi.ArrayTypeImpl@39832} "array class byte[] (no class loader)"
8 = {com.sun.tools.jdi.ArrayTypeImpl@39833} "array class byte[][] (no class loader)"
9 = {com.sun.tools.jdi.ArrayTypeImpl@39834} "array class char[] (no class loader)"
10 = {com.sun.tools.jdi.ArrayTypeImpl@39835} "array class int[] (no class loader)"
11 = {com.sun.tools.jdi.ArrayTypeImpl@39836} "array class short[] (no class loader)"
12 = {com.sun.tools.jdi.ArrayTypeImpl@39837} "array class long[] (no class loader)"
13 = {com.sun.tools.jdi.ArrayTypeImpl@39838} "array class long[][] (no class loader)"
14 = {com.sun.tools.jdi.ArrayTypeImpl@39839} "array class float[] (no class loader)"
15 = {com.sun.tools.jdi.ArrayTypeImpl@39840} "array class double[] (no class loader)"
0 = {com.sun.tools.jdi.ClassTypeImpl@40142} "class user$eval147 (loaded by instance of clojure.lang.DynamicClassLoader(id=2790))"
1 = {com.sun.tools.jdi.ClassTypeImpl@40143} "class clojure.lang.AFunction (loaded by instance of sun.misc.Launcher$AppClassLoader(id=2677))"
2 = {com.sun.tools.jdi.ClassTypeImpl@40133} "class user$eval147$fn__148 (loaded by instance of clojure.lang.DynamicClassLoader(id=2790))"
3 = {com.sun.tools.jdi.ArrayTypeImpl@40144} "array class boolean[] (no class loader)"
4 = {com.sun.tools.jdi.ArrayTypeImpl@40145} "array class byte[] (no class loader)"
5 = {com.sun.tools.jdi.ArrayTypeImpl@40146} "array class byte[][] (no class loader)"
6 = {com.sun.tools.jdi.ArrayTypeImpl@40147} "array class char[] (no class loader)"
7 = {com.sun.tools.jdi.ArrayTypeImpl@40148} "array class int[] (no class loader)"
8 = {com.sun.tools.jdi.ArrayTypeImpl@40149} "array class short[] (no class loader)"
9 = {com.sun.tools.jdi.ArrayTypeImpl@40150} "array class long[] (no class loader)"
10 = {com.sun.tools.jdi.ArrayTypeImpl@40151} "array class long[][] (no class loader)"
11 = {com.sun.tools.jdi.ArrayTypeImpl@40152} "array class float[] (no class loader)"
12 = {com.sun.tools.jdi.ArrayTypeImpl@40153} "array class double[] (no class loader)"

So, many of the base JDK classes are visible, but Object is not. I have no idea why, though.

0
Comment actions Permalink

It seems that only the generated bytecode version matters, if I change it back to 49 and leave the rest as is (COMPUTE_FRAMES etc.) it starts working.

Starting with version 50 it stops working... Will try to investigate further.

My be connected with changed bytecode verification in java 6, but not sure.

0
Comment actions Permalink

I think I moved a little further:

Compile.eval changes the classloader for the generated class, so when we evaluate it in the debugger jdi checks arguments in one (new) classloader but then tries to load the Object class into another (context) classloader. Not sure how it worked before :) But will try to find a way to make it work.

0
Comment actions Permalink

The workaround that seems to be working:

add

context.setClassLoader(invoke.declaringType().classLoader());

into ClojureCodeFragmentFactory right before the main invokeMethod (invoke invocation :)

0
Comment actions Permalink

Fantastic, thank you Egor! I haven't had a chance to try this today but I'll try it tomorrow and let you know.

0
Comment actions Permalink

After testing this today, I can confirm that the workaround fixes the issue - thank you again! I would never have worked that out on my own.

Are there likely to be any side effects of this? Is the EvaluationContext created for each evaluation, or might there be weirdness later if the context is later used with a different classloader?

0
Comment actions Permalink

ClassLoader in EvaluationContext is mostly used for loading not yet loaded classes and searching for classes (for example to invoke a static method). I'm not sure about possible side effects but if you call a method of the class that is loaded in a separate class loader, it seems correct to set it as the evaluation context class loader.

Unfortunatelly I was not able to find out the exact change in java 6 class loading/verification that caused this difference in the behavior :(

0

Please sign in to leave a comment.