EAP 992 - now we can't even import openapi.vfs.VirtualFile!

There are a couple of bugs, http://www.intellij.net/forums/thread.jsp?forum=13&thread=45300 and http://www.intellij.net/forums/thread.jsp?forum=13&thread=48597 that describe some serious problems we plugin developers are experiencing, namely, we can't use or include certain classes from openapi.jar or idea.jar. The classes appear corrupted, and this is apparently due to a bug in the obfuscator.

Now, with build 992, we can't even use VirtualFile. Whether I look in openapi.jar or idea.jar for this class (and why is it in both places?), I get the "bad class file, bad signature" error.

This is getting really hard to work around. The open API isn't open if you can't use its classes!

IntelliJ, is there a possibility that

1) a new version of the obfuscator is going to be available, say, later today? :) -OR-

2) I could sign a non-disclosure agreement and get an un-obfuscated copy of idea.jar and openapi.jar? I just want to continue plugin development work.

Thanks!

-Dave

0
18 comments
Avatar
Permanently deleted user

On Tue, 09 Dec 2003 19:47:02 +0000, Dave Kriewall wrote:

Now, with build 992, we can't even use VirtualFile. Whether I look in
openapi.jar or idea.jar for this class (and why is it in both places?), I
get the "bad class file, bad signature" error.


I noticed several classes are being reported as having generics attached
to them ( such as the Key class used in putUserData() - this will probably
trip us up as well.


--
Discouragement is a dissatisfaction with the past, a distaste for the
present, and a distrust of the future - Maree De Jong, CLCA.

0
Avatar
Permanently deleted user
0
Avatar
Permanently deleted user

Thanks, Paul! I'll give it a try.
-Dave

0
Avatar
Permanently deleted user

Paul,

Thank you for solution! Can you, please, give short description about issue?

Why only VirtualFile have broken signature from generics side, and have VALID signature from non-generics code...

The bcel in %IDEA_HOME%/lib is up-to-date, i use last generics - 2.2. That wrong?

Thanks!

0
Avatar
Permanently deleted user

I post a little hotfix with description to your topic.
Thanks!

0
Avatar
Permanently deleted user

In one of the bug report descriptions a Jetbrain engineer said this was a problem with the obfuscator -- that the obfuscator doesn't work that well with generic'd classes.

That theory fits with the behavior I've seen. My understanding is that generic classes have a bit of extra metadata stored in the .class file. Theory goes that the obfuscator is corrupting that metadata...

The normal javac compiler just ignores this metadata attribute, so it doesn't matter if it's corrupt or not.

But, the EA generics javac tries to use the metadata attribute and, since it contains garbage, it complains and gives up.

The patch strips out that metadata attribute (although I do it an awkward way because I'm too lazy to read about how the metadata is really implemented and just strip that, so I just rebuild the classes from scratch without including the metadata in the first place).

The problem certainly isn't just with VirtualFile - there are a bunch of classes that have this problem. I originally ran into one with PsiTreeUtil. I first noticed it back in build 95x and slowly more and more classes became affected - I'm not sure what's going on to make it get worse like this, unless JetBrains has been adding generics to classes that previously didn't have any (or upgraded a version of a compiler or something).

Hope this helps - I'm happy with the explanation that it's a obfuscator bug... I think with a workaround people won't complain too much, they're using a whole bunch of early-access stuff doing what they're doing... :)

0
Avatar
Permanently deleted user

Seems that you are right! :)

But it surprise for me, i think before, that generics compiler just convert generics to JDK 1.4 code... I think that metadata is not used, but i was wrong, as it hapend.

In any case, thank you very much for rewriter, it allow me continue my work :)

0
Avatar
Permanently deleted user

For your convenience I uploaded stub .jar files (openapi.jar and idea.jar) for build 996 at http://www.intellij.org/twiki/bin/view/Main/StubbingIdeaJar.

0
Avatar
Permanently deleted user

Hi Paul,

There's a problem with the stub generator. It is apparently stripping constants out of classes; e.g. in idea.jar,

is no longer recognized.

IntelliJ folks, this is still a showstopper for my plugin development. Even if I manage to compile with Paul's stubbed .jar files and then revert the classpath to the original idea.jar and openapi.jar files, I get null pointer exceptions whenever I attempt to run a JUnit test, as follows. (I actually get 12 identical NPE's with interleaved output, but I've shortened it to just one here.) I'm suspicious that this is related somehow to the compilation procedure.

(Reader.java:61)
	at java.io.InputStreamReader.(InputStreamReader.java:80)
	at com.intellij.util.JDOMUtil.loadDocument(JDOMUtil.java:110)
	at com.intellij.openapi.components.a.a.a(a.java:167)
	at com.intellij.openapi.components.a.a.a(a.java:175)
	at com.intellij.openapi.components.a.a.e(a.java:165)
	at com.intellij.openapi.application.a.b.s(b.java:233)
	at com.intellij.openapi.application.a.b.(b.java:155)
	at com.intellij.openapi.application.b.b.a(b.java)
	at com.intellij.idea.a.(a.java:7)
	at com.intellij.idea.b.(b.java:3)
	at com.intellij.idea.b.a(b.java:1)
	at com.intellij.testFramework.LightIdeaTestCase.a(LightIdeaTestCase.java:54)
	at com.intellij.testFramework.LightIdeaTestCase.setUp(LightIdeaTestCase.java:70)
	at com.wrq.tabifier.IfStmtAlignmentTest.setUp(IfStmtAlignmentTest.java:33)
	at com.intellij.testFramework.LightIdeaTestCase.d(LightIdeaTestCase.java:31)
	at com.intellij.testFramework.LightIdeaTestCase.access$400(LightIdeaTestCase.java:26)
	at com.intellij.testFramework.LightIdeaTestCase$5.run(LightIdeaTestCase.java)
	at com.intellij.util.LaterInvokator$1.run(LaterInvokator.java)
	at com.intellij.util.LaterInvokator$FlushQueue.run(LaterInvokator.java:9)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:178)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:454)
	at java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:145)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:137)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:100)

java.lang.NullPointerException
	at com.intellij.testFramework.LightIdeaTestCase.b(LightIdeaTestCase.java:37)
	at com.intellij.testFramework.LightIdeaTestCase.a(LightIdeaTestCase.java:4)
	at com.intellij.testFramework.LightIdeaTestCase.setUp(LightIdeaTestCase.java:70)
	at com.wrq.tabifier.IfStmtAlignmentTest.setUp(IfStmtAlignmentTest.java:33)
	at com.intellij.testFramework.LightIdeaTestCase.d(LightIdeaTestCase.java:31)
	at com.intellij.testFramework.LightIdeaTestCase.access$400(LightIdeaTestCase.java:26)
	at com.intellij.testFramework.LightIdeaTestCase$5.run(LightIdeaTestCase.java)
	at com.intellij.util.LaterInvokator$1.run(LaterInvokator.java)
	at com.intellij.util.LaterInvokator$FlushQueue.run(LaterInvokator.java:9)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:178)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:454)
	at java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:145)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:137)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:100)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:100)

Process terminated with exit code -1]]>

May I renew my request to sign an NDA or something so that I can get an un-obfuscated version of the .jar files? I would at least like to determine if the "compile with stubs, run with original jars" approach is the cause of the NPE.

Thanks,
-Dave

0
Avatar
Permanently deleted user

Ahh, yes. Sorry about that. I've posted a fixed program on the wiki: http://www.intellij.org/twiki/bin/view/Main/StubbingIdeaJar

Let me know if they let you sign an NDA, that'd make my life a lot easier, too!

:)

0
Avatar
Permanently deleted user

I have no problem since I don't use Generics anymore.

I removed all generics from my plugins right after I filed http://www.intellij.net/forums/thread.jsp?forum=13&thread=45300. This was necessary due to the same problems you have.
Since the gjc generates standard jdk1.3 bytecode, idea.jar and openapi.jar are fine using standard javac. So AS LONG AS you do not enable gjc IN YOUR IDEA projects you will be fine. KlassMaster, IDEA obfuscator, seems to only corrupt generics specific data.

As a treat, as of 1024 you won't have the IDEA editor flag your class that extends/implements a generic type (UserDataHolder, Comparable...) because it would miss your non-generic implementation (almost EVERY SINGLE PSI mock class I have was marked red in the editor !!!)

So this is my advice: Just comment out your "]]>" (so you can remove the comment with a regexp find&replace) and turn your "T" into Object (gjc does that type erasure anyway) and do not use generics anymore. They are just not worth the trouble IMHO.

Jacques

0
Avatar
Permanently deleted user

Yeah, Jacques, I think I'll have to cave in and strip the generics from my code for now. I understand that the production version of 4.0 will not contain any generics in the openapi.jar and idea.jar files, so we can probably go back to them then.

Thanks for the suggestion,
-Dave

0
Avatar
Permanently deleted user

the production version of 4.0 will not contain any generics in the openapi.jar and idea.jar files

What do you mean? Are they going to remove their usage of generics in their own code?
Or are you refering to the fact that we can use generics compiled code with a non-generics compiler even right now?

Jacques

0
Avatar
Permanently deleted user

I was told (by EugeneB) that "in the release version of IDEA 4.0 there will be no class files using generics." Sounds like they are going to remove usage of generics in their own code. But they haven't done it yet (as of build 1032) since I still get corrupted class errors. The sooner the better, of course. :) I'm in the process of stripping out my generics usages. I suppose I'll be able to replace them safely with the 4.0 production version.

-Dave

0
Avatar
Permanently deleted user

Does my new patch not work for you?

0
Avatar
Permanently deleted user

Hi Paul,

I am trying to find out why none of my JUnit tests will run (I get an NPE deep in IDEA code). I suspected that maybe it had to do with compiling with the stubs, or not getting the classpath right for execution, or something. So for the moment I am trying to eliminate generics as a factor. (So far, it doesn't seem to help, but I'll try a few more things before totally giving up on plugin development. :))

Anyway, I haven't tried your new patch yet, sorry. Between flu, holidays, and obfuscated generics, I haven't been very productive lately! (Acronym for that would be FHOG; as in, I've been in a FHOG.)

-Dave

0
Avatar
Permanently deleted user

Ahh, right. Well, give it a go, I built it just for you.

When I read your original problem, I immediately knew what was wrong with the stub generator - because it wasn't putting constants into the .class files it built. So it would turn

Blah.EQ = 5;

into

Blah.EQ = 0;

(Just because I'd never coded that bit up).

The problem is that javac doesn't (and hasn't since the 1.1 days, I think) put a reference to "Blah.EQ" into your classfiles when it compiles them. Instead, it actually puts the constant "5" in there. And, when you compiled against my stubs, it would be putting in "0" instead, which would cause all sorts of problems, I'm sure.

So, I fixed the stub generator to properly put in initialization values for primitive types. I'm sure it will get you up and running.

0
Avatar
Permanently deleted user

Compliled all requered classes into one jar.
http://www.intellij.org/twiki/bin/view/Main/StubbingIdeaJar

0

Please sign in to leave a comment.