Re: Release date of IntelliJ IDEA 5

Marcus,
I really understand your point here, but I think that what you call a new
"wow" factor is more a matter of marketing work.
Previewing the 5.0 release, can you name an IDE that provides the level of
support we do for JSP/JSPX?
Does any IDE provide completion/resolve, even partial for (dynamically
typed) javascript?
What level of support is provided by existing IDEs to help edit html/css?
Speaking about refactorings, NetBeans provides only the basics for java
(probably more will be added in the near future)
We are going to provide those basics inter-language (i.e. rename, safe
delete and move). For example moving an image file with automatic references
correction from html.

Having said that, my personal opinion is that we (and not only=) are on the
wrong track when trying to add support to some really old technologies,
that in spite of being popular now, were not designed bearing in mind the
possible support from the tools.
Indeed it is impossible to make IDEA as smart when editing say Javascript,
as it is for Java, because statically typing is essentially the spices
of what we are doing. So what I want to say is that the progress of classic
development tools depends greatly on the progress in programming languages.
There is of course the inverse idea of making the language specific for
particular domain, but that is just another story (and I really don't know
any news on MPS:)

Eugene.

"Marcus Brito" <mbrito@gmail.com> wrote in message
news:2376322.1115041408888.JavaMail.itn@is.intellij.net...

<rant-mode>
The IDE arena is surprisingly active since the release of IDEA 4.5

(Pallada).

When Pallada was released, IDEA was on a very comfortable position as the

best

IDE out there, hands down. There were some contenders, of course, but none

would

get close to the greater user experience provided by IDEA.

>

Has this changed? In my opinion, yeah, significantly. The latest releases

of

Eclipse, including the Web Tools Platform brings Eclipse to a level of
functionality only achieved by commercial plugins before. This means there

will

be one less reason to buy a commercial product, unfortunately. The new

releases

for NetBeans are also much more polished and incorporates a great deal of
functionality. OmniCore released their new cross platform IDE, which still
has a long way to go but already shows a promising product. Even Microsoft

is

raising the bar, previewing the new refactoring functionality in Visual

Studio.
>

At the same time, what happened to IDEA, really? If some cosmic force

fixed all

critical bugs on Irida and it was ready for release tomorrow, what do we

have?

Primitive support for javascript (but still better than what most other

tools

have to offer), a couple of new XML features (schema validation) and a few

new

inspections. Maybe a new refactoring or two.

>

What do I want to say with all this nonsense? Just think about the

context: we'

re talking about release dates. So, closing this wrecked train of thought,

I

really hope Irida is not released any time soon. I'd love to see

Irida

feature chart reworked to better fit the new IDE arena. We need a release

with

a "wow" factor again... just don't ask me what this could be.
</rant-mode>



58 comments
Comment actions Permalink

Yeah it's definitely not for everyone. I would like it if Java's types were
optional when type could be inferred.

A design-by-contract system and nullity specifications are the
only big gaps that I feel.


By the way, my nullness plugin is coming along very well. I'm preparing the
code, website, and a paper about it (for my advisor to grade). You can see
a screenshot from the website at http://conio.net/tmp/demo2.png.

It supports @NonNull method return value, local variable, and parameter.
It shows in-editor warnings whenever a possibly null is being placed into
a @NonNull, as well as compilation message warnings. It also inserts code
during compilation to enforce @NonNull at runtime using null checks and runtime
exceptions.

The main limitation is that it requires IDEA for the compile warnings and
runtime check insertion, no Ant builds. Maybe I can convince JB to write
an Ant task which loads idea.jar or something. The second main limitation
is that no one else uses @NonNull so it will probably need a system of loading
libraries of nullness specifications for the JDK and popular libraries.

I want to release it by the end of this week. It has a lot of bugs but I
think I can do it.


0
Comment actions Permalink

Keith,
I'm looking forward to seeing this - this was the first thing I thought
of when I saw annotations, since my Javadocs are full of "may be
null"/"never null" comments and I'd longed for a way to enforce them for
a long time. Glad someone (with such a good pedigree - Dad must be proud
;) has got around to it,
R

0
Comment actions Permalink

+
By the way, my nullness plugin is coming along very well. +

Excellent. I've been looking forward to it (and not just because I want example code for an instrumenting pre-compiler).

+It supports @NonNull method return value, local variable, and parameter. +

Sweet. And since the highest form of applause is a feature request, I'll put in the first one: any guesses on LOE to extend it to work with fields? In that case @NonNull would mean "Not null once the object (or class, for statics) is initialized, and forever thereafter". Big pain or easy extension?

+The second main limitation
is that no one else uses @NonNull so it will probably need a system of loading libraries of nullness specifications for the JDK and popular libraries.+

I wonder if it would be possible to infer annotations from the library bytecode in any sort of reasonable way. For libraries, what you really need is the method return annotations. The parameter annotations are just gravy. Should be possible to (conservatively) infer cases where method returns can never be null by analyzing all of the return clauses of the method. Plus any possible sub- super- or cousin- methods. Ouch.

--Dave Griffith

0
Comment actions Permalink

+It supports @NonNull method return value, local variable, and
parameter. +

Sweet. And since the highest form of applause is a feature request,
I'll put in the first one: any guesses on LOE to extend it to work
with fields? In that case @NonNull would mean "Not null once the
object (or class, for statics) is initialized, and forever
thereafter". Big pain or easy extension?


It's not trivial, at least for the static checking part of the plugin. I
plan on implementing it, but it's not trivial.

I read a paper by K. Rustan Leino from Microsoft about a similar non-null
type system experimentally added to C#. For fields, he implemented a concept
of "raw null type" for fields. It's been a while since I read it, but I think
it basically works like this: a NonNull field is considered "raw" until the
class's constructor has completed. During this time, only methods marked
as Raw can access NonNull fields, and they are not guaranteed of any fields'
nullness. All constructors are automatically also Raw. Raw methods / constructors
can only call other Raw methods in the class.

I might implement something like this. In the short term, I might also implement
@NonNull for fields without doing any static checking, or doing some kind
of messy conservative checking.

+The second main limitation
is that no one else uses @NonNull so it will probably need a system of
loading libraries of nullness specifications for the JDK and popular
libraries.+
I wonder if it would be possible to infer annotations from the library
bytecode in any sort of reasonable way. For libraries, what you
really need is the method return annotations. The parameter
annotations are just gravy. Should be possible to (conservatively)
infer cases where method returns can never be null by analyzing all of
the return clauses of the method. Plus any possible sub- super- or
cousin- methods. Ouch.


Yeah, I plan on doing something like this. I think it would work for most
methods in most libraries. I don't think checking for parameters should be
too bad either, because I think it would be pretty easy to determine whether,
in all cases, passing null for each parameter would throw an exception.

I think the Extended Static Checking for Java people have already done something
like this, for use with JML assertions. I'll probably look into that too.



0
Comment actions Permalink

Keith,
I'm looking forward to seeing this - this was the first thing I
thought
of when I saw annotations, since my Javadocs are full of "may be
null"/"never null" comments and I'd longed for a way to enforce them
for
a long time. Glad someone (with such a good pedigree - Dad must be
proud
;) has got around to it,
R


This was one of the first things I wanted to use annotations for too. And
yet few people seem interested in this functionality when I explain it to
them. Maybe it's boring, but I think it's important.


0
Comment actions Permalink


>Maybe it's boring, but I think it's important.

It is important. Very. A huge percentage of coding errors in Java have an NPE as their initial manifestation. In most cases the NPE isn't the root cause, just the symptom. A suitably powerful, usable, and proactive NPE-prevention tool will move the fix time for all of these errors from test-phase (or later) to code-phase. Since there are so many errors that nullity-checking will find, that's an enormous potential value to be reaped. Frankly, a high quality solution to this problem could be worth all the rest of the IDEA+IG inspections combined.

Someone really should be working on a JCP submission for this stuff. Not the tooling, but the annotation(s) and their intended semantics. That way multiple vendors could provide their own interoperable tooling, and library providers would have a better reason to annotate their own products. JetBrains would actually be a great lead for it, since they are probably the largest and most popular vendor of static analysis tooling. Max? Sergey? Any interest?

--Dave Griffith

0
Comment actions Permalink

Max? Sergey? Any interest?

Sure...

One improovment idea is to have @nullable annotation in addition to @NotNull.
An assumption made that everything not annotated with @NotNull is actually
nullable is quite strong and would make actual application of the idea into
existing code quite a heavy task. It's all or nothing. Having both @Nullable
and @NotNull will make transition much smoother so tool isn't allowed to
make assumptions about declarations which aren't annotated at all.

P.S.: It moved from 'Release date of IntelliJ IDEA 5' quite far :)
-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"


0
Comment actions Permalink

>> Max? Sergey? Any interest?
>>

Sure...

One improovment idea is to have @nullable annotation in addition to
@NotNull. An assumption made that everything not annotated with
@NotNull is actually nullable is quite strong and would make actual
application of the idea into existing code quite a heavy task. It's
all or nothing. Having both @Nullable and @NotNull will make
transition much smoother so tool isn't allowed to make assumptions
about declarations which aren't annotated at all.


I think that's a good idea. However, I don't think I see what you mean by
"assumption that everything not annotated with @NotNull is actually nullable
is quite strong and would make application to existing code a heavy task."
It seems the opposite to me - this assumption allows you to transition, because
it keeps the original semantics of your code for unannotated declarations,
and only adds new semantics for annotated declarations. Do I misinterpret
what you mean?


0
Comment actions Permalink

Someone really should be working on a JCP submission for this stuff.
Not the tooling, but the annotation(s) and their intended semantics.
That way multiple vendors could provide their own interoperable
tooling, and library providers would have a better reason to annotate
their own products. JetBrains would actually be a great lead for it,
since they are probably the largest and most popular vendor of static
analysis tooling. Max? Sergey? Any interest?


I agree that a JCP submission should be made. I will help all I can if JB
decides to propose it. I don't think I'm in a position to propose it, considering
that I am so new to static analysis.

I feel funny supporting this as a JSR because I am against the concept of
this being an annotation rather than part of the language, conceptually.
One big reason is that if you compile class which uses these declarations
with javac, the runtime checks will not be inserted, and your assertions
will not even be checked at runtime.

But, I think an annotation would be a good start. Maybe if people start using
it, they will add nonnull and nullable keywords to Java 10 or so :)


0
Comment actions Permalink

I presume it would depend on your coding style. If hardly any of your
methods return null, then to get the full benefit of the annotation, you
would have to label them all as @NotNull, otherwise you will get hit by
a lot of false positives. In this case it would obviously make sense to
assume that methods cannot return null, unless told otherwise. Not sure
how you would know just be looking at the code which convention was
being used, though.

R

0
Comment actions Permalink

Ok, let's see the sample:

public void foo(PsiReference ref) {
PsiElement elt = ref.resolve();
PsiFile file = elt.getContainingFile(); // Bug here. Any reference.resolve()
is known to return null, it's normal.
}

PsiReference.resolve() method is perfect candidate to have @nullable annotation.


-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

>>> Max? Sergey? Any interest?
>>>
>> Sure...
>>
>> One improovment idea is to have @nullable annotation in addition to
>> @NotNull. An assumption made that everything not annotated with
>> @NotNull is actually nullable is quite strong and would make actual
>> application of the idea into existing code quite a heavy task. It's
>> all or nothing. Having both @Nullable and @NotNull will make
>> transition much smoother so tool isn't allowed to make assumptions
>> about declarations which aren't annotated at all.
>>

I think that's a good idea. However, I don't think I see what you mean
by "assumption that everything not annotated with @NotNull is actually
nullable is quite strong and would make application to existing code a
heavy task." It seems the opposite to me - this assumption allows you
to transition, because it keeps the original semantics of your code
for unannotated declarations, and only adds new semantics for
annotated declarations. Do I misinterpret what you mean?



0
Comment actions Permalink


Probably be best to have both, and let tools decide the policy for non-annotated methods/vars on a per-tool basis. For instance, I could imagine a tool that generated test cases from code, including generating mocks for library classes(a couple already exist). It might generated different test cases for code that calls library methods, based on whether those calls are nullable or have nullable parameters. Such a tool might very well have a "how paranoid are you" flag, indicating whether mocks should be generated for cases where the library methods return null, or where they NPE when passed null, when the method/parameter is unannotated.

My head frankly hurts just thinking about such a tool.

--Dave Griffith

0
Comment actions Permalink

Ok, let's see the sample:

public void foo(PsiReference ref) {
PsiElement elt = ref.resolve();
PsiFile file = elt.getContainingFile(); // Bug here. Any
reference.resolve()
is known to return null, it's normal.
}

PsiReference.resolve() method is perfect candidate to have @nullable
annotation.


Yeah okay. I'll make sure I understand with examples:

class PsiReference {
@Nullable PsiElement resolve();
}

This has the same semantics as:

class PsiReference {
PsiElement resolve();
}

I think you are saying that @Nullable should be present only as documentation
which tells your API users that you, PsiReference developer, have thought
about whether resolve() can return null or not, and you have decided that
it can. Is that right? So it's for documentation, not for semantics?


0
Comment actions Permalink

I would also offer to help in any way I can, with the caveat that I'm probably contractually prohibitted from actually being on a JCP Expert Group (I'll check with legal here, if anyone cares). Any behind-the-scenes work, spec, reference implementation, reviews, or whatever, I'm available.

+
But, I think an annotation would be a good start. Maybe if people start using
it, they will add nonnull and nullable keywords to Java 10 or so :)+

Keywords are too long. I want ! and ? as type modifiers. Otherwise, I totally agree. I just have the feeling an standard annotation JSR is a lot more likely to pass in the forseeable future than a language change JSR.

--Dave Griffith

0
Comment actions Permalink

Not only documentation puroposes of course though those are important as
well.

If resolve() had @nullable then we could report a warning (or error) in our
dfa analysis for any code that calls methods against result of the resolve()
method without checking for null first. Again,

PsiElement elt = ref.resolve();
PsiFile file = elt.getContainingFile(); // Bug here. Reported by inspection

, but

PsiElement elt = ref.resolve();
if (elt != null) {
PsiFile file = elt.getContainingFile(); // OK, no warnings here.
}


-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

>> Ok, let's see the sample:
>>
>> public void foo(PsiReference ref) {
>> PsiElement elt = ref.resolve();
>> PsiFile file = elt.getContainingFile(); // Bug here. Any
>> reference.resolve()
>> is known to return null, it's normal.
>> }
>> PsiReference.resolve() method is perfect candidate to have @nullable
>> annotation.
>>

Yeah okay. I'll make sure I understand with examples:

class PsiReference {
@Nullable PsiElement resolve();
}
This has the same semantics as:

class PsiReference {
PsiElement resolve();
}
I think you are saying that @Nullable should be present only as
documentation which tells your API users that you, PsiReference
developer, have thought about whether resolve() can return null or
not, and you have decided that it can. Is that right? So it's for
documentation, not for semantics?



0
Comment actions Permalink

For an example of the value of this, even the extremely liberal analysis Max is describing would have cut down my development time on IG by about 10%, and probably lowered the number of bugs reported from the field by over 70%. Almost all of my issues have been due to not being paranoid enough about PSI return values, or passing values to the PSI that were null but shouldn't have been. Admittedly the PSI is just about the worst case scenario for creating NPEs...

--Dave Griffith

0
Comment actions Permalink

Should annotations be uppercased like classes or lowercased ones is OK?

The reason I asking is I feel
@nullable PsiElement resolve();

looks better than
@Nullable PsiElement resolve()

IHMO.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"


0
Comment actions Permalink

Sun standard seems to be uppercased, but I don't know if that's published anywhere. All of the annotations I've seen in JSR are uppercased.

--Dave Griffith

0
Comment actions Permalink

If resolve() had @nullable then we could report a warning (or error)
in our dfa analysis for any code that calls methods against result of
the resolve() method without checking for null first. Again,


I understand now. My plugin treats all un-annotated declarations as possibly
null, and shows a warning for each possibly null use. I understand that you
would want built in IDEA support to only analyze annotated declarations,
like it does now for values which have been checked for null. I, however,
want all possible null misuses to be reported, even from un-annotated declarations.
It may be annoying but I'd prefer to know.

So I understand now, a @nullable would allow for even more inspections to
take place, without annoying the user.


0
Comment actions Permalink

I'm starting to lean towards @nullable over @Nullable. They are modifiers,
after all. Also, it's easier to type @nullable and @nonnull than @Nullable
and @NonNull.

I was also thinking of allowing @NN as a short-hand for my plugin, which
I think would be just as easy to type, but not as easy to read.

Should annotations be uppercased like classes or lowercased ones is
OK?

The reason I asking is I feel
@nullable PsiElement resolve();
looks better than
@Nullable PsiElement resolve()
IHMO.

-------------------
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"




0
Comment actions Permalink

It could be implemented as a class annotation:

@DefaultNullness(NOT_NULL)
public class SomeClass { .. }

I presume it would depend on your coding style. If hardly any of your
methods return null, then to get the full benefit of the annotation,
you would have to label them all as @NotNull, otherwise you will get
hit by a lot of false positives. In this case it would obviously make
sense to assume that methods cannot return null, unless told
otherwise. Not sure how you would know just be looking at the code
which convention was being used, though.

R




0
Comment actions Permalink

By the way Dave, I used the Soot framework for Java code analysis for my
project. Until IDEA opens its code analysis code into OpenAPI I highly suggest
it, for any dataflow analyses you want to implement. It makes dataflow easy
and has many built-in analyses and analysis helper tool classes. Unfortunately
the tradeoff is performance. To analyse a method, you have to:

1. Make copy of PSI tree
2. Preprocess copy:
a. Delete body of all but the inspected method
b. Convert Java 5 constructs to Java 4 (until Soot supports Java 5, supposedly
later this year)
c. Remove any lines from the code which contain errors (my plugin currently
doesn't do this, I don't know how hard it would be)
3. Pass copy to Soot
a. Soot compiles the source code to Soot IR with line/col numbers for each
value and stmt
4. Run analyses on Soot IR
5. Convert line/column numbers from Soot IR for desired elements, back to
PsiElement in copy, back to PsiElement in original

I'm thinking of writing a plugin "IDEA Soot helper" which automates this
process but it still makes things very slow. An analysis of a small class
can take half a second, although usually it's less.

It could be optimized however. First, any Java 5 constructs could have cached
Java 4 versions as user data on the original, so repeated analys doesn't
continually convert the same data to the same result. Second, a system could
be developed for compiling PSI tree directly to Soot IR. It would probably
require modification of Soot because currently it depends on the Polyglot
Java source parser.

Anyway if you're interested I will be glad to help you. I think some cool
inspections could be made using dataflow analysis.


0
Comment actions Permalink

I'm starting to lean towards @nullable over
@Nullable. They are modifiers,
after all. Also, it's easier to type @nullable and
@nonnull than @Nullable
and @NonNull.


While we're on the convenient syntax subject, am I the only one that finds @notnull a lot better than @nonnull? Maybe it's just my SQL background...

0
Comment actions Permalink

Karsten Wagner wrote:

Something I have to add to my previous post:

In Javascript every-Browser has it's own runtime-model. So even if you're able to infer every type of your program you still don't know exactly what attributes and methods i.e. 'document' has. And since most js-programs 'bootstrap' from these system defined objects you're in a mess.

What could be possible is to assume, that there are certain thing true (like certain attributes of 'document' for example) and reason from this on. But you'll never have the same possibilities as in Java or even in Smalltalk.

OTOH: Limited support for well known objects like 'document', 'window' etc should be possible (assuming, that no one messed with them somewhere) and would be nice. Also simple type-propagation like

var el = document.getElementById("element")
.. // nothing changes el here
el.| // here IDEA nows that el is a 'Element'

should be possible - even if it could fail and give you the wrong options somehow.

So don't expect the impossible. Anyhow, there're some limited opportunities also.

Which is why I have an open proposal for "javascript includes"
(http://www.jetbrains.net/jira/browse/IDEADEV-117)
Feel free to vote for it :)

0
Comment actions Permalink

not null is two words, not modifies the verb "is" in the phrase "x is not
null". non-null is an adjective. Would you say "he is a nonviolent guy" or
"he is a not violent guy"?

>> I'm starting to lean towards @nullable over
>> @Nullable. They are modifiers,
>> after all. Also, it's easier to type @nullable and
>> @nonnull than @Nullable
>> and @NonNull.

While we're on the convenient syntax subject, am I the only one that
finds @notnull a lot better than @nonnull? Maybe it's just my SQL
background...




0
Comment actions Permalink

KL> not null is two words, not modifies the verb "is" in the phrase "x
KL> is not null". non-null is an adjective. Would you say "he is a
KL> nonviolent guy" or "he is a not violent guy"?

he is not violent and you mispelled: he's not gay at all.


But my english is look and feel only :)


0
Comment actions Permalink

not null is two words, not modifies the verb "is" in the phrase "x is
not null". non-null is an adjective. Would you say "he is a nonviolent
guy" or "he is a not violent guy"?


I didn't mention the significance of "non-null" being an adjective. "public"
"static" etc. are adjectives, so "nonnull" matches Java's style for declaration
modifiers.


0
Comment actions Permalink

I didn't mention the significance of "non-null" being
an adjective. "public"
"static" etc. are adjectives, so "nonnull" matches
Java's style for declaration
modifiers.


I'm just being dead pragmatic: "notnull" sounds better to SQL-battered ears :)

0

Please sign in to leave a comment.