JetBrains: What we're busy with right now.

Hello All,

A long awaited status report from IDEA team follows:

1. Language API. This stuff begin implemented finally will allow a 3rd party
plugin writers to embed particulary any language support to IDEA and benefit
from it's features. Sytax highlighting, custom PSI, select word, folding,
reference completion, Ctrl+B, error highlighting, rename refactoring, find
usages, brace matching highlights and navigation and code formatter hooks
are implemented so far. We take that really serious so any feedback is urgely
awaited.

2. JavaScript plugin implements an API mentioned. While quite rough on edges
seem to benefit from all that stuff already. Plugin and it's source code
will be available with the next EAP build.

3. Formatter API and implementations (for HTML, XML, JSPX, JSP, JavaScript,
CSS and their various embeddments) are being developed quite actively.

4. Analyze backward dependencies and package cycle dependencies implemented.

5. Some refactorings and code cleanups in debugger and usage view.

6. Improoving stability and performance of JSPX editing



-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"


40 comments
Comment actions Permalink

Thanks Max for this sneak peak.

In addition to the stuff below, which is great, Will Irida eventually be
packaged with JRE 5.0?

Amnon

max@jetbrains.com wrote:

Hello All,
A long awaited status report from IDEA team follows:

1. Language API. This stuff begin implemented finally will allow a 3rd
party plugin writers to embed particulary any language support to IDEA
and benefit from it's features. Sytax highlighting, custom PSI, select
word, folding, reference completion, Ctrl+B, error highlighting, rename
refactoring, find usages, brace matching highlights and navigation and
code formatter hooks are implemented so far. We take that really serious
so any feedback is urgely awaited.

Great! And you read my mind with the next item...


2. JavaScript plugin implements an API mentioned. While quite rough on
edges seem to benefit from all that stuff already. Plugin and it's
source code will be available with the next EAP build.

Cool. Since I am now working heavily in JavaScript I'll benefit from
that (or get exceptions :)). BTW, are cross-file references analyzed as
well? Is that configurable somehow (at the lack of JavaScript real import)?


3. Formatter API and implementations (for HTML, XML, JSPX, JSP,
JavaScript, CSS and their various embeddments) are being developed quite
actively.

That's great news. Looks like an interesting ride for us EAPers ahead.


4. Analyze backward dependencies and package cycle dependencies
implemented.

5. Some refactorings and code cleanups in debugger and usage view.

Do you see refactorings starting to appear for JavaScript?


6. Improoving stability and performance of JSPX editing



-------------------
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

0
Comment actions Permalink

1. Language API. This stuff begin implemented finally will allow a 3rd

party

Wow, cool.
Of course you're prepared for the obvious question: when? :)

Vince.


0
Comment actions Permalink

2. JavaScript plugin implements an API mentioned. While quite rough on
edges seem to benefit from all that stuff already. Plugin and it's
source code will be available with the next EAP build.

Can't wait to see this :).

You are saying that the JavaScript plug-in "seem to benefit from all that
stuff", and that the "Language API" supports "reference completion, Ctrl+B,
error highlight, find usages".
Does that mean that you managed to implement the above for JavaScript too?
(especially the "reference" stuff for javascript ? ).

Thanks in advance,

Ahmed.

0
Comment actions Permalink

References stuff is quite weak right now and is only able to resolve to local
variables, function parameters and functions themselves. Accurate reference
resolve after dot is not possible at all due to dynamic javascript nature.
HTML context implicit variables as well as standard objects and properties
are in plans.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

>> 2. JavaScript plugin implements an API mentioned. While quite rough
>> on edges seem to benefit from all that stuff already. Plugin and it's
>> source code will be available with the next EAP build.
>>

Can't wait to see this :).

You are saying that the JavaScript plug-in "seem to benefit from all
that
stuff", and that the "Language API" supports "reference completion,
Ctrl+B,
error highlight, find usages".
Does that mean that you managed to implement the above for JavaScript
too?
(especially the "reference" stuff for javascript ? ).
Thanks in advance,

Ahmed.




0
Comment actions Permalink

Thx for the update. One word: Outstanding :D

6. Improoving stability and performance of JSPX editing

What about "normal" JSPs? Editing those is, umm, a little unpleasant right now.
Is this in the works or scheduled as well?

Sascha

0
Comment actions Permalink

+1

Norris Shelton
Sun Certified Java Programmer




Sascha Weinreuter wrote:

Thx for the update. One word: Outstanding :D

>
>> 6. Improoving stability and performance of JSPX editing
>

What about "normal" JSPs? Editing those is, umm, a little unpleasant
right now.
Is this in the works or scheduled as well?

>

Sascha

0
Comment actions Permalink

Ahmed Mohombe wrote:

Can't wait to see this :).


Yup, I think Max's message caused a severe case of anticEAPation :)

0
Comment actions Permalink

What about "normal" JSPs? Editing those is, umm, a little unpleasant
right now. Is this in the works or scheduled as well?

Of course. We just want to be 100% sure new language intersecting (XML/Java)
implementation scheme really works before we start re-implementing existing
JSP stuff. Once we finally get JSPX working fine it wouldn't take too much
time to transfer all the things to plain JSP.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"




0
Comment actions Permalink

Thank you very much for the information. Now it is understandable for us,
why new builds take their time.

Tom

0
Comment actions Permalink

Hello Maxim,

Of course. We just want to be 100% sure new language intersecting
(XML/Java) implementation scheme really works before we start
re-implementing existing JSP stuff.


Any chance we'll see the schema metadata annotations anytime soon?



0
Comment actions Permalink

1. Language API. This stuff begin implemented finally will allow a 3rd
party plugin writers to embed particulary any language support to IDEA
and benefit from it's features. Sytax highlighting, custom PSI, select
word, folding, reference completion, Ctrl+B, error highlighting,
rename refactoring, find usages, brace matching highlights and
navigation and code formatter hooks are implemented so far. We take
that really serious so any feedback is urgely awaited.

Is it, or will it be possible to "map" more than one "language" to the same
file extension? (I don't know, to make distinction based on DTD or Schema
(for xml), or just a "file name pattern", - but not only the extension) ?

Most of the problems come with XML files. Almost all the frameworks that
use XML for configuration, let the file exentsion to default *.xml. However
these files are "lanuguages" on their own with "references" to other files(mostly
java :) ). Interpreting these files as "other type of files than xml", thru'
the Language API could ease greatly the implementation of plug-ins to support
those frameworks. Considering the small amounth of plug-ins for IntelliJ,
such a feature could boost the development of plugins for IntelliJ (for the
large number of framworks out there :) ).

Thanks in advance,

Ahmed.

0
Comment actions Permalink

I wouldn't say any kind of language embedment is feasible right away. We
do plan to provide hooks to embed a language into XML tag content and XML
attribute value as well as javadoc content in java so most cases would be
covered.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"


>> 1. Language API. This stuff begin implemented finally will allow a
>> 3rd party plugin writers to embed particulary any language support to
>> IDEA and benefit from it's features. Sytax highlighting, custom PSI,
>> select word, folding, reference completion, Ctrl+B, error
>> highlighting, rename refactoring, find usages, brace matching
>> highlights and navigation and code formatter hooks are implemented so
>> far. We take that really serious so any feedback is urgely awaited.
>>

Is it, or will it be possible to "map" more than one "language" to the
same file extension? (I don't know, to make distinction based on DTD
or Schema (for xml), or just a "file name pattern", - but not only the
extension) ?

Most of the problems come with XML files. Almost all the frameworks
that use XML for configuration, let the file exentsion to default
*.xml. However these files are "lanuguages" on their own with
"references" to other files(mostly java :) ). Interpreting these files
as "other type of files than xml", thru' the Language API could ease
greatly the implementation of plug-ins to support those frameworks.
Considering the small amounth of plug-ins for IntelliJ, such a feature
could boost the development of plugins for IntelliJ (for the large
number of framworks out there :) ).

Thanks in advance,

Ahmed.




0
Comment actions Permalink

I think a rule should be that in order for us to get these nice updates,
that if we ask questions on the status, that we not get frustrated with
not getting answers back, otherwise Max will spend his whole time
answering questions and not making things work :)

R

0
Comment actions Permalink

With http://www.jetbrains.net/jira/browse/IDEADEV we always will know what happening right now. Well done.
Such reports is very useful too, if anyone interested in specific feature it's better to discuss it in RFE thread instead of work report thread :)

TIA,
Dmitry

0
Comment actions Permalink

Updating http://www.intellij.net/eap/products/idea/irida.jsp from time to
time would be nice too :)

With http://www.jetbrains.net/jira/browse/IDEADEV we always will know
what happening right now. Well done. Such reports is very useful too,
if anyone interested in specific feature it's better to discuss it in
RFE thread instead of work report thread :)




0
Comment actions Permalink

... otherwise Max will spend his whole time
answering questions and not making things work :)


cracks whip

0
Comment actions Permalink

In article <13844017.1107810251819.JavaMail.itn@is.intellij.net>,
Colin Fleming <no_mail@jetbrains.com> wrote:

... otherwise Max will spend his whole time
answering questions and not making things work :)


cracks whip


haha oh come on :)

0
Comment actions Permalink

It's not the same, dev-log is more dynamic.

0
Comment actions Permalink

Haha, that was me, asking for JRE 1.5 support :)

0
Comment actions Permalink

Ahmed Mohombe wrote:

Is it, or will it be possible to "map" more than one "language" to the
same file extension? (I don't know, to make distinction based on DTD or
Schema (for xml), or just a "file name pattern", - but not only the
extension) ?


I'm not sure if mapping multiple languages to a single filetype is a
good idea. On the other hand, the XML namespaces provide for exactly
this - you are supposed to distinguish the different schemas by the
namespace and not by the actual schema file.

What I think is better is to provide an API for plugging XML language
plugins, mapped to one or more schema URI's. The XML language pluggins
would share the same API as the "normal" ones, differing only in the
code that registers them.

Dimitar

0
Comment actions Permalink

I'm not sure if mapping multiple languages to a single filetype is a
good idea.

I think it is. I'm not talking only about XML files. Only that this extension
is the "most overloaded" :).

On the other hand, the XML namespaces provide for exactly
this - you are supposed to distinguish the different schemas by the
namespace and not by the actual schema file.

Yeah, in theory :).
But the in practice it looks totaly differend: Just take a look to the most
of the opensource frameworks and you will see that the most of the xml files
does not contain DTD or schema or something like this :).

Ahmed.

0
Comment actions Permalink

There are, of course, cases when multiple languages exist in 1 file.
JSPs are probably the most horrible example of a language salad, but
there are cases when XML markup is mixed with JavaScript code or other
toolkit or framework specific mixes.

Amnon

0
Comment actions Permalink

Ahmed Mohombe wrote:

I think it is. I'm not talking only about XML files. Only that this
extension is the "most overloaded" :).


In fact I cannot think of any other overloaded extension. And in XML,
the overloading is by design, that's why you have the namespaces. The
namespaces serve only to identify the actual type of the document and
the origin of the element in multi-schema documents. If the proposed
solution is implemented, you would be able to embed HTML mixed with
Jelly in XSLT. Isn't it cool ;-P


>> On the other hand, the XML namespaces provide for exactly
>> this - you are supposed to distinguish the different schemas by the
>> namespace and not by the actual schema file.


Yeah, in theory :). But the in practice it looks totaly differend: Just
take a look to the most of the opensource frameworks and you will see
that the most of the xml files does not contain DTD or schema or
something like this :).


The namespace URI is not nessesarily related to a physical schema file.
You can use namespace without having a schema, and without having a
website. All you need is URI (e.g. "donut:///mister-donut-service" is
perfectly valid namespace URI.) Another example is that there is no any
official schema file for XSLT, even though there is an official
namespace URI since 1999.

-- -- --
Here are two excerpts from: http://www.w3.org/TR/REC-xml-names/

An XML namespace is a collection of names, identified by a
URI reference , which are used in XML documents as element
types and attribute names. XML namespaces differ from the "namespaces"
conventionally used in computing disciplines in that the XML version has
internal structure and is not, mathematically speaking, a set. These
issues are discussed in "A. The Internal Structure of XML Namespaces".

URI references which identify namespaces are considered
identical when they are exactly the same character-for-character. Note
that URI references which are not identical in this sense may in fact be
functionally equivalent. Examples include URI references which differ
only in case, or which are in external entities which have different
effective base URIs.
-- -- --

cheers,
Dimitar

0
Comment actions Permalink

In fact I cannot think of any other overloaded extension.

e.g.: all the other file types when they get "templated" with the diff template
lanugages.

In a lot of cases one "may not change the extension" to use a unique type:
e.g. the file is templated with velocity, but has the "html" extension.

The namespace URI is not nessesarily related to a physical schema
file. You can use namespace without having a schema, and without
having a website. All you need is URI (e.g.
"donut:///mister-donut-service" is perfectly valid namespace URI.)
Another example is that there is no any official schema file for XSLT,
even though there is an official namespace URI since 1999.

You don't understand what I wanted to outline :) :
In reality, in a lot of projects, the file has the extenstion XML, but no
other useful information in it's header.
Mostly starts only with a tag.
I'm not talking about how it would be right to make it. I'm talking about
a matter of fact, something that can't be changed (or it's not desired by
the authors). This situation exists in a lot of very popular frameworks.

Ahmed.

0
Comment actions Permalink

In reality, in a lot of projects, the file has the extenstion XML, but
no
other useful information in it's header.
Mostly starts only with a tag.

Apache Ant is a perfect sample...


0
Comment actions Permalink

Ahmed Mohombe wrote:

>> I think it is. I'm not talking only about XML files. Only that this
>> extension is the "most overloaded" :).
>>

In fact I cannot think of any other overloaded extension.


I can think of lots of file types for which multiple editors would be cool:
1. HTML - edit as HTML, edit as RIFE template, GUI edit, preview (embedded
browser window)
2. TXT - edit as text, edit as robots.txt file
3. CLASS - view as outline, view as bytecode, view as decompiled source


0
Comment actions Permalink

You convinced me that this would be a pragmatic decision. I guess that
part of the reason that the OS projects so much underuse the namespace
mechanism is that there is no good support in the leading IDE's, so it's
going in circle. Ant is a very good example, ever since 1.6 you can use
namespaces, but nobody cares because there are no immediate benefits.

0
Comment actions Permalink

>> Yup, I think Max's message caused a severe case of anticEAPation :) <<

<groan/> .... :)

0
Comment actions Permalink

Hello All,

A long awaited status report from IDEA team follows:

1. Language API. This stuff begin implemented finally will allow a 3rd
party plugin writers to embed particulary any language support to IDEA
and benefit from it's features. Sytax highlighting, custom PSI, select
word, folding, reference completion, Ctrl+B, error highlighting,
rename refactoring, find usages, brace matching highlights and
navigation and code formatter hooks are implemented so far. We take
that really serious so any feedback is urgely awaited.


This is very cool.

I was looking through the source of the Javascript language plug-in - it
uses a lexer generated by JFlex. I don't know much about lexing/parsing,
but does IDEA reparse the entire file every time the file is changed? Or
does JFlex lexer provide a way to incrementally re-lex?

Also, how powerful is this language API? How powerful do you intend it to
be? Could IDEA's current Java support be reimplemented using this API? Do
you plan to provide this much power?

Could preprocessed languages like C and C++ be implemented using this API?

Thanks,
-Keith


0
Comment actions Permalink

I was looking through the source of the Javascript language plug-in -
it uses a lexer generated by JFlex. I don't know much about
lexing/parsing, but does IDEA reparse the entire file every time the
file is changed? Or does JFlex lexer provide a way to incrementally
re-lex?

We wrap lexers provided with one able to re-lex incrementally. That's why
getState() and getLastState() methods need to be exposed from the lexer.

Also, how powerful is this language API? How powerful do you intend it
to be? Could IDEA's current Java support be reimplemented using this
API? Do you plan to provide this much power?

Theoretically one is able to extends javascript functionality to match java
one-to-one where appropriate right away. It just may be necessary to reimplement
the stuff we've already implemented for java. The main idea though is to
reuse the functionality where it's possible by enforcing language implementation
to expose it's features and semantics via specially dedicated API in Language
class. Thus once control flow builder and some other stuff is exposed one
would get extract variable/inline/extract method refactorings for free for
example.
In fact, we'll be trying to expose those features one-by-one, slowly, painfully
step-by-step making others be able to implement the same stuff we have for
java.

Could preprocessed languages like C and C++ be implemented using this API?

That could be very uneasy. Have to think about. The main problem is in current
architecture every parser builds a syntax tree that matches file being edited
in editor one-to-one cause operations on that tree are intended to take immediate
reflection on the document being edited. Thus constructs like macro substitution
should not be expanded to their content by parser but rather PSI built upon
that tree will logically represent expanded content.

Thanks for questions, Keith. Hope together we'll be able to build something
really useful out of this idea.

Max.






0

Please sign in to leave a comment.