JetBrains: What we're busy with right now.

Hello All,

A long awaited status report from IDEA team follows:

1. Language API. This stuff begin implemented finally will allow a 3rd party
plugin writers to embed particulary any language support to IDEA and benefit
from it's features. Sytax highlighting, custom PSI, select word, folding,
reference completion, Ctrl+B, error highlighting, rename refactoring, find
usages, brace matching highlights and navigation and code formatter hooks
are implemented so far. We take that really serious so any feedback is urgely
awaited.

2. JavaScript plugin implements an API mentioned. While quite rough on edges
seem to benefit from all that stuff already. Plugin and it's source code
will be available with the next EAP build.

3. Formatter API and implementations (for HTML, XML, JSPX, JSP, JavaScript,
CSS and their various embeddments) are being developed quite actively.

4. Analyze backward dependencies and package cycle dependencies implemented.

5. Some refactorings and code cleanups in debugger and usage view.

6. Improoving stability and performance of JSPX editing



-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"


40 comments
Comment actions Permalink

>> I was looking through the source of the Javascript language plug-in -
>> it uses a lexer generated by JFlex. I don't know much about
>> lexing/parsing, but does IDEA reparse the entire file every time the
>> file is changed? Or does JFlex lexer provide a way to incrementally
>> re-lex?
>>

We wrap lexers provided with one able to re-lex incrementally. That's
why getState() and getLastState() methods need to be exposed from the
lexer.


So you need special JFlex config file or something? Will you release yours
that you used for JavascriptLanguage?

Thanks for questions, Keith. Hope together we'll be able to build
something really useful out of this idea.


Me too, I'm excited about it.


0
Comment actions Permalink

If you meant template file - it's there, attached.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

>>> I was looking through the source of the Javascript language plug-in
>>> - it uses a lexer generated by JFlex. I don't know much about
>>> lexing/parsing, but does IDEA reparse the entire file every time the
>>> file is changed? Or does JFlex lexer provide a way to incrementally
>>> re-lex?
>>>
>> We wrap lexers provided with one able to re-lex incrementally. That's
>> why getState() and getLastState() methods need to be exposed from the
>> lexer.
>>

So you need special JFlex config file or something? Will you release
yours that you used for JavascriptLanguage?

>> Thanks for questions, Keith. Hope together we'll be able to build
>> something really useful out of this idea.
>>

Me too, I'm excited about it.


begin 777 idea-skeleton
M("`O*BH@:6YI=&EA;"!S:7IE(&]F('1H92!L;V]K86AE860@8G5F9F5R("HO
M"BTM+2!P<FEV871E('-T871I8R!F:6YA;"!I;G0@6EI?0E5&1D524TE:12`]
M("XNCL*"B`@RHJ(&QE>&EC86P@<W1A=&5S("HO"BTM+2`@;&5X:6-A;"!S
M=&%T97,L(&-H87)M87`"B`@+RH@97)R;W(@8V]D97,@B\*("!P<FEV871E
M('-T871I8R!F:6YA;"!I;G0@6EI?54Y+3D]73E]%4E)/4B`](#`["B`@<')I
M=F%T92!S=&%T:6,@9FEN86P@:6YT(%I:7TY/7TU!5$-((#T@,3L*("!P<FEV
M871E('-T871I8R!F:6YA;"!I;G0@6EI?4%532$)!0TM?,D))1R`](#(["B`@
M<')I=F%T92!S=&%T:6,@9FEN86P@8VAA<EM=($5-4%197T)51D9%4B`](&YE
M=R!C:&%R6S!=.PH@('!R:79A=&4@<W1A=&EC(&9I;F%L(&EN="!9645/1B`]
M("TQ.PH("`OB!E<G)O<B!M97-S86=E<R!F;W(@=&AE(&-O9&5S(&%B;W9E
M("HO"B`@<')I=F%T92!S=&%T:6,@9FEN86P@4W1R:6YG(%I:7T524D]27TU3
M1UM=(#T@>PH@("`@(E5N:V]W;B!I;G1E<FYA;"!S8V%N;F5R(&5R<F]R(BP*
M("`@(")%<G)O<CH@8V]U;&0@;F]T(&UA=&-H(&EN<'5T(BP*("`@(")%<G)O
M<CH@<'5S:&)A8VL@=F%L=64@=V%S('1O;R!L87)G92(("!].PH+2TM(&ES
M1FEN86P@;&ES=`H@("\J*B!T:&4@8W5R<F5N="!S=&%T92!O9B!T:&4@1$9!
M("HO"B`@<')I=F%T92!I;G0@>GI3=&%T93L*"B`@+RHJ('1H92!C=7)R96YT
M(&QE>&EC86P@<W1A=&4@B\("!P<FEV871E(&EN="!Z>DQE>&EC86Q3=&%T
M92`](%E924Y)5$E!3#L*"B`@+RHJ('1H:7,@8G5F9F5R(&-O;G1A:6YS('1H
M92!C=7)R96YT('1E>'0@=&\@8F4@;6%T8VAE9"!A;F0@:7,*("`@("`@=&AE
M('-O=7)C92!O9B!T:&4@>7ET97AT*"D@<W1R:6YG("HO"B`@<')I=F%T92!C
M:&%R('IZ0G5F9F5R6UT@/2!%35!465]"549&15(["@H@("\J*B!T:&4@=&5X
M='!O<VET:6]N(&%T('1H92!L87-T(&%C8V5P=&EN9R!S=&%T92`J+PH@('!R
M:79A=&4@:6YT('IZ36%R:V5D4&]S.PH("`OBH@=&AE('1E>'1P;W-I=&EO
M;B!A="!T:&4@;&%S="!S=&%T92!T;R!B92!I;F-L=61E9"!I;B!Y>71E>'0@
MB\("!P<FEV871E(&EN="!Z>E!U<VAB86-K4&]S.PH("`OBH@=&AE(&-U
M<G)E;G0@=&5X="!P;W-I=&EO;B!I;B!T:&4@8G5F9F5R("HO"B`@<')I=F%T
M92!I;G0@>GI#=7)R96YT4&]S.PH("`OBH@<W1A<G1296%D(&UA<FMS('1H
M92!B96=I;FYI;F<@;V8@=&AE('EY=&5X="@I('-T<FEN9R!I;B!T:&4@8G5F
M9F5R("HO"B`@<')I=F%T92!I;G0@>GI3=&%R=%)E860["@H@("\J*B!E;F12
M96%D(&UA<FMS('1H92!L87-T(&-H87)A8W1E<B!I;B!T:&4@8G5F9F5R+"!T
M:&%T(&AA<R!B965N(')E860*("`@("`@9G)O;2!I;G!U="`J+PH@('!R:79A
M=&4@:6YT('IZ16YD4F5A9#L*"B`@+RHJ"B`@("H@>GI!=$)/3"`]/2!T<G5E
M(#P]/B!T:&4@<V-A;FYE<B!I<R!C=7)R96YT;'D@870@=&AE(&)E9VEN;FEN
M9R!O9B!A(&QI;F4("`@B\*("!P<FEV871E(&)O;VQE86X@>GI!=$)/3"`]
M('1R=64["@H@("\J*B!Z>D%T14]&(#T]('1R=64@/#T^('1H92!S8V%N;F5R
M(&ES(&%T('1H92!%3T8@B\("!P<FEV871E(&)O;VQE86X@>GI!=$5/1CL*
M"BTM2!U<V5R(&-L87-S(&-O9&4*"B`@RHJ"B`@("H@0W)E871E<R!A(&YE
M=R!S8V%N;F5R"B`@("H@5&AE<F4@:7,@86QS;R!A(&IA=F$N:6\N26YP=713
M=')E86T@=F5R<VEO;B!O9B!T:&ES(&-O;G-T<G5C=&]R+@H@("`J"B`@("H@
M0'!A<F%M("`@:6X@('1H92!J879AFEOE)E861E<B!T;R!R96%D(&EN<'5T
M(&9R;VTN"B`@("HO"BTM+2!C;VYS=')U8W1O<B!D96-L87)A=&EO;@H*"B`@
MRHJ"B`@("H@4F5F:6QL<R!T:&4@:6YP=70@8G5F9F5R@H@("`J"B`@("H@
M0')E='5R;B`@("`@(#QC;V1E/F9A;'-E/"]C;V1E/BP@:69F('1H97)E('=A
M<R!N97<@:6YP=70N"B`@("H@"B`@("H@0&5X8V5P=&EO;B`@(&IA=F$N:6\N
M24]%>&-E<'1I;VX@(&EF(&%N>2!)+T\M17)R;W(@;V-C=7)S"B`@("HO"B`@
M<')I=F%T92!B;V]L96%N('IZ4F5F:6QL*"D@=&AR;W=S(&IA=F$N:6\N24]%
M>&-E<'1I;VX@>PH@("`@<F5T=7)N('1R=64["B`@?0H("`@(`H@("\J@H@
M("`J($-L;W-E<R!T:&4@:6YP=70@<W1R96%M@H@("`JPH@('!U8FQI8R!F
M:6YA;"!V;VED('EY8VQO<V4H*2!T:')O=W,@:F%V82YI;RY)3T5X8V5P=&EO
M;B!["B`@("!Z>D%T14]&(#T@=')U93L@("`@("`@("`@("`O*B!I;F1I8V%T
M92!E;F0@;V8@9FEL92`J+PH@("`@>GI%;F1296%D(#T@>GI3=&%R=%)E860[
M("`OB!I;G9A;&ED871E(&)U9F9E<B`@("`J+PH@('T"@H@("\J*@H@("`J
M(%)E<V5T<R!T:&4@<V-A;FYE<B!T;R!R96%D(&9R;VT@82!N97<@:6YP=70@
M<W1R96%M@H@("`J($1O97,@;F]T(&-L;W-E('1H92!O;&0@<F5A9&5R@H@
M("`J"B`@("H@06QL(&EN=&5R;F%L('9A<FEA8FQE<R!A<F4@<F5S970L('1H
M92!O;&0@:6YP=70@<W1R96%M(`H@("`J(#QB/F-A;FYO=#PO8CX@8F4@<F5U
M<V5D("AI;G1E<FYA;"!B=69F97(@:7,@9&ES8V%R9&5D(&%N9"!L;W-T2X
M("`@*B!,97AI8V%L('-T871E(&ES('-E="!T;R`\='0^6EI?24Y)5$E!3#PO
M='0^+@H@("`J"B`@("H@0'!A<F%M(')E861E<B`@('1H92!N97<@:6YP=70@
M<W1R96%M(`H@("`J+PH@('!U8FQI8R!F:6YA;"!V;VED('EY<F5S970H:F%V
M82YI;RY296%D97(@<F5A9&5R*2!["B`@("!Z>D%T0D],("`]('1R=64["B`@
M("!Z>D%T14]&("`](&9A;'-E.PH@("`@>GI%;F1296%D(#T@>GI3=&%R=%)E
M860@/2`P.PH@("`@>GI#=7)R96YT4&]S(#T@>GI-87)K9610;W,@/2!Z>E!U
M<VAB86-K4&]S(#T@,#L*("`@('IZ3&5X:6-A;%-T871E(#T@65E)3DE424%,
M.PH@('T"@H@("\J@H@("`J(%)E='5R;G,@=&AE(&-U<G)E;G0@;&5X:6-A
M;"!S=&%T92X("`@B\*("!P=6)L:6,@9FEN86P@:6YT('EY<W1A=&4H*2![
M"B`@("!R971U<FX@>GI,97AI8V%L4W1A=&4["B`@?0H*"B`@+RHJ"B`@("H@
M16YT97)S(&$@;F5W(&QE>&EC86P@<W1A=&4("`@@H@("`J($!P87)A;2!N
M97=3=&%T92!T:&4@;F5W(&QE>&EC86P@<W1A=&4("`@B\*("!P=6)L:6,@
M9FEN86P@=F]I9"!Y>6)E9VEN*&EN="!N97=3=&%T92D@>PH@("`@>GI,97AI
M8V%L4W1A=&4@/2!N97=3=&%T93L("!]"@H("`OBH("`@*B!2971U]]>'!R
M97-S:6]N@H@("`JPH@('!U8FQI8R!F:6YA;"!3=')I;F<@>7ET97AT*"D@
M>PH@("`@<F5T=7)N(&YE=R!3=')I;F<H('IZ0G5F9F5R+"!Z>E-T87)T4F5A
M9"P@>GI-87)K9610;W,M>GI3=&%R=%)E860@3L("!]"@H("`OBH*("`@
M*B!2971U71E>]]>'0H2YC:&%R070H<&]S2P@8G5T
M(&9A<W1E<@H@("`J"B`@("H@0'!A<F%M('!O<R!T:&4@<&]S:71I;VX@;V8@
M=&AE(&-H87)A8W1E<B!T;R!F971C:"X@"B`@("H@("`@("`@("`@("!!('9A
M;'5E(&9R;VT@,"!T;R!Y>6QE;F=T:"@I+3$N"B`@("H("`@B!`<F5T=7)N
M('1H92!C:&%R86-T97(@870@<&]S:71I;VX@<&]S"B`@("HO"B`@<'5B;&EC
M(&9I;F%L(&-H87(@>7EC:&%R870H:6YT('!O<RD@>PH@("`@<F5T=7)N('IZ
M0G5F9F5R6WIZ4W1A<G1296%DW!O<UT["B`@?0H"B`@+RHJ"B`@("H@4F5T
M=7)N<R!T:&4@;&5N9W1H(&]F('1H92!M871C:&5D('1E>'0@<F5G:6]N+@H@
M("`J+PH@('!U8FQI8R!F:6YA;"!I;G0@>7EL96YG=&@H*2!["B`@("!R971U
M<FX@>GI-87)K9610;W,M>GI3=&%R=%)E860["B`@?0H*"B`@+RHJ"B`@("H@
M4F5P;W)T<R!A;B!E<G)O<B!T:&%T(&]C8W5R960@=VAI;&4@<V-A;FYI;F<N
M"B`@("H("`@B!);B!A('=E;&QF;W)M960@<V-A;FYE<B`H;F\@;W(@;VYL
M>2!C;W)R96-T('5S86=E(&]F(`H@("`J('EY<'5S:&)A8VLH:6YT*2!A;F0@
M82!M871C:"UA;&P@9F%L;&)A8VL@<G5L92D@=&AI<R!M971H;V0@"B`@("H@
M=VEL;"!O;FQY(&)E(&-A;&QE9"!W:71H('1H:6YG<R!T:&%T(")#86XG="!0
M;W-S:6)L>2!(87!P96XB+@H@("`J($EF('1H:7,@;65T:&]D(&ES(&-A;&QE
M9"P@<V]M971H:6YG(&ES('-E<FEO=7-L>2!W<F]N9PH@("`J("AE+F<N(&$@
M2D9L97@@8G5G('!R;V1U8VEN9R!A(&9A=6QT>2!S8V%N;F5R(&5T8RXI+@H@
M("`J"B`@("H@57-U86P@<WEN=&%X+W-C86YN97(@;&5V96P@97)R;W(@:&%N
M9&QI;F<@<VAO=6QD(&)E(&1O;F4("`@B!I;B!E<G)O<B!F86QL8F%C:R!R
M=6QE<RX("`@@H@("`J($!P87)A;2`@(&5R<F]R0V]D92`@=&AE(&-O9&4@
M;V8@=&AE(&5R<F]R;65S<V%G92!T;R!D:7-P;&%Y"B`@("HO"BTM+2!Z>E-C
M86Y%<G)O<B!D96-L87)A=&EO;@H@("`@4W1R:6YG(&UE<W-A9V4["B`@("!T
M<GD@>PH@("`@("!M97-S86=E(#T@6EI?15)23U)?35-'6V5R<F]R0V]D95T[
M"B`@("!]"B`@("!C871C:"`H07)R87E);F1E>$]U=$]F0F]U;F1S17AC97!T
M:6]N(&4I('L*("`@("`@;65S<V%G92`](%I:7T524D]27TU31UM:6E]53DM.
M3U=.7T524D]273L("`@('T"BTM+2!T:')O=W,@8VQA=7-E"B`@?2`*"@H@
M("\J*@H@("`J(%!U<VAE<R!T:&4@<W!E8VEF:65D(&%M;W5N="!O9B!C:&%R
M86-T97)S(&)A8VL@:6YT;R!T:&4@:6YP=70@<W1R96%M+@H@("`J"B`@("H@
M5&AE>2!W:6QL(&)E(')E860@86=A:6X@8GD@=&AE;B!N97AT(&-A;&P@;V8@
M=&AE('-C86YN:6YG(&UE=&AO9`H@("`J"B`@("H@0'!A<F%M(&YU;6)E<B`@
M=&AE(&YU;6)E<B!O9B!C:&%R86-T97)S('1O(&)E(')E860@86=A:6XN"B`@
M("H@("`@("`@("`@("`@("`@5&AI<R!N=6UB97(@;75S="!N;W0@8F4@9W)E
M871E<B!T:&%N('EY;&5N9W1H*"DA"B`@("HO"BTM+2!Y>7!U<VAB86-K(&1E
M8VP@&-O;G1A:6YS('IZ4V-A;D5R<F]R(&5X8V5P=&EO;BD("`@(&EF("@@
M;G5M8F5R(#X@>7EL96YG=&@H2`I"B`@("`@('IZ4V-A;D5R<F]R%I:7U!5
M4TA"04-+7S)"24<I.PH("`@('IZ36%R:V5D4&]S("T](&YU;6)E<CL("!]
M"@H+2TM('IZ1&]%3T8("`OBH("`@*B!297-U;65S('-C86YN:6YG('5N
M=&EL('1H92!N97AT(')E9W5L87(@97AP<F5S<VEO;B!I<R!M871C:&5D+`H@
M("`J('1H92!E;F0@;V8@:6YP=70@:7,@96YC;W5N=&5R960@;W(@86X@22]/
M+45R<F]R(&]C8W5R<RX("`@@H@("`J($!R971U<FX@("`@("!T:&4@;F5X
M="!T;VME;@H@("`J($!E>&-E<'1I;VX@("!J879AFEODE/17AC97!T:6]N
M("!I9B!A;GD@22]/45R<F]R(&]C8W5R<PH@("`JPHM+2T@>7EL97@@9&5C
M;&%R871I;VX("`@(&EN="!Z>DEN<'5T.PH@("`@:6YT('IZ06-T:6]N.PH
M("`@("\O(&-A8VAE9"!F:65L9',Z"B`@("!I;G0@>GI#=7)R96YT4&]S3#L*
M("`@(&EN="!Z>DUA<FME9%!O<TP["B`@("!I;G0@>GI%;F1296%D3"`]('IZ
M16YD4F5A9#L*("`@(&-H87(@6UT@>GI"=69F97),(#T@>GI"=69F97(["B`@
M("!C:&%R(%M=('IZ0TUA<$P@/2!:6E]#34%0.PH*+2TM(&QO8V%L(&1E8VQA
M<F%T:6]N<PH("`@('=H:6QE("AT<G5E2!["B`@("`@('IZ36%R:V5D4&]S
M3"`]('IZ36%R:V5D4&]S.PH*2TM('-T87)T(&%D;6EN("AL:6YE"!C:&%R
M"!C;VP@8V]U;G0I"B`@("`@('IZ06-T:6]N(#T@3$["@H@("`@("!Z>D-U
M<G)E;G10;W-,(#T@>GI#=7)R96YT4&]S(#T@>GI3=&%R=%)E860@/2!Z>DUA
M<FME9%!O<TP["B`@"BTM+2!S=&%R="!A9&UI;B`H;&5X<W1A=&4@971C0H
M("`@("`@>GI&;W)!8W1I;VXZ('L*("`@("`@("!W:&EL92`H=')U92D@>PH@
M("`@"BTM2!N97AT(&EN<'5T"!L:6YE+"!C;VPL(&-H87(@8V]U;G0L(&YE
M>'0@=')A;G-I=&EO;BP@:7-&:6YA;"!A8W1I;VX*("`@("`@("`@("`@>GI!
M8W1I;VX@/2!Z>E-T871E.PH@("`@("`@("`@("!Z>DUA<FME9%!O<TP@/2!Z
M>D-U<G)E;G10;W-,.PHM+2T@;&EN92!C;W5N="!U<&1A=&4*("`@("`@("`@
M('T*"B`@("`@("`@?0H@("`@("!]"@H@("`@("`O+R!S=&]R92!B86-K(&-A
M8VAE9"!P;W-I=&EO;@H@("`@("!Z>DUA<FME9%!O<R`]('IZ36%R:V5D4&]S
M3#L*2TM(&-H87(@8V]U;G0@=7!D871E"@HM2T@86-T:6]N<PH@("`@("`@
M(&1E9F%U;'0Z(`H@("`@("`@("`@:68@*'IZ26YP=70@/3T@65E%3T8@)B8@
M>GI3=&%R=%)E860@/3T@>GI#=7)R96YT4&]S*2!["B`@("`@("`@("`@('IZ
M071%3T8@/2!T<G5E.PHM+2T@96]F=F%L=64*("`@("`@("`@('T@"B`@("`@
M("`@("!E;'-E('L+2TM(&YO(&UA=&-H"B`@("`@("`@("!]"B`@("`@('T
7("`@('T("!]"@HM+2T@;6%I;@H?0IH
`
end

0
Comment actions Permalink

Hi Maxim,

1. Language API. This stuff begin implemented finally
will allow a 3rd party
plugin writers to embed particulary any language
support to IDEA and benefit
from it's features.


is the new Language API a first step towards your Metaprogramming System?

0
Comment actions Permalink

Thanks.

I wonder, why not use a generated scanner/parser as well? Your Javascript
parser is not trivial, over 1,000 lines, all appears to be hand-written,
and it doesn't appear to do anything too complicated which some grammar based
parser generator could not do, don't you think? Manual parser may be okay
for Javascript, but other languages (like Groovy, for example) may have more
complicated syntax.


0
Comment actions Permalink

Also - most parts of JavascriptLanguage plugin are clear, but what is the
purpose of WordsScanner?


0
Comment actions Permalink

We do build search caches for find usages (as well as refactorings) with
help of WordsScanner. Basically we need to know what words certain file
contains.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

Also - most parts of JavascriptLanguage plugin are clear, but what is
the purpose of WordsScanner?




0
Comment actions Permalink

Thought so too. FWIW, a Groovy integration is in the works. The current ANTLR grammar for Groovy is modelled after J2SE5. Are you interested in joining the effort?

Regards,

Franck

0
Comment actions Permalink

FWIW, a Groovy integration is in the works.

Great news!

As the matter of fact we do not insist parsers to be written in recursive
down method we've done for javascript. The only thing we need is input token
stream to be marked with PsiBuilder.Marker calls to build a syntax tree that
maps to original text. ANTLR grammar (probably?) could do the same in its
semantic parts. In the worst case one can build its own whatever format tree
and then translate it to IDEA's format by means of traversing and calling
PsiBuilder's methods while traversing.

-


Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"

Thought so too. FWIW, a Groovy integration is in the works. The
current ANTLR grammar for Groovy is modelled after J2SE5. Are you
interested in joining the effort?

Regards,

Franck




0
Comment actions Permalink

Thought so too. FWIW, a Groovy integration is in the works. The
current ANTLR grammar for Groovy is modelled after J2SE5. Are you
interested in joining the effort?


No, it was only an example, if I develop a language plugin, I think it will
be for Ruby.


0
Comment actions Permalink

late reply, I know, but +1 for ruby. ;o)

0

Please sign in to leave a comment.