Annotator not called when lexing problems present?

I'm working on Language plugin, and it seems that my Annotators are only
run once there are no lexing problems in the whole file. Is this
intended behavior? Should I try to be more cautious about lexing
problems, and convert them to parsing probelms?

7 comments
Comment actions Permalink

I mis-spoke. When I said lexing problems, I meant problems with the
lexed document, i.e. parsing problems :) That is, if builder.error() has
been called during parse, my annotators are not run. So, is this
intended behavior? Should parsing problems only be unrecoverable problems?

Keith Lea wrote:

I'm working on Language plugin, and it seems that my Annotators are only
run once there are no lexing problems in the whole file. Is this
intended behavior? Should I try to be more cautious about lexing
problems, and convert them to parsing probelms?

0
Comment actions Permalink

Now in build 3419 it appears that the annotator isn't even being called
when there are annotator-created errors in subelements. Could someone
describe how to use the annotator, when it's called, and so on?

Keith Lea wrote:

I mis-spoke. When I said lexing problems, I meant problems with the
lexed document, i.e. parsing problems :) That is, if builder.error() has
been called during parse, my annotators are not run. So, is this
intended behavior? Should parsing problems only be unrecoverable problems?

Keith Lea wrote:

>> I'm working on Language plugin, and it seems that my Annotators are
>> only run once there are no lexing problems in the whole file. Is this
>> intended behavior? Should I try to be more cautious about lexing
>> problems, and convert them to parsing probelms?

0
Comment actions Permalink

That's really true (an optimization):
if you have an error in subelement, then the annotator is not called for
parent elements.
If you need annotator to be nevertheless called in some cases (like we do
sometimes in java), write the checks for some leaf element.

Eugene.

"Keith Lea" <keith@cs.oswego.edu> wrote in message
news:dbr58s$gna$1@is.intellij.net...

Now in build 3419 it appears that the annotator isn't even being called
when there are annotator-created errors in subelements. Could someone
describe how to use the annotator, when it's called, and so on?

>

Keith Lea wrote:

I mis-spoke. When I said lexing problems, I meant problems with the
lexed document, i.e. parsing problems :) That is, if builder.error() has
been called during parse, my annotators are not run. So, is this
intended behavior? Should parsing problems only be unrecoverable

problems?

>

Keith Lea wrote:

>
>> I'm working on Language plugin, and it seems that my Annotators are
>> only run once there are no lexing problems in the whole file. Is this
>> intended behavior? Should I try to be more cautious about lexing
>> problems, and convert them to parsing probelms?



0
Comment actions Permalink

Okay. I think it shouldn't be that way. I try to keep a valid PSI tree
even with parsing and other errors. It's especially true for things like
missing semicolon or misformatted literal, which certainly should not
stop annotator. In fact, I think most errors in my language should not
prevent parents from being checked for error. Are other languages not
this way?

What if the annotator were called all the time, with a flag
"hasErrorsInSubElements"? Or what if there was a method on annotator
shouldCheckParentsWhenErrorsExist()? This would make a lot more sense
for my language.

Eugene Vigdorchik (JetBrains) wrote:

That's really true (an optimization):
if you have an error in subelement, then the annotator is not called for
parent elements.
If you need annotator to be nevertheless called in some cases (like we do
sometimes in java), write the checks for some leaf element.

Eugene.

"Keith Lea" <keith@cs.oswego.edu> wrote in message
news:dbr58s$gna$1@is.intellij.net...

>>Now in build 3419 it appears that the annotator isn't even being called
>>when there are annotator-created errors in subelements. Could someone
>>describe how to use the annotator, when it's called, and so on?
>>
>>Keith Lea wrote:
>>
>>>I mis-spoke. When I said lexing problems, I meant problems with the
>>>lexed document, i.e. parsing problems :) That is, if builder.error() has
>>>been called during parse, my annotators are not run. So, is this
>>>intended behavior? Should parsing problems only be unrecoverable


problems?

>>>Keith Lea wrote:
>>>
>>>
>>>>I'm working on Language plugin, and it seems that my Annotators are
>>>>only run once there are no lexing problems in the whole file. Is this
>>>>intended behavior? Should I try to be more cautious about lexing
>>>>problems, and convert them to parsing probelms?


0
Comment actions Permalink

Keith,
you are the first who complains on this. And even you started it only when
you began implementing your own
language, not when you were experiencing some inconveniences as a user - so,
I don't really see the problem, and why on earth
this would require reimplementation.


"Keith Lea" <keith@cs.oswego.edu> wrote in message
news:dbritm$h2t$1@is.intellij.net...

Okay. I think it shouldn't be that way. I try to keep a valid PSI tree
even with parsing and other errors. It's especially true for things like
missing semicolon or misformatted literal, which certainly should not
stop annotator. In fact, I think most errors in my language should not
prevent parents from being checked for error. Are other languages not
this way?

Apart from few exceptional cases, all languages IDEA supports are this way.

Eugene.


0
Comment actions Permalink

This is a question of API, not user experience. Obviously IDEA doesn't behave this way for Java. If I'm missing a semicolon in a method, IDEA still checks the class, and other methods, and just about everything, even that statement itself. So I don't see why the language API should promote the opposite behavior.

Anyway, so you say I should make the annotator run for parents in deeper nodes. So now I have to figure out what all of the leaf nodes in my PSI tree are, and if any one of them is run by the annotator, I walk up the tree and annotate all parents, but making sure I haven't already annotated elements for a previous leaf. Is that correct? How am I supposed to do this? I can't think of how, because the annotator is not given information about an annotation run, it is mostly stateless. It seems that I would end up re-annotating things dozens of times.

0
Comment actions Permalink

I have filed IDEA-4026 about this behavior.

Eugene Vigdorchik (JetBrains) wrote:

Keith,
you are the first who complains on this. And even you started it only when
you began implementing your own
language, not when you were experiencing some inconveniences as a user - so,
I don't really see the problem, and why on earth
this would require reimplementation.


"Keith Lea" <keith@cs.oswego.edu> wrote in message
news:dbritm$h2t$1@is.intellij.net...

>>Okay. I think it shouldn't be that way. I try to keep a valid PSI tree
>>even with parsing and other errors. It's especially true for things like
>> missing semicolon or misformatted literal, which certainly should not
>>stop annotator. In fact, I think most errors in my language should not
>>prevent parents from being checked for error. Are other languages not
>>this way?


Apart from few exceptional cases, all languages IDEA supports are this way.

Eugene.

0

Please sign in to leave a comment.