Parser w/o ASTNode
To get started, I would like to be able to have some functionality from my parser. However, since it doesn't currently spit out AST trees, I would like to be able to just hightlight any errors. So my question is: Is it possible to get a handle to the error handler and hightlight code if I can find out the line/col number, etc? If so, could I get a pointer in the right direction? Or do I haveto return the ASTNode and let it be handled for me. It seems as though implementing AST trees would effectively be re-implementing the parser.
thanks again!
adam
Please sign in to leave a comment.
Yep, that's possible.
You'll have to implement getAnnotator() method in your language class and
return Annotator interface implementation. Even if you do not implement the
full blown parser you still can have single PsiElement for the whole file
that will be passed to Annotator.annotate() method along with AnnotationHolder
method. Latter can be populated with error or warning messages using create<*>Annotation()
methods.
-
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"
Thanks Maxim! This helps a lot. I'm still not sure what to do in the parse method. Returning null doesn't appear to be ok;)
adam
I've found reasonable "mock" implementation in Groovy plugin by Dierk at
https://svn.canoo.com/trunk/groovyj/src/java/groovy/idesupport/idea/parsing/GroovyParser.java
Hopefully I'm not doing anything bad by posting this link as it have been
already posted here in the list.
-
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"
Thanks a lot for all your help!
adam
Maxim,
I was able to more done using this method. However, I am getting many exceptions and out of mem errors when trying to load even an empty file. Could you possibly provide a little more info on the necessary methods in the lexer? I think I must be doing something wrong there. I am currently creating a new lexer each time the parse method gets called and passing in the text from the builder. Also, I am not doing anything in the 3 start methods because of this. If you can provide some info, I can relay this info to the wiki for others to use and it won't be lost.
Adam,
creating new Lexer every time you need one or required for is fine. Not implementing
start methods - that's what causes problems most probably. Since Lexer is
an interface there's no way to enforce initialization and pass required parameters
at object creation time. That's why initialization is performed in start
methods.
-
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"
ok - but what exactly needs to be done in the start methods? Should I wait and only create the new lexer here?
And sorry for barrage of questions. I'm getting close to a deadline...
/**
Prepare for lexing character data from <code>buffer</code> passed. Lexing
should be performed starting from offset 0 and
terminated (EOFed) at offset buffer.length. Internal lexer state is
supposed to be initial. YY_INITIAL for JLex and JFlex generated
lexer.
@param buffer character data for lexing.
*/
void start(char[] buffer);
/**
Prepare for lexing character data from <code>buffer</code> passed. Lexing
should be performed starting from offset startOffset>]]> and
terminated (EOFed) at offset <code>endOffset</code>. Internal lexer
state is supposed to be initial. YY_INITIAL for JLex and JFlex generated
lexer.
@param buffer character data for lexing.
@param startOffset offset to start lexing from
@param endOffset offset to stop lexing at
*/
void start(char[] buffer, int startOffset, int endOffset);
/**
Prepare for lexing character data from <code>buffer</code> passed. Lexing
should be performed starting from offset startOffset>]]> and
terminated (EOFed) at offset <code>endOffset</code>. Internal lexer
state is supposed to be initialState]]>. It is guaranteed
that value of initialState have been returned by {@link #getState()}
method of this Lexer at condition
]]>This method is used to incrementally relex changed characters using
lexing data acquired from this particular lexer sometime in the past.
@param buffer character data for lexing.
@param startOffset offset to start lexing from
@param endOffset offset to stop lexing at
*/
void start(char[] buffer, int startOffset, int endOffset, int initialState);
-
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"
That's OK. We're very interested for this API to be used. This way we'll
be able to make it better where it is still rough on edges.
What is the language you're implementing the plugin for if it's not a secret?
-
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"
And now it's clear Lexer interface has to be reworked. Thanks!
-
Maxim Shafirov
http://www.jetbrains.com
"Develop with pleasure!"
>> And sorry for barrage of questions. I'm getting close to a
>> deadline...
>>
Its actually a toy language for a master's project I'm working on called mini-java, which is used to generate native assembly from java. I chose this project because I wanted to get get some practice in doing a plugin before doing an industrial strength one at work.
And I really, really appreciate all the help - especially on Saturday!
Maxim,
I think I'm doing the right thing in the start methods, but when I try to load a simple file, I still can't get past the out of mem error. I also get the following:
java.lang.IllegalArgumentException
at java.nio.Buffer.position(Buffer.java:218)
at com.intellij.util.io.ByteBufferRADataInput.setPosition(ByteBufferRADataInput.java:24)
at com.intellij.util.io.ByteBufferMap.(ByteBufferMap.java:33) at com.intellij.psi.impl.cache.impl.idCache.ImmutableIdCacheImpl.a(ImmutableIdCacheImpl.java:61) at com.intellij.psi.impl.cache.impl.idCache.ImmutableIdCacheImpl.getFilesWithTodoItems(ImmutableIdCacheImpl.java:29) at com.intellij.psi.impl.cache.impl.idCache.IdCacheImpl.getFilesWithTodoItems(IdCacheImpl.java:94) at com.intellij.psi.impl.cache.impl.CacheManagerImpl.getFilesWithTodoItems(CacheManagerImpl.java:36) at com.intellij.psi.impl.cache.impl.CompositeCacheManager.getFilesWithTodoItems(CompositeCacheManager.java:20) at com.intellij.psi.impl.search.PsiSearchHelperImpl.findFilesWithTodoItems(PsiSearchHelperImpl.java:242) at com.intellij.ide.todo.AllTodosTreeBuilder.rebuildCache(AllTodosTreeBuilder.java:13) at com.intellij.ide.todo.TodoTreeBuilder.init(TodoTreeBuilder.java:192) at com.intellij.ide.todo.TodoView$2.createTreeBuilder(TodoView.java:0) at com.intellij.ide.todo.TodoPanel.(TodoPanel.java:42) at com.intellij.ide.todo.TodoView$2.]]>(TodoView.java:1)
at com.intellij.ide.todo.TodoView$1.run(TodoView.java:4)
at com.intellij.ide.startup.impl.StartupManagerImpl.a(StartupManagerImpl.java:7)
I think I narrowed down the problem. The code seems to be bouncing back and forth between getTokenStart and getTokenEnd infinitely. Any ideas what could cause this?
More info: It looks like builder.eof is never true. Is there a token or something I need to return to indicate this?
adam
Return null as getTokenType() when you finish.
Thanks Maxim, but I am still getting a OOB exception that I mentioned in the next thread. Any ideas?
adam