Is the same lexer instance called by multiple threads?

Hello Siddharth,

If you're returning the same lexer instance from every call of a method returning
a lexer in ParserDefinition and similar APIs, then yes, your lexer instance
will be called concurrently for multiple threads. Both indexing and inspections
happen in multiple threads in parallel.

In method advance(), I access a list. From the error it seems that two
threads are trying to access/modify the list concurrently. I am going
to add a synchronized block (or is there a better alternative?) but I
want to also understand why this exception occurs -

• Do multiple threads access a lexer instance and if yes, when does

this occur?

thanks
Siddharth
----- Snip from stacktrace ----

To reindex this file IDEA has to be restarted
java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:782)
at java.util.ArrayList$Itr.next(ArrayList.java:754)
at
*myplugins.HighlightingLexerAdapter.advance(HighlightingLexerAdapter.j
ava:68)*
at
com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.doSetText(L
exerEditorHighlighter.java:312)
at
com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.setText(Lex
erEditorHighlighter.java:287)
at
com.intellij.psi.impl.cache.impl.id.IdTableBuilding$TokenSetTodoIndexe
r.map(IdTableBuilding.java:320)
at
com.intellij.psi.impl.cache.impl.id.IdTableBuilding$TokenSetTodoIndexe
r.map(IdTableBuilding.java:298)
at
com.intellij.psi.impl.cache.impl.todo.TodoIndex$4.map(TodoIndex.java:1
00)
at
com.intellij.psi.impl.cache.impl.todo.TodoIndex$4.map(TodoIndex.java:9
4)
at
com.intellij.util.indexing.MapReduceIndex.update(MapReduceIndex.java:2
00)
at
com.intellij.util.indexing.FileBasedIndex$20.run(FileBasedIndex.java:1
402)
at
com.intellij.openapi.progress.impl.ProgressManagerImpl.executeNonCance
lableSection(ProgressManagerImpl.java:132)
at
com.intellij.util.indexing.FileBasedIndex.updateSingleIndex(FileBasedI
ndex.java:1399)
at
com.intellij.util.indexing.FileBasedIndex.indexFileContent(FileBasedIn
dex.java:1365)
at
com.intellij.util.indexing.UnindexedFilesUpdater.processFile(Unindexed
FilesUpdater.java:61)
at
com.intellij.openapi.project.CacheUpdateSession.processFile(CacheUpdat
eSession.java:102)
at
com.intellij.openapi.project.CacheUpdateRunner$MyRunnable$1.run(CacheU
pdateRunner.java:228)
at
com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(Ap
plicationImpl.java:790)
at
com.intellij.openapi.project.CacheUpdateRunner$MyRunnable.run(CacheUpd
ateRunner.java:232)
at
com.intellij.openapi.application.impl.ApplicationImpl$7.run(Applicatio
nImpl.java:386)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471
)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j
ava:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:603)
at java.lang.Thread.run(Thread.java:636)
at
com.intellij.openapi.application.impl.ApplicationImpl$1$1.run(Applicat
ionImpl.java:130)
---
Original message URL:
http://devnet.jetbrains.net/message/5322215#5322215

--
Dmitry Jemerov
Development Lead
JetBrains, Inc.
http://www.jetbrains.com/
"Develop with Pleasure!"

I found changing my highlighter helped cut down on the concurrency issues with my lexer, try lang.syntaxHighlighterFactory

http://devnet.jetbrains.net/thread/292610?tstart=30



        <!-- Syntax Hilighting extensions -->
         <lang.syntaxHighlighterFactory key="Lua"  implementationClass="com.sylvanaar.idea.Lua.editor.highlighter.LuaSyntaxHighlighterFactory"  />
        <!-- OLD EP <syntaxHighlighter key="Lua"   implementationClass="com.sylvanaar.idea.Lua.editor.highlighter.LuaSyntaxHighlighter"  />-->

Implemented as:

public class LuaSyntaxHighlighterFactory extends SingleLazyInstanceSyntaxHighlighterFactory {
  @NotNull
  protected SyntaxHighlighter createHighlighter() {
    return new LuaSyntaxHighlighter();
  }
}