I'm one of the developers of the GLSL plugin, which adds support for a C-like language to IntelliJ/CLion. The language includes a C-like preprocessor, and I'd like the plugin to support understanding code which relies on #define.
My current solution is to use a PsiBuilderAdapter which overrides getTokenType/getTokenText/advance to automatically add phantom tokens to the parse tree which is builder with a PsiBuilder. This partially works but it means I can't ever replace the token text or output ForeignLeafElement tokens like CLion does.
So I figured that a better way to handle this would be to use a DelegateLexer instead. My current implementation of this, based on LookAheadLexer, is here. It generates the tokens correctly (ie. it generates the same tokens CLion does), but something seems to go catastrophically wrong and many exceptions are repeatedly thrown.
So how does CLion handle this? I was hoping a developer would be able to shed some light or point me in the right direction.