Expanding macros with LazyParseablePsiElement or TemplateDataElementType

Hi there,

I've been extending my M68k plugin (https://plugins.jetbrains.com/plugin/17268-mc68000-assembly-language-support source https://github.com/chrisly42/mc68000-asm-plugin ) and after trying for several days, I'm now stuck at a point where I dearly need help.

The assembly language allows definition of macros in the source that can be used for arbitrary text replacement (similar to #define in C language. E.g.

BLTCON_SET     MACRO
        move.l #(((BLTEN_\1+((\2)&$ff))|(\3<<12))<<16)|(\4<<12),bltcon0(a5)    ;BLTCON0/BLTCON1
        ENDM

Invocation of the macro is specified like this:

 BLTCON_SET D,BLT_A,0,0

... where the backslash placeholders are replaced by the parameters.

The language should be able to expand these macros by evaluating it, because these macros are frequently used to e.g. define symbols and labels and large blocks of code. For references and code flow analysis, it is crucial that these macros expand to statements as if they were written out at that spot.

I've got the macro definition and macro invocation covered in the plugin and it works nicely. I've been looking at GroovyDocComment and other uses of ILazyParseableElementType.

Thus, I have defined a M68kLazyElementType that inherits from ILazyParseableElementType and that can be produced by the Lexer and consumed by the Parser. However, it is an empty token (without content) that appears after the Macro Invocation:

MacroCall ::= MACRO_INVOCATION PlainOperands? <<parseMacroBody>>

@JvmStatic
fun parseMacroBody(b: PsiBuilder, l: Int): Boolean {
    if (!recursion_guard_(b, l, "MacroBody")) return false
    if (!nextTokenIs(b, M68kElementTypes.MACRO_BODY)) return false
    val r: Boolean
    val m = enter_section_(b)
    r = consumeToken(b, M68kElementTypes.MACRO_BODY)
    exit_section_(b, m, M68kElementTypes.MACRO_BODY, r)
    return r
}

I've got a M68kMacroBody PSI-Element that extends LazyParseablePsiElement, which is returned by the M68kLazyElementType.createNode() method.

fake MacroBody ::= MACRO_BODY {
    implements="de.platon42.intellij.plugins.m68k.psi.M68kLazyMacroBody"
    extends="de.platon42.intellij.plugins.m68k.psi.impl.M68kLazyMacroBodyImpl"
    elementTypeClass="de.platon42.intellij.plugins.m68k.psi.M68kMacroBodyElementType"
}

When I run this, it fails because somewhere on the road (when the children are added), it attempts to create the PSI element from the token and there is no factory method (Unknown element type: MACRO_BODY). If I patch the element factory to produce the PSI element on the MACRO_BODY

override fun createElement(node: ASTNode): PsiElement {
    if (node.elementType == M68kElementTypes.MACRO_BODY) {
        return (node.elementType as ILazyParseableElementType).createNode("foobar") as LazyParseablePsiElement
    }
    return M68kTypes.Factory.createElement(node)
}

... it fails because the visitor wants to add the element, calls ensureParsed(), detects that there is no parent assigned yet and shows the error "Chameleons must not be parsed till they're in file tree: M68kMacroBody".

I'm not sure I'm following the right path here. I've come across TemplateDataElementType, but seems to be a different but maybe related mechanism.

Maybe I haven't understood yet how these Lazy Psi elements get their parent as the it is not specified during creation of the node, unlike normal Psi elements.

Do you have any suggestions on how I can implement the macro expansion the right way?

Thank you!

Cheers

Chris