Grammar-Kit explicit handling of spaces and ambiguity
Hi there,
I'm working on an assembly language support plugin and about to bang my head against the wall after several days of fruitless attempts to Get Things Right(TM).
The language has some things I have a hard time to get working in a BNF / Lexer way.
Legal examples of defining a label:
GlobalLabel
GlobalLabel:
GlobalLabel::
GlobalLabel:
GlobalLabel::
.locallabel
locallabel$
.locallabel:
locallabel$:
.locallabel:
locallabel$:
The label itself is optional on the line. After the optional label, an assembly instruction may be posted e.g.
GlobalLabel: move.l d0,d1
add.l d0,d1
rts
With the Parser skipping the whitespaces I explicitly defined in the BNF to distinguish lines with spaces before the first token it makes it impossible to distinguish the various forms of legal label definitions and assembly mnemonics.
To make things extra complicated, one can also define macros (via preprocessor keyword) and use these instead of assembly mnemonics, e.g.
PUTMSG MACRO
someinstructions
ENDM
PUTMSG 10,<"Foobar">
The ambiguity between assembly language mnemonics (which I don't want to define within the BNF) and macro calls with a different syntax for the parameters makes parsing hard.
I'm stuck here, but I guess I will need to write a manual lexer to handle these cases? Or is there a less awful way?
Thanks, Chris
Please sign in to leave a comment.
I've uploaded the BNF under Upload id: 2021_07_14_NMXr6kRw8hH2vFgD (file: m68k.bnf)
Do you want to join my efforts to build m68k Assembly support :-) https://github.com/YannCebron/m68kplugin
Oh my. Your work never showed up in my plugin searches. Seems like you have been working on your plugin without a release yet? Since I was struggling with coding in CLion for more than a year, I wanted to give it a try myself. I think I might have a different approach on some things than you and I want to check how far I will get. You should get a version to the marketplace sooner than later. I think ppl (retro and demoscener) would appreciate.
As for the issue, I think I now have understood that I need to be adding states to my .flex-File and remove all whitespace-stuff from the BNF.
It's not published anywhere yet as I still want to fix some missing pieces and quirks before going public.
All the label/whitespace/macro stuff indeed is quite tricky to get right (I'm still missing some bits and pieces in this area) and needs thorough testing. I ended up with a more complex Lexer than I ever wanted to have, but there's no way around that.