Formatter NoneIndent

已回答

创建于 2022年09月29日 16:01

Hello,
I'm trying to build a Formatter, but can't figure out Indents.

When I simply try to build blocks, giving every single one Indent: NONE, it still formats to 8 spaces.
I suppose, that it has something to do with ASTree.

The tree in screenshoot has WhiteSpace character to those 8 spaces, before it is NewLine WhiteSpace and Colon, after is "Get" token... but bnf grammar is little different, new line is actually a separate token, including an IndentToken, this has size 0 and is not visible there, there screen of parsed Tokens before AST:

I'm not sure, why those 2 tokens are not in resulting AST, but I suspect, they are the reason for wrong indentation.

Simplified grammar for it looks like:

varDeclaration ::= VAR IDENTIFIER setgetDecl
setgetDecl ::= COLON NEW_LINE INDENT getDecl DEDENT
getDecl ::= GET COLON stmtOrSuite

Does anyone have an idea what I might be doing wrong, so formatter is this broken?

Or is it even ok, for a grammar to check newLines & indents like this?

6 条评论

Karol Lewandowski

创建于 2022年09月30日 07:09

Hi,

Are the whitespaces significant in your language? Is there any need for NEW_LINE or INDENT tokens? Usually, they are unnecessary and only needed in languages like Python, where indents matter.

H Raven

创建于 2022年09月30日 07:32

Yes, unfortunately this is Python like language, where whitespace indents specify code blocks.

Valid if/else is like:

if true:
    dosomething();
else:
    somethingelse();
outsideOfIf();

H Raven

创建于 2022年09月30日 16:10

I've tried to mess with it a little bit, but still no success.

All empty token (WhiteSpace, NewLine, Indent, Dedent) are ignored when building child blocks.

I've also changed Lexer, because I thought that the reason can be, that there was IndentToken, then WhiteSpace and only after it GetToken, so now the order is reversed - first WhiteSpace, then Indent:

But still it adds 12 instead of 4 spaces even when (well it seems that 8 spaces there always, while last 4 differ on settings -> so if I set indent to 2, there's 10 spaces total, even when in BlockTree is only one block having a NormalIndent, while all the others have NoneIndent.

H Raven

创建于 2022年10月02日 08:23

Does anyone have at least an idea, why formatter would force 8 spaces, even when all blocks have NoneIndent?

H Raven

创建于 2022年10月04日 19:39

Ok, I've found the culprit I guess:

This SETGET_DECL node does contain both colon at the end of the line and definition on the next line -> so those 8 spaces where just where this block started.

But I still don't know how to solve this issue -> is it completly wrong, that 1 block is on multiple lines or is there some way to say to ignore this?

Problem with splitting is, that when there is not that "get: ..." line, there should not even be that colon. :/ That's why it's a single block

H Raven

创建于 2022年10月06日 07:35

Finally solved it today,
All I had to do, was to skip that element over multiple lines, when building child blocks, and instead build it from it's children.

请先登录再写评论。