Parsing Techniques for big files ???

Hi,

Are there any techniques to ease the parsing pain of big files ~ 10/15.000 lines ~ 3-5000 statements per file ?

Usually a statement in this language has between 4 and 10-15 child nodes.

Sincerely,
ToMiC

3 comments

Avoid very deep and very wide trees - they will cause you performance
problems later.

0


Hmm,

What would the "very deep/wide" mean ? Is 5.000 node too wide ? What can
I do if that's how the language is built ?
Are there any ways to create child nodes on demand ?

Sincerely,
ToMiC

Avoid very deep and very wide trees - they will cause you performance
problems later.



0

> What would the "very deep/wide" mean ? Is 5.000 node too wide ?

I think it's enough to cause you problems.

> What can I do if that's how the language is built ?

You can introduce artificial tree nodes to group your statements. I.e.
instead of:

statements
- s1
- s2
- s3
- s4

you can parse into:

statements
- statementGroup1
- s1
- s2
- statementGroup2
- s3
- s4


That way you'll get logarithmic growth of tree size.

In any case you can try linear structure first, and in the case of
performance problems caused by tree navigation, optimize it this way.

> Are there any ways to create child nodes on demand ?

Yes, there is. See IChameleonElementType class.

0

Please sign in to leave a comment.