[ANN] Tabifier version 4.3

Are you planning to apply your last changes in tabifier for IDEA production build (3.0.*, unfortunately I have to work with production build in current project)?. It would be very useful...

Hi Pavel,

I'm very sorry, but I don't have plans to retrofit the current version of the tabifier to 3.0.x. The Psi and OpenAPI changes are significant, and it would be a lot of work. I figure about the time I would be finished, Aurora would be shipping! :)

If anyone knew what the real ship date for Aurora will be, and it was a significant amount of time (say 4 months), I'd certainly rethink this. But last I heard they were hoping for November, and maybe it had slipped to January. That was probably all just rumor, though.

Sorry,
-Dave

Hey, maybe we could offer a "Tabification Service" for those folks who aren't using the EAP. You send me your source, I'll tabify it and return it to you! :-D

I'll have to give this idea to my marketing department! They haven't had much originality lately.

-Dave

Hey, maybe we could offer a "Tabification Service" for those folks who

aren't using the EAP. You send me your source, I'll tabify it and return it
to you! :-D
>

I'll have to give this idea to my marketing department! They haven't had

much originality lately.

Let's just expose a Web Service ;)

Guillaume Laforge

I've submitted the following feature request, but perhaps this belongs in the tabifier plugin. I would love tosee an option to turn partial chopping into complete chopping, before tabification.

so...


What do you think?

They've got that in the EAP already. Under Options...Code Style...Work in Progress..."Method call argument wrapping", choose "wrap always." (I suppose they could have hidden it a few levels deeper... :) Then just run the IDEA code layout tool before tabifying and you're all set!

-Dave

BTW - Tabifier seems to reset my code to having 4 spaces per tab rather than my desired 2 (as I have it in code layout).

Sorry for peppering you with messages but I'm focusing on my layout at the moment...

I don't appear to be able to use the align parameters option without selecting Align Expressions also. I get the following when I try...



But When I use both options I get the following (which is also not desired)...




what I'd like to see is...

Ooohhh...yuk -- messy! :) OK, I'll take a look.
-Dave

No, that's great. I love it when all the bugs come in at once! (So I don't have to release the plugin as often.)

I really like getting code examples like these; I just make them into another JUnit test. Real-life examples work better than my contrivances.

Thanks for the feedback,
-Dave

Here's another one for you.... ;)



This came when I tabified the whole file...

When I just tried to tabify the selection I got...

java.lang.NullPointerException

at com.wrq.tabifier.columnizer.DocumentParser.isCurrentLineBlank(DocumentParser.java:509)

at com.wrq.tabifier.columnizer.NestedParser.isCurrentLineBlank(NestedParser.java:115)

at com.wrq.tabifier.columnizer.ClassParser.visitClass(ClassParser.java:150)

at com.intellij.psi.impl.source.s.accept(s.java:72)

at com.wrq.tabifier.columnizer.DocumentParser.visitClass(DocumentParser.java:169)

at com.intellij.psi.impl.source.s.accept(s.java:72)

at com.intellij.psi.impl.source.p.acceptChildren(p.java:54)

at com.intellij.psi.PsiRecursiveElementVisitor.visitElement(PsiRecursiveElementVisitor.java:1)

at com.wrq.tabifier.columnizer.NestedParser.visitElement(NestedParser.java:53)

at com.wrq.tabifier.columnizer.DocumentParser.visitElement(DocumentParser.java:214)

at com.intellij.psi.JavaElementVisitor.visitFile(JavaElementVisitor.java:39)

at com.intellij.psi.JavaElementVisitor.visitJavaFile(JavaElementVisitor.java:156)

at com.wrq.tabifier.columnizer.DocumentParser.visitJavaFile(DocumentParser.java:224)

at com.intellij.psi.impl.source.z.accept(z.java:182)

at com.wrq.tabifier.TabifierActionHandler.tabifyPsiFile(TabifierActionHandler.java:329)

at com.wrq.tabifier.TabifierActionHandler.tabify(TabifierActionHandler.java:239)

at com.wrq.tabifier.TabifierActionHandler.executeWriteAction(TabifierActionHandler.java:158)

at com.intellij.openapi.editor.actionSystem.EditorWriteActionHandler$1.run(EditorWriteActionHandler.java:2)

at com.intellij.openapi.application.a.b.runWriteAction(b.java:7)

at com.intellij.openapi.editor.actionSystem.EditorWriteActionHandler.execute(EditorWriteActionHandler.java:5)

at com.intellij.openapi.editor.actionSystem.EditorAction$1.run(EditorAction.java:2)

at com.intellij.openapi.command.b.a.executeCommand(a.java:88)

at com.intellij.openapi.editor.actionSystem.EditorAction.actionPerformed(EditorAction.java:21)

at com.intellij.openapi.editor.actionSystem.EditorAction.actionPerformed(EditorAction.java:25)

at com.intellij.openapi.a.c.d.a(d.java:118)

at com.intellij.openapi.a.c.d.b(d.java:9)

at com.intellij.openapi.a.c.d.a(d.java:134)

at com.intellij.ide.s.dispatchEvent(s.java:57)

at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.run(Unknown Source)

Actually I'm after something extra. I don't want to have 2 parameters gratuitously chopped as in...



But if there's partial chopping going on already (as in my previous example) then I need the whole lot chopped in order for Tabifier to deal with things properly.

Does that make sense?

Would the "Chop down if exceeds right margin" setting do what you want? If the length of the method call with all parameters on the first line (with the method name and left parend) would exceed the right margin, then all parameters after the first are "chopped" off and moved to subsequent lines of their own. Otherwise they all stay on the first line.

not quite unfortunately. The situation I'm dealing with is that a previous person has partially chopped the parameters so they don't exceed the margin, but they mess with tabifier's head.

Hi Dave

Tabifier v4.3 works great. Damned good job!!!

- Jens

--
<= Nail here for a new monitor!

PGP fingerprint: 5209 35D1 772F 639A 4EB0 D475 3049 D1C1 718D E1D1
Get public key: gpg --keyserver pgp.mit.edu --recv-keys 718DE1D1

hi Dave,

This exception is happening every time I tabify...

java.lang.NullPointerException

at com.wrq.tabifier.columnizer.DocumentParser.isCurrentLineBlank(DocumentParser.java:509)

at com.wrq.tabifier.columnizer.NestedParser.isCurrentLineBlank(NestedParser.java:115)

at com.wrq.tabifier.columnizer.ClassParser.visitClass(ClassParser.java:150)

at com.intellij.psi.impl.source.s.accept(s.java:72)

at com.wrq.tabifier.columnizer.DocumentParser.visitClass(DocumentParser.java:169)

at com.intellij.psi.impl.source.s.accept(s.java:72)

at com.intellij.psi.impl.source.p.acceptChildren(p.java:54)

at com.intellij.psi.PsiRecursiveElementVisitor.visitElement(PsiRecursiveElementVisitor.java:1)

at com.wrq.tabifier.columnizer.NestedParser.visitElement(NestedParser.java:53)

at com.wrq.tabifier.columnizer.DocumentParser.visitElement(DocumentParser.java:214)

at com.intellij.psi.JavaElementVisitor.visitFile(JavaElementVisitor.java:39)

at com.intellij.psi.JavaElementVisitor.visitJavaFile(JavaElementVisitor.java:156)

at com.wrq.tabifier.columnizer.DocumentParser.visitJavaFile(DocumentParser.java:224)

at com.intellij.psi.impl.source.z.accept(z.java:182)

at com.wrq.tabifier.TabifierActionHandler.tabifyPsiFile(TabifierActionHandler.java:329)

at com.wrq.tabifier.TabifierActionHandler.tabify(TabifierActionHandler.java:239)

at com.wrq.tabifier.TabifierActionHandler.executeWriteAction(TabifierActionHandler.java:158)

at com.intellij.openapi.editor.actionSystem.EditorWriteActionHandler$1.run(EditorWriteActionHandler.java:2)

at com.intellij.openapi.application.a.b.runWriteAction(b.java:7)

at com.intellij.openapi.editor.actionSystem.EditorWriteActionHandler.execute(EditorWriteActionHandler.java:5)

at com.intellij.openapi.editor.actionSystem.EditorAction$1.run(EditorAction.java:2)

at com.intellij.openapi.command.b.a.executeCommand(a.java:88)

at com.intellij.openapi.editor.actionSystem.EditorAction.actionPerformed(EditorAction.java:21)

at com.intellij.openapi.editor.actionSystem.EditorAction.actionPerformed(EditorAction.java:25)

at com.intellij.openapi.a.c.d.a(d.java:118)

at com.intellij.openapi.a.c.d.b(d.java:9)

at com.intellij.openapi.a.c.d.a(d.java:134)

at com.intellij.ide.s.dispatchEvent(s.java:57)

at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.run(Unknown Source)



here's the code I selected that threw the error, though it seems that anything will do it...

Yes. Sorry, just SPOD (stupid programmer on device). Didn't realize it was happening that often.

I have a simple fix for it, but wasn't going to upload that by itself, and the code is in the middle of development. Can you wait a day or two, or is this severely hampering your productivity or enjoyment of life?

-Dave

:)

I can wait. My code can remain untabified till Monday! Enjoyment of life remains high.

cheers

Hey Dave!

I find myself waiting for the next version of tabifier the way I wait for the next EAP release! Do you have an ETA to satiate my lust? ;)

Hi Joe,

Just about to release another Rearranger plugin version, I expect today. I wasn't making as much progress on Tabifier as I had hoped; it will be probably a few days, though you could help me speed that up if you'd be willing to give me your thoughts as follows. They have to do directly with an example you submitted.

I've been struggling with just trying to figure out what desired behavior would be in situations where unaligned tokens wrap to a new line. My assumption is that the unaligned tokens would shift left to the appropriate indent level, which is (2 * editor indent size) for each level of nesting.

I've tried two different approaches at handling them and am not really satisfied with results of either. The basic design problem I am struggling with is how to align things at a given indentation level (or sets of levels) without unduly affecting the first (unindented) line. Before I invest time in another approach just to find out I'm not happy, I'd like some feedback, if you're willing.

Let me provide a few scenarios. Let's say that method call parameters are not aligned, and that the source contains nested method calls spanning multiple lines. But suppose that method call open and close parends ARE aligned. Each time a new method call is entered, the indentation level goes up (by 8 spaces in this case). The question is, are close parends on a given indentation level aware of open parends on a different indentation level? Can they be aligned to the open parends? Is this even desirable?

EXAMPLE 1

Where does the final close parend belong? Left shifted against the parameter, aligned under the open parend, or aligned after the open parend?

EXAMPLE 2

(So the parameters are essentially tabified in a block by themselves, namely at an indent level of 8 and independent of anything on line 1.)

What's the desired behavior? Line 4A aligns the close parend with the open parend. Line 4B aligns the close parend to the right of the longest parameter. Line 4C says, "hey, the open parend was at a totally different indent level, I'm not even aware of where it is."

EXAMPLE 3
Now take the same example and add another level of nesting (and another level of indentation).

Where should the close parends on lines 6 and 7 go? If the parend on line 6 aligns to the right of the end of line 5, should the parend on line 7 align to the right of that (line 6 parend)?

If you could suggest a rule that everybody likes and is easy to implement, I would be very happy, and the tabifier will release sooner. :)

Thanks very much!
-Dave

Happy to throw in my 2 cents...

example 1 - Personally, I'd like to see the right parenthesis be left justified against the end of the param. That looks the prettiest and neatest to me.

example 2

If possible, I'd like to see



but if that is still over the end of the line, then my vote would be for



and your final example...



I would again choose to tie the closing paren to the code.

As I said, IMHO...

thanks again Dave.
again, IMHO that dangling last paren is just an eyesore.

Hi Joe,

I think you'll be happy then with aligned params and unaligned close parends. Those cases work fine.

The tricky one is what an "aligned" close parend means when "unaligned" parameters are continued to a new line. The unaligned parameters slide left all the way to the indentation level. But it's not clear what the aligned close parend should align with. It sort of seems to me that it shouldn't align with the open parend (which was in a totally different indentation block) but should align past the end of any parameters. Well, I'll give it some more thought.

Thanks for your input!
-Dave

sorry I wasn't more help. I'm no use on that question as I would never choose that alignment option.

Hi Joe,

You've been so patient, thanks. I'd like to fix your other tab size bug before releasing.

Do I understand the problem correctly? You have a Java file that contains tab characters, and your settings in Code Style "Tabs and Indents" are: Use tab character (checked), smart tabs (doesn't matter), tab size 2, indent size 4, continuation indent (doesn't matter).

Then after you tabify, is there just one tab character instead of the two necessary to achieve indent = 4? Or are there two tab characters but indent = 8? Did the tab size setting actually change from 2 to 4, or do you just mean the tabifier treats every tab as if it was worth 4 spaces?

Thanks very much,
-Dave

Actually, I can't confirm what was happening then because my tab settings are now set to 4 and 8 and I can't use tabifier because it throws and NPE every time with the isCurrentLineBlank() problem.

However, if I recall correctly I wasn't using the tab character (I'm certainly not using it now).

Joe,

I've put a version of tabifier.jar up on the Wiki site at http://www.intellij.org/twiki/bin/view/Main/TabifierPlugin that has a fix for the NPE. I was hoping to fix all this stuff at once but it isn't converging as I hoped. :)

Sorry to have delayed you a week. I should have just fixed it right away.

I did not test this version as it was a trivial fix, but you know what that means. Keep the old version around (or you can download it again from the plugin manager.)

-Dave

thanks a lot! It's great. All is tabified again.

More thoughts... Sometimes, when tabifying there's one line that screws it up for the rest of them in a block. That line means that all (or some) of the others go over the end of the page.

What I'd like is the option to ignore that line in the tabification of that block so the others tabified nicely.

here's a code snippet...



in this example the top line makes the others less readable and makes them overflow the line.

Also, I think it would be great if I could set a MAX_ADJUSTMENT value for tabifier. If more spaces were needed to tabify that line than my specified value, then it would be considered a new tabification block.

Actually, come to think of it, that might take care of my first issue also.

What do you think?

thanks,

Joe

I think you're on to something there. I have been struggling with the increasing complexity required to "just make things look right." I've thought about this "conditional alignment" a little in the past (but have been too busy with immediate bugs to put enough thought into it.)

The problem is really to figure out what we're trying to achieve. Is it:

- not exceeding the right margin
- not inserting too many extra spaces
- grouping the wrong things together

and what to do about it. In your example, if we somehow identified the first line as causing "too many spaces" to appear in the next couple and forcing a line past the right margin, we could change grouping and align the second two together, like this:

which eliminates 14 spaces from each line.

Let's say the rule is, "if tabified lines are going to exceed the right margin, break them up into more subgroups (which are aligned independently) if possible to prevent that." I am assuming that lines don't already exceed the right margin -- the IDEA code layout tool takes care of that for me.

I could try different groupings just to see what the result was. Let's say the first thing to try is breaking the one big group into two small groups (with 3 lines, that's all you can do and still have at least two lines to tabify together.)

So I try doing "holdingStream" as one group, and "writer" and "reader" as another. We get the results I put above.

Then I try grouping "holdingStream" and "writer" together, leaving reader alone. That result is:

saving us 4 spaces from the "writer" line and 22 spaces from the "reader" line.

Is this an acceptable choice? How would I know which was "better"? The first grouping saves us 28 spaces, the second saves us 26 spaces. Perhaps that makes the first grouping better. It looks nicer to me for some reason.

Anyway, for groups of more than 3 lines, I could try breaking into 2 subgroups, then 3, etc. and taking the minimum number of subgroups that saves me the most spaces while preventing any line from exceeding the right margin. This would essentially guarantee that the maximum tabification possible without exceeding the right margin would be performed.

Thoughts? If you think you like it, we should do some code tabification prototyping by hand to see if it really will do what folks want.

-Dave

I think it would be great. As for thoughts on tabifying philosophy, here's something to start the ball rolling...

For a given tabification block...

Tabify normally, noting the extra spaces put into each line.
If any line has added more than x spaces then (of the line that Caused the spaces to be added and the line that Had the spaces added) break the tabification block on the line that is closest to the end of the block.

Rerun the process above for both tabification sub-blocks.

This may result in sub-optimal tabification, but it might do the job. In many cases, if too much extra spacing is required then the lines should probably not be considered part of the same block.

However, there are exceptions to every case and the rule that would result in the prettiest code would depend upon what options are chosen in the tabifier settings.

The MAX_EXTRA_SPACES value might be best applied to different parts of the statement also. ie. I may say that I don't want a line to have more than 15 spaces before the assignment operator, but no more than 5 spaces between parameters.

Things are getting sticky at that point though and I'm not sure how many different types of 'space sections' there are that might require a MAX_VALUE. Though if it's only 2 or 3 then you could use the breaking-blocks-into-sub-blocks rule if any of the max_values are exceeded.

What do you think?

Hmm, one thing that would be hard to determine in your algorithm is "the line that caused the spaces to be added." Several lines could combine to make lots of spaces added; e.g.

resulting in

I've thought about the "max extra spaces" setting, saying "if more than X spaces precede this aligned column, break it into a separate group." So more than 10 spaces before the assignment operator in the example above would have caused the first two lines to be regrouped as one.

-D