messed up encoding in HTML template when exporting

Answered

I use IntelliJ for actionscript. I create basic actionscript-only app and then it offers me to use html template. It's just ridiculous that it fails if I don't use ${swf} there, WTF?! Can't I simply have a normal page where nothing needs to be preprocessed or replaced?! The other point, that "template" processing code is pure crap, it destroys the template. And the way it does so is just pure mystery: it destroys only some characters, just doesn't make any sense to me. For your example I include sample project that you can use, I uploaded it to https://uploads.jetbrains.com/, name broken-html.7z. There I use utf8 html template. For the example I added some russian text so you can read what it means. In template (html-template/index.template.html) I have this text: "<body>${swf} Функция пробегает по всем плеерам...</body>" and processed result supposed to replace that ${swf} with "Main", but resulting text is: "<body>Main Функциѿ пробегает по вѿем плеерам...</body>" Note, the broken chars are broken second time in the forum message (after template is processed it's actually different), but you see chars aren't the same and it just random behavior there... why in the first place they are even touched, just doesn't make sense.

I've been using Visual Studio for around 15 years, and it's by far the best IDE imo, it has issues but at least stuff that I needed never had some weird roadblocks.

If in that sample project, try to remove ${swf}: it will fail (complete WTF?) the other point is that it creates a message:

Error:[broken-html]: HTML wrapper template must contain ${swf} macro that is substituted at compile time:
C:\RT\broken-html\html-template\index.template.html

How is that even possible that this text isn't selectable with mouse, message cannot be clicked to go to this file? What a joke.

The other point: as I say, I don't want any templates and that dumb error saying that "wrapper template must contain ${swf}". How can I simply avoid preprocessing, of any html templates and simply open existing html code that I've done myself? Or better, I'd prefer generated swf to be copied to my site directly and then open real live site in browser for debugging. Why aren't such obvious things aren't available: the only option in actionscript is to use some dumb template. Just doesn't make sense. At least it should have offered custom url top open browser, and allow build steps: after Main.swf generated in post build step I could run my custom commands to copy the file to mars or moon or anywhere else I want.

 

Then there is that complete craze with all kinds of windows going on: I see "Messages" window where I see messages that cannot be selected with mouse like normal text?!?!? Then there is "Even log" window, and funny part is that in both of them I don't show the most important part: what is being compiled or not.

Try to check Visual Studio when debugging C# or C++ and hover mouse over variables and how you can expand object properties and compare that to how intelliJ does that. I think it copies that weird way eclipse does that.

 

30 comments
Comment actions Permalink

and by the way... why can't I edit my forum posts? Why don't you add stackoverflow-style format (`markdown`) support?

0
Comment actions Permalink

 And the other point: why can't I open another instance of editor?! WHY?? eclipse behavior?

0
Comment actions Permalink

I can't reproduce the problem with the project that you have provided. What OS and AIR SDK versions do you use? What browser do you test in?

(tested with the latest AIR SDK from http://www.adobe.com/devnet/air/air-sdk-download.html).

> If in that sample project, try to remove ${swf}: it will fail (complete WTF?)

It's by design, see https://www.jetbrains.com/help/idea/2016.2/create-html-wrapper-template-dialog.html. ${swf} placeholder is required for this feature.If you have a use case where you must use a template without this placeholder, please submit an issue at https://youtrack.jetbrains.com/issues/IDEA and we'll consider making it optional.

> How is that even possible that this text isn't selectable with mouse, message cannot be clicked to go to this file?

You are right, it's a usability problem. Navigation was not implemented for this type of the error, I've reported a bug: https://youtrack.jetbrains.com/issue/IDEA-160079. Please follow it for updates. See https://intellij-support.jetbrains.com/hc/articles/207241135 if you are not familiar with YouTrack. Thank you for the report.

> How can I simply avoid preprocessing, of any html templates and simply open existing html code that I've done myself?

You need to change the run/debug configuration to open URL or local file instead of using the HTML wrapper:

 

> Or better, I'd prefer generated swf to be copied to my site directly and then open real live site in browser for debugging.

You can do it by configuring deployment (https://www.jetbrains.com/help/idea/2016.2/deployment.html) or by running some external tool/task before launch. Your custom tool can copy the .swf file to a web site root.

 

0
Comment actions Permalink

> Why don't you add stackoverflow-style format (`markdown`) support?

We are using a third-party engine that doesn't support Markdown, unfortunately. Many users asked to provide Markdown support for the community forums, but they are refusing to add it.

0
Comment actions Permalink

> why can't I open another instance of editor?

You can open multiple projects in different IDE frames. Use File | Open or New. Is that what you need? If not, please clarify what is "another instance of editor".

0
Comment actions Permalink

> Try to check Visual Studio when debugging C# or C++ and hover mouse over variables and how you can expand object properties and compare that to how intelliJ does that.

Please see

https://www.jetbrains.com/help/idea/2016.2/evaluating-expressions.html#d1657933e298
https://www.jetbrains.com/help/idea/2016.2/inline-debugging.html

0
Comment actions Permalink

Regarding "I can't reproduce the problem with the project that you have provided", I'm sure that the problem is there. I don't think it has anything to do with airsdk: airsdk compiles actionscript, that part works ok, HTML template is prepared by IDEA and this part is buggy and results in messed up html. In my case I open html directly: out\production\broken-html\Main.html and compare it to unprocessed original. I guess it works for you because your regional options are set to Russian code page by default for non unicode apps in windows?.. maybe that's where the bug is: code translates encoding incorrectly. To prove that this code is completely messed up I tried to save that tempalate file in other encodings (USC2 for exmaple), but in that case it doesn't work at all. I uploaded that project as broken-html-2.7z, you can open as see template html-template/index.template.html in text editor, but when you build it will say:

Error:[broken-html]: HTML wrapper template must contain ${swf} macro that is substituted at compile time:
C:\RT\broken-html\html-template\index.template.html

By the way, I use IDEA 2016.2, I didn't update yet to 2.2. I use Win 10.

 

EDIT: encoding issue verified. When I convert file to ASCI and set it to cyrilic encoding, then processing works properly and I get the same result. So, the problem is in handling encodings. First of all, IMO it shouldn't touch encodings: it's not multi-byte encoding then it should treat it like binary and replace ${swf} with "Whatever", otherwise I think it's going to be buggy. Imagine somebody saved html in some local code page, when you load it you cannot know if it's UTF8 or some local code page, if you try to modify it you won't get correct results, since you don't even know know what encoding was used. The other point: I used utf8, this has to work, I don't know how somebody managed to mess it up on your side.

0
Comment actions Permalink

"You need to change the run/debug configuration to open URL or local file instead of using the HTML wrapper:"

 

- obviously I tried that :) Did you? It doesn't work at all. Browser ends up opening "http:///" and I tried all kinds of stuff there. IMO it should support there local file: "C:\myshit\index.html" or web url: "http://example.com/". Regarding requirement of ${swf} - I get the idea, still, it must not be an error! A warning perhaps?.. what if I already hardcoded swf name in my tempalte, then it won't work? There are other template params that get replaces, like width, height, color etc, even though I don't see where they get defined (which is another WTF)

 

0
Comment actions Permalink

Try settings IDE and Project encodings to UTF-8: http://blogs.jetbrains.com/idea/2013/03/use-the-utf-8-luke-file-encodings-in-intellij-idea/.

>  I guess it works for you because your regional options are set to Russian code page by default for non unicode apps in windows?

I don't see how it would matter since the file is already in UTF-8.

> I uploaded that project as broken-html-2.7z

Yes, it reproduces the problem. To fix it I've changed Project encoding to UTF-16BE. Note that you need to close and open the project again for this setting to take effect (or restart IDEA).

So, Project Encoding setting should match the HTML template encoding.

0
Comment actions Permalink

Another annoyance by the way.. there is no clear project file for IDEA. So, I cannot open project simply from explorer. I don't put shortcuts to IDEA on my desktop an everywhere else, so to open a project it's a pain: I need to search to idea, open it, then from idea open a project. You should have some meta file that could open idea with one click, similar to visual studio where you get .sln that opens it all.

 

0
Comment actions Permalink

> - obviously I tried that :) Did you?

Sure:

1
Comment actions Permalink

> Another annoyance by the way.. there is no clear project file for IDEA. So, I cannot open project simply from explorer.

Vote for https://youtrack.jetbrains.com/issue/IDEA-80673.

0
Comment actions Permalink

> There are other template params that get replaces, like width, height, color etc, even though I don't see where they get defined (which is another WTF)

Please submit a request at https://youtrack.jetbrains.com/issues/IDEA.

0
Comment actions Permalink

> I don't see how it would matter since the file is already in UTF-8.

 

Exactly, I don't see how it's possible that it doesn't work in the first place. I checked and my encoding settings are OK: both are set to UTF8, that's what it should be. The template file is also utf8 (not from broken-html-2.7z, where I converted it to USC2 BE just for example) and I don't edit it in IDEA, I used other editor, so it shouldn't matter what settings I have in editor.

0
Comment actions Permalink

Sorry, can't reproduce the encoding issue.

0
Comment actions Permalink

>  Imagine somebody saved html in some local code page, when you load it you cannot know if it's UTF8 or some local code page, if you try to modify it you won't get correct results, since you don't even know know what encoding was used.

IDE knows this file encoding via the user setting or via meta charset tag:

0
Comment actions Permalink

Wow weird, I don't see dialog that you show. I have only this one:

 

Only after messing with all the menus I see where it is. That's SOOO WEIRD that project specific settings aren't shown in project settings! I right click project, select Properties (I don't know why it's F4 shortcut there, in VisualStudio layout it's been Alt+F7 for 20 years already) and in properties I don't see configurations for my project?? :) why is that even called project properties then?

 

0
Comment actions Permalink

I have identical encoding settings as you. In any case, source template is utf8, it cannot possibly end up with some chars messed up. Even if it was any other encoding, it shouldn't modify chars at all. And it does for me. I don't know why it works for you. I just installed IDEA and didn't so anything special: simple default clean install.

 

0
Comment actions Permalink

Not sure what you mean. The dialog where you set URL to open is the run/debug configuration, not the project or module setting. You can have multiple different run/debug configurations that will open different pages. It would be impossible if it was a project or module setting. Also IDEA project can contain multiple different modules (Ruby/Java/Python). that is why the settings you show on your screenshot are displayed for the module in the project structure and not in the project settings or IDE settings.

If you don't know how to edit run/debug configurations, please refer to https://www.jetbrains.com/help/idea/2016.2/creating-and-editing-run-debug-configurations.html.

0
Comment actions Permalink

> I have identical encoding settings as you. In any case, source template is utf8, it cannot possibly end up with some chars messed up. Even if it was any other encoding, it shouldn't modify chars at all. And it does for me. I don't know why it works for you. I just installed IDEA and didn't so anything special: simple default clean install.

OK, I've reported a bug at https://youtrack.jetbrains.com/issue/IDEA-160081 and asked the testers to reproduce it in different environments. Maybe the testers will have better luck than me. Please follow the issue for updates.

0
Comment actions Permalink

 Regarding encoding, I don't know what's going on, when opened the same dialog as you it didn't show anything for index.template.html (the encoding field was empty). Everything was set to utf8, I tried to rebuild and I got messed up output. Then I changed file encoding to UTF8 with BOM (thinking maybe for some weird reason IDEA thinks that BOM is needed), after that I went back to IDEA and now ti showed me UTF8 as detected encoding. I tried to rebuild and now output was correct finally.

Then I deleted all output, changed file back to regular UTF8, opened IDEA, built it again and now it works. Now I take that original zipped project that I submitted, extracted it to another folder, opened and off course the problem is there:

 

0
Comment actions Permalink

Please move the encoding issue discussion to IDEA-160081. Also check if you can reproduce it with 2016.2.2 (though, I doubt that it's version specific). For me the problem is still not reproducible using the original project that you have submitted.

 

0
Comment actions Permalink

I added there a comment: I was able to create two identical projects where one produces error and the other one is ok. Still, I think you said that even without that enconding.xml you got correct output, while I didn't. Could it be that your default code page in windows set to russian and that's why even without these encodings.xml it works ok for you?

 

0
Comment actions Permalink

> You can do it by configuring deployment (https://www.jetbrains.com/help/idea/2016.2/deployment.html) or by running some external tool/task before launch. Your custom tool can copy the .swf file to a web site root.

 

Many of my questions resulted in poor (my opinion) design of project settings and configurations. Here's my expectation (based on 15+ years use of visual studio and other IDEs): when I see project, I expect to be able to right click it and select there project settings, build it, clean it etc. If I rigth click, I only see module settings and there re setting for my actionscript project. I don't see anything else related to actionscript, so, I have all reasons to assume that there is no other way to configure anything. What a surprise that it appears that most of project settings are buried somewhere in multilevel menus. There has to be one stop project config that has allows configuring all aspects of a project. It's like "editor" for configuration files. Visual studio does shit load of complex stuff all in one simple dialog box (project settings) and there is no other place where some other project settings might located. I know it's not going to be changed on IDEA because everybody is used it, but IMO it's so messed up, when I open project structure settings, I see some settings related to that HTML wrapper template and nothing else and then half a day later, because I was pointed to ... what a surprise somewhere else there are also that html template settings?..

0
Comment actions Permalink

I understand that it's always hard to migrate to a different product that was built from the ground up with different concepts, especially after working with another product for 15 years.

To those who work 15 years in IntelliJ IDEA transferring to Visual Studio or some other IDE will cause even more pain.

And you are right, IDE settings/project structure settings/module settings/run-debug configuration settings are probably not going to change much and will not move between these categories.

What you see in the project structure for the flash module are the facets that allow to define multiple different build configurations. See https://www.jetbrains.com/help/idea/2016.2/build-configuration-page-for-a-flash-module.html.

It allows you to define different build options (like debug/release) or different architectures and operating systems for the same module. It seems logical that these settings are defined in the module configuration and not in the run/debug configuration or project configuration, or IDE configuration.

0
Comment actions Permalink

> Vote for https://youtrack.jetbrains.com/issue/IDEA-80673.

I voted there, but it's obviously useless request, it's been hanging around for 5 years, as if nobody wants that. Just doesn't make sense. The dialog to open a project itself makes me hate this tool before I even open anything. Is there command line params in idea so I could run it and tell it to open some project in new idea instance? I'd even write my own ".ideaproj" extension with empty contents that would trigger that idea command to open project where that .ideaproj file located, that would be good enough already.

 

0
Comment actions Permalink

Yes, you can specify the project directory as the command line parameter and IDEA will open it.

I also use Open Project Fast plug-in that opens the directory path in the clipboard as a project by a single hotkey.

0
Comment actions Permalink

By the way, it feels somewhat like eclipse, which is total failure IMO, was it at the beginning influenced by eclipse maybe and have that weird all over the place config that somehow considered "logical"?

It's good that android main dev tool finally moved on to intellij from eclipse. Even java devs who never used anything other than eclipse hated it :)

 

Anyways, there isn't much (my opinion) that can compare with visual studio, but IDEA is at the top IMO, at least it supports languages that aren't supported by visual studio, and debugging actionscript works (main point for me). I trying to get used it, but some stuff just outlandish: for example breakpoints window. WTF? standalone dialog, for real?? Why not a tabbed pane just like all others there? Something like that (in visual studio):

 

 

> You can do it by configuring deployment (https://www.jetbrains.com/help/idea/2016.2/deployment.html) or by running some external tool/task before launch. Your custom tool can copy the .swf file to a web site root.

 

I tried that, even here it's all messed up. There is at least 3 output windows for all kinds of different stuff. How is that when I add custom build step to copy compiled output to my remote drive all I get is super cryptic message box (why they are even used?!) and I don't see any clue WTF is wrong.

 

nothing is logged to any of the three or more windows. I switch every time between many windows to see where if the output

It seems that I'm trying to copy output.swf into my remote location and IDEA tries to run command like a regular executable?! :)) haha, wtf?? Shouldn't it execute in cmd context? instead of starting "copy" process :)).

The other point: there are a few useless macro when I try to definecustom build step, and there no single macro that defines build output?..

 

0
Comment actions Permalink

Use External Tool, cmd.exe as an app, /C .... as the command you want to run, $OutputPath$ macro for the output folder, add file name manually. Add external tool in before launch.

https://www.jetbrains.com/help/idea/2016.2/external-tools.html.

0
Comment actions Permalink

This is also wrong, who on earth adds build steps like that?! :) what a joke, it has to be simple multiline text field where you enter regular commands and IDEA should take that text and feed ti to CMD.exe or bash, no need for workign dirs, params, process names etc. Plain script filed. That's how it's done in visual studio.

That way for simple commands like copy a file it's very simple command, exactly what you'd write in shell. For external tool where you need to use some your own executable you just write it all the same way you'd run it from command line (it's also easier to verify manually the command, since you run exactly the same stuff). For working dir obviously default is project dir, but there is always `cd` to change the default.

I reported it: https://youtrack.jetbrains.com/issue/IDEA-160084

0

Please sign in to leave a comment.