Multimodule Flex Maven

Hello.

I have a large (50+ module) flex/java/webapp project that builds with maven.  While I can usually get it to eventually work, we waste hours every time the dependencies change, with no real understanding of why it doesn't build in idea but builds fine from the command line.

  • Reimporting poms whenever the project structure changes takes 8-10 minutes and often results in build errors or "can't find compile config."  (we have configurationReport=true in our poms).  We usually have to run the "force reimport" multiple times in a row.  Sometimes this is solved by running a command line build outside idea, but often it's necessary to delete all the IML/IPR files and start over.
  • Even after reimport, we often see all sorts of good-code-is-red errors.  Idea sometimes stops prompting for imports (code is red, no import popup), even though we can ctrl/command click on the actionscript symbol and go to it's source.
  • The flex configuration reports generated by idea often do not match (are missing dependencies or point to the wrong dependencies) those generated by the maven command line tool, even when we set our M2_HOME in idea to the same as the command line (this one makes no sense to me).  Even if I turn off the config reports in the poms, idea still generates them (incorrectly) !?  If we could just disable idea generating these files and generate them manually when the config changes, we would be so much faster!
  • Config reports are not generated in dependency order, causing particular problems for link reports and rsls.
  • IDEA's war artifacts don't correctly include the swf artifacts (instead include the zipped classes dir, named .swf); manually adding the swf file to the war doesn't cause the war to rebuild when the flex project changes.
  • IDEA doesn't install generated artifacts in the local maven repo; so if we execute maven goals in our run configurations, they don't get the latest outputs as made by idea.
  • We could build by running maven in idea; but then we loose the speed advantage from idea knowing what's changed; this means a 3.5 minute build every time we change a single file.  This doesn't address the good-code-is-red issues (which we can only fix by endless cycles of reimport and command line builds).


We spend literally hours trying to get idea to reconfigure after changing dependencies.

I'm working on putting together a sample project that demonstrates this but it's quite tedious to replicate all these issues; is there any way that we could maybe provide our existing source code under NDA so that you could see what's going on here in a real project?  I've attached the poms to
http://youtrack.jetbrains.net/issue/IDEA-66875 .

This is so close to being seemless, it seems like maybe a couple of days of your effort on a real-sized flex/maven/java project would clean up some of these last niggling bugs.  We have 20 developers who are seeing these issues.

Thanks for making a really great IDE!

-c

http://youtrack.jetbrains.net/issue/IDEA-66874
http://youtrack.jetbrains.net/issue/IDEA-66875
http://youtrack.jetbrains.net/issue/IDEA-66876

6 comments
Comment actions Permalink

Hi Christopher,
thank you for your feedback! We'll do our best to resolve your issues. Some first considerations are:

  • Maven reimport should take seconds, not minutes (only first import can be long, other should be quick). Is the longest part of reimport process "Generating Flex config file" or some other?
  • We suggest not to put configurationReport=true into pom because in this case mvn install generates incorrect Flex configs (dependencies are resolved into local repository rather that to project target). configurationReport should be generated only when doing mvn compile
  • For some technical reasons as well as possible bugs IDEA is not always able to generate 100% correct Flex compiler configuration files. The most reliable way is mvn compile -DconfigurationReport=true from command line. In case of 'good code is red' File | Synchronize can be used to make sure that IDEA picked up all changes. Close/open IDEA project is the strongest 'reset' you should use. You don't need to consequently force reimport or compiling from command line several times in a row.
  • Currently it is impossible to ask IDEA not to generate Flex configs on reimport. Thus it is possible that IDEA rewrites correct configs with incorrect ones. We'll add ability to disable configs generation: http://youtrack.jetbrains.net/issue/IDEA-56002
  • You can disable 'Clean output directories on rebuild' at File | Settings | Compiler to make sure that Flex configs and other auxiliary files are not accidentally removed
  • There's no feature in IDEA to automatically include SWF in the artifact yet. So you need to edit IDEA's artifacts manually. Artifact editor allows to create any file structure. Check 'Build on make' option in artifact editor or set 'Before launch: build artifact' at Run configuration setting to make sure that artifact will be up-to-date.
0
Comment actions Permalink


  • Maven reimport should take seconds, not minutes (only first import can be long, other should be quick). Is the longest part of reimport process "Generating Flex config file" or some other?


It really does take minutes, every time.  Definitely the compiler config is the longest part by far.  Is there some way to generate the compiler config without actually running the compile?  I see flexmojos maven goals for generate-config-swc and -swf but don't see a way to run it on an aggregator that may include both swfs and swcs.  I'm not sure I understand what this process really does.  You're relying on flexmojos to parse the configuration from the pom and turn it into a config report you can run the compiler against?  Why not just parse the options from the pom directly?  Even so, why does parsing the options from the pom have to take so long?  What's with all the stuff in target/classes?  Is that to speed up compiles by caching font stuff, etc?  In other words: why do you need the config reports and why do they take so long to generate?


  • For some technical reasons as well as possible bugs IDEA is not always able to generate 100% correct Flex compiler configuration files. The most reliable way is mvn compile -DconfigurationReport=true from command line. In case of 'good code is red' File | Synchronize can be used to make sure that IDEA picked up all changes. Close/open IDEA project is the strongest 'reset' you should use. You don't need to consequently force reimport or compiling from command line several times in a row.


I think the good-code-is-red is caused by idea not respecting transitive swc dependencies the same way the maven command line tool does.  When you compile a swc that depends on another swc, you've implicitly imported those symbols into your swc; but the maven dependencies are still transitive: if my project depends on A.swc and the A.swc pom specifies B.swc as a dependency, maven on the command line still includes B.swc in the compile; the config reports generated by idea do not; so I get errors about missing symbols for anything from B that A didn't use.  See IDEA-66875 for an example.


Could you please explain "some technical reasons?"  Why does idea not use exactly the same flexmojos code that runs when the config reports are generated from the command line with mvn?

  • Currently it is impossible to ask IDEA not to generate Flex configs on reimport. Thus it is possible that IDEA rewrites correct configs with incorrect ones. We'll add ability to disable configs generation: http://youtrack.jetbrains.net/issue/IDEA-56002

This would be really useful!

  • There's no feature in IDEA to automatically include SWF in the artifact yet. So you need to edit IDEA's artifacts manually. Artifact editor allows to create any file structure. Check 'Build on make' option in artifact editor or set 'Before launch: build artifact' at Run configuration setting to make sure that artifact will be up-to-date.

The problem is that manually inserting the swf in the war doesn't cause the war to depend on the flex facet; so changes to the project don't cause the war to be rebuilt and changes to the code don't cause the right flex modules to be rebuilt.  In the flex run configuration, I arbitrarily select one of the flex swfs (there are many, due to runtime loading) as the "Run configuration for module" but none of the other ones are built.  In idea's artifacts there's no dependency between the war and the flex, so only the one swf that I selected as "run configuration for module" is rebuilt; none of the other ones are.  I have to manually trigger a "make" of the full project.  If I forget to manually "make," then I don't see my changes in the running app.

How else can we help track this stuff down?  Would an real sized (20-30 flex modules) example project be helpful?  I could also cut down the poms to not include any deps not in a public repo?


Thanks again,


-c

0
Comment actions Permalink
  • IDEA uses generate-config-swc and generate-config-swf goals to generate configs when importing pom. These goals to not run real compilation and should work fast. For projects that we used to test this functionality first generation takes ~5-10 seconds, 2nd and further - few milliseconds. And this should not depend on project complexity. We'll try to figure out why performance is so bad for you.
  • 'Good code is red' is not connected with config generations directly. Generated configs are used for compilation. Code highlighting is done according to IDEA project model that is imported from your pom. (Truly speaking generated configs also affect highlighting , but only manifests/namespaces and conditional compiler definitions are taken from configs. Dependencies for highlighting are taken from IDEA project model). And as far as I see when project A (swf) depends on project И (swc) which in turn depends on project C (swc) then in IDEA we'll see at Dependencies tab of imported module A that it depends both on B and C. So highlighting is good for me whereas compilation fails because of incorrect cionfigs (unless I generate configs by mvn compile -DconfigurationReport=true).
  • As for 'technical reasons' - for example see http://youtrack.jetbrains.net/issue/IDEA-62430
  • I figured out a sort of hack to make sure than when you run Flex run configuration all other modules that will be loaded in runtime are also built. In IDEA module settings, Dependencies tab add dependency of one SWF module on the other with 'Runtime' scope. This will not affect highlighting/completion or compilation, but will guarantee рефе dependent module will be built (if changed) when launching run configuration. Problem remains that maven reimport will kill manually set dependency. You may try to play with 'runtime' dependency in pom and check if IDEA imports it in the same way.


Of course sample project, big or small, would be great (small is better :)). You can contact me directly by email firstName.lastName@jetbrains.com

0
Comment actions Permalink

About speed: If your project contains many snapshots (artifact with snapshot version), don't forget to set "Snapshot update policy" = "Do Not Update" (see Settins -> Maven).

0
Comment actions Permalink

We're still fighting with this. In 106.447, with the fix to http://youtrack.jetbrains.net/issue/IDEA-66875 (thanks!) I can at least reliably reimport the maven project and still have idea build.  This is great!  However, the reimport is perhaps even slower than previously; about 15 minutes for this build.  Profiler snapshot attached to http://youtrack.jetbrains.net/issue/IDEA-67994 .

Any progress on http://youtrack.jetbrains.net/issue/IDEA-56002 ?  That would be really helpful.  Alternatively, would you consider open sourcing flexmojos idea support so we can help?

Also, we've attempted to upgrade to flexmojos 4 (to fix problems with asdoc) but we can't because the swf compiler configs generated by flexmojos 4 don't include the main mxml file path.  See http://youtrack.jetbrains.net/issue/IDEA-67993 .

I'm unfortunately not going to be able to give our source code.  I'm working on putting together a non-toy-sized project that you can play with.

Finally, in talking with velo of flexmojos, it seems there's an alternative way to generate the compiler configs that he added for jetbrains specifically in version 4. Are you aware of this?

Thanks,

-c

0
Comment actions Permalink

However, the reimport is perhaps even slower than previously; about 15 minutes for this build

Will take a look at the snapshot and comment the issue. Does setting "Snapshot update policy" to "Do Not Update" has an effect?
Any progress on http://youtrack.jetbrains.net/issue/IDEA-56002

Will fix it for 10.5 release

As for Flexmojos 4 - indeed, it is not fully supported now because IDEA doesn't fully support Maven 3. See Vladimir's comment to the issue.
0

Please sign in to leave a comment.