Are modules and projects redundant?
Of the many praises/criticisms of the new modules support, three in particular stand out:
1. Adds too much confusing complexity
2. Adds great flexibility for large projects
3. XXX should be configurable on a per-project basis
What if there were no modules? What if there were just projects, but projects could depend on other projects?
Basically, promote the concept of "module" all the way up to a full-fledged "project". Then add all the features unique to modules, like dependency checking and resource sharing, to projects.
Please sign in to leave a comment.
Russell Egan wrote:
Excellent plan! And so simple...
Replace "project" with "module" in your post, and that is exactly what
currently happens. Unfortunately, whatever you call those entities
(projects, modules whatever), adding "all features unique to modules"
takes some time.
Friendly,
Dmitry
--
Dmitry Lomov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"
Perhaps I just misinterpreted the course of action you guys took to implement modules. It seems to me:
1. You created a new entity, called "module", which was a very limited subset of the entity "project"
2. You implemented the dependency feature
3. You're slowly porting the features of "projects" to "modules" until the two entities are essentially the same.
I was suggesting that maybe it would have been more intuitive to users to just add the dependency features to projects. Ultimately, I think this would be great (eliminate either "projects" or "modules" and just have one thing).
Russell Egan wrote:
If we followed your advice, we would have
1. created a new entity, collection of projects
2. Implemented a dependency feature
3. Slowly made all features collections-of-projects aware.
I am afraid one needs to have a "collection-of-projects" entities.
Otherwise many things will not work.
Friendly,
Dmitry
--
Dmitry Lomov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"
Egan, I must agree with Dmitry.
Users typically need an entity above "projects" (using your terminology).
I don't want the dependencies to be automatic. I want to specify explicitly
what parts of the whole system I'm going to work with. That would be a
super-project. And, from that, it is just renaming step to modules and project.
(Well, along this line, I still wish I was able to specify just one
package to be in the module, but that is another story.)
r.
Russell Egan wrote:
But the only property of "collection-of-projects" is that it can contain/depend on other projects.
It's like MutableTreeNodes. All nodes have the capability of containing nodes. Whether the node is a leaf or not is determined solely on whether it contains children. You don't need two entities to represent containers and leaves. Isn't this another classic application of the Composite pattern?
>But the only property of "collection-of-projects" is that it can contain/depend on other projects.
From the sounds of it, there is going to be "project default" information (JDK, for instance), and there are already project libraries. More importantly, the project object manages central administration of the module objects. If you rename a module object, any references to the module will be renamed as well (when renaming is supported, presumably). If projects depended on projects, there would be no place to put this sort of central administration, and renaming your project could break other projects which you don't even know about.
Not advocating one way or another, just pointing out the complexities.
--Dave
I think they'll face renaming problems anyway. Modules can be shared between projects, so presumably renaming a module while it's opened in one project will break other projects which rely on it.
There is something to be said for default settings, like JDK, VCS, code format, etc, being inherited when not specifically overwritten by a child. While modules do not need to be autonomous, and therefore can leave these settings blank, projects must be autonomous, and so must have default values for all these settings. This would mean you'd need to set each of these options to "Inherit from parent" or "Override children" or something. That would be tedious.
"Russell Egan" <russegan@email.com> wrote in message news:26036536.1069264147924.JavaMail.itn@is.intellij.net...
>
>
>
modules, like dependency checking and resource sharing, to projects.
Just to add some fuel to the flame :)
Here are two excerpts of my two months old postings in another forum.
-
... There are only two basic fundamental/orthogonal concepts we have to deal with:
1. Project (don't confuse with IDEA project)
This is essentially a description of your final software product.
Includes:
1.1. Base project if any
1.2. References to other projects this project depends on.
1.3. References to all necessary source/resource files
1.4. Build procedure:
1.4.1. Compiler options
1.4.2. Custom pre/post build steps (multiple output paths go here)
1.5. Run/Debug/Test environment settings (ideally, might also define pre/post steps)
2. Workspace (don't confuse with *.iws)
NOTE: This does not have anything to do with actual projects and/or software product!
This is (ideally) is your IDEA's desktop, merely a container for projects
you're working with at the moment. This is your current view on your developer's
environment. It does not produce results, but controls how results are produced.
Includes:
2.1. References to included projects
2.2. Active/current project marker
2.3. All local user/machine/environment specific info which is not shared
between developers. (For example you can include some project into
your workspace, but exclude it from refactoring/build etc.)
That's it!!! There is nothing else!
Now if you do have some specific reasons (I can't imagine any) why you want to treat
someone else's code differently, then you can split projects into projects and libraries.
If this is the case I'd rather suggest to allow projects to have tags (i.e. free form text strings
attached) and let IDEA to recognize some predefined set of tags (that is, you can have "Library" tag)
and act accordingly (show in different color in workspace, for example).
Remarks on Project/BaseProject relationships (1.1.).
The main purpose of this is to be able to mutate projects . For example,
if I have a project with some output path and I need to change it in my
local environment I'd derive a new project, override the output path and include
it into my workspace instead of original version.
The extent of allowed "mutability" can be different for different project attributes, but
I'd expect that generally prepend/append/substract/substitute types of mutations
allowed.
Remarks on Custom pre/post steps (1.4.2.)
The "multiple output paths" functionality is nothing else, but a smart post build step.
Why don't call it by it's natural name then? And why don't allow any pre/post build
steps to be defined? This is a natural extension/grow point for the limited built in
functionality. I can run a smart Ant script as a pre step to generate the sources or do whatever
I want with build results in post build, not to mention to perform packaging, deployment etc..etc. It would be
really great if there was a way to communicate with IDEA about build results. For
example if there was a macro %recompiled.class% meaning all class files actually
recompiled by IDEA build process then I'd define my post build as
copy %recompiled.class% MyDestinationPath
and avoid copying the whole 10000 class files every time I do a build.
Remarks on workspaces (2.)
The level of smartness here can be also rather high. For example it can
seamlessly mutate (i.e. create local mutated project and use it instead of
origianl) projects in case of simpe changes (like compiler options or
output paths, or base output paths etc.) It can either keep explicit files
for mutated projects or store it in my local .iws instead. It can also track
project compatibility. For example if base project major version number
changes then all local mutated projects can be automatically invalidated
if desired.
Now tell me what real life configurations this simple scheme can't reasonably
describe? Is there any confusion/misunderstanding?
Of course we can work out some implementation details, but my primary
intention was to illustrate the idea. It can be simple. It MUST be simple.
-
........
The major missing part is a clear concept of workspace. I'm still confused
if your concept of project is supposed to substitute it somehow. Even if it's
true, the naming is totally misleading. If you're intentionally trying to get rid
of concept of workspace (or desktop if your prefer) then it's a terrible design
flaw. No one I know works with just a single project at a time these days. You
always have tree-five related (or unrelated) projects on your desktop
(workspace). The main reason for this is tight code integration and intensive
code reuse between projects. When I need to refactor something
in some of my project (your module?) I need to be sure that the refactoring
will propagate everywhere correctly so I need these projects on my table
within easy reach and with full refactoring support. Other developers might have
another working sets, so the concept of workspace is absolutely vital and
seems to be totally missed now.
What is the difference between project and module anyway? The typical
case scenario is that one of modules of my project is a someoneelse's
project. Do I need to redefine his/her project to be my module? Can I use his
project as a module? Questions go on and on just because there is no
reliable way to assign static roles in dynamic context.
The same goes for libraries. They seem to be projects (or modules?) with
disabled change/refactoring/compile support. I agree this is in most cases the
normal way of events. Unfortunately the real life is quite often goes abnormal.
What if I need to fix a simple error in a library (or even easier, recompile it for
debug as I often do for swing for example)? Correct me if I'm wrong
but in you case I'll need to remove the library from my project, redefine it as
a module, include again, recompile and turn everything back. Will I do all this
hassle? No way! I'd rather define everything as a module in the first place
and keep it this way at all times, probably overloading IDEA with tons of
unnecessary information and getting a lot of spam while refactoring.
BTW this is one of my biggest problems in 3.0.5 since I have JDK in my project
recompiled once and excluded from compilation. When I try to change
a string constant chances are that I'll get a lot of crap from JDK in preview.
In my model I'd just temporary clear "read-only" flag from the JDK project,
recompile and then put read-only flag back again. I don't need to change
the type of the entity. Anything which results in executable code is a
project anything else is not. Simple isn't it?
So the main difference is that my model is homogeneous. It uses
minimal number of concepts (just one to be precise) to dynamically handle a
situation of arbitrary complexity while in your heterogeneous model you
have to decide in advance what is what despite the fact that you can't
see the difference or you know that it's all going to change soon.
My model might be a bit more complicated to implement (although I doubt
it) but it's surely much less complicated conceptually and no doubt will result
to much simple UI and documentation (I can propose some idea of UI if you wish).
Yet again, you're trying to provide a "universal solution" to the problem which
you hope will suit all needs, cluttering UI with stuff for every occasion.
I know I'm weird but I'm not using JUnit to test my applications and I don't
want to see "Test folders" in my projects (although I reserve the right to change
my mind). The same goes for Web apps and for million of other individual cases.
The very attempt to satisfy everyone with the brute force is a guaranteed failure.
You don't have to solve the problem, you must not solve it, but allow for
everyone to create and maintain their own local solutions optimized
for their local needs.
BTW a set of project templates for typical cases (like web apps) would be
of a great use here, I think.
--Oleg
After reading this I got the impression that JDK settings could be set at the project level - I believe this is not the case, I find it a little annoying that I need to specify the JDK for each module - although I can understand the reasoning behind it. I assume in time that I will be able to specify JDK once per project...
cheers
dim
Oleg,
The more I think about it, the more I like your idea. Nevertheless, I
would like to be able to add "sub-projects" (in your meaning of
projects) to a "project" in the "workspace"; just to group them.
How we can apply this to IDEA? First, a current "IDEA-project" must be
replaced by a "workspace". "Projects" are more independent of the
"workspace" than current "modules" are from "IDEA-projects". One can
create "Projects" and add them to the "workspace" or to other (parent)
"projects". That's it.
Tom
Dmitri Colebatch wrote:
Coming soon.
Friendly,
Dmitry
--
Dmitry Lomov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"
"Thomas Singer" <idea@NOregnisSpam.de> wrote in message news:bpkl58$l6h$1@is.intellij.net...
>
I'm not sure about your meaning of sub-projects. My meaning is that
a sub-project is an another project the current project depends on,
i.e. if I have project A depending on projects B and C then from the
project A point of view B and C are it's sub-projects. So for any given
project you can build a tree of it's sub-projects. Evidently, they must be
compiled in leaf-to-root (deepest first) order.
All above does not prevent you from working with a "forest" of projects
(in your workspace) providing that you have a single "current" project.
Nothing prevents you from adding as many related projects to your
workspace as you want and group them as you wish. Any project
can participate in a number of different hierarchies and/or selected as
"current" at any time. I think that even some syclic dependencies can be
handled this way.
>
This is just a matter of convenience. I would expect that if I have added
some project to my workspace, it's sub-projects would be also added
automatically, but this is not a requirement. They may remain (or made)
invisible in the workspace.
As per independence from each other, then my idea is to make them
as loosely dependent as possible. Basically, I'd implement both projects and
workspaces as loose maps, i.e. every entity consist of a number of tags with
values (multiple values are allowed). Then I can use tag inheritance for any
given hierarchy. For example, here is my workspace:
workspace
+ project A
|- project B
|- project C
+ project D
|- project F
|- project B
As you can see, sub-project B is shared between A and D.
Now first of all I'd define tag Library="Library B" in project B and have IDEA to
dispaly project B in different font/colour and as "Library B" rather than "project B".
Then, let's suppose I need project A to be compiled with JDK 1.1 and project D
with JDK 1.4. In order to do this I'd define tag JDK= in project A and JDK=]]> in project D and do not define this tag in B so forcing
this tag to be inherited from the parent (i.e. either from A or D respectively).
I could also define JDK tag in the workspace itself providing a default JDK in case if
some project does not have it defined (i.e. does not care).
This simple scheme gets just a little more complex (a few more tags are needed) to cover
the case when I can treat JDKs themselves as projects/libraries. Allowing sub-tags
(i.e. Library.JDK or such) and combining this with a simple template processor which
can process expressions like --> "c:\Java\j2sdk1.4.2" we can get
exceptionally powerful tool to easily handle every case imaginable.
This is not something just off the top of my head. I've been using this technology for
several years now with tremendous success. One example is an activation framework
which allows to get source code from CVS, build a complex enterprise distributed app
configure, deploy and start it in desired environment/mode even from within IDEA
for debug. All it really does is just generate a proper set of ant and config files from
distributed activation spec. Another example is a GUI framework.
The amount of code to write is tremendously small (few hundred lines) since I already
have all major building blocks (ListMap and TemplateProcessor) in place.
--Oleg
That's it!!
That is exactly it
There is no need for anything other than a top level work space containing projects, that can contain other projects.
JDK settings etc. inherited from the parent!
Yes, that's it!
+10
+10 indeed!!
Gosh, it really is that simple...
Well, but what's the difference of the above approach and the current IDEA's
one (if we just replace "workspace" with "project" and "project" with
"module")?
--
Valentin Kipiatkov
JetBrains, Inc
http://www.intellij.com
"Develop with pleasure!"
"Robin van het Hof" <no_mail@jetbrains.com> wrote in message
news:18329586.1069749453801.JavaMail.itn@is.intellij.net...
>
Since I generally trust the technical decisons taken the IntelliJ team - and since things are sometimes not as simple as they appear to be - I am interested in JetBrains honest opinion:
Is it really that simple? Are there some important things overlooked by the previous posts? Is IDEA's model more complicated than needed (while not flexible enough for some people)? Can JetBrains make IDEA's model more user friendly - and better understood? (This does not necessarily mean a radical change of design and implementation. Don't underestimate the importance of good documentation and tutorials!) If yes, will this be addressed at some point (in Aurora or later)?
Thanks!
Vlad
Sorry, I hadn't seen Valentin's reply above. However, I would still be interested in a somewhat more detailed answer.
Valentin Kipiatkov (JetBrains) wrote:
>Well, but what's the difference of the above approach and the current IDEA's
>one (if we just replace "workspace" with "project" and "project" with
>"module")?
>
You can work in, and run a project in isolation, but not a module.
This is mostly what motivated this suggestion :
http://www.intellij.net/forums/thread.jsp?forum=22&thread=53611
Alain
Why not? You can create a new project and add this module to it. (If this
module depend on some others it would make sense to add those modules as
well.)
--
Valentin Kipiatkov
JetBrains, Inc
http://www.intellij.com
"Develop with pleasure!"
"Alain Ravet" <alain.ravet.list@wanadoo.be> wrote in message
news:bpvik0$lue$2@is.intellij.net...
>
IDEA's
>
>
>
>
>
Valentin Kipiatkov (JetBrains) wrote:
>>You can work in, and run a project in isolation, but not a module.
>>
>>
>
>Why not? You can create a new project and add this module to it. (If this
>module depend on some others it would make sense to add those modules as
>well.)
>
What is missing in a module, that requires us to wrap it in an - empty
(1-module) - project, to start working on it?
Once module sharing is implemented, we'll be able to wrap 1 single
module in multiple "empty" (1-module) project.
That will be possible, but I can't see any advantage/use for this feature?!
Alain
With respect to tags which are/are not defined/inherited, is it sufficient to simply inherit from the parent when the tag is not defined? Or does each tag need to specify whether it may be inherited even if it is defined, or whether it overrides children.
JDK is a good example. I could see defining a little project and setting the jdk to 1.3. Then I use that project in a larger project which uses 1.4. And I want to force all the subprojects to use 1.4.
Is there also a need to allow for saying "Must use 1.3 or greater"?
It seems that each tag may have a slightly complex set of rules regarding its inheritability/overrideability.
"Valentin Kipiatkov (JetBrains)" <valentin@intellij.com> wrote in message news:bpvftf$qd8$1@is.intellij.net...
>
...Then why not to do it? It would eliminate a lot of confusion....
But I guess the main problem/difference is your general approach to solve
this problem. You're trying to solve the problem indeed for every possible
scenario which, I think, is a futile attempt. IMHO you would be much better
off if you provide means to solve the problem instead and (optionally) a set
of ready-to-use solution for typical cases.
I've already mentioned a possible (and proven) way to achieve this. You could
turn your project/module/library architecture into a simple-to-understand
macro generator which would use xml-based project/workspace/whatever
specs/templates to generate an ant (or maven, or other widely used build/deploy
utility) file, for example, and then run it. Anyone (well, at least majority) nowdays
knows ant. There are a lot of people working to add functionality to this tool and,
ultimately, anyone can do it himself easily since it's an open source thing.
So you would not need to offer a solution for every possible case while
achieving ultimate flexibility (TRIZ-ideal system).
The required modification to the ant is very minimal. You only need an extra option
not to create a different classloader every time if you're going to debug it from within
IDEA. It took me about 10 minutes to fix and I'm sure you can work this out with
the ant team.
Just to give a real-life example, I've attached an exerpt from one of my projects.
It contains two folders "config" and "runtime". The former contains some project
templates and the latter contains the generated stuff. The notation
designates macro. As a simplest thing you can
compare config/AntActivate.ant (source) with the runtime/med/AntActivate.ant
(result). The contents of config/ConfigDefaults.any might seem somewhat scary at
first, but here is an incomplete list of things it does:
1. It controls compilation, deployment and runtime of several related client-server web apps.
2. Each app exists in multiple versions represented by different CVS branches
3. It is possible to run multiple instances of the same app/version at the same time
4. Each app can be built and run in several different roles (i.e. as server, client)
5. Each component can be deployed in different environments (inside/outside firewall)
6. Each component can communicate with different database (Oracle, MsSql) server.
7. Each component can store information in different tablespaces (especially for debug)
8. Each component can be compiled/run in different modes (Production, LocalDebug, RemoteDebug)
9. Each run mode assumes different app settings, tablespaces, log location/settings etc.
10. Each web component needs it's own web settings and run either on local or remote box.
11. Every app consists of multiple plugins which must be configured separately and consistently.
12 Each developer has it's own preferred way to do development yet this project spec must be shared.
13 etc, etc. etc.
Now try to achieve this with your project/module architecture On the other hand, I can do it
easily using a set of command line arguments and local config files which makes extremly
easy to personalize my work environmet (I'd just put my favorite options, logins, etc into
my local (app-wise, project-wise, location-wise, user-wise or machine-wise) config files).
The activation framework would use these options/files to override/inherit values from
a shared project spec. The best of all is that it's not carved in stone in any way. Anyone
from the team can locally override anythig including the very way how runtime configs
are produced. For example, anyone can introduce his own set of debug modes which might
or might not be descendants of predefined ones.
--Oleg
>
>
>
>
>
Attachment(s):
MED.ZIP
+Why not? You can create a new project and add this module to it. (If this module depend on some others it would make sense to add those modules as
well.)+
That's the difference!! ... :)
Why should we need to wrap anything? It's much more intuitive to just work on a project either standalone, and then just add it to another project when I require. I shouldn't have to have the same set of files as a module, and then again as a module wrapped in a project. They're the same thing.
The workspace (and I hope you don't mind me calling it that, but using Oleg's terminology, the whole thing has actually started to make sense to me) should hold a hiearcharchy of projects.
Each project contains content folders (and I'm not sure we need this level either) which holds src and test trees which can be added from anywhere, rather than just from a single root. The project should also contain compiler settings, or (as Oleg said) just inherit them from the parent project if none are set.
I've been struggling with this project/module thing for quite some time, and I think the problem I was having, was that modules appeared to do very little, apart from add an extra level of complexity.
Is there also a need to allow for saying "Must use 1.3 or greater"?
I think it might be better to keep things simple; a JDK must be specifically set, otherwise we may fall foul to a sudden change in name/numbering schemes later on.
I was going to disagree with you. I was going to point out that there are
a number of things which it makes sense to set at the project level but not
at the module level (to use the current terminology). But then I thought
about it and I could only come up with two things: the compiler and the JDK.
Even with those, there is a case to be made for setting them per-module. So
instead I find myself starting to agree...
Vil.
Rayz wrote:
--
Vilya Harvey
vilya.harvey@digitalsteps.com / digital steps /
(W) +44 (0)1483 469 480
(M) +44 (0)7816 678 457 http://www.digitalsteps.com/
+I was going to disagree with you. I was going to point out that there are
a number of things which it makes sense to set at the project level but not
at the module level (to use the current terminology). But then I thought
about it and I could only come up with two things: the compiler and the JDK.
Even with those, there is a case to be made for setting them per-module. So
instead I find myself starting to agree...+
Yes, project settings are set in each of the included projects; if they are not, then they are inherited from the parent.
As someone has already suggested, each project has a set of checkboxes that allows it to override the settings in its child projects.
That way I could have a set of JARS for a project, then it can automatically pick up a separate set of JARS when it is part of another project ...
+which holds src and test trees which can be added from anywhere, rather than just from a single root. +
... which would also aloow folk more freedom in how they set up there projects on disk.
At the moment, IDEA is starting to dictate the structure of folks' projects.
>Is there also a need to allow for saying "Must use 1.3 >or greater"?
Seems unlikely, simply because there are just enough backwards incompatible changes between 1.4 and 1.3 as to make such a statement problematic. I just spent several days tracing down an issue where the following code got accidentally compiled under 1.4 that was meant to run in 1.3, and ended up getting NoSuchMethod errors.
StringBuffer foo = new StringBuffer();
StringBuffer bar = new StringBuffer();
bar.append(foo);
A no-prize to the first person who posts what the incompatibility issue with that code is. In any case, this extremely simple fragment should show the problem with saying "Requires 1.3 or greater".
--Dave
Dave Griffith wrote:
>StringBuffer foo = new StringBuffer();
>StringBuffer bar = new StringBuffer();
>bar.append(foo);
>
>A no-prize to the first person who posts what the incompatibility issue with that code is. In any case, this extremely simple fragment should show the problem with saying "Requires 1.3 or greater".
>
>--Dave
>
Ctrl-Q on 'append' => Since 1.4
Right, but the gotcha is that there was a method append(Object) in 1.3.
1.4 only introduced append(StringBuffer). I've seen this on lists a
couple of times, never experienced it thankfully since we build and
deploy on the same platform. Phew!