Relative paths in libraries

I asked a similar question recently in this group, and didn't get any
response, so here it goes again:

Suppose I have my local source directories arranged like this:

src
sharedlib
componentA
componentB

I have IDEA projects for sharedlib, componentA, and componentB. Both
componentA and componentB depend on sharedlib. I have defined a Library
in IDEA named "sharedlib", which is referenced in the .ipr files for
both componentA and componentB.

My source code is stored in a branch-capable SCM like CVS, etc. The
project files (.ipr) are stored in CVS.

Some time after release 1.0 of my product, I need to make some bug fixes
to the 1.0 release. The 1.0 release is now on a branch, with the main
trunk being ongoing 2.0 work. So, I check out the 1.0 branch into a
separate local directory, let's say, "branch". My branch source tree
looks like

branch
sharedlib
componentA
componentB

Now, the .ipr files in branch/componentA and branch/componentB both
refer to a Library named "sharedlib". However, currently (in Ariadna),
library definitions can only hold absolute paths, so, I'm screwed. I
cannot safely build from this branch because the "sharedlib" Library
definition referenced by "componentA/componentA.ipr" and
"componentB/componentB.ipr" refer to "/src/sharedlib" whereas I need it
to refer to "/branch/sharedlib" (actually what I really need is for the
Library definition to refer to "../sharedlib"- this would enable me to
check out my source to whatever local directory I wanted to).

I thought Aurora would add relative path capability in Library
definitions, but, my current understanding is that only Project Library
definitions can hold relative paths. Since Project Library definitions
are stored in the .ipr file, I cannot share the definition of a
"sharedlib" Project Library between componentA and componentB- I have to
redefine it in each project.

JetBrains gurus- is my understanding correct? Will there be any way in
Aurora to have relative paths in Global Library definitions? If not,
doesn't this common use-case argue that Aurora needs this capability?

Thanks for any advice-

Danny Burkes

13 comments
Comment actions Permalink

Daniel,
I am afraid I miss the point a bit...
Is sharedlib something you develop or external library?

Friendly,
Dmitry

Daniel Burkes wrote:

I asked a similar question recently in this group, and didn't get any
response, so here it goes again:

Suppose I have my local source directories arranged like this:

src
sharedlib
componentA
componentB

I have IDEA projects for sharedlib, componentA, and componentB. Both
componentA and componentB depend on sharedlib. I have defined a Library
in IDEA named "sharedlib", which is referenced in the .ipr files for
both componentA and componentB.

My source code is stored in a branch-capable SCM like CVS, etc. The
project files (.ipr) are stored in CVS.

Some time after release 1.0 of my product, I need to make some bug fixes
to the 1.0 release. The 1.0 release is now on a branch, with the main
trunk being ongoing 2.0 work. So, I check out the 1.0 branch into a
separate local directory, let's say, "branch". My branch source tree
looks like

branch
sharedlib
componentA
componentB

Now, the .ipr files in branch/componentA and branch/componentB both
refer to a Library named "sharedlib". However, currently (in Ariadna),
library definitions can only hold absolute paths, so, I'm screwed. I
cannot safely build from this branch because the "sharedlib" Library
definition referenced by "componentA/componentA.ipr" and
"componentB/componentB.ipr" refer to "/src/sharedlib" whereas I need it
to refer to "/branch/sharedlib" (actually what I really need is for the
Library definition to refer to "../sharedlib"- this would enable me to
check out my source to whatever local directory I wanted to).

I thought Aurora would add relative path capability in Library
definitions, but, my current understanding is that only Project Library
definitions can hold relative paths. Since Project Library definitions
are stored in the .ipr file, I cannot share the definition of a
"sharedlib" Project Library between componentA and componentB- I have to
redefine it in each project.

JetBrains gurus- is my understanding correct? Will there be any way in
Aurora to have relative paths in Global Library definitions? If not,
doesn't this common use-case argue that Aurora needs this capability?

Thanks for any advice-

Danny Burkes


--
Dmitry Lomov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Comment actions Permalink

Hi Dmitry-

sharedlib is something I develop. It contains classes that are used by
both componentA and componentB. Further, it's contents might change
between release 1.0 and release 2.0 of my product, so I need IDEA to use
the 1.0 source for sharedlib when building the 1.0 branch of my product,
and use the 2.0 source for sharedlib when building the 2.0 branch of my
product.

If I can explain anything else, please let me know, because I am really
keen to get a solution to this problem!

Regards,

Danny

Daniel,
I am afraid I miss the point a bit...
Is sharedlib something you develop or external library?

Friendly,
Dmitry

Daniel Burkes wrote:

I asked a similar question recently in this group, and didn't get any
response, so here it goes again:

Suppose I have my local source directories arranged like this:

src
sharedlib
componentA
componentB

I have IDEA projects for sharedlib, componentA, and componentB. Both
componentA and componentB depend on sharedlib. I have defined a Library
in IDEA named "sharedlib", which is referenced in the .ipr files for
both componentA and componentB.

My source code is stored in a branch-capable SCM like CVS, etc. The
project files (.ipr) are stored in CVS.

Some time after release 1.0 of my product, I need to make some bug fixes
to the 1.0 release. The 1.0 release is now on a branch, with the main
trunk being ongoing 2.0 work. So, I check out the 1.0 branch into a
separate local directory, let's say, "branch". My branch source tree
looks like

branch
sharedlib
componentA
componentB

Now, the .ipr files in branch/componentA and branch/componentB both
refer to a Library named "sharedlib". However, currently (in Ariadna),
library definitions can only hold absolute paths, so, I'm screwed. I
cannot safely build from this branch because the "sharedlib" Library
definition referenced by "componentA/componentA.ipr" and
"componentB/componentB.ipr" refer to "/src/sharedlib" whereas I need it
to refer to "/branch/sharedlib" (actually what I really need is for the
Library definition to refer to "../sharedlib"- this would enable me to
check out my source to whatever local directory I wanted to).

I thought Aurora would add relative path capability in Library
definitions, but, my current understanding is that only Project Library

0
Comment actions Permalink

Daniel,
as soon as we have multiple modules, you will be able to make a sharedlib,
a componentA and a componentB three separate modules - and things will work
as you want.

Libraries are not supposed to be something you develop - they are something
that comes from somwhere else.

Friendly,
Dmitry

Daniel Burkes wrote:

Hi Dmitry-

sharedlib is something I develop. It contains classes that are used by
both componentA and componentB. Further, it's contents might change
between release 1.0 and release 2.0 of my product, so I need IDEA to use
the 1.0 source for sharedlib when building the 1.0 branch of my product,
and use the 2.0 source for sharedlib when building the 2.0 branch of my
product.

If I can explain anything else, please let me know, because I am really
keen to get a solution to this problem!

Regards,

Danny

>> Daniel,
>> I am afraid I miss the point a bit...
>> Is sharedlib something you develop or external library?
>>
>> Friendly,
>> Dmitry
>>
>> Daniel Burkes wrote:
>>
>> > I asked a similar question recently in this group, and didn't get any
>> > response, so here it goes again:
>> >
>> > Suppose I have my local source directories arranged like this:
>> >
>> > src
>> > sharedlib
>> > componentA
>> > componentB
>> >
>> > I have IDEA projects for sharedlib, componentA, and componentB. Both
>> > componentA and componentB depend on sharedlib. I have defined a
>> > Library in IDEA named "sharedlib", which is referenced in the .ipr
>> > files for both componentA and componentB.
>> >
>> > My source code is stored in a branch-capable SCM like CVS, etc. The
>> > project files (.ipr) are stored in CVS.
>> >
>> > Some time after release 1.0 of my product, I need to make some bug
>> > fixes
>> > to the 1.0 release. The 1.0 release is now on a branch, with the main
>> > trunk being ongoing 2.0 work. So, I check out the 1.0 branch into a
>> > separate local directory, let's say, "branch". My branch source tree
>> > looks like
>> >
>> > branch
>> > sharedlib
>> > componentA
>> > componentB
>> >
>> > Now, the .ipr files in branch/componentA and branch/componentB both
>> > refer to a Library named "sharedlib". However, currently (in Ariadna),
>> > library definitions can only hold absolute paths, so, I'm screwed. I
>> > cannot safely build from this branch because the "sharedlib" Library
>> > definition referenced by "componentA/componentA.ipr" and
>> > "componentB/componentB.ipr" refer to "/src/sharedlib" whereas I need it
>> > to refer to "/branch/sharedlib" (actually what I really need is for the
>> > Library definition to refer to "../sharedlib"- this would enable me to
>> > check out my source to whatever local directory I wanted to).
>> >
>> > I thought Aurora would add relative path capability in Library
>> > definitions, but, my current understanding is that only Project Library

--
Dmitry Lomov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Comment actions Permalink

Thanks for the info, Dmitry. Let me make sure I understand you.

You are saying that module path information will be stored as relative
paths, so that src/componentA will build with src/sharedlib, and
branch/componentA will build with branch/sharedlib, even if the
src/componentA/componentA.ipr and branch/componentA/componentA.ipr files
are identical?

Also, I understand your statement that "Libraries are not supposed to be
something you develop - they are something that comes from somewhere
else", but this still does not obviate the need for relative paths in
Library definitions. Even in the case of third-party libraries that are
developed elsewhere, these jarfiles can change between revision 1.0 and
revision 2.0 of my product. For example, let's extend the physical
project structure I mentioned before:

src
sharedlib
externalLibraries
componentA
componentB

and

branch
sharedlib
externalLibraries
componentA
componentB

where the "externalLibraries" directory contains third-party jarfiles
that are needed by "sharedlib", "componentA", and "componentB", and
these jarfiles are stored in CVS. Suppose revision 1.0 of my product
uses revision 5.0 of SomeApacheLibrary, then I upgrade to
SomeApacheLibrary 6.0 for revision 2.0 of my product. So, when I build
src/componentA, I need for it to use
src/externalLibraries/SomeApacheLibrary.jar, and when I build
branch/componentA, I need for it to use
branch/externalLibraries/SomeApacheLibrary. Are you saying that Library
definitions can never have relative paths? If so, I guess I will have
to define and use a module for SomeApacheLibrary, rather than a Library?

Thanks for the clarification-

Danny

- Danny

Daniel,
as soon as we have multiple modules, you will be able to make a sharedlib,
a componentA and a componentB three separate modules - and things will work
as you want.

Libraries are not supposed to be something you develop - they are something
that comes from somwhere else.

Friendly,
Dmitry

Daniel Burkes wrote:

Hi Dmitry-

sharedlib is something I develop. It contains classes that are used by
both componentA and componentB. Further, it's contents might change
between release 1.0 and release 2.0 of my product, so I need IDEA to use
the 1.0 source for sharedlib when building the 1.0 branch of my product,
and use the 2.0 source for sharedlib when building the 2.0 branch of my
product.

If I can explain anything else, please let me know, because I am really
keen to get a solution to this problem!

Regards,

Danny

>> Daniel,
>> I am afraid I miss the point a bit...
>> Is sharedlib something you develop or external library?
>>
>> Friendly,
>> Dmitry
>>
>> Daniel Burkes wrote:
>>
>> > I asked a similar question recently in this group, and didn't get any
>> > response, so here it goes again:
>> >
>> > Suppose I have my local source directories arranged like this:
>> >
>> > src
>> > sharedlib
>> > componentA
>> > componentB
>> >
>> > I have IDEA projects for sharedlib, componentA, and componentB. Both
>> > componentA and componentB depend on sharedlib. I have defined a

0
Comment actions Permalink

Can anyone from JetBrains answer the follow-up questions I posted above?

Thanks-

Danny

0
Comment actions Permalink

Daniel,
sorry for the long time it took me to reply.

Daniel Burkes wrote:

Thanks for the info, Dmitry. Let me make sure I understand you.

You are saying that module path information will be stored as relative
paths, so that src/componentA will build with src/sharedlib, and
branch/componentA will build with branch/sharedlib, even if the
src/componentA/componentA.ipr and branch/componentA/componentA.ipr files
are identical?


Well, it will work so that modules will refer to each other by their names,
that is, componentA will say, "I depend on 'sharedlib'" and componentB will
say "I depend on 'sharedlib'". When you add all three to one project, they
will find each other :)

Also, I understand your statement that "Libraries are not supposed to be
something you develop - they are something that comes from somewhere
else", but this still does not obviate the need for relative paths in
Library definitions. Even in the case of third-party libraries that are
developed elsewhere, these jarfiles can change between revision 1.0 and
revision 2.0 of my product. For example, let's extend the physical
project structure I mentioned before:

src
sharedlib
externalLibraries
componentA
componentB

and

branch
sharedlib
externalLibraries
componentA
componentB

where the "externalLibraries" directory contains third-party jarfiles
that are needed by "sharedlib", "componentA", and "componentB", and
these jarfiles are stored in CVS. Suppose revision 1.0 of my product
uses revision 5.0 of SomeApacheLibrary, then I upgrade to
SomeApacheLibrary 6.0 for revision 2.0 of my product. So, when I build
src/componentA, I need for it to use
src/externalLibraries/SomeApacheLibrary.jar, and when I build
branch/componentA, I need for it to use
branch/externalLibraries/SomeApacheLibrary. Are you saying that Library
definitions can never have relative paths? If so, I guess I will have
to define and use a module for SomeApacheLibrary, rather than a Library?


You have two options: either to include library as part of project (project
level library), or have 'versioned' global libraries, the way Gordon
suggested in the other reply.

Cheers,
Dmitry

--
Dmitry Lomov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Comment actions Permalink

Dmitry Lomov wrote:
>>src
>> sharedlib
>> externalLibraries
>> componentA
>> componentB
>>
>>and
>>
>>branch
>> sharedlib
>> externalLibraries
>> componentA
>> componentB
>>
>>where the "externalLibraries" directory contains third-party jarfiles
>>that are needed by "sharedlib", "componentA", and "componentB", and
>>these jarfiles are stored in CVS. Suppose revision 1.0 of my product
>>uses revision 5.0 of SomeApacheLibrary, then I upgrade to
>>SomeApacheLibrary 6.0 for revision 2.0 of my product. So, when I build
>>src/componentA, I need for it to use
>>src/externalLibraries/SomeApacheLibrary.jar, and when I build
>>branch/componentA, I need for it to use
>>branch/externalLibraries/SomeApacheLibrary. Are you saying that Library
>>definitions can never have relative paths? If so, I guess I will have
>>to define and use a module for SomeApacheLibrary, rather than a Library?


You have two options: either to include library as part of project (project
level library), or have 'versioned' global libraries, the way Gordon
suggested in the other reply.


I did?

Hmmm... Oh I think I understand what you're referring to.

The way we organise the PerformaSure project is that everything needed
to build the project is in its CVS module. So we have the following
directory structure (pruned for brevity):

D:\dev\performasure\head (my checkout point for the HEAD branch)
src\ -- PerformaSure module from CVS (yeh, imaginative, eh?)
build.xml -- Monolithic Ant build script - 50KB, 1200 lines ;)
build\ -- Build support
lib\ -- Jars needed to build but not needed at runtime
dist\ -- Template for distribution archive
lib\ -- Jars needed to build and needed at runtime
java\ -- Java source
com\...
java-test\ -- Java test source (unit tests, etc.)
com\...

So in the two lib directories, we have jars checked into CVS, and it's
simply a matter of checking out the module and running ant to build the
whole system.

Some of the jars are libraries developed in-house but are not
sub-projects of PerformaSure, e.g. licensing, JClass. We use particular
versions of the libraries for particular versions of PerformaSure. When
we need to use a newer version, we get a new jar and check it into CVS.

The java and java-test directories contains ALL the java source that
constitutes the project and they are compiled into separate classes and
classes-test directories. Separation of packages into jars is done in
the build script, so for example, com/sitraka/pas/common goes into
performasure-common.jar and com/sitraka/pas/agent goes into
performasure-agent.jar (they're more complicated than that but you get
the idea). We use Classpath entries in the jar manifest files to set
runtime dependencies between the jars, so we only need to reference one
main jar (performasure-agent, performasure-nexus, etc.) on the Java
command line classpath.

As an indication of how well this works, we've got 8 major branches (for
each release of the product), 2 of which are currently active. As a
developer I have most of these branches checked out and I am able to
develop and build any of them independently of the others. The only
external dependency is Ant, of which we're using 1.5 which luckily is
compatible with all the versions of our build scripts (well as far back
as I've checked ;).

Ciao,
Gordon

--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919

0
Comment actions Permalink

Daniel,
sorry for the long time it took me to reply.

>

No problem - thanks for your time! :)

You have two options: either to include library as
part of project (project
level library),


That means I have to redefine the library definition for SomeApacheLibrary in every project that uses it. I am trying to avoid this, because our product has hundreds of sub-projects, and I don't want to redefine the library definition hundreds of times.

or have 'versioned' global libraries,
the way Gordon
suggested in the other reply.


That means I have to have duplicate every global library definition for each branch of my product. Also, since global library definitions contain only absolute paths, it means that every developer must checkout the source code to the exact same absolute path on his local machine.

Don't you find all of these limitations unacceptable? All of these limitations would be eliminated if global libraries could hold relative paths, which seems simple to implement for JetBrains. If this were implemented, shared global library definitions would work regardless of the absolute path where the developer checked out the source tree.

I find it hard to believe that no one else watching this thread feels the severe impact of these limitations!

Regards,

Danny

0
Comment actions Permalink

Hi Gordon-

Thanks for the detailed reply!

Hmmm... Oh I think I understand what you're referring
to.

The way we organise the PerformaSure project is that
everything needed
to build the project is in its CVS module. So we have
the following
directory structure (pruned for brevity):

D:\dev\performasure\head (my checkout point for the
HEAD branch)
src\ -- PerformaSure module from CVS (yeh,
h, imaginative, eh?)
build.xml -- Monolithic Ant build script -
pt - 50KB, 1200 lines ;)
build\ -- Build support
lib\ -- Jars needed to build but not
ut not needed at runtime
dist\ -- Template for distribution archive
lib\ -- Jars needed to build and needed at
ded at runtime
java\ -- Java source
com\...
java-test\ -- Java test source (unit tests,
sts, etc.)
com\...

>

Our directory structure is logically the same, so we face the same issues...

So in the two lib directories, we have jars checked
into CVS, and it's
simply a matter of checking out the module and
running ant to build the
whole system.


I think you just mentioned the most salient point- that you don't use IDEA for building, you use Ant. It's apparent from Dmitry's replies that using IDEA for building in this type of system would be very difficult, because

1. One would have to duplicate every shared library definition per branch.

2. Every developer would have to check out the source to some fixed, prescribed absolute path on his local machine, since shared library definitions can only hold absolute paths.

So I guess you just don't use the IDEA build system at all, even in your daily development work?

Regards,

Danny

0
Comment actions Permalink

Daniel Burkes wrote:

Our directory structure is logically the same, so we face the same issues...


I was actually under the impression that your library jars are in a
separate CVS module which needs to be checked out in addition to your
project source.

I think you just mentioned the most salient point- that you don't use IDEA for building, you use Ant. It's apparent from Dmitry's replies that using IDEA for building in this type of system would be very difficult, because

1. One would have to duplicate every shared library definition per branch.


The Ariadna IDEA project that I use, which is checked into CVS and
mostly everybody else that uses IDEA also uses it, has all the jars from
those two lib directories in the Classpath list. Since it's in CVS, it's
versioned between branches, and since all the jars are relative to the
project directory, all the paths in the project file are relative too.

2. Every developer would have to check out the source to some fixed, prescribed absolute path on his local machine, since shared library definitions can only hold absolute paths.


I have converted our Ariadna project successfully though to Aurora by
doing pretty much the same thing as the Ariadna Classpath and adding all
the jars from the lib directories into the Library list as un-named
libraries.

So I guess you just don't use the IDEA build system at all, even in your daily development work?


Nope. I have keybindings which invoke various compile targets in the
build script. Our build process is too complicated for IDEA to handle.
And that's okay, because we have build systems like Ant.

But, as I described above, I still need to have all the jars referenced
in the IDEA project so that I don't get red wiggly lines all over the
editor.

Ciao,
Gordon

--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919

0
Comment actions Permalink

The Ariadna IDEA project that I use, which is checked
into CVS and
mostly everybody else that uses IDEA also uses it,
has all the jars from
those two lib directories in the Classpath list.
Since it's in CVS, it's
versioned between branches, and since all the jars
are relative to the
project directory, all the paths in the project file
are relative too.

>

I see- so essentially you just use the Classpath tab rather than shared library definitions. I think being able to share library definitions would be nicer- the "library" concept ties together jars, source, and javadocs nicely, so you don't have to respecify this with each project.

I guess the most frustrating thing about this issue is that it seems like it would be easy for JetBrains to fix- just allow relative paths in Global library definitions.

Thanks again for your time in sharing your solutions!

Regards,

Danny

0
Comment actions Permalink

Daniel Burkes wrote:
>>The Ariadna IDEA project that I use, which is checked
>>into CVS and
>>mostly everybody else that uses IDEA also uses it,
>>has all the jars from
>>those two lib directories in the Classpath list.
>>Since it's in CVS, it's
>>versioned between branches, and since all the jars
>>are relative to the
>>project directory, all the paths in the project file
>>are relative too.


I see- so essentially you just use the Classpath tab rather than shared library definitions. I think being able to share library definitions would be nicer- the "library" concept ties together jars, source, and javadocs nicely, so you don't have to respecify this with each project.


Yeah, I guess my situation is different to yours in that I work on only
one project (albeit multiple branches). If I were working on many
different projects, my solution may not be the best.

I guess the most frustrating thing about this issue is that it seems like it would be easy for JetBrains to fix- just allow relative paths in Global library definitions.


Relative to what? Global library definitions are defined at the IDE
level. They're not related to a particular project. Global library
defintions should be absolute. Project library definitions should be
relative.

Ciao,
Gordon

--
Gordon Tyler (Software Developer)
Quest Software <http://java.quest.com/>
260 King Street East, Toronto, Ontario M5A 4L5, Canada
Voice: 416-643-4846 | Fax: 416-594-1919

0
Comment actions Permalink

Yeah, I guess my situation is different to yours in
that I work on only
one project (albeit multiple branches). If I were
working on many
different projects, my solution may not be the best.

>

Well, currently, I think your solution is the only way :)

I guess the most frustrating thing about this issue

is that it seems like it would be easy for JetBrains
to fix- just allow relative paths in Global library
definitions.

Relative to what? Global library definitions are
defined at the IDE
level. They're not related to a particular project.
Global library
defintions should be absolute. Project library
definitions should be
relative.


My point is, there should be some way to define a shared library definition (shared meaning you can define it once and use it in many projects) that uses relative paths (relative to the project directory, just like the current Project Library).

Actually, when I typed the previous paragraph, it got me thinking that perhaps the Default Project Properties could play into this somehow- meaning, could I define all my library definitions as Project Libraries in the Default Project Properties, so, when I create a new Project, it inherits these definitions. I tried it in Aurora and it seems to work! This might be a workaround- I will research it a little further.

Thanks for nudging me in the right direction :)

Regards,

Danny

0

Please sign in to leave a comment.